If that paper about astra being able to make sense of other models’ weights has any merit, at some point they became so desperate they actually started adding serialized open weight models to the training data.