Two small AI models read a protein's amino-acid sequence and predict, for every position, whether it sits in a helix, a strand or a loop, shown on real 3D structures.
Can a small protein model learn structure without ever predicting amino acids? I pretrained eleven 7.4M-parameter encoders on the same 486M-token budget, comparing masked language modelling with latent prediction against a self-distilled target and against frozen ESM-IF1 structure embeddings. The self-distilled version collapsed as predicted and latent prediction alone did not beat masked language modelling, but distilling the structure embeddings directly did on every task. The demo runs two of the encoders in your browser and predicts secondary structure for any sequence.