# Protein shape from sequence

Two small AI models read a protein's amino-acid sequence and predict, for every position, whether it sits in a helix, a strand or a loop, shown on real 3D structures.

Can a small protein model learn structure without ever predicting amino acids? I pretrained eleven 7.4M-parameter encoders on the same 486M-token budget, comparing masked language modelling with latent prediction against a self-distilled target and against frozen ESM-IF1 structure embeddings. The self-distilled version collapsed as predicted and latent prediction alone did not beat masked language modelling, but distilling the structure embeddings directly did on every task. The demo runs two of the encoders in your browser and predicts secondary structure for any sequence.

Interactive tool: eight well-known proteins (green fluorescent protein, calmodulin, leghemoglobin, T4 lysozyme, TNF-alpha, an antibody heavy chain, BPTI and carbonic anhydrase I), each drawn in 3D from its PDB structure and coloured by its real or predicted secondary structure. Predictions for these are precomputed. A pasted sequence downloads the two models (C5b, trained to match ESM-IF1 structure embeddings; C1, masked language modelling) and runs them in the browser with onnxruntime-web. The page ends with the study's result tables.

Part of [Jacob Davies](https://jacob-davies.com/).