MiniFold: Simple, Fast, and Accurate Protein Structure Prediction

Link: full article

Authors: Jeremy Wohlwend, Mateo Reveiz, Matt McPartlon, Axel Feldmann, Wengong Jin, Regina Barzilay

Transactions on Machine Learning Research, February 2, 2025

Since AlphaFold2, protein structure prediction has become remarkably usable. But there is still a huge gap between ‘usable’ and ‘affordable’: scanning 10 million antibody sequences, even at just a few seconds per sequence, would consume close to 1 GPU-year.

MIT’s Regina Barzilay team published MiniFold in TMLR—a significantly streamlined protein structure prediction architecture. Core finding: the Evoformer and IPA structural modules inherited from AlphaFold2 in ESMFold account for over 95% of inference time, yet many components are not essential. After removing them, with the same ESM-2 language model, MiniFold maintains essentially comparable accuracy on CAMEO/CASP, with 10-20× speed improvement and substantially reduced memory usage.

Part 1: Research Question

The applications of protein structure prediction are expanding rapidly: large-scale virtual screening, antibody library scanning, mutational effect assessment, filtering de novo design candidates, and downstream fine-tuning. All of these demand high throughput, and the computational cost of current models (AlphaFold2, ESMFold) severely limits deployment at scale.

PLM-based models such as ESMFold replace AlphaFold2’s expensive MSA search with a protein language model (ESM-2), cutting inference from minutes to seconds. But the authors point out that the real bottleneck is not the language model itself:

Breakdown of ESMFold inference time

ESM-2 protein language model: < 5%

Evoformer (including triangular operations): ~80%

Structure module (IPA): ~15%

In other words, ESMFold has already dropped the MSA search, but the heavy folding trunk it inherits from AlphaFold2 remains almost untouched. The authors’ central question is:

Once the input switches from an MSA to protein language model embeddings, should the downstream folding architecture be redesigned as well?

Part 2: Methodology

MiniFold keeps the three-stage architecture (embedding → folding trunk → structure module) but greatly simplifies the latter two stages.

Miniformer: a slimmed-down Evoformer

Removing the sequence track: ESMFold’s Evoformer contains two pathways, a sequence track and a pairwise track, and the sequence track alone holds 600M+ parameters. MiniFold simply removes it and keeps only the pairwise track. After removal, fewer than 25M parameters remain in the Evoformer part, with almost no effect on accuracy. This suggests that the Evoformer’s reasoning ability is determined mainly by the pairwise operations themselves and has little to do with parameter count.

Removing triangular attention: AlphaFold-style architectures contain two kinds of operations, triangular attention and the triangular multiplicative update. The authors found that triangular attention consumes a lot of memory but contributes little; the truly critical operation is the triangular multiplicative update. Triangular attention is therefore removed entirely.

Redesigning the triangular multiplicative update: the previously separate incoming and outgoing modules are merged into a single bi-directional triangular update. Concretely, the pair embedding channels are split into 4 chunks, matrix multiplications are performed along the two directions, and the outputs are concatenated. This design is simpler and also better suited to GPU kernel fusion.

In the end, a Miniformer block has only two sublayers: the bi-directional triangular update plus a two-layer FFN.

Structure Realizer: redefining the role of the structure module

The authors put forward a thought-provoking hypothesis: the main structural information has already been encoded by the pairwise distogram module, and the subsequent structure module acts more like a coordinate ‘realizer’.

To test this hypothesis, they ran a clever control experiment: they used MDS (multidimensional scaling), a method with no learnable parameters at all, to recover Cα coordinates directly from the distogram. Remarkably, this non-parametric method recovered the Cα trace about as well as a lightweight transformer realizer.

Building on this, MiniFold replaces AlphaFold2’s heavy IPA module with an 8-layer gated transformer with pair bias. Unlike IPA, it does not iteratively update coordinates layer by layer; instead it predicts the N, Cα and C atom coordinates in one shot at the end and then constructs local frames via Gram-Schmidt.

GPU kernel optimization: reducing memory traffic

MiniFold’s speedup comes not only from architectural simplification but also from two custom GPU kernels written in Triton:

1. Self-gating kernel: fuses the linear projection, sigmoid and elementwise multiplication in the triangular update into a single kernel, reducing HBM reads and writes.

2. Feed-forward kernel: the FFN’s hidden dimension is 4x the input dimension, which would consume a great deal of memory if materialized explicitly. The new kernel scans the weight matrices block by block, avoiding writing the large intermediate activations back to HBM.

The idea is the same as FlashAttention: what is reduced is not FLOPs but expensive memory traffic. The two kernels provide an additional 2-3x speedup on long sequences.

Part 3: Key Figure Analysis

Figure 1: MiniFold architecture overview

What this figure asks: What does MiniFold’s overall architecture look like?

Figure 1. MiniFold architecture: ESM-2 extracts sequence embeddings and pairwise attention maps → the Miniformer (48 blocks) updates the pairwise representation → the Structure Realizer (8-layer gated transformer) predicts all-atom coordinates.

How to read this figure

Top: the Embedding Module uses ESM-2 (3 billion parameters) to extract sequence embeddings and pairwise attention maps, which are concatenated into the initial pair representation.

Top right: the Miniformer Module consists of 48 minimal blocks, each containing only a bi-directional triangular multiplicative update plus a transition (FFN). Note that recycling is applied only to the pairwise blocks.

Bottom: the Realizer Module uses an 8-layer gated transformer with pair bias to output backbone frames and torsion angles in one shot.

Takeaway: Compared with ESMFold, MiniFold drops the Evoformer’s sequence track and triangular attention and replaces IPA with a lightweight transformer. The overall pipeline keeps its three stages, but each stage is systematically slimmed down.

Table 1: Accuracy comparison

What this table asks: With this much simplification, how much accuracy is actually lost?

What this figure asks: How far is MiniFold’s prediction accuracy from the full AlphaFold2 on standard benchmarks?

Table 1. MiniFold (12/48 layers) compared with ESMFold, OmegaFold, AlphaFold2 and RosettaFold on CAMEO, CASP14 and CASP15.

Key numbers

CAMEO: MiniFold-48 reaches a TM-score of 0.846 vs 0.837 for ESMFold. MiniFold is actually slightly better than ESMFold, while runtime drops from 11.4 s/seq to 0.8 s/seq, 14x faster.

CASP14: MiniFold-48 TM-score 0.641 vs 0.656 for ESMFold. Slightly weaker, but the gap is small, and it is about 9x faster.

CASP15: MiniFold-48 TM-score 0.657 vs 0.659 for ESMFold. Essentially on par, and about 8x faster.

12-layer version: on CAMEO, MiniFold-12 takes only 0.36 s/seq, 1/32 of ESMFold’s runtime, with accuracy still in an acceptable range.

Takeaway: Because MiniFold and ESMFold share the same PLM (ESM-2), the comparison is very fair. The conclusion is clear: the folding trunk of PLM-based structure predictors can be greatly simplified. A gap remains relative to MSA-based models such as AlphaFold2, but it stems mainly from the difference between PLM and MSA inputs.

Figure 4: Speed and memory comparison

What this figure asks: How does the speedup change with sequence length?

Figure 4. Left: inference time as a function of sequence length (191 CAMEO targets). Right: peak memory as a function of sequence length.

How to read this figure

Left (inference time): MiniFold in green, ESMFold in red. The gap between the two curves widens sharply as sequences get longer: about 10x faster for a length-400 protein and about 20x faster for a length-1000 protein.

Right (peak memory): MiniFold’s memory grows more gently. On long sequences, memory drops by nearly 10x. This matters a lot for real deployment, since it means the same GPU can handle longer proteins.

Takeaway: MiniFold’s advantage is more pronounced on long sequences, because the cost of the removed triangular attention grows faster with length. This shows that the architectural simplification and kernel optimization deliver more than a constant-factor speedup; they also improve scaling behavior.

Figure 5: Ablation study

What this figure asks: How much does each simplification step contribute?

Figure 5. Stepwise ablation from ESMFold’s Evoformer to the Miniformer. Left: memory improvement ratio. Right: speed improvement ratio. Cyan = original Evoformer, green = final Miniformer.

Contribution of each step

Removing the sequence track: parameter count falls sharply (600M → 25M), but inference speed barely improves. This suggests that the sequence track’s parameters are largely ‘redundant’.

Removing triangular attention: both speed and memory improve markedly; this is the largest single-step gain.

Bi-directional fusion + down-projection: further speedup.

Custom GPU kernels: an additional 2-3x speedup. At a protein length of 1024, the overall speed improvement reaches 10-20x.

Takeaway: The efficiency gain is the cumulative result of architectural simplification plus kernel fusion, with each step making a clear contribution. The speedup is systematic and follows a reproducible engineering path.

Part 4: Key Contributions

The MDS control: the distogram already contains the main structural information

The authors used MDS (multidimensional scaling) as a non-parametric structure-realization control. MDS has no learnable parameters at all; starting only from the distance matrix predicted by the distogram, it recovers Cα coordinates through shortest-path completion, eigendecomposition and LBFGS refinement.

The accuracy of Cα trace recovery is almost identical. This is one of the most illuminating experiments in the paper, because it shows that the Miniformer’s pairwise distogram already contains the vast majority of the structural information, and the subsequent structure module merely ‘realizes’ the coordinates. This provides conceptual support for drastically simplifying the structure module.

Uncertainty estimation

MiniFold also trains a pLDDT predictor, whose Pearson correlation with true lDDT is 0.76 on CAMEO and 0.90 on CASP14. The authors further found that distogram entropy can also serve as an uncertainty indicator: low-entropy predictions tend to correspond to more accurate structures, with a correlation of 0.90 on CASP14.

Training efficiency

MiniFold is trained on a single node of 8 × A100 GPUs for about 250K steps (in two stages: first the distogram, then adding the structure realizer and recycling). Because the architecture is lighter and uses less memory, training is also much faster and can use larger batch sizes.

Innovations and limitations

4 core innovations

1. Shows that a PLM-based folding trunk need not copy the AlphaFold2 architecture wholesale

2. Proposes the Miniformer: a minimal pairwise folding trunk (no sequence track, no triangular attention)

3. Redefines the structure module as a ‘coordinate realizer’ and validates this conceptually with the MDS experiment

4. Builds custom GPU kernels specifically for protein structure models, fusing operations to address the IO bottlenecks of self-gating and the FFN

Main limitations

Still weaker than strong MSA-based models: on CASP14/15, MiniFold, like ESMFold, trails AlphaFold2 and RosettaFold2 overall. The gap stems mostly from the difference in information content between single-sequence PLM input and multiple sequence alignments.

Conclusions may not transfer to MSA-based models: the authors themselves stress that these observations may not apply to MSA models such as AlphaFold2. In the MSA setting, sequence/pair interactions may matter more.

Focus on single-chain proteins: validation so far has mainly covered single-chain structure prediction; multi-chain complexes and protein-nucleic acid/small-molecule complexes have not yet been systematically tested.

Open questions worth following

1. Can the Miniformer design be transferred to AlphaFold2/Pairformer? If MSA-based models can be slimmed down in a similar way, the impact would be enormous.

2. MiniFold as a fast structure filter for generative models: use MiniFold-12 for large-scale coarse screening of de novo design candidates, then evaluate the survivors carefully with AlphaFold2. This may be the most practically valuable usage pattern.

3. Can it be extended to multi-chain proteins and complexes? If pairwise-only reasoning is strong enough, it may be just as effective for complex prediction.

4. Can a smaller PLM compress things further? Since the PLM accounts for only 5% of inference time, swapping in a smaller distilled PLM might lower the deployment barrier even more.

In one sentence: MiniFold shows that many expensive components that protein folding models inherited from AlphaFold2 are not necessary. Its contribution lies in re-examining the question of ‘which components really matter’, and the answer is pairwise triangular multiplicative reasoning: everything else can be slimmed down around it.

Paper: MiniFold: Simple, Fast, and Accurate Protein Structure Prediction

Authors: Jeremy Wohlwend*, Mateo Reveiz*, Matt McPartlon, Axel Feldmann, Wengong Jin, Regina Barzilay

Affiliations: MIT CSAIL / University of Chicago / Broad Institute of MIT and Harvard

Published: Transactions on Machine Learning Research (TMLR), 04/2025

Senior Author

The paper does not indicate a corresponding author; the senior author is Regina Barzilay, School of Engineering Distinguished Professor of AI & Health, Department of Electrical Engineering and Computer Science, MIT, and also serves as MIT Jameel Clinic AI Faculty Lead and is a member of CSAIL. She received her bachelor’s and master’s degrees from Ben-Gurion University of the Negev in Israel, her PhD from the Department of Computer Science at Columbia University, and completed her postdoctoral research at Cornell University. Her research spans natural language processing and AI for Science, with recent focus on applying deep learning to drug discovery and clinical AI. Representative achievements include the discovery of a novel antibiotic halicin through deep learning (published in Cell, 2020), and a series of AI models for early detection of breast cancer. She received the MacArthur Fellowship in 2017 (the “Genius Award”), the AAAI Squirrel AI Award for AI for the Benefit of Humanity in 2022, and was selected for the TIME100 AI list in 2025. Over 50,000 citations on Google Scholar with an h-index exceeding 100. In the protein structure prediction field, her team is also a major developer of Boltz-1 (a biomolecular interaction modeling model).

Citation

Wohlwend, J., Reveiz, M., McPartlon, M., Feldmann, A., Jin, W., & Barzilay, R. (2025). MiniFold: Simple, Fast, and Accurate Protein Structure Prediction. Transactions on Machine Learning Research. https://openreview.net/forum?id=1p9hQTbjgo