Solved on the Set It Was Scored On

On 10 October, Latent Space published an episode with the title Why AlphaFold Didn’t Solve Protein Folding. The guests were Pushmeet Kohli of Google DeepMind and Sal Candido of Biohub. The title is not a critic’s: Kohli worked on AlphaFold at DeepMind. He says, of the original project, that “what we were trying to do was replicate a structure that somebody had obtained and deposited in the PDB,” and a few lines later: “that doesn’t mean we’ve understood all of protein dynamics.”

Later in the episode he gives the score. “AlphaFold 2 had a GDT score of about 90 on that set at the time.” Then the sentence I found more useful than the score, “even if it had a GDT score of 95, if its pLDDT score were completely uncalibrated, who would trust it?” (The timestamps I could check are 00:15:28 and 00:23:53; I read the transcript, not the audio.)

So “solved” is a claim about a particular test, and a person who worked on the scored model is pointing at what the test leaves out. That is a measurement question, and 2026 has produced some measurements. None of them says the structure-prediction work failed. They show what the pass mark was.

What a pass looks like

GDT_TS averages the share of C-alpha positions in a predicted structure that land within a set of distance cutoffs (1, 2, 4 and 8 Å) of an experimentally determined one. It rewards getting one structure right. The experimental structures behind it come from the Protein Data Bank, which holds what could be crystallized, imaged or measured by NMR and was chosen for solving, in the state the protein happened to be in at the time.

A recent perspective from Chakravarty and colleagues puts the achievement in those terms: AI has “largely solv[ed] the problem of identifying a dominant conformation from sequence” (arXiv, August 2026). It is a perspective, not a measurement. But the qualifier, dominant conformation, is the whole scope of the claim, and the rest of this piece is about what happens at its edge. I wrote about the same habit in another corner of DeepMind’s work in a post on AlphaGenome’s variant predictions: a headline score, and then a quieter question about which benchmark was the one measured in a lab.

The model leans on whichever state the PDB has more of

The cleanest 2026 test I found is in PNAS. Yu and colleagues took 82 enzymes whose domains move from open to closed when a ligand binds, and generated ensembles of AlphaFold 3 models with and without the triggering ligand (PNAS, March 2026).

If the model were reading physics, taking the ligand away should move the ensemble toward the open state for each enzyme the ligand closes. The result instead depended on what the PDB held. For enzymes with more apo than holo structures, 64.8% of the no-ligand models sat closer to the open apo state. For enzymes with more holo than apo structures, 75.5% of no-ligand models were still in the holo conformation, which the authors call “strong memorization.” In both groups, adding the ligand had “only a moderate impact,” except for enzymes with few structures in the training set. AlphaFold 2 showed the same pattern for the holo-heavy group.

Share of no-ligand AlphaFold 3 models in the state the PDB has more of Two horizontal bars from Yu and colleagues, PNAS, March 2026, on 82 enzymes. For enzymes with more apo than holo structures in the PDB, 64.8 percent of models generated without ligand are closer to the open apo state than to the closed holo state. For enzymes with more holo than apo structures, 75.5 percent of models generated without ligand are in the holo conformation. The remainder of each bar, 35.2 and 24.5 percent, is the other outcome. A reference line marks 50 percent. NO-LIGAND ALPHAFOLD 3 MODELS · 82 ENZYMES · YU ET AL., PNAS, MARCH 2026 More apo than holo structures in the PDB More holo than apo structures in the PDB 50% 64.8% closer to the open apo state — Yu et al., PNAS, March 2026 35.2% remainder — computed from the reported 64.8% 75.5% in the holo conformation — Yu et al., PNAS, March 2026 24.5% remainder — computed from the reported 75.5% 64.8% 75.5% filled: open apo state filled: holo (closed) state Unfilled remainder is the other outcome (writer's subtraction). The ligand was absent in every model shown.
With the trigger removed, the ensembles still lean toward whichever state has more entries in the training data.

The numbers are for enzymes that move between two states, and 82 enzymes is a modest sample. But the direction is the one Kohli’s own sentence predicts. A model rewarded for reproducing deposited structures will reproduce the deposited structures.

An inward-facing state that the training data contains

The sharpest case I found is a July preprint. Ye, Gumbart and colleagues ran AlphaFold 3, Boltz-2, Chai-1 and BioEmu on four proteins known to adopt more than one state: Pf-MATE, LAO, SecA and β2AR (bioRxiv, July 2026). It is a preprint and four proteins is a small set, so I am treating it as a worked example, not a rate.

On Pf-MATE, AlphaFold 3’s models sit 0.57 ± 0.03 Å from the outward-facing structure and 5.42 ± 0.05 Å from the inward-facing one. Chai-1 and Boltz-2 land 0.79 and 0.74 Å from the outward-facing state, and 5.10 and 4.93 Å from the inward-facing one. Apart from a single BioEmu frame at 1.8 Å, the authors report no tool recovering the inward-facing state; most of their distances to it exceed 3.5 Å. The inward-facing structure was released in 2019, and the authors note it is in the training data of all the models tested. These are not predictors that never saw a state. They were shown it, and still returned the other one. For SecA, the authors report that the four tools’ apo predictions split between closed and wide-open clamp states with none near the open reference, a reminder that the failure is not always so tidy.

Distance to the two Pf-MATE states by predictor Dot plot of root-mean-square deviation in angstroms on Pf-MATE, from Ye and colleagues, bioRxiv, July 2026. AlphaFold 3 is 0.57 angstroms from the outward-facing structure and 5.42 from the inward-facing structure. Chai-1 is 0.79 angstroms from the outward-facing structure and 5.10 from the inward-facing one; Boltz-2 is 0.74 and 4.93. BioEmu's best single frame is 1.8 angstroms from the inward-facing structure; its outward-facing distance is not plotted. A reference line marks 3.5 angstroms. PF-MATE · RMSD TO EACH EXPERIMENTAL STATE · YE ET AL., BIORXIV, JULY 2026 outward-facing state inward-facing state (in training data) AlphaFold 3 Chai-1 Boltz-2 BioEmu (best single frame) 3.5 Å 0.57 Å from outward-facing — AlphaFold 3, Ye et al., July 2026 5.42 Å from inward-facing — AlphaFold 3, Ye et al., July 2026 5.10 Å from inward-facing — Chai-1, Ye et al., July 2026 4.93 Å from inward-facing — Boltz-2, Ye et al., July 2026 0.79 Å from outward-facing — Chai-1, Ye et al., July 2026 0.74 Å from outward-facing — Boltz-2, Ye et al., July 2026 1.8 Å from inward-facing — BioEmu single frame, Ye et al., July 2026 0.57 5.42 0.79 5.10 4.93 0.74 1.8 0 2 4 6 BioEmu: best single inward-facing frame only (its mean distances are 1.22 Å outward and 3.56 Å inward).
About five angstroms (5.42 minus 0.57) separate AlphaFold 3's distances to the two states. BioEmu's one close frame is the exception the authors report.

Averages that agree while the interesting parts do not

A third paper looks at a different edge. Díaz and colleagues compared AlphaFold2, OpenFold, ColabFold and the AlphaFold3 server on 10 proteins lacking experimental structures, covering 222 single-residue variants (Briefings in Bioinformatics, September 2026). Global metrics, in the abstract’s wording, “frequently suggest comparable performance,” while the large local differences sit in flexible or disordered regions. This paper compares predictors with each other, not against experiment, and I only read the abstract, so I have no effect sizes to report. The point stands without them: a whole-chain score is an average, and disorder is exactly where an average is least informative.

The same shape turns up for ligands. Škrinjar and colleagues built a test from 2,600 protein–ligand systems released after the methods’ training cutoffs, had 229,887 predictions from four methods (AlphaFold 3, Boltz-1, Chai-1 and Protenix) for its common-subset analysis, and concluded that co-folding methods “largely memorize ligand poses” (Nature Structural & Molecular Biology, May 2026). They also report that 87% of PoseBusters complexes have a pose-and-pocket (SuCOS–pocket) similarity score above 50 to something in the training set, which goes a long way toward explaining the high headline numbers. That paper is about ligands, not folds, but it is the best post-cutoff test I found, and its lesson transfers: a benchmark that overlaps with training tests recall.

A pass mark Levinthal never set

One piece of history changes how I read “solved”. The folding problem is usually introduced through Levinthal’s paradox: a chain has so many possible configurations that it cannot find its structure by search, yet proteins fold in seconds. The 1969 talk often cited for it survives as a short transcription in a conference volume, and I read the scan (Levinthal, 1969). It estimates something like 10^300 configurations and says at most about 10^8 can be tried. It never uses the word “paradox.” And it does not say the native state is the global energy minimum: it says the final state “obviously must be a metastable state,” and that the lowest-energy state, if reached, is a product of evolution.

That reading and the Anfinsen-style slogan, that sequence determines the structure by minimum energy, are not the same claim. If the native fold is a kinetically reachable state, then a database of snapshots is a database of what was reached under the conditions someone chose, and a model scored against it has been scored on that.

What would count as a different test

There is a different test on the calendar. The CASP17 page says registration opened on 31 March and that the prediction season was scheduled to close on 31 August (CASP17). Its ensemble category, which CASP first ran in 2022 and says it is expanding, again has two target types: one type with two or a few discrete, well-determined conformations, and another scored against low-resolution data such as cryo-ET, SAXS, NMR, FRET and cross-linking, by how well the submitted ensembles agree with values computed from the data. A banner on the page gives the conference start as 30 November, though its timetable says only “December 2026”. I found no CASP17 results as of 19 October, and the page gives no target count.

I also looked for the check I most wanted: a 2026 study of how much of the remaining error in structure prediction is the experimental uncertainty in the reference, through B-factors or resolution. I found no such study in an OpenAlex sweep as of 19 October

  1. That is a bounded statement and not a claim that none exists.

So the careful reading is narrower than either slogan. For the dominant conformation of a well-represented protein, the headline is earned. Kohli says about 90 on the set, and nothing here disputes it. For the question the field now asks, which states a protein occupies and how often, the 2026 evidence I could find says the scores measure how much the PDB already contains. CASP’s ensemble track, running since CASP15 and expanding in CASP17, is the public test built for the second question, and its results should arrive around the conference, which the page dates to late November or December.

References

  1. Kohli, P., Candido, S. & Anderson, B. (2026). Why AlphaFold Didn’t Solve Protein Folding. Latent Space, 10 October 2026.
  2. Chakravarty, D. et al. (2026). Expanding Protein Structure Prediction into Conformational State Space. arXiv:2608.02866, 3 August 2026.
  3. Yu, H. et al. (2026). Bias in the AlphaFold3 prediction of ligand-induced domain motion in enzymes. Proceedings of the National Academy of Sciences, 4 March 2026.
  4. Ye, M. et al. (2026). Benchmarking AI Protein Structure Predictors Reveals a Persistent Bias in Multi-State Proteins. bioRxiv, 11 July 2026 (preprint).
  5. Díaz, F. R. et al. (2026). Comprehensive evaluation of AlphaFold/OpenFold prediction of experimentally unresolved proteins through novel metrics. Briefings in Bioinformatics, online 3 September 2026.
  6. Škrinjar, P. et al. (2026). Evaluating generalization in protein–ligand cofolding methods. Nature Structural & Molecular Biology, May 2026.
  7. Protein Structure Prediction Center (2026). CASP17. Page read 19 October 2026.
  8. Levinthal, C. (1969). How to fold graciously. Transcribed talk, Mössbauer Spectroscopy in Biological Systems, University of Illinois Bulletin 67(41), pp. 22–24. Historical background.