← Gautam Parab

Robot Foundation Models Generalize. The Experiments Can't Tell You How Much.

A ten-trial success rate has a 95% confidence interval roughly fifty percentage points wide. Most manipulation papers run ten to twenty-five trials per condition. The instrument this field uses to judge its generalist policies is coarser than the effects it is asked to detect.

What that instrument is measuring is good. Robot foundation models generalize, and the effect is real enough to have survived thousands of physical trials. RT-2 executed instructions on objects it had never seen. π₀.₅ ran in kitchens and bedrooms that were not in its training set. Open X-Embodiment pooled a million trajectories from twenty-one institutions and shipped it. None of that is vendor theater.

What the headlines leave out is the level all of this happens at, and how fast it decays once you chain tasks together. Cross-embodiment pretraining, meanwhile, has been measured hurting performance as readily as helping it, depending on the data regime.

Levels, not deltas

Papers report gains. Gains are ratios, and ratios hide levels. RT-2 is the best-documented case because Google ran roughly 6,000 real-world trials and published the split-by-split table rather than an average.

RT-2-PaLI-X-55B real-world success rate by evaluation split Horizontal bar chart of RT-2-PaLI-X-55B success rates across four evaluation conditions: seen tasks 91 percent, unseen objects on the hard split 62 percent, unseen backgrounds on the hard split 48 percent, and unseen environments on the hard split 35 percent. The bars shorten steeply from left to right as conditions move further from the training distribution. RT-2-PALI-X-55B · REAL-WORLD SUCCESS · ARXIV 2307.15818 Seen tasks Unseen objects (hard) Unseen backgrounds (hard) Unseen environments (hard) 91% — RT-2 paper, Jul 2023 62% — RT-2 paper, Jul 2023 48% — RT-2 paper, Jul 2023 35% — RT-2 paper, Jul 2023 91% 62% 48% 35% Scale runs 0–100%. "Hard" splits are the paper's own designation for the more aggressive out-of-distribution variants.
The generalization is real and it is also a steep decline. A model that hits 91% on what it was trained for hits 35% in a room it has not seen.

RT-1, the smaller predecessor, tells the same story with a different shape: 97% on seen tasks, 76% on novel instructions, 83% with distractors, and 59% against unseen backgrounds — measured across more than 3,000 real trials. The pattern is consistent across the family. In-distribution, these systems are close to solved. One axis away from the training data, they are coin-flip-adjacent.

OpenVLA, the open 7B model trained on 970,000 demonstrations, averaged 70.6% on the WidowX/BridgeData V2 evaluation against RT-2-X’s 50.6%. Across 29 tasks spanning both the WidowX and Google Robot embodiments it beats RT-2-X by 16.5 points absolute — a model that still fails roughly three attempts in ten.

Then chain the tasks together

Single-task rates flatter these systems. Compounding error is the old, known failure of behavior cloning, and it arrives the moment you ask for more than one thing in a row.

CALVIN success rate at one task versus five consecutive tasks, by action representation, backbone held fixed Grouped horizontal bar chart holding the LLaVA backbone fixed and varying only the action representation. One-step discrete action tokens fall from 80.9 percent on a single task to 10.3 percent across five consecutive tasks. Continuous actions with a policy head fall from 87.3 percent on a single task to 27.5 percent across five consecutive tasks. Both representations lose most of their success over the horizon; the continuous one retains about two and a half times as much. CALVIN LONG-HORIZON DECAY · NAT MACH INTELL 2026 Discrete tokens · 1 task Discrete tokens · 5 in a row Policy head · 1 task Policy head · 5 in a row 80.9% — RoboVLMs, Nat Mach Intell 2026 10.3% — RoboVLMs, Nat Mach Intell 2026 87.3% — RoboVLMs, Nat Mach Intell 2026 27.5% — RoboVLMs, Nat Mach Intell 2026 80.9% 10.3% 87.3% 27.5% Simulation benchmark. Scale runs 0–100%. LLaVA backbone held fixed; only the action representation changes. The study's best configuration overall uses a different backbone and reaches 82.6% at five tasks. Real-world evaluation in the same paper used 3 rollouts per condition — see the interval chart below.
An eighty-percent single-task policy is a ten-percent five-task policy. Holding the backbone fixed, the action representation alone changes the five-step survival rate by more than a factor of two.

The discrete-versus-continuous gap here interests me more than the absolute levels. Hold the backbone fixed, change only the action head, and the five-step number moves from 10.3% to 27.5% — the paper’s best configuration overall, on a different backbone, gets to 82.6%. That is an architecture result masquerading as a scaling result, the kind of thing that gets lost when a field reports averages. It rhymes with what I wrote about small models on device: the gains that survive contact with reality are usually structural, not sheer size.

The data story is not the language-model data story

The standard prescription for both of those problems is more data, and robotics has assembled a corpus to supply it. Open X-Embodiment is the field’s ImageNet moment on paper: over a million trajectories, 22 robot embodiments, 527 skills, 160,266 tasks, pooled from 60 pre-existing robot datasets contributed by 21 institutions across 34 labs. It is an extraordinary act of coordination.

It is also where the analogy to language pretraining breaks. The Open X-Embodiment paper itself reports that in its data-rich domains — Bridge, and the RT-1 dataset — RT-1-X trained on the pooled cross-embodiment corpus does not beat an RT-1 baseline trained only on its own embodiment’s data, which the authors attribute to underfitting at that model capacity. In the small-data domains the pooling worked well, beating single-embodiment baselines on four of five datasets. So the flagship dataset paper reports its own corpus helping exactly where you already lack data and failing where you don’t — a transfer story, not a scaling one.

The 2026 Nature Machine Intelligence study that systematically ablated VLA design choices found the same thing more sharply. Co-training on Open X-Embodiment without post-training did not help significantly, and post-training on it improved high-frequency skills like pick-and-place while decreasing performance on lower-frequency skills — “move near,” “open/close drawer.” The corpus is dominated by pick-and-place, so training on it is partly training the model to forget everything else. Cross-embodiment pretraining did help in the data-scarce regime: a 17.2% improvement in few-shot CALVIN. Which is roughly the opposite of the language-model story, where scale is the thing that helps most when you already have plenty.

The sharpest single data point comes from TinyVLA, published in RA-L last year: a 1.3B model fine-tuned on 100 demonstrations per task hit 94.0% average success across five real tasks over 20 trials each, against OpenVLA’s 68.3% on the same setup — with 5.5× fewer parameters and no Open X-Embodiment pretraining at all. And on bimanual tasks, OpenVLA — pretrained on a corpus containing only single-arm data — scored 0%. Every trial. That is not a small transfer penalty; that is a corpus teaching a model something actively wrong about the body it is in.

The one clean scaling result I trust points somewhere else entirely. A 2024 study collected over 40,000 demonstrations and ran more than 15,000 real-world rollouts to measure what actually scales, and found a power-law relationship with the number of environments and objects, not with raw demonstration count. Thirty-two diverse environment-object pairs at about fifty demonstrations each — four people, one afternoon — got two tasks to roughly 90% in novel settings. If that replicates, the lesson is that robot data has a diversity exponent, not a volume one, and most of the field has been optimizing the wrong axis.

Three rollouts per condition

Every number above came out of an instrument. Manipulation evaluations are physical, expensive, and therefore small. The Nature Machine Intelligence ablation study used three rollouts per real-world condition. TinyVLA used twenty per task for single-arm and ten for bimanual. Ten to twenty-five is the field norm.

Run the binomial arithmetic on that. (This next figure is my own calculation, not a citation — Wilson score intervals at 95%.)

95 percent Wilson confidence intervals for three outcomes of a ten-trial evaluation Interval plot on a zero to one hundred percent axis. Ten successes out of ten gives a point estimate of 100 percent with an interval from 72 to 100 percent. Seven of ten gives 70 percent with an interval from 40 to 89 percent. Five of ten gives 50 percent with an interval from 24 to 76 percent. The seven-of-ten and five-of-ten intervals overlap across most of their range. WHAT 10 TRIALS CAN RESOLVE · WILSON 95% CI · AUTHOR CALCULATION 10 of 10 succeed 7 of 10 succeed 5 of 10 succeed true rate: 72–100% true rate: 40–89% true rate: 24–76% point estimate 100% point estimate 70% point estimate 50% 0% 25% 50% 75% 100% A 7-of-10 result and a 5-of-10 result are not distinguishable at this sample size. Ten to twenty-five trials is the field norm.
The interval is the finding. At ten trials, a policy that looks 20 points better may not be better at all.

Separating a genuine 70% policy from a genuine 50% one at conventional power takes on the order of ninety trials per arm — again my arithmetic, not a published figure. Almost nobody runs that. A 2025 study of repeatable robot measurement makes the same point from the metrology side: ISO 9283 recommends 20–50 samples per reference point for industrial-arm repeatability, and the authors found even that insufficient for humanoid evaluation, using 250–700 samples in response to the variability they observed.

Then there is the setup itself. The MIRRER work on reproducibility in robot learning quotes the field’s blunt working assumption — that building precisely reproducible robotic setups is impossible, and therefore absolute benchmark numbers are close to meaningless — and cites the RB2 benchmark’s observation of roughly 20% performance variation when the same experiment was reproduced across two labs. A 2025 Nature Machine Intelligence piece on reproducible robotics notes that a 2009 survey found zero of ten representative SLAM and navigation papers fully met reproducibility requisites, with a 2023 surgical-robotics study finding much the same. The remedy it endorses is not better absolute numbers but RB2’s local relative ranking: compare a method against your own lab’s baseline and stop pretending the cross-lab number means anything.

That is a reasonable adaptation. It is also an admission that the field’s headline leaderboard is not a measurement.

Five hours to cross a room

In 1979 Hans Moravec’s Stanford Cart crossed twenty-meter obstacle courses using stereo vision. His own write-up in Proceedings of the IEEE records the pace: the Cart “moved one meter every ten to fifteen minutes, in lurches,” stopping after each lurch to take pictures and “thought about them for a long time” on a lightly loaded DEC KL-10. Each course took about five hours. Effective velocity: three to five meters per hour.

Moravec would formulate his paradox nine years later. He had already measured it. A machine that could reason about a scene needed five hours to walk across a room — roughly a thousandth of human walking speed. The numbers in this essay are the same paradox with better hardware: the semantic part is nearly free now, and the physical part still costs everything.

So I’m watching three things. Whether the diversity-not-volume scaling result replicates outside its original lab, because it reorganizes where the money should go. Whether anyone publishes real-world manipulation results with enough trials to carry confidence intervals — and whether reviewers start asking. And whether cross-embodiment corpora get rebalanced by skill rather than aggregated by availability, since the negative transfer observed so far looks like a composition problem, not a fundamental one.

The models are better than the numbers suggest in some ways and worse in others, and right now the evaluation apparatus cannot tell us which. That is a solvable problem, and it is cheaper to solve than another million trajectories.

References

  1. Brohan et al. RT-1. arXiv:2212.06817.
  2. Brohan et al. RT-2. arXiv:2307.15818.
  3. Open X-Embodiment Collaboration. arXiv:2310.08864.
  4. Kim et al. OpenVLA. arXiv:2406.09246.
  5. Team et al. Octo. arXiv:2405.12213.
  6. Black et al. π₀. Physical Intelligence. pi.website/blog/pi0.
  7. Intelligence et al. π₀.₅. arXiv:2504.16054.
  8. Bjorck et al. GR00T N1. arXiv:2503.14734.
  9. Lin et al. Data scaling laws in imitation learning. arXiv:2410.18647.
  10. Li et al. (2026). “What matters in building vision–language–action models for generalist robots.” Nature Machine Intelligence 8(2).
  11. Wen et al. (2025). TinyVLA. IEEE RA-L 10(4).
  12. Kawaharazuka et al. (2025). VLA review. IEEE Access 13.
  13. Bonsignorio et al. (2025). “Towards reproducible robotics research.” Nature Machine Intelligence 7(10).
  14. MIRRER. arXiv:2408.04736.
  15. RB2. arXiv:2203.08098.
  16. Statistical query for repeatable robot performance. arXiv:2505.08216.
  17. Aljalbout et al. (2025). “The reality gap in robotics.” Annual Review of Control, Robotics, and Autonomous Systems.
  18. Moravec, H. (1983). “The Stanford Cart and the CMU Rover.” Proc. IEEE.