Robotics / Foundations
verifiedManipulation
Picking things up is the problem that has stayed hard. A policy that folds one towel reliably will often fail on a different towel, because what changed is not the task but the object — and the space of objects is not something a training set covers.
Generalisation is measured across objects, positions, lighting and distractors, and a paper reporting a success rate without saying which of those varied has reported very little. The practical divide is between quasi-static tasks, where the robot can stop and think, and dynamic ones, where it cannot — and almost all published success is on the first kind.
Success is evaluated over a distribution of object instances rather than repeated trials on one, so the standard error depends on the number of distinct objects and not on trial count. Ten trials on one object is a measurement of that object; ten objects once each is a measurement of the policy, and the two are reported in the same units.
Loss over 2000 training steps, starting near 7.2. It falls to about 1.95, with 93% of the total improvement arriving in the first half. A second line shows held-out, ending higher at about 2.22.
Error on the objects the policy was trained with against error on objects it has never seen. Drag the object shift up to watch the second curve leave the first — a success rate that does not say which objects varied is reporting the first curve alone.
Reviewed by opendroid · 2026-08-18
- arXiv:2212.06817 — RT-1: Robotics Transformer for Real-World Control at Scale
- arXiv:1808.00177 — Learning Dexterous In-Hand Manipulation