Embodiment Gap

A policy trained on one robot rarely works on another. Different arms have different kinematics, different grippers, different cameras in different places — so the same instruction requires different motion, and the data collected on one body does not straightforwardly describe the other.

Cross-embodiment datasets attack this by pooling across bodies and letting the model learn what is shared, which works better than expected and is why pooled datasets exist at all. What transfers is the semantic layer — which object, which goal — while the low-level control stays body-specific, which suggests the split most architectures now make rather than being a limitation of them.

Actions live in a body-specific space, so pooling requires either a shared action representation or per-body heads on a shared trunk. Positive transfer then depends on the shared part carrying most of the task information — which is exactly the claim vision-language-action models make, and the reason pooling across bodies helps rather than merely adding noise.