Vision Language large Models (VLMs) are powerful, but have a fatal shortcoming: they essentially view 2D images, while real-world robot manipulation requires precise 3D spatial perception — where objects are, how they're oriented, how far away, whether they can be grasped. This gap has long plagued embodied-intelligence landing.
Zhejiang Humanoid Robot Innovation Center, jointly with the Chinese University of Hong Kong, Zhejiang University, and other institutions, published the RAM (Retrieval-Augmented Manipulation) 3D spatial understanding and manipulation model in the international top journal Science Robotics, systematically solving VLM's 3D spatial perception deficiencies for the first time.
RAM's core idea is retrieval augmentation: instead of having the model hard-code all 3D knowledge, it constructs an external 3D knowledge base that dynamically retrieves task-relevant spatial information at runtime. This is equivalent to adding an external brain to VLM, specifically for spatial reasoning. Compared to end-to-end retraining of VLMoE, this approach has lower cost, faster migration, and doesn't need to break the original model's capabilities.
Real-machine verification results are impressive. On a humanoid robot platform, language-instruction-driven manipulation achieves an average success rate of 89.17%, image-guided manipulation 92%. RAM also supports multiple mainstream VLMs including GPT and Qwen-VL, with good model compatibility — meaning existing models can gain spatial intelligence with virtually no retraining.
Why is this noteworthy? In the past, for robots to achieve reliable grasping and manipulation, they needed either expensive end-to-end training or rule engines with limited precision. RAM represents a middle path: via retrieval mechanisms, foundation models can gain near-specialized spatial capabilities without complete retraining. This external-knowledge-base paradigm has the potential to migrate to other scenarios requiring precise spatial reasoning — such as autonomous driving environment perception, industrial robot precision assembly.
Embodied intelligence has reached today, lacking not general language ability, but physical-world spatial commonsense. RAM has taken an interesting step.