Over the last few months, we’ve been thinking about how to learn from “off-domain” data - data from non-robot sources like video or simulation. These data sources are not quite good enough to learn policies (even monolithic VLA models) directly, but they still contain lots of information that can... be useful for generalizable robot control. How can we develop robot learning models that are able to make use of this type of data for generalizable control? In new work, that we call HAMSTER, we show that VLMs can be useful for enabling robotic learning from off-domain data, but specifically when used through hierarchical VLA architectures. We show that this class of models can learn generalizable robot policies for the real world from large-scale, off-domain data. A 🧵 (1/10)show more

The key idea behind HAMSTER is simple - use off-domain data to train hierarchical VLA architectures. The high-level controller is a VLM that produces 2D paths for desired motion of the end effector, as text, given images and desired tasks. These paths are generated in the training data through simple point tracking or annotation, and can be easily and quickly generated for a variety of data. In our work, we use a diversity of video datasets and simulation data as off-domain data (2/10)

What I found particularly interesting here was just how generalizable the resulting VLM is - @Yi showed that the resulting VLM was surprisingly proficient at generalizing to new embodiments, tasks, scenes and objects. This is cool because it retains the fundamental generalization and steerability properties of VLMs, while still producing spatially relevant predictions. Play around with the demo yourself at (3/10)

Ok, but we care about robots - how does this communicate with low-level control? The generated paths from the high-level VLM can then be used as conditioning for a low-level policy to guide closed-loop action generation. What’s cool about this separation is that we can use richer context for the low-level, for instance 3D point clouds inputs or proprioception, while still retaining the generalization benefits of the VLM. The paths generated by the VLM make the job of the low-level significantly easier, since it primarily needs to perform local corrections around the predicted paths to generate actions. (4/10)

Ok the architecture seems reasonable, but Abhishek, aren’t you just selling us RT-trajectory again? Yes and no - while the architecture does bear a resemblance (with the primary separation being 3D vs 2D low level policies), the key insight for me in this work is how the hierarchical separation let’s these models make use of off-domain data more effectively. It even follows for the usage of quite unrealistic-looking data such as RLBench! (5/10)

Ok, so how well does this work? On a number of controlled evaluations across a number of tabletop manipulation tasks, with variation across objects, language, visual appearance and other environmental factors - HAMSTER models outperform monolithic VLA models such as openVLA and imitation learning methods like RVT-2 or 3DDA. (6/10)

Importantly, we find that HAMSTER style models are able to utilize off-domain data more effectively than monolithic models. This suggests that even when the data can be used just for monolithic co-training, as is commonly done, there is a real benefit to using it for finetuning models hierarchically. (7/10)

Ok, but can we do more than just tabletop manipulation? Because the low-level policy is arbitrarily expressive, we can solve more dexterous and diverse tasks, including multi-step long-horizon tasks. (8/10)

The resulting model retains interesting capabilities of the base model, such as semantic generalization and multi-view consistency. (9/10)

Ok so whats the takeaway here - VLMs can be effective tools for absorbing off-domain data such as videos or simulation, but they need to be used with the appropriate hierarchical models. Now these models certainly have their limitations - still requiring non-trivial amounts of real-world data to learn low-level controllers, but I think they’re a real step in the right direction for scalable robot learning. As pure teleoperation data becomes untenably expensive, the ability to learn from this kind of off-domain data will become more and more necessary. Read our paper for more details: Paper: Website: Demo: (play around with it, it’s fun!) This was my first foray into VLA models, many thanks to the excellent @fox_dieter17849 for getting us initiated to @Yi_Li_UW , @Jesse_Y_Zhang, @nild0000 for leading and to @AnqiLi24, @imankitgoyal, @fox_dieter17849 for advising, with excellent collaborators @CaelanGarrett, @jang_yoel, @memmelma, Raymond Yu, Fabio Ramos at UW and NVIDIA! This work will be presented at #ICLR2025 🎉(10/10)

Every day, we generate mountains of data—our likes, our searches, our daily routines. But while data is often called “the new oil,” few realize who truly benefits from its extraction. Under the current Web2 model, we are simply the “oil fields” feeding the giant tech corporations. These companies collect our data, monetize it, and pocket the profit—all without our consent or compensation. In return, we receive “free” services like social media platforms and search engines. But is it really free if our digital identities are being sold for billions?

Very nicely summarised thread Abhishek.