Turns out that doing In-context learning for robots is not that hard...

it's right behind me isn't it

@chris_j_paxton I feel this is a not very well-defined area where we know how “OOD” the task is.

What's the difference between in-context learning and those one-shot imitation learning papers from several years ago? I think there demo are very similar:

I like the robot wearing meta glasses. Also wuji hands?

nice @BillyYYan

Oh my! Third in context learner robot in the last few weeks and first I’ve seen with a humanoid. Nice!

@BillyYYan this is great!

@chris_j_paxton are those Aria glasses?

Very true. But the real question is: how do we scale this data, and how accurate and reliable are these actions once they’re deployed in real production environments?

What happens if you move the bowl?

How sensitive is it to the context you give it? Picking an example that actually disambiguates the task seems like it could become half the work.

@chris_j_paxton Any reason for having your hands in that particular starting/neutral position?

man, 'not that hard' on a clean table with two objects. real world generalization is the hard part. my cats would break this in ten seconds flat.

In-context adaptation is the most promising route past per-task finetuning. The eval that would convince me: a demo given in one room, then executed in a room the policy never saw, with the object swapped for a same-category variant.

do a more complex grasp, like a hammer or something. Squishing a soft toy is trivial.

Impressive. The product question is no longer whether a robot can learn in context, but whether a non-expert user can provide the right demonstration reliably. The teams that turn that teaching loop into a simple, observable UX will have a real adoption advantage.

@xiaolonw This is cool

And how does it look without the context?

can it learn to wash the dishes this way?

Just add some legs to increase the cost and complexity and you’ve got a humanoid

VLA model?

What robot is this. The arms look very human like

Now I understand where robots get such robotic movements. Infact robot movements are more human like than the person teaching it.

what is the Glasses, it is the secrete , how this demo work

u need another bot, thats taking it out