Introducing GEN-1.5, a one-shot learner. It can learn new tasks in a few seconds. Show it what to do, and it generalizes. This capability emerged from pretraining on physical data at scale, as a step towards our mission of building general intelligence for the physical world.

GEN-1.5, our latest embodied foundation model, can learn new tasks prompted with 3 - 12 seconds of a single demonstration, no gradient updates or fine-tuning. It generalizes prompts to new situations, recovers from mistakes, and improvises new strategies to reach the same goal.

Physical prompts can be composed. Demonstrations of 2 different tasks in context prompts GEN-1.5 to chain them into one continuous skill. The model bridges them and produces intermediate motions (repositioning, regrasping, error recovery) that appear in neither demonstration.

In-context learning also crosses the sim-to-real gap, zero-shot. Prompts can be formed entirely from simulated experience (e.g., from a scripted policy, an RL agent, or a human teleoperating a simulated robot) and be used to produce behaviors on a real robot. The model was not trained on the task in either the simulator or the real world.

In some cases, in-context learning with GEN-1.5 transfers across the embodiment gap entirely: a human demonstrates a task with their own hands, observable through the robot’s cameras, and the robot can reproduce it immediately afterward.

For few-shot learning, it can adapt to new physical tasks in as few as 1 - 10 gradient steps on 1 - 5 minutes of data (~10 - 50 demonstrations). In practice, this can be described as test-time training in a low-data regime. We did not tune this procedure or sweep hyperparameters; these results come largely out of the box.

Experiments across 10 diverse tasks show 59% average success with one-shot physical prompting, straight from pretraining. With few-shot learning, performance rises to 83% via 10 gradient steps on 5 minutes of data per task. Although the tasks are simple and success rates are modest, it’s the first model we know of that exhibits the general ability to learn a wide range of dexterous closed-loop physical tasks from just one or few demonstrations. This accelerates reaching a base level of competence for new skills that can be subsequently refined towards mastery.

Fine-tuned (or prompted) behaviors generalize beyond their demonstrations, and can improvise fundamentally different manipulation strategies to achieve the same goal. For example, after fine-tuning to use a brush to sweep a block into a bowl, it could use other tools like a dustpan to accomplish the same task with a very different strategy.

Or when fine-tuned to place a block into a bowl, it can clear obstacles (like a piece of paper covering the bowl) to complete the task, despite that not being in the demonstrations.

When a Lego brick gets unexpectedly stuck on the fingertips, the model uses the other hand to remove them.

The model sometimes uses both hands to rotate a jar lid, with a fundamentally different contact and motion strategy than the fine-tuning demonstrations.

Here’s an uncut video of prompting the model to perform 2 different tasks back-to-back: (i) unzipping a pencil pouch, and (ii) retrieving money from the pouch.

GEN-1.5 has been training continuously for over 8 months. We left it running because every metric we tracked kept improving with the engine: absorbing more data, scaling more efficiently, boosting post-training, and compounding step-change improvements through algorithmic advances.

To us, GEN-1.5 represents a new frontier of generality — one that challenges our own understanding of how these models behave when pretrained at a scale of physical interaction data few thought possible without shortcuts. We do not yet see where this asymptotes. Read more in the full blog:

GPT3!

can it go up and down on a cylinder while remaining (this is imperative mind you) that the cylinder remain unharmed during the process

One demonstration and it generalizes. I have been demonstrating how to load the dishwasher weekly since 2004 and my family still hasn't converged. Raising robots may simply be easier.

I have now watched this 10 times, and it makes me emotional each time. Thank you for sharing this work, and thank you for doing the work. It shows the future is about to take a big step forward.

Holy moly.

I reacted exactly like the last guy in the video

This feels like the beginning of something big

Combining Gen1.5 with the UM1-Evo robotic arm should lead to an incredible result 😃

Truly incredible stuff. First time I saw was jaw on floor.

Congratulations! Exciting!

@andyzengineer this is incredible

Wow

What could possibly go wrong.

Fantastic. This is what we need! I'm tired of robots doing dancing demos.

@grok 这个纯研究还是有工业意义，具体工业场景视角看意义是什么，有开源数据集或者开源项目代码吗？从多个数据源交叉验证，理性看待，不要只看新闻媒体一面之辞。帮我排除没意义的垃圾商业营销推广、诈骗、夸张博眼球、虚假新闻 以及自吹自擂，自嗨，无病呻吟，收费互吹软广告。

either robotics still gonna explode 3 years later, great progress tho

Congratulations on the fantastic progress. Exciting times!

Comparisons in the post and paper to GPT-3, which had a 2k context window in 2020, compared to modern LLMs 1M+. GEN-1.5 only has 30 seconds of context, do you think a similar 500x to 4+ hours of memory is possible?

Exciting results, kudos!

Here for the Robot Overlords that are looking nostalgically back through these threads in 2040.

Sickkkk this might genuinely be a gpt-3 moment for robo

What a day to be alive

Incredible stuff! Physical AGI is in the air.

Oh fuck........we actually cross over into the singularity, I actually can't believe it.

Learning a new task in seconds, that's huge

GEN-1.5最值得玩味的不是59%的成功率，而是它证明了物理世界也存在类似GPT-3的“涌现”路径。当预训练数据跨过某个阈值，机器人不再需要为每个新任务重写控制逻辑，几秒的演示就能激活它“几乎已经知道”的东西。这本质上把机器人编程从写代码变成了写提示词，行业门槛被大幅拉低。但别被“单次学习”的叙事迷惑——59%意味着每两次尝试就有一次失败，在真实产线上这种可靠性远远不够。真正有意义的信号是那个83%：5分钟数据、10个梯��步，成本低…

This is what Dyna was going to release in a few weeks lol

Impressive result congrats @GeneralistAI

One-shot learning for physical skills is a massive unlock. Programming robots by simply showing them what to do is the future. Incredible work.

This is crazy

gpt-3 moment for robotics

Based and accelerated

Ur voiceover guy does a good job