It can be taught extremely long-horizon tasks, over 10 minutes long. By composing skills learned in pretraining or creating new ones, it can make coffee, pot a plant, fry pancakes, and more – tasks never seen during pretraining.

Introducing S1, our new foundation model that learns from one example. It can be taught 10-minute long tasks that it has never seen before, from one video prompt without any fine-tuning. Watch S1 operate in real-time via in-context learning:

The first time S1 flipped a pancake, we assumed pancake flipping must have been in its pre-training data. We searched our whole pre-training data and found no examples of flipping. S1 inferred the out-of-distribution task from one video prompt.

S1 learns new tasks like a language model. You prompt it with a video demonstration, and it outputs robot actions to complete the task in any environment and in any embodiment.

Compared to conventional VLAs, S1 is a step-change improvement. For known tasks, S1 can match the performance of language-prompted VLAs. For novel tasks, S1 exponentially outperforms any existing language-prompted VLAs as we scale the pre-training. We believe this is a promising direction towards establishing scaling laws for robotics.

S1 does not blindly replay the video demonstration. Instead, it displays common-sense understanding that goes beyond the video prompt: - can withstand perturbations - improvise upon mistakes, even if the human didn’t - at times execute with more precision than the human in the video prompt

We found that to match the accuracy that S1 can achieve with just one example of prompting, current VLA models would need to be post-trained with 50-100 hours of data collection followed by fine-tuning!

Last year, we showed in-context learning for the first time for mobility. Locoformer can control any robot, and adapt to disturbances like disabled legs, locked wheels and more. S1 expands that capability to general manipulation.

We're excited to deploy S1 with our limited industrial partners today, rolling out to more customers over the next few months. S1 is the first light on a new path towards general intelligence grounded in the physical world. Technical blog:

Ofc I would try a food robot that learns a new dish from one video. I just wouldn't put it on a line until the same plate comes out the same way all week.

Tmf