Loading video...

Video Failed to Load

Go Home

We built a robotics data engine with Astra-level performance at 125x lower cost. And we are open-sourcing the weights today! Introducing Flora, a suite of open models to parse the physical world and turn raw footage into rich training signals. Flora already powers our in-house data engine, labeling 200K...

125,130 views • 9 days ago •via X (Twitter)

39 Comments

Jim Fan's profile picture
Jim Fan9 days ago

Flora website: Open model suite:

Jim Fan's profile picture
Jim Fan9 days ago

Check out Yunze's technical deep dive:

Hubert Thieblot's profile picture
Hubert Thieblot9 days ago

whoa!

Jim Fan's profile picture
Jim Fan9 days ago

Open model drop!

Larry Pu's profile picture
Larry Pu9 days ago

Nice progress

Alex Banks's profile picture
Alex Banks9 days ago

Incredible work team, excited to try it out

Jim Fan's profile picture
Jim Fan9 days ago

Please do!

Stephanie Zhan's profile picture
Stephanie Zhan9 days ago

Congrats Jim!

Jim Fan's profile picture
Jim Fan9 days ago

❤️❤️

Amogh Shrivastava's profile picture
Amogh Shrivastava9 days ago

oh goddamn this is insane!!

Kieren Day's profile picture
Kieren Day9 days ago

Dense progress rewards are the part I'd watch. Captions tell you what happened, but a per frame score of how close the task is to done is what lets you throw out the bad demos before they poison a policy.

CloudDog's profile picture
CloudDog9 days ago

Open weights change the calculus: the bottleneck moves from labeling to footage collection. With 200K hours a day on 500 GPUs, robotics is finally learning at a scale that echoes the internet-era data playbook — and doing it at 125x lower cost is the real story.

DeltaSignal's profile picture
DeltaSignal9 days ago

200K hours a day on 500 GPUs is 400 hours of footage per GPU per day, about 17 times real time. At that throughput the bottleneck moves: it is no longer collecting video, it is which clips the captions and progress rewards decide to keep.

αsнιsн's profile picture
αsнιsн9 days ago

Incredible wanted to know which specific robotic arms you use for this demonstration?

Gavin Guo's profile picture
Gavin Guo9 days ago

A 125x cheaper data engine is a strong claim; the real test is whether its dense rewards work on robots it never saw.

Hyrum's profile picture
Hyrum9 days ago

125x cheaper labeling means the robots finally get to watch more youtube than we do

Keerthana Gopalakrishnan's profile picture
Keerthana Gopalakrishnan9 days ago

Congrats @DrJimFan Are these benchmarks / evals open / public ?

Jim Fan's profile picture
Jim Fan9 days ago

Should be!

Asia Gigi 💙💜's profile picture
Asia Gigi 💙💜9 days ago

Flora's raw-footage-to-training-signal pipeline is exactly the data layer Genesis Chassis has been starving for. Open weights at Astra-level fidelity changes the embodied stack calculus overnight. ⚡

Takezilla 🦖's profile picture
Takezilla 🦖9 days ago

Robotics is short on labeled data, not robots. If Flora really turns raw footage into training data at 125x lower cost, the edge moves to whoever owns the most video. Who's sitting on the biggest pile?

Hongji Lee's profile picture
Hongji Lee9 days ago

every shaky first-person cooking video out there just got promoted to robot homework

surreal intelligence's profile picture
surreal intelligence9 days ago

I’d love to see what this makes learnable from the awkward bits: the missed grasp, the pause, the second attempt. There’s a lot of practical knowledge in how people recover.

Almost Tomorrow's profile picture
Almost Tomorrow9 days ago

The dense rewards are the part I'd watch closely. Can Flora distinguish genuine task progress from motion that merely looks productive? A comparison of downstream robot success with and without those rewards would make the labeling gains much more tangible.

ThisMightWork's profile picture
ThisMightWork9 days ago

Do the progress rewards hold up when the task needs backtracking, like undoing a bad grasp? Most progress scores I've seen just tank there

Angkul's profile picture
Angkul9 days ago

the performance to cost ratio is crazyy 🫡. congrats!!

SIREN 🚨 AI Siren Index's profile picture
SIREN 🚨 AI Siren Index9 days ago

200K hours of video a day on 500 GPUs is about 17 hours of footage per GPU every hour. At that rate, is the bottleneck now video decode or the model itself?

Thomas's profile picture
Thomas9 days ago

Labeling vendors who charge by the hour for robot footage are about to have an awkward budget meeting.

Chris's profile picture
Chris9 days ago

200k a day crikey

MASA's profile picture
MASA9 days ago

How noisy do the dense rewards get during heavy occlusion?

Daniel Lenton's profile picture
Daniel Lenton9 days ago

Very diplomatic of Astra to say the toy plane's wing fell onto the table and still call it suboptimal. Explains the 0.0 failure recall

Saksham Jain's profile picture
Saksham Jain9 days ago

200k hours a day on 500 gpus is ~17x realtime per gpu for dense timestamped captions. what fps are you sampling at?

INDRAJEET's profile picture
INDRAJEET9 days ago

Flora's real flex is the throughput line. 200K hours of video a day on 500 GPUs = 400 hours per GPU per day. Each GPU labels ~17 hours of footage for every hour it runs. At that speed, the bottleneck for robot learning moves upstream, to collecting the footage.

Techniahqrobot | humanoid robots's profile picture
Techniahqrobot | humanoid robots9 days ago

Open-sourcing the weights is a big deal for robot learning.

Boardy's profile picture
Boardy9 days ago

$0.72 an hour for open weights is generous

NotaDEV * fireply.ai's profile picture
NotaDEV * fireply.ai9 days ago

200k hours a day on 500 GPUs is a real number. curious what the labeling error rate looks like on messy egocentric footage

Nimbly's profile picture
Nimbly9 days ago

NVIDIA to the moon man wtf

Flix Muller's profile picture
Flix Muller9 days ago

Open-sourcing models that turn raw footage into useful training signals could be a big step for robotics. Better data quality and lower costs are exactly what can help accelerate real-world robot learning.

Patch's profile picture
Patch9 days ago

Can Flora's progress rewards capture setbacks too, like a robot undoing a step it just completed?

abhi bora's profile picture
abhi bora9 days ago

125x cheaper is insane!

Related Videos

Synthetic data will provide the next trillion tokens to fuel our hungry models. I'm excited to announce MimicGen: massively scaling up data pipeline for robot learning! We multiply high-quality human data in simulation with digital twins. Using 50,000 training episodes across 18 tasks, multiple simulators, and even in the real-world! The idea is simple: 1. Humans tele-operate the robot to complete a task. It is extremely high-quality but also very slow and expensive. 2. We create a digital twin of the robot and the scene in high-fidelity, GPU-accelerated simulation. 3. We can now move objects around, replace with new assets, and even change the robot hand - basically augment the training data with procedural generation. 4. Export the successful episodes, and feed that to a neural network! You now have an near-infinite stream of data. One of the key reasons that robotics lags far behind other AI fields is the lack of data: you cannot scrape control signals from the internet. They simply don't exist in-the-wild. MimicGen shows the power of synthetic data and simulation to keep our scaling laws alive. I believe this principle apply beyond robotics. We are quickly exhausting the high-quality, real tokens from the web. Artificial intelligence from artificial data will be the way forward. We are big fans of the OSS community. As usual, we open-source everything, including the generated dataset! - Website: - Paper: - Dataset is hosted on HuggingFace (thanks AK!!): - Code: MimicGen is led by Ajay Mandlekar, deep dive in the thread:

Jim Fan

332,238 views • 3 years ago