Chelsea Finn's banner

Chelsea Finn

@chelseabfinn • 97,148 subscribers

Asst Prof of CS & EE @Stanford Co-founder of Physical Intelligence @physical_int PhD from @Berkeley_EECS, EECS BS from @MIT

Shorts

Disappointed with your ICLR paper being rejected? Ten years ago today, Sergey and I finished training some of the first end-to-end neutral nets for robot control 🤖 We submitted the paper to RSS on January 23, 2015. It was rejected for being "incremental" and "unlikely to have much impact" Our resubmission to NeurIPS was also rejected It now has >4,000 citations (and more importantly, end-to-end training is widely accepted!) It's also cool to think about what's changed and what's the same -- - The network was 92k parameters and trained on ~15 minutes of data - The code was a combination of matlab, caffe, ROS, a custom CUDA kernel for speed, and a low-level 20 Hz controller in C++, all talking to each other. ROS+matlab was as bad as it sounds. - We pre-trained the encoder and did inference off-board on a workstation with a larger GPU. - We were paranoid about varying lighting messing up the network, so we did all the experiments after sunset (so long nights running experiments on the robot past 3 am) Now, we have manipulation policies that are far more dextrous, far more generalizable, and maybe on the cusp of breaking into the real world. :) (the paper:

Disappointed with your ICLR paper being rejected? Ten years ago today, Sergey and I finished training some of the first end-to-end neutral nets for robot control 🤖 We submitted the paper to RSS on January 23, 2015. It was rejected for being "incremental" and "unlikely to have much impact" Our resubmission to NeurIPS was also rejected It now has >4,000 citations (and more importantly, end-to-end training is widely accepted!) It's also cool to think about what's changed and what's the same -- - The network was 92k parameters and trained on ~15 minutes of data - The code was a combination of matlab, caffe, ROS, a custom CUDA kernel for speed, and a low-level 20 Hz controller in C++, all talking to each other. ROS+matlab was as bad as it sounds. - We pre-trained the encoder and did inference off-board on a workstation with a larger GPU. - We were paranoid about varying lighting messing up the network, so we did all the experiments after sunset (so long nights running experiments on the robot past 3 am) Now, we have manipulation policies that are far more dextrous, far more generalizable, and maybe on the cusp of breaking into the real world. :) (the paper:

169,024 views

Why is action chunking crucial for robot dexterity? 🤖 - We identify a natural tradeoff between temporal consistency and reactivity - New policy decoding technique that is *both* temporally consistent & fully reactive ICLR 2025 paper: A short thread 🧵

Why is action chunking crucial for robot dexterity? 🤖 - We identify a natural tradeoff between temporal consistency and reactivity - New policy decoding technique that is both temporally consistent & fully reactive ICLR 2025 paper: A short thread 🧵

36,073 views

Videos

Anya Rossi

sweetdream.ai

SweetDream.ai•Sponsored•Livecam

Watch Anya Live

Anya is streaming live right now! Join her private show and enjoy exclusive content.

Exclusive private shows

1.2k viewers online

Private Show

Join now for exclusive access

Free preview available • Premium content

LLM post-training used to mean fine-tuning to a downstream task Robotics has been stuck in this setting, needing task-specific fine-tuning for best performance π07 changes this: It works out of the box & outperforms fine-tuned specialists Details:

LLM post-training used to mean fine-tuning to a downstream task Robotics has been stuck in this setting, needing task-specific fine-tuning for best performance π07 changes this: It works out of the box & outperforms fine-tuned specialists Details:

63,780 views • 3 months ago

How does test-time scaling impact robots? We find that larger models, more thinking, and more context help significantly for some prompts but not others. Like LLMs, we can also train a router to for a better performance/latency tradeoff! Paper:

How does test-time scaling impact robots? We find that larger models, more thinking, and more context help significantly for some prompts but not others. Like LLMs, we can also train a router to for a better performance/latency tradeoff! Paper:

23,121 views • 1 month ago

For robots to be actually useful, they need to be reliable. We’re sharing an RL recipe for VLA models that takes a step in this direction, allowing robots to operate autonomously for hours at a time. Blog & paper:

For robots to be actually useful, they need to be reliable. We’re sharing an RL recipe for VLA models that takes a step in this direction, allowing robots to operate autonomously for hours at a time. Blog & paper:

74,450 views • 8 months ago

We introduce a system for fine-grained robotic manipulation! 🤖 What’s new? * We can control cheap robots to do surprisingly dexterous tasks * New technique that allows robots to learn fine motor skills A short thread 🧵

We introduce a system for fine-grained robotic manipulation! 🤖 What’s new? * We can control cheap robots to do surprisingly dexterous tasks * New technique that allows robots to learn fine motor skills A short thread 🧵

264,399 views • 3 years ago

Our robot can now make you coffee 🤖☕ A short 🧵 on how it works ⬇️

Our robot can now make you coffee 🤖☕ A short 🧵 on how it works ⬇️

210,028 views • 3 years ago

Pi models are now running in production settings, in collab with Ultra and Weave Robotics. We see: - much higher autonomy with pi-0.6 over using pi-0.5 - fewer mistakes & higher throughput from incorporating data in pre-training Blog post:

Pi models are now running in production settings, in collab with Ultra and Weave Robotics. We see: - much higher autonomy with pi-0.6 over using pi-0.5 - fewer mistakes & higher throughput from incorporating data in pre-training Blog post:

28,620 views • 4 months ago

Introducing a new, fully open robotics dataset! - 76k episodes - 564 unique scenes - 100 contributors - 13 labs/institutions - 3 continents A short 🧵 on the backstory

Introducing a new, fully open robotics dataset! - 76k episodes - 564 unique scenes - 100 contributors - 13 labs/institutions - 3 continents A short 🧵 on the backstory

98,616 views • 2 years ago

Data curation is crucial for post-training recipes. But how do we curate? Curation is usually manual & tedious. And, it's hard to tell if a strategy in the data will be reliable! We introduce an automatic way to curate, informed by the robot's policy learning.

Data curation is crucial for post-training recipes. But how do we curate? Curation is usually manual & tedious. And, it's hard to tell if a strategy in the data will be reliable! We introduce an automatic way to curate, informed by the robot's policy learning.

19,910 views • 1 year ago

No more content to load