Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

Thrilled to announce Octo 🐙, an open-source robot foundation model! Octo is a sota generalist robot policy based on transformer+diffusion. Most importantly, you can finetune Octo today with flexible observation and action spaces on your robot setup!

Oier Mees

2,296 subscribers

44,944 views • 2 years ago •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

7 Comments

Oier Mees2 years ago

Octo was an incredible effort w/ the fantastic @KarlPertsch @its_dibya @kvablack @Homerwalker @SudeepDasari @JoeyHejna Tobias Kreiman, Charles Xu @jianlanluo You Liang @DorsaSadigh @chelseabfinn @svlevine It was inspiring how everybody pushed hard together to make Octo happen!

Oier Mees2 years ago

@its_dibya @kvablack @homerwalker @SudeepDasari @JoeyHejna @jianlanluo @DorsaSadigh @chelseabfinn @svlevine Huge shout-out to my amazing co-leads @KarlPertsch @its_dibya @kvablack @HomerWalke who poured their hearts and energy into this project and spent countless nights in the arena. I feel immensely grateful to get to do this with you'all ❤️ Excited for the next steps of Octo!

Alamin Ibrahim2 years ago

@KarlPertsch @its_dibya @kvablack @HomerWalke i wonder how this compared to @Tesla 's optimus in terms of approach

Julian Harris2 years ago

@KarlPertsch @its_dibya @kvablack @HomerWalke Amazing! Can the brain learn in virtual spaces?

Joel Ye2 years ago

@svlevine @KarlPertsch @its_dibya @kvablack @HomerWalke Do you have thoughts on initializing diffusion either during training or inference with the previous time step’s action due to strong autocorrelation? I’d expect only minor steps needed over computing from scratch?

Ant A2 years ago

@KarlPertsch @its_dibya @kvablack @HomerWalke Using this will expose those arcade claw games that only stay closed after it has made enough money from the people who lost.

jackhuang1 year ago

@KarlPertsch @its_dibya @kvablack @HomerWalke I have recently developed a DC brushless motor that is capable of achieving 92% energy efficiency and IP68 water and dust resistance. I wonder if it can be applied to robots

Related Videos

✨ Introducing 𝐎𝐩𝐞𝐧𝐕𝐋𝐀 — an open-source vision-language-action model for robotics! 👐 - SOTA generalist policy - 7B params - outperforms Octo, RT-2-X on zero-shot evals 🦾 - trained on 970k episodes from OpenX dataset 🤖 - fully open: model/code/data all online 🤗 🧵👇

✨ Introducing 𝐎𝐩𝐞𝐧𝐕𝐋𝐀 — an open-source vision-language-action model for robotics! 👐 - SOTA generalist policy - 7B params - outperforms Octo, RT-2-X on zero-shot evals 🦾 - trained on 970k episodes from OpenX dataset 🤖 - fully open: model/code/data all online 🤗 🧵👇

Moo Jin Kim

227,044 views • 2 years ago

$OCTO OctonetAI has been given an AssetDash Gold Boost ⚡ 📈 Buying $OCTO on AssetDash now comes with a 10% bonus distribution of Gold 🎁 You can spend your Gold to open a $OCTO Loot Box in the Marketplace

$OCTO OctonetAI has been given an AssetDash Gold Boost ⚡ 📈 Buying $OCTO on AssetDash now comes with a 10% bonus distribution of Gold 🎁 You can spend your Gold to open a $OCTO Loot Box in the Marketplace

AssetDash

11,914 views • 1 year ago

🚀 We’re excited to announce LingBot-VA, a new state-of-the-art robot policy model from Robbyant ! LingBot-VA is built on a causal, autoregressive video-action world model for generalist robot control. Highlights: (1) First unified autoregressive video-action world model for robot control (2) Low-latency inference with a new asynchronous execution pipeline (3) SOTA on RoboTwin (92.9%, firstever > 90%) and LIBERO (98.5%) (4) +20% over π0.5 on challenging real-world long-horizon & high-precision tasks

🚀 We’re excited to announce LingBot-VA, a new state-of-the-art robot policy model from Robbyant ! LingBot-VA is built on a causal, autoregressive video-action world model for generalist robot control. Highlights: (1) First unified autoregressive video-action world model for robot control (2) Low-latency inference with a new asynchronous execution pipeline (3) SOTA on RoboTwin (92.9%, firstever > 90%) and LIBERO (98.5%) (4) +20% over π0.5 on challenging real-world long-horizon & high-precision tasks

Yinghao Xu

46,962 views • 6 months ago

NVIDIA Isaac GR00T N1 is an open generalist foundation model for #humanoidrobots. 🤖 Discover how the model can easily generalize common tasks, boosting robot reasoning and skills. ▶️

NVIDIA Isaac GR00T N1 is an open generalist foundation model for #humanoidrobots. 🤖 Discover how the model can easily generalize common tasks, boosting robot reasoning and skills. ▶️

NVIDIA Robotics

53,497 views • 1 year ago

LingBot-VLA 2.0 is an impressive new embodied model. Open source and is trained across diverse robot configurations, from single-arm robots to humanoid platforms. It packs 60K hours of curated robot and human data into one generalist policy. It improves robots on difficult long-horizon tasks. Great release by Robbyant.

LingBot-VLA 2.0 is an impressive new embodied model. Open source and is trained across diverse robot configurations, from single-arm robots to humanoid platforms. It packs 60K hours of curated robot and human data into one generalist policy. It improves robots on difficult long-horizon tasks. Great release by Robbyant.

elvis

10,464 views • 23 days ago

📢 First contact between a frontier model and robots! Gemini Robotics is a SOTA generalist Vision-Language-Action model bringing frontier model intelligence to the physical world. It's an extremely capable model enabling dexterous, steerable, and general robot control. 🧵⬇️

📢 First contact between a frontier model and robots! Gemini Robotics is a SOTA generalist Vision-Language-Action model bringing frontier model intelligence to the physical world. It's an extremely capable model enabling dexterous, steerable, and general robot control. 🧵⬇️

Ted Xiao

152,458 views • 1 year ago

What happens when robot world models learn from human experience at scale? 🤔 DreamDojo from NVIDIA Research is a generalist robot world model pretrained on 44K hours of egocentric human videos and then post-trained on robot data to generalize across new objects and environments. After distillation, it runs at 10 FPS for live teleoperation, policy evaluation, and model-based planning. Read the ICML paper to learn more 📄

What happens when robot world models learn from human experience at scale? 🤔 DreamDojo from NVIDIA Research is a generalist robot world model pretrained on 44K hours of egocentric human videos and then post-trained on robot data to generalize across new objects and environments. After distillation, it runs at 10 FPS for live teleoperation, policy evaluation, and model-based planning. Read the ICML paper to learn more 📄

NVIDIA Robotics

22,413 views • 1 month ago

Excited to release τ0-WM: an open-source unified video-action world model for robotic manipulation. It's a 5B-parameter robotic foundation model trained on 27.3K hours of real-robot teleoperation, UMI-style demonstrations, and egocentric interaction videos.

Excited to release τ0-WM: an open-source unified video-action world model for robotic manipulation. It's a 5B-parameter robotic foundation model trained on 27.3K hours of real-robot teleoperation, UMI-style demonstrations, and egocentric interaction videos.

Jianlan Luo

63,831 views • 2 months ago

We’re excited to share DiT4DiT, an end-to-end Video-Action Model for robot learning that unifies a video Diffusion Transformer and an action Diffusion Transformer in a single cascaded framework. By leveraging the rich spatiotemporal and physical dynamics learned through video generation, rather than static image-text priors, DiT4DiT achieves state-of-the-art results on LIBERO (98.6%) and RoboCasa GR1 (50.8%) with far less training data, delivering over 10× better sample efficiency and up to 7× faster convergence. Real-world deployment on a humanoid robot further shows robust generalization. We believe this is a step toward making video generation a powerful backbone for robot policy learning. This work builds upon the brilliant foundations laid by Nvidia's GR00T and Cosmos. Project: Paper: Code: Coming soon. In the meantime, you can ask your coding agent to reproduce the method based on GR00T/Cosmos.

We’re excited to share DiT4DiT, an end-to-end Video-Action Model for robot learning that unifies a video Diffusion Transformer and an action Diffusion Transformer in a single cascaded framework. By leveraging the rich spatiotemporal and physical dynamics learned through video generation, rather than static image-text priors, DiT4DiT achieves state-of-the-art results on LIBERO (98.6%) and RoboCasa GR1 (50.8%) with far less training data, delivering over 10× better sample efficiency and up to 7× faster convergence. Real-world deployment on a humanoid robot further shows robust generalization. We believe this is a step toward making video generation a powerful backbone for robot policy learning. This work builds upon the brilliant foundations laid by Nvidia's GR00T and Cosmos. Project: Paper: Code: Coming soon. In the meantime, you can ask your coding agent to reproduce the method based on GR00T/Cosmos.

Shuo Yang

31,872 views • 4 months ago

We release Cosmos Policy 💫: a state-of-the-art robot policy built on a video diffusion model backbone. - policy + world model + value function — in 1 model - no architectural changes to the base video model - SOTA in LIBERO (98.5%), RoboCasa (67.1%), & ALOHA tasks (93.6%) 🧵👇

We release Cosmos Policy 💫: a state-of-the-art robot policy built on a video diffusion model backbone. - policy + world model + value function — in 1 model - no architectural changes to the base video model - SOTA in LIBERO (98.5%), RoboCasa (67.1%), & ALOHA tasks (93.6%) 🧵👇

Moo Jin Kim

149,398 views • 6 months ago

1/ We just released π0.7 — a steerable generalist robot model with emergent capabilities. I want to share a bit of the backstory, because π0.7 taught me something surprising about where robot learning is heading. A thread on bittersweet lessons 🧵

1/ We just released π0.7 — a steerable generalist robot model with emergent capabilities. I want to share a bit of the backstory, because π0.7 taught me something surprising about where robot learning is heading. A thread on bittersweet lessons 🧵

Lucy Shi

87,736 views • 3 months ago

1/5 🚀 Thrilled to open-source OSCAR 🤖 — an action-conditioned world model for robotics, led by the visiting student in my group Zhuoyuan Wu! It generalizes across different robot embodiments with precise action controllability. All trained on a single GH200 GPU, and outperforms existing open-sourced baselines, which have larger model capacity and need more compute. Everything is public, including training data. 📄 Paper: 🌐 Project: 💻 Code: 🤗 Robot data: 🤗 Human data: 🤗 Weights: #Robotics #WorldModels #AI #OpenSource

1/5 🚀 Thrilled to open-source OSCAR 🤖 — an action-conditioned world model for robotics, led by the visiting student in my group Zhuoyuan Wu! It generalizes across different robot embodiments with precise action controllability. All trained on a single GH200 GPU, and outperforms existing open-sourced baselines, which have larger model capacity and need more compute. Everything is public, including training data. 📄 Paper: 🌐 Project: 💻 Code: 🤗 Robot data: 🤗 Human data: 🤗 Weights: #Robotics #WorldModels #AI #OpenSource

Jun Gao

104,471 views • 1 month ago

The official user manual for the Dreamina Octo is finally here! The perfect companion for Dreamina Seedance 2.0 is Dreamina Octo, your AI agent. We call this Vibe Create. Now you can: · Think and create with Dreamina Octo, your AI partner. It combines intuitive thinking and deep creative skills, staying in sync with your ideas every step of the way. · Bring your ideas to life across text, image, and video. Inspiration is captured, enhanced, and shared in one collaborative space. · From the first spark to the finished work, Dreamina Octo's intelligent workflow keeps pace with you, helping ideas grow naturally into real creations. Advanced subscribers can start creating now, and everyone else can explore a demo project for a first look as access continues to roll out. Come to Dreamina AI and try Dreamina Octo today!

The official user manual for the Dreamina Octo is finally here! The perfect companion for Dreamina Seedance 2.0 is Dreamina Octo, your AI agent. We call this Vibe Create. Now you can: · Think and create with Dreamina Octo, your AI partner. It combines intuitive thinking and deep creative skills, staying in sync with your ideas every step of the way. · Bring your ideas to life across text, image, and video. Inspiration is captured, enhanced, and shared in one collaborative space. · From the first spark to the finished work, Dreamina Octo's intelligent workflow keeps pace with you, helping ideas grow naturally into real creations. Advanced subscribers can start creating now, and everyone else can explore a demo project for a first look as access continues to roll out. Come to Dreamina AI and try Dreamina Octo today!

Dreamina AI

2,083,652 views • 2 months ago

🔥 Hot release: Aloha unleashed World first demonstration of a robot able to tie shoelaces or hang t-shirts autonomously! They trained a diffusion policy at scale: 26,000 demonstrations over 5 tasks on Aloha 2 robot Retweet if you'd like them to open-source 😝 (video x4) 1/🧵

🔥 Hot release: Aloha unleashed World first demonstration of a robot able to tie shoelaces or hang t-shirts autonomously! They trained a diffusion policy at scale: 26,000 demonstrations over 5 tasks on Aloha 2 robot Retweet if you'd like them to open-source 😝 (video x4) 1/🧵

Remi Cadene

198,426 views • 1 year ago

An interactive world model developed by NVIDIA in collaboration with academic partners. - DreamDojo turns egocentric human video data into physical intelligence. - Human data is more scalable than robotics data but lacks action labels. - To solve this, a dedicated action model extracts latent actions by identifying physics and motion deltas between frames. Training - A massive 44k hours of video data are used for pre-training. - Post-training on small-scale robot datasets maps human physics to specific robot embodiments. - An additional distillation stage converts the model into an autoregressive, few-step diffusion model, enabling real-time, action-controllable simulation. Primary Use Cases - Live Teleoperation: Controlling a robot inside a world simulation in real-time. - Model-based Planning: Previewing and curating the best actions for improved success. - Policy Evaluation: Testing robot policies in realistic, out-of-distribution scenarios. Everything that's open-sourced: weights, code, post-training dataset, eval set, and details to reproduce.

An interactive world model developed by NVIDIA in collaboration with academic partners. - DreamDojo turns egocentric human video data into physical intelligence. - Human data is more scalable than robotics data but lacks action labels. - To solve this, a dedicated action model extracts latent actions by identifying physics and motion deltas between frames. Training - A massive 44k hours of video data are used for pre-training. - Post-training on small-scale robot datasets maps human physics to specific robot embodiments. - An additional distillation stage converts the model into an autoregressive, few-step diffusion model, enabling real-time, action-controllable simulation. Primary Use Cases - Live Teleoperation: Controlling a robot inside a world simulation in real-time. - Model-based Planning: Previewing and curating the best actions for improved success. - Policy Evaluation: Testing robot policies in realistic, out-of-distribution scenarios. Everything that's open-sourced: weights, code, post-training dataset, eval set, and details to reproduce.

The Humanoid Hub

11,575 views • 5 months ago

.One Robot builds world model-based simulations that are realistic to see and realistic to interact with, so robotics teams can train and evaluate VLAs without being bottlenecked by robot time. Congrats on the launch, Hemanth Sarabu and Elton Bear!

.One Robot builds world model-based simulations that are realistic to see and realistic to interact with, so robotics teams can train and evaluate VLAs without being bottlenecked by robot time. Congrats on the launch, Hemanth Sarabu and Elton Bear!

Y Combinator

25,902 views • 5 months ago

Excited to announce GR00T N1, the world’s first open foundation model for humanoid robots! We are on a mission to democratize Physical AI. The power of general robot brain, in the palm of your hand - with only 2B parameters, N1 learns from the most diverse physical action dataset ever compiled and punches above its weight: - Real humanoid teleoperation data. - Large-scale simulation data: we are open-sourcing 300K+ trajectories! - Neural trajectories: we apply SOTA video generation models to “hallucinate” new synthetic data that features accurate physics in pixels. Using Jensen’s words, “systematically infinite data”! - Latent actions: we develop novel algorithms to extract action tokens from in-the-wild human videos and neural generated videos. GR00T N1 is a single end-to-end neural net, from photons to actions: - Vision-Language Model (System 2) that interprets the physical world through vision and language instructions, enabling robots to reason about their environment and instructions, and plan the right actions. - Diffusion Transformer (System 1) that “renders” smooth and precise motor actions at 120 Hz, executing the latent plan made by System 2. We deploy N1 on GR1 robot, 1X Neo robot, and a large collection of simulation benchmarks. N1 achieves up to +30% boost in diverse manipulation tasks for household and industrial settings. While humanoid robots are the main focus of N1, our model also supports cross-embodiment. We finetune it to work on the $110 HuggingFace LeRobot SO100 robot arm! Open robot brain runs on open hardware. Sounds just right. Let’s solve robotics, together, one token at a time. Links to our Whitepaper, Github repo, HuggingFace model, and open dataset page in the thread: 🧵

Excited to announce GR00T N1, the world’s first open foundation model for humanoid robots! We are on a mission to democratize Physical AI. The power of general robot brain, in the palm of your hand - with only 2B parameters, N1 learns from the most diverse physical action dataset ever compiled and punches above its weight: - Real humanoid teleoperation data. - Large-scale simulation data: we are open-sourcing 300K+ trajectories! - Neural trajectories: we apply SOTA video generation models to “hallucinate” new synthetic data that features accurate physics in pixels. Using Jensen’s words, “systematically infinite data”! - Latent actions: we develop novel algorithms to extract action tokens from in-the-wild human videos and neural generated videos. GR00T N1 is a single end-to-end neural net, from photons to actions: - Vision-Language Model (System 2) that interprets the physical world through vision and language instructions, enabling robots to reason about their environment and instructions, and plan the right actions. - Diffusion Transformer (System 1) that “renders” smooth and precise motor actions at 120 Hz, executing the latent plan made by System 2. We deploy N1 on GR1 robot, 1X Neo robot, and a large collection of simulation benchmarks. N1 achieves up to +30% boost in diverse manipulation tasks for household and industrial settings. While humanoid robots are the main focus of N1, our model also supports cross-embodiment. We finetune it to work on the $110 HuggingFace LeRobot SO100 robot arm! Open robot brain runs on open hardware. Sounds just right. Let’s solve robotics, together, one token at a time. Links to our Whitepaper, Github repo, HuggingFace model, and open dataset page in the thread: 🧵

Jim Fan

466,333 views • 1 year ago

Meet Dreamina Octo 🐙 The creative agent unlocking the full potential of Dreamina Seedance 2.0. Tell Octo what you're thinking, and it： ✅ Reads your canvas ✅ Keeps your context ✅ Picks up where your last idea left off When your vision shifts, Octo shifts with it — not just following, but thinking alongside you in real time. This is Creative Resonance — and it's just the beginning of Vibe Create. ⌛ Coming soon on Dreamina AI. 🎁 RT + Comment within 12h to get 1000 extra credits via DM! First 500 users ONLY! #DreaminaOcto #DreaminaSeedance2 #DreaminaOctoVibeCreate #DreaminaAI

Meet Dreamina Octo 🐙 The creative agent unlocking the full potential of Dreamina Seedance 2.0. Tell Octo what you're thinking, and it： ✅ Reads your canvas ✅ Keeps your context ✅ Picks up where your last idea left off When your vision shifts, Octo shifts with it — not just following, but thinking alongside you in real time. This is Creative Resonance — and it's just the beginning of Vibe Create. ⌛ Coming soon on Dreamina AI. 🎁 RT + Comment within 12h to get 1000 extra credits via DM! First 500 users ONLY! #DreaminaOcto #DreaminaSeedance2 #DreaminaOctoVibeCreate #DreaminaAI

Dreamina AI

48,483 views • 3 months ago

introducing lattice a low-cost robot arm with a modular, detachable gripper so the robot can swap its own tools the adapter is $14 if you already have an so101 arm, and about $140 to build the full system from scratch it’s now fully open source

introducing lattice a low-cost robot arm with a modular, detachable gripper so the robot can swap its own tools the adapter is $14 if you already have an so101 arm, and about $140 to build the full system from scratch it’s now fully open source

ahad

40,184 views • 2 months ago