正在加载视频...

视频加载失败

加载此视频时出现问题。这可能是由于临时网络问题，或视频可能不可用。

We got a robot to clean up homes that were never seen in its training data! Our new model, π-0.5, aims to tackle open-world generalization. We took our robot into homes that were not in the training data and asked it to clean kitchens and bedrooms. More below⤵️

Physical Intelligence

50,575 subscribers

536,611 次观看 • 1 年前 •via X (Twitter)

健康养生科学技术教育

Anya Rossi• Live Now

Private livecam show

11 条评论

Physical Intelligence 的头像

Physical Intelligence1 年前

π-0.5 performs hierarchical inference, inferring high-level semantic subtasks ("pick up the plate") followed by actions. It uses a co-training recipe with data from other robots, high-level commands, verbal instructions, and multimodal data from the web.

Physical Intelligence 的头像

Physical Intelligence1 年前

97.6% of the π-0.5 data does not come from the mobile robots we use in our experiments. Most comes from other robots: cross-embodiment data in the lab, non-mobile (static) robots in the wild. It also includes web data. Each piece of data is important for good results.

Physical Intelligence 的头像

Physical Intelligence1 年前

To learn more about π-0.5 (pronounced “pi oh five”), check out our blog post here: A research paper about π-0.5 describing the model, training, and experiments is here:

Yang 的头像

Yang1 年前

Want to learn how practical AI skills and automations for your business and work? Check out our 50+ step-by-step video tutorials 100% FREE 20+ hours of Ai and Automation goodness absolutely free 🥳

kojichuuba 的头像

kojichuuba1 年前

inb4 can you put the fries in the bag

Luke Igel 的头像

Luke Igel1 年前

Let's go @donaldjewkes

Mocib 的头像

Mocib1 年前

But can it place the lotion in the basket?

AngelBattles.eth 🦇🔊 的头像

AngelBattles.eth 🦇🔊1 年前

quite impressive

Felix 的头像

Felix1 年前

Always super impressive

Maxx Yung 的头像

Maxx Yung1 年前

damn

Vikram Mishra 的头像

Vikram Mishra1 年前

Good stuff!

相关视频

We took a robot to RSS in LA running our new Gemini Robotics On-Device VLA model. People interacted with the model with new objects and instructions in a brand new environment and the results were amazing!

Ayzaan Wahid

19,710 次观看 • 1 年前

In my experience, robot 'generalists' are often jacks of all trades but masters of none. In training across multiple tasks and environments, robot policies fail to generalize robustly and effectively to each particular test setting. What if at test time, we non-parametrically *retrieved* “relevant” data from the training set and used it to significantly improve the performance of few-shot imitation learning to be robust to various test time scenes. Notably, we are *not* collecting lots of new data, just training more on sub-components of the same training data! Now, we’re certainly not the first to suggest retrieval, but in our new work - STRAP, we show how retrieving relevant *sub-trajectories* from offline datasets can significantly increase data reuse across tasks, when paired with an appropriate metric space. A 🧵 (1/7)

In my experience, robot 'generalists' are often jacks of all trades but masters of none. In training across multiple tasks and environments, robot policies fail to generalize robustly and effectively to each particular test setting. What if at test time, we non-parametrically retrieved “relevant” data from the training set and used it to significantly improve the performance of few-shot imitation learning to be robust to various test time scenes. Notably, we are not collecting lots of new data, just training more on sub-components of the same training data! Now, we’re certainly not the first to suggest retrieval, but in our new work - STRAP, we show how retrieving relevant sub-trajectories from offline datasets can significantly increase data reuse across tasks, when paired with an appropriate metric space. A 🧵 (1/7)

Abhishek Gupta

12,045 次观看 • 1 年前

Jason Ma reveals the first true scaling law in robotics: robots trained on 1 million hours of first-person human video predictably get better without ever seeing robot data "Yesterday we announced our new flagship robot foundation model called Dyna-2. It's very significant for the entire field for many different reasons. First of all, it's the first robot foundation model trained on at least one million hours of data. That in itself is a very challenging infrastructure challenge." "What we demonstrated is that even if the one million hours of data is entirely first-person video of humans doing manipulation tasks, we actually see scaling law for transferring to robot embodiments that the model has never seen before." "By training on more human data, we actually see predictable performance improvement on robots. That's a big deal because the biggest challenge facing robot foundation models is that we don't have enough data." "Unlike language models, there are just not enough robotics data out there, and it's very difficult to collect these robot data. Only by passively observing humans doing things can we scale this kind of data, and this is the first time we showed there is a lot of improvement that can transfer." Jason Ma Dyna Robotics

Jason Ma reveals the first true scaling law in robotics: robots trained on 1 million hours of first-person human video predictably get better without ever seeing robot data "Yesterday we announced our new flagship robot foundation model called Dyna-2. It's very significant for the entire field for many different reasons. First of all, it's the first robot foundation model trained on at least one million hours of data. That in itself is a very challenging infrastructure challenge." "What we demonstrated is that even if the one million hours of data is entirely first-person video of humans doing manipulation tasks, we actually see scaling law for transferring to robot embodiments that the model has never seen before." "By training on more human data, we actually see predictable performance improvement on robots. That's a big deal because the biggest challenge facing robot foundation models is that we don't have enough data." "Unlike language models, there are just not enough robotics data out there, and it's very difficult to collect these robot data. Only by passively observing humans doing things can we scale this kind of data, and this is the first time we showed there is a lot of improvement that can transfer." Jason Ma Dyna Robotics

MTS

17,740 次观看 • 1 个月前

Imagine robots learning new skills—without any robot data. Today, we're excited to release EgoZero: our first steps in training robot policies that operate in unseen environments, solely from data collected through humans wearing Aria smart glasses. 🧵👇

Imagine robots learning new skills—without any robot data. Today, we're excited to release EgoZero: our first steps in training robot policies that operate in unseen environments, solely from data collected through humans wearing Aria smart glasses. 🧵👇

Lerrel Pinto

42,673 次观看 • 1 年前

Figure took its humanoid into 30 homes it had never seen and asked it to tidy toys, fold towels and make beds. Helix 2.5 completed 56% of full-task trials in Figure’s evaluation. The important distinction: it had learned these chores elsewhere. No additional training was done for these homes or their objects. That’s a meaningful step toward a robot you don’t have to retrain for every house. 🎥 Figure

Figure took its humanoid into 30 homes it had never seen and asked it to tidy toys, fold towels and make beds. Helix 2.5 completed 56% of full-task trials in Figure’s evaluation. The important distinction: it had learned these chores elsewhere. No additional training was done for these homes or their objects. That’s a meaningful step toward a robot you don’t have to retrain for every house. 🎥 Figure

Humanoids daily

27,612 次观看 • 12 天前

ego-to-robot data pipeline is one key component of Qwen-RobotManip （ Today, we’re sharing the full technical details behind it: Ego2Robot 🤖 It turns egocentric human videos into scalable robot training data and improves VLA generalization across tasks and embodiments. page: arxiv: huggingface upvoting 🚀:

ego-to-robot data pipeline is one key component of Qwen-RobotManip （ Today, we’re sharing the full technical details behind it: Ego2Robot 🤖 It turns egocentric human videos into scalable robot training data and improves VLA generalization across tasks and embodiments. page: arxiv: huggingface upvoting 🚀:

barry (xiong-hui) chen

17,329 次观看 • 1 个月前

A humanoid robot policy trained solely on synthetic data generated by a world model. Research Scientist Joel Jang presents NVIDIA's DreamGen pipeline: ⦿ Post-train the world model Cosmos-Predict2 with a small set of real teleoperation demos. ⦿ Prompt the world model to generate synthetic video data with verbs and scenarios not used in the world model’s post-training. ⦿ Auto-label synthetic video data with action sequences. ⦿ Train robot policies using only synthetic data. That's it. Deploy zero-shot to a real humanoid robot.

A humanoid robot policy trained solely on synthetic data generated by a world model. Research Scientist Joel Jang presents NVIDIA's DreamGen pipeline: ⦿ Post-train the world model Cosmos-Predict2 with a small set of real teleoperation demos. ⦿ Prompt the world model to generate synthetic video data with verbs and scenarios not used in the world model’s post-training. ⦿ Auto-label synthetic video data with action sequences. ⦿ Train robot policies using only synthetic data. That's it. Deploy zero-shot to a real humanoid robot.

The Humanoid Hub

20,968 次观看 • 1 年前

When will robots help us with our household chores? TidyBot++ brings us closer to that future. Our new open-source mobile manipulator makes it more accessible and practical to do robot learning research outside the lab, in real homes!

When will robots help us with our household chores? TidyBot++ brings us closer to that future. Our new open-source mobile manipulator makes it more accessible and practical to do robot learning research outside the lab, in real homes!

Jimmy Wu

188,753 次观看 • 1 年前

How to use simulation data for real-world robot manipulation? We present sim-and-real co-training, a simple recipe for manipulation. We demonstrate that sim data can significantly enhance real-world performance, even with notable differences between the sim and the real. (1/n)

How to use simulation data for real-world robot manipulation? We present sim-and-real co-training, a simple recipe for manipulation. We demonstrate that sim data can significantly enhance real-world performance, even with notable differences between the sim and the real. (1/n)

Zhenyu Jiang

44,844 次观看 • 1 年前

An interactive world model developed by NVIDIA in collaboration with academic partners. - DreamDojo turns egocentric human video data into physical intelligence. - Human data is more scalable than robotics data but lacks action labels. - To solve this, a dedicated action model extracts latent actions by identifying physics and motion deltas between frames. Training - A massive 44k hours of video data are used for pre-training. - Post-training on small-scale robot datasets maps human physics to specific robot embodiments. - An additional distillation stage converts the model into an autoregressive, few-step diffusion model, enabling real-time, action-controllable simulation. Primary Use Cases - Live Teleoperation: Controlling a robot inside a world simulation in real-time. - Model-based Planning: Previewing and curating the best actions for improved success. - Policy Evaluation: Testing robot policies in realistic, out-of-distribution scenarios. Everything that's open-sourced: weights, code, post-training dataset, eval set, and details to reproduce.

An interactive world model developed by NVIDIA in collaboration with academic partners. - DreamDojo turns egocentric human video data into physical intelligence. - Human data is more scalable than robotics data but lacks action labels. - To solve this, a dedicated action model extracts latent actions by identifying physics and motion deltas between frames. Training - A massive 44k hours of video data are used for pre-training. - Post-training on small-scale robot datasets maps human physics to specific robot embodiments. - An additional distillation stage converts the model into an autoregressive, few-step diffusion model, enabling real-time, action-controllable simulation. Primary Use Cases - Live Teleoperation: Controlling a robot inside a world simulation in real-time. - Model-based Planning: Previewing and curating the best actions for improved success. - Policy Evaluation: Testing robot policies in realistic, out-of-distribution scenarios. Everything that's open-sourced: weights, code, post-training dataset, eval set, and details to reproduce.

The Humanoid Hub

11,575 次观看 • 7 个月前

We brought Dynasaur to The Clean Show and ran it live, 8 hours a day. Each deployment fuels our robotics foundation model — scaling data, accelerating iteration, and enabling real-world generalization. Embodied intelligence won’t come from lab demos, but from deployment-first robotics running continuously in the wild.

We brought Dynasaur to The Clean Show and ran it live, 8 hours a day. Each deployment fuels our robotics foundation model — scaling data, accelerating iteration, and enabling real-world generalization. Embodied intelligence won’t come from lab demos, but from deployment-first robotics running continuously in the wild.

Dyna Robotics

24,901 次观看 • 1 年前

Putting it in perspective. We have every reason to be PROUD, we just need to clean up our homes. APC, PDP, CABALS, CRIMINALS, YAHOO boys and men, May your days expire. I believe in a New Nigeria.

Putting it in perspective. We have every reason to be PROUD, we just need to clean up our homes. APC, PDP, CABALS, CRIMINALS, YAHOO boys and men, May your days expire. I believe in a New Nigeria.

Charly Boy Area Fada 1

23,690 次观看 • 3 年前

Today’s data shows that when it comes to growing our economy, building more homes, incentivising investment and boosting wages and incomes, we are making a lot of progress together.

Today’s data shows that when it comes to growing our economy, building more homes, incentivising investment and boosting wages and incomes, we are making a lot of progress together.

Jim Chalmers MP

24,597 次观看 • 10 个月前

So I heard we need more data for robot learning :) Purely real world teleop is expensive and slow, making large scale data collection challenging. I’ve been excited about getting more data into robot learning, going beyond just real-world teleop data. To this end, we’ve been scaling up data generation with RL in realistic simulations generated on the fly from crowdsourced videos. Enables realistic data collection, much more cheaply than purely real world teleop. Importantly, data collection becomes even*cheaper* with more environments, allowing training with over 100x more data. Transfers to real robots for generalizable manipulation. A 🧵 (1/N)

So I heard we need more data for robot learning :) Purely real world teleop is expensive and slow, making large scale data collection challenging. I’ve been excited about getting more data into robot learning, going beyond just real-world teleop data. To this end, we’ve been scaling up data generation with RL in realistic simulations generated on the fly from crowdsourced videos. Enables realistic data collection, much more cheaply than purely real world teleop. Importantly, data collection becomes evencheaper with more environments, allowing training with over 100x more data. Transfers to real robots for generalizable manipulation. A 🧵 (1/N)

Abhishek Gupta

13,358 次观看 • 1 年前