Video wird geladen...

Video konnte nicht geladen werden

Beim Laden dieses Videos ist ein Problem aufgetreten. Dies könnte an einem vorübergehenden Netzwerkproblem liegen oder das Video ist möglicherweise nicht verfügbar.

We got a robot to clean up homes that were never seen in its training data! Our new model, π-0.5, aims to tackle open-world generalization. We took our robot into homes that were not in the training data and asked it to clean kitchens and bedrooms. More below⤵️

Physical Intelligence

50,167 subscribers

535,698 Aufrufe • vor 1 Jahr •via X (Twitter)

Gesundheit & Wellness Wissenschaft & Technologie Bildung

Anya Rossi• Live Now

Private livecam show

11 Kommentare

Profilbild von Physical Intelligence

Physical Intelligencevor 1 Jahr

π-0.5 performs hierarchical inference, inferring high-level semantic subtasks ("pick up the plate") followed by actions. It uses a co-training recipe with data from other robots, high-level commands, verbal instructions, and multimodal data from the web.

Profilbild von Physical Intelligence

Physical Intelligencevor 1 Jahr

97.6% of the π-0.5 data does not come from the mobile robots we use in our experiments. Most comes from other robots: cross-embodiment data in the lab, non-mobile (static) robots in the wild. It also includes web data. Each piece of data is important for good results.

Profilbild von Physical Intelligence

Physical Intelligencevor 1 Jahr

To learn more about π-0.5 (pronounced “pi oh five”), check out our blog post here: A research paper about π-0.5 describing the model, training, and experiments is here:

Profilbild von Yang

Yangvor 1 Jahr

Want to learn how practical AI skills and automations for your business and work? Check out our 50+ step-by-step video tutorials 100% FREE 20+ hours of Ai and Automation goodness absolutely free 🥳

Profilbild von kojichuuba

kojichuubavor 1 Jahr

inb4 can you put the fries in the bag

Profilbild von Luke Igel

Luke Igelvor 1 Jahr

Let's go @donaldjewkes

Profilbild von Mocib

Mocibvor 1 Jahr

But can it place the lotion in the basket?

Profilbild von AngelBattles.eth 🦇🔊

AngelBattles.eth 🦇🔊vor 1 Jahr

quite impressive

Profilbild von Felix

Felixvor 1 Jahr

Always super impressive

Profilbild von Maxx Yung

Maxx Yungvor 1 Jahr

damn

Profilbild von Vikram Mishra

Vikram Mishravor 1 Jahr

Good stuff!

Ähnliche Videos

We took a robot to RSS in LA running our new Gemini Robotics On-Device VLA model. People interacted with the model with new objects and instructions in a brand new environment and the results were amazing!

Ayzaan Wahid

19,710 Aufrufe • vor 1 Jahr

In my experience, robot 'generalists' are often jacks of all trades but masters of none. In training across multiple tasks and environments, robot policies fail to generalize robustly and effectively to each particular test setting. What if at test time, we non-parametrically *retrieved* “relevant” data from the training set and used it to significantly improve the performance of few-shot imitation learning to be robust to various test time scenes. Notably, we are *not* collecting lots of new data, just training more on sub-components of the same training data! Now, we’re certainly not the first to suggest retrieval, but in our new work - STRAP, we show how retrieving relevant *sub-trajectories* from offline datasets can significantly increase data reuse across tasks, when paired with an appropriate metric space. A 🧵 (1/7)

In my experience, robot 'generalists' are often jacks of all trades but masters of none. In training across multiple tasks and environments, robot policies fail to generalize robustly and effectively to each particular test setting. What if at test time, we non-parametrically retrieved “relevant” data from the training set and used it to significantly improve the performance of few-shot imitation learning to be robust to various test time scenes. Notably, we are not collecting lots of new data, just training more on sub-components of the same training data! Now, we’re certainly not the first to suggest retrieval, but in our new work - STRAP, we show how retrieving relevant sub-trajectories from offline datasets can significantly increase data reuse across tasks, when paired with an appropriate metric space. A 🧵 (1/7)

Abhishek Gupta

12,045 Aufrufe • vor 1 Jahr

Jason Ma reveals the first true scaling law in robotics: robots trained on 1 million hours of first-person human video predictably get better without ever seeing robot data "Yesterday we announced our new flagship robot foundation model called Dyna-2. It's very significant for the entire field for many different reasons. First of all, it's the first robot foundation model trained on at least one million hours of data. That in itself is a very challenging infrastructure challenge." "What we demonstrated is that even if the one million hours of data is entirely first-person video of humans doing manipulation tasks, we actually see scaling law for transferring to robot embodiments that the model has never seen before." "By training on more human data, we actually see predictable performance improvement on robots. That's a big deal because the biggest challenge facing robot foundation models is that we don't have enough data." "Unlike language models, there are just not enough robotics data out there, and it's very difficult to collect these robot data. Only by passively observing humans doing things can we scale this kind of data, and this is the first time we showed there is a lot of improvement that can transfer." Jason Ma Dyna Robotics

Jason Ma reveals the first true scaling law in robotics: robots trained on 1 million hours of first-person human video predictably get better without ever seeing robot data "Yesterday we announced our new flagship robot foundation model called Dyna-2. It's very significant for the entire field for many different reasons. First of all, it's the first robot foundation model trained on at least one million hours of data. That in itself is a very challenging infrastructure challenge." "What we demonstrated is that even if the one million hours of data is entirely first-person video of humans doing manipulation tasks, we actually see scaling law for transferring to robot embodiments that the model has never seen before." "By training on more human data, we actually see predictable performance improvement on robots. That's a big deal because the biggest challenge facing robot foundation models is that we don't have enough data." "Unlike language models, there are just not enough robotics data out there, and it's very difficult to collect these robot data. Only by passively observing humans doing things can we scale this kind of data, and this is the first time we showed there is a lot of improvement that can transfer." Jason Ma Dyna Robotics

MTS

17,740 Aufrufe • vor 1 Monat

Imagine robots learning new skills—without any robot data. Today, we're excited to release EgoZero: our first steps in training robot policies that operate in unseen environments, solely from data collected through humans wearing Aria smart glasses. 🧵👇

Imagine robots learning new skills—without any robot data. Today, we're excited to release EgoZero: our first steps in training robot policies that operate in unseen environments, solely from data collected through humans wearing Aria smart glasses. 🧵👇

Lerrel Pinto

42,673 Aufrufe • vor 1 Jahr

Figure took its humanoid into 30 homes it had never seen and asked it to tidy toys, fold towels and make beds. Helix 2.5 completed 56% of full-task trials in Figure’s evaluation. The important distinction: it had learned these chores elsewhere. No additional training was done for these homes or their objects. That’s a meaningful step toward a robot you don’t have to retrain for every house. 🎥 Figure

Figure took its humanoid into 30 homes it had never seen and asked it to tidy toys, fold towels and make beds. Helix 2.5 completed 56% of full-task trials in Figure’s evaluation. The important distinction: it had learned these chores elsewhere. No additional training was done for these homes or their objects. That’s a meaningful step toward a robot you don’t have to retrain for every house. 🎥 Figure

Humanoids daily

27,612 Aufrufe • vor 6 Tagen

ego-to-robot data pipeline is one key component of Qwen-RobotManip （ Today, we’re sharing the full technical details behind it: Ego2Robot 🤖 It turns egocentric human videos into scalable robot training data and improves VLA generalization across tasks and embodiments. page: arxiv: huggingface upvoting 🚀:

ego-to-robot data pipeline is one key component of Qwen-RobotManip （ Today, we’re sharing the full technical details behind it: Ego2Robot 🤖 It turns egocentric human videos into scalable robot training data and improves VLA generalization across tasks and embodiments. page: arxiv: huggingface upvoting 🚀:

barry (xiong-hui) chen

17,254 Aufrufe • vor 1 Monat

A humanoid robot policy trained solely on synthetic data generated by a world model. Research Scientist Joel Jang presents NVIDIA's DreamGen pipeline: ⦿ Post-train the world model Cosmos-Predict2 with a small set of real teleoperation demos. ⦿ Prompt the world model to generate synthetic video data with verbs and scenarios not used in the world model’s post-training. ⦿ Auto-label synthetic video data with action sequences. ⦿ Train robot policies using only synthetic data. That's it. Deploy zero-shot to a real humanoid robot.

A humanoid robot policy trained solely on synthetic data generated by a world model. Research Scientist Joel Jang presents NVIDIA's DreamGen pipeline: ⦿ Post-train the world model Cosmos-Predict2 with a small set of real teleoperation demos. ⦿ Prompt the world model to generate synthetic video data with verbs and scenarios not used in the world model’s post-training. ⦿ Auto-label synthetic video data with action sequences. ⦿ Train robot policies using only synthetic data. That's it. Deploy zero-shot to a real humanoid robot.

The Humanoid Hub

20,968 Aufrufe • vor 1 Jahr

When will robots help us with our household chores? TidyBot++ brings us closer to that future. Our new open-source mobile manipulator makes it more accessible and practical to do robot learning research outside the lab, in real homes!

When will robots help us with our household chores? TidyBot++ brings us closer to that future. Our new open-source mobile manipulator makes it more accessible and practical to do robot learning research outside the lab, in real homes!

Jimmy Wu

188,682 Aufrufe • vor 1 Jahr

How to use simulation data for real-world robot manipulation? We present sim-and-real co-training, a simple recipe for manipulation. We demonstrate that sim data can significantly enhance real-world performance, even with notable differences between the sim and the real. (1/n)

How to use simulation data for real-world robot manipulation? We present sim-and-real co-training, a simple recipe for manipulation. We demonstrate that sim data can significantly enhance real-world performance, even with notable differences between the sim and the real. (1/n)

Zhenyu Jiang

44,813 Aufrufe • vor 1 Jahr

An interactive world model developed by NVIDIA in collaboration with academic partners. - DreamDojo turns egocentric human video data into physical intelligence. - Human data is more scalable than robotics data but lacks action labels. - To solve this, a dedicated action model extracts latent actions by identifying physics and motion deltas between frames. Training - A massive 44k hours of video data are used for pre-training. - Post-training on small-scale robot datasets maps human physics to specific robot embodiments. - An additional distillation stage converts the model into an autoregressive, few-step diffusion model, enabling real-time, action-controllable simulation. Primary Use Cases - Live Teleoperation: Controlling a robot inside a world simulation in real-time. - Model-based Planning: Previewing and curating the best actions for improved success. - Policy Evaluation: Testing robot policies in realistic, out-of-distribution scenarios. Everything that's open-sourced: weights, code, post-training dataset, eval set, and details to reproduce.

An interactive world model developed by NVIDIA in collaboration with academic partners. - DreamDojo turns egocentric human video data into physical intelligence. - Human data is more scalable than robotics data but lacks action labels. - To solve this, a dedicated action model extracts latent actions by identifying physics and motion deltas between frames. Training - A massive 44k hours of video data are used for pre-training. - Post-training on small-scale robot datasets maps human physics to specific robot embodiments. - An additional distillation stage converts the model into an autoregressive, few-step diffusion model, enabling real-time, action-controllable simulation. Primary Use Cases - Live Teleoperation: Controlling a robot inside a world simulation in real-time. - Model-based Planning: Previewing and curating the best actions for improved success. - Policy Evaluation: Testing robot policies in realistic, out-of-distribution scenarios. Everything that's open-sourced: weights, code, post-training dataset, eval set, and details to reproduce.

The Humanoid Hub

11,575 Aufrufe • vor 7 Monaten

We brought Dynasaur to The Clean Show and ran it live, 8 hours a day. Each deployment fuels our robotics foundation model — scaling data, accelerating iteration, and enabling real-world generalization. Embodied intelligence won’t come from lab demos, but from deployment-first robotics running continuously in the wild.

We brought Dynasaur to The Clean Show and ran it live, 8 hours a day. Each deployment fuels our robotics foundation model — scaling data, accelerating iteration, and enabling real-world generalization. Embodied intelligence won’t come from lab demos, but from deployment-first robotics running continuously in the wild.

Dyna Robotics

24,901 Aufrufe • vor 1 Jahr

Putting it in perspective. We have every reason to be PROUD, we just need to clean up our homes. APC, PDP, CABALS, CRIMINALS, YAHOO boys and men, May your days expire. I believe in a New Nigeria.

Putting it in perspective. We have every reason to be PROUD, we just need to clean up our homes. APC, PDP, CABALS, CRIMINALS, YAHOO boys and men, May your days expire. I believe in a New Nigeria.

Charly Boy Area Fada 1

23,690 Aufrufe • vor 3 Jahren

Today’s data shows that when it comes to growing our economy, building more homes, incentivising investment and boosting wages and incomes, we are making a lot of progress together.

Today’s data shows that when it comes to growing our economy, building more homes, incentivising investment and boosting wages and incomes, we are making a lot of progress together.

Jim Chalmers MP

24,597 Aufrufe • vor 9 Monaten

So I heard we need more data for robot learning :) Purely real world teleop is expensive and slow, making large scale data collection challenging. I’ve been excited about getting more data into robot learning, going beyond just real-world teleop data. To this end, we’ve been scaling up data generation with RL in realistic simulations generated on the fly from crowdsourced videos. Enables realistic data collection, much more cheaply than purely real world teleop. Importantly, data collection becomes even*cheaper* with more environments, allowing training with over 100x more data. Transfers to real robots for generalizable manipulation. A 🧵 (1/N)

So I heard we need more data for robot learning :) Purely real world teleop is expensive and slow, making large scale data collection challenging. I’ve been excited about getting more data into robot learning, going beyond just real-world teleop data. To this end, we’ve been scaling up data generation with RL in realistic simulations generated on the fly from crowdsourced videos. Enables realistic data collection, much more cheaply than purely real world teleop. Importantly, data collection becomes evencheaper with more environments, allowing training with over 100x more data. Transfers to real robots for generalizable manipulation. A 🧵 (1/N)

Abhishek Gupta

13,358 Aufrufe • vor 1 Jahr