Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

Introducing an approach to directly ground video generation models to policy execution without needing any action labels! Our approach uses a generic goal-conditioned exploration procedure to learn a policy that works across robots / embodiments!

Yilun Du

12,180 subscribers

21,392 views • 1 year ago •via X (Twitter)

Education Science & Technology Health & Wellness

Anya Rossi• Live Now

Private livecam show

2 Comments

sonu1 year ago

Great work 👍

Yilun Du1 year ago

Ohh haven't created an account there yet -- didn't realize lots of people were moving over.

Related Videos

By unifying prediction and control, NavWAM acts as a closed-loop policy out of the box. In evaluations, it outperforms planning-based world models without needing test-time action search, matches a much larger 7B VLA policy, and transfers successfully to real mobile robots.

By unifying prediction and control, NavWAM acts as a closed-loop policy out of the box. In evaluations, it outperforms planning-based world models without needing test-time action search, matches a much larger 7B VLA policy, and transfers successfully to real mobile robots.

Robots Digest 🤖

11,076 views • 1 month ago

The most frustrating part of imitation learning is collecting huge amounts of teleop data. But why teleop robots when robots can learn by watching us? Introducing Point Policy, a novel framework that enables robots to learn from human videos without any teleop, sim2real, or RL.

The most frustrating part of imitation learning is collecting huge amounts of teleop data. But why teleop robots when robots can learn by watching us? Introducing Point Policy, a novel framework that enables robots to learn from human videos without any teleop, sim2real, or RL.

Siddhant Haldar

69,056 views • 1 year ago

Respect the call. Introducing a stronger policy to protect our referees. Learn More >

Respect the call. Introducing a stronger policy to protect our referees. Learn More >

U.S. Soccer

15,914 views • 1 year ago

Vision-language models perform diverse tasks via in-context learning. Time for robots to do the same! Introducing In-Context Robot Transformer (ICRT): a robot policy that learns new tasks by prompting with robot trajectories, without any fine-tuning. [1/N]

Vision-language models perform diverse tasks via in-context learning. Time for robots to do the same! Introducing In-Context Robot Transformer (ICRT): a robot policy that learns new tasks by prompting with robot trajectories, without any fine-tuning. [1/N]

Max Fu

40,447 views • 1 year ago

How to teach robots to perform long-horizon tasks efficiently and robustly🦾? Introducing MimicPlay - an imitation learning algorithm that uses "cheap human play data". Our approach unlocks both real-time planning through raw perception and strong robustness to disturbances!🧵👇

How to teach robots to perform long-horizon tasks efficiently and robustly🦾? Introducing MimicPlay - an imitation learning algorithm that uses "cheap human play data". Our approach unlocks both real-time planning through raw perception and strong robustness to disturbances!🧵👇

Chen Wang

288,642 views • 3 years ago

Long-horizon visual goals remain surprisingly hard for robot manipulation. We introduce Act2Goal, a goal-conditioned policy that uses a visual world model to reason about progress toward a goal, and practice it autonomously in the real world.

Long-horizon visual goals remain surprisingly hard for robot manipulation. We introduce Act2Goal, a goal-conditioned policy that uses a visual world model to reason about progress toward a goal, and practice it autonomously in the real world.

Jianlan Luo

95,100 views • 6 months ago

A credible, clear and effective development policy is key to delivering a world free from poverty on a liveable planet. Today we’re launching a Development Review to help in that mission. With an approach based on respect, genuine partnership and evidence of what works.

A credible, clear and effective development policy is key to delivering a world free from poverty on a liveable planet. Today we’re launching a Development Review to help in that mission. With an approach based on respect, genuine partnership and evidence of what works.

Anneliese Dodds

35,622 views • 1 year ago

A real-to-sim-to-real approach for more precise home robots. Full video:

A real-to-sim-to-real approach for more precise home robots. Full video:

MIT CSAIL

50,619 views • 2 years ago

Here’s why the lessons from Iraq support a balanced approach to foreign policy, one that lies between interventionism and isolationism.

Here’s why the lessons from Iraq support a balanced approach to foreign policy, one that lies between interventionism and isolationism.

Dinesh D'Souza

50,248 views • 1 year ago

Our CoRL 2024 paper shows Reinforcement Learning can allow robots to learn skills via real-world practice, without any demonstrations or simulation engineering. Rewards are provided using language/vision models, and mobility of robots enables autonomous exploration. 1/N

Our CoRL 2024 paper shows Reinforcement Learning can allow robots to learn skills via real-world practice, without any demonstrations or simulation engineering. Rewards are provided using language/vision models, and mobility of robots enables autonomous exploration. 1/N

Russell Mendonca

38,454 views • 1 year ago

We’re bringing powerful AI directly onto robots with Gemini Robotics On-Device. 🤖 It’s our first vision-language-action model to help make robots faster, highly efficient, and adaptable to new tasks and environments - without needing a constant internet connection. 🧵

We’re bringing powerful AI directly onto robots with Gemini Robotics On-Device. 🤖 It’s our first vision-language-action model to help make robots faster, highly efficient, and adaptable to new tasks and environments - without needing a constant internet connection. 🧵

Google DeepMind

819,309 views • 1 year ago

Presenting DemoDiffusion: An extremely simple approach enabling a pre-trained 'generalist' diffusion policy to follow a human-demonstration for a novel task during inference One-shot human imitation *without* requiring any paired human-robot data or online RL 🙂 1/n

Presenting DemoDiffusion: An extremely simple approach enabling a pre-trained 'generalist' diffusion policy to follow a human-demonstration for a novel task during inference One-shot human imitation without requiring any paired human-robot data or online RL 🙂 1/n

Homanga Bharadhwaj

32,830 views • 1 year ago

Industrial policy is back and greener than ever. Green industrial policy is how countries can continue to grow, while tackling the climate crisis head on. Open Society’s Luis Godoy explains how it is also an opportunity to build a fairer, cleaner, and stronger economy that benefits everyone—especially countries in the Global South. Learn more about our approach to economic and climate prosperity:

Industrial policy is back and greener than ever. Green industrial policy is how countries can continue to grow, while tackling the climate crisis head on. Open Society’s Luis Godoy explains how it is also an opportunity to build a fairer, cleaner, and stronger economy that benefits everyone—especially countries in the Global South. Learn more about our approach to economic and climate prosperity:

Open Society Foundations

14,699 views • 7 months ago

NEWS: Humanoid robot startup Figure announced Helix today, their "in-house AI that reasons like a human." "Our robots equipped with Helix can now pick up virtually any household object without any code or prior training. Helix uses a single set of neural network weights to learn all behaviors." "We're introducing Helix, a generalist Vision-Language-Action (VLA) model that unifies perception, language understanding, and learned control to overcome multiple longstanding challenges in robotics."

NEWS: Humanoid robot startup Figure announced Helix today, their "in-house AI that reasons like a human." "Our robots equipped with Helix can now pick up virtually any household object without any code or prior training. Helix uses a single set of neural network weights to learn all behaviors." "We're introducing Helix, a generalist Vision-Language-Action (VLA) model that unifies perception, language understanding, and learned control to overcome multiple longstanding challenges in robotics."

Sawyer Merritt

181,177 views • 1 year ago

Testified at the Senate of Canada this morning, explaining to Senators why our current carbon pricing approach is flawed policy — and why the Grocery Code of Conduct is sound policy. Have a listen.

Testified at the Senate of Canada this morning, explaining to Senators why our current carbon pricing approach is flawed policy — and why the Grocery Code of Conduct is sound policy. Have a listen.

The Food Professor

21,806 views • 5 months ago

Gen2Act: Casting language-conditioned manipulation as *human video generation* followed by *closed-loop policy execution conditioned on the generated video* enables solving diverse real-world tasks unseen in the robot dataset! 1/n

Gen2Act: Casting language-conditioned manipulation as human video generation followed by closed-loop policy execution conditioned on the generated video enables solving diverse real-world tasks unseen in the robot dataset! 1/n

Homanga Bharadhwaj

71,154 views • 1 year ago

The holy grail of robotics is to be able to perform previously-unseen, out-of-distribution manipulation tasks “zero shot” in a new environment. NovaFlow proposes an approach which (1) generates a video, (2) computes predicted flow — how points move through the scene — and (3) uses this flow as an objective to generate a motion. Using this procedure, NovaFlow generates motions in unseen scenes, for unseen tasks, and can transfer across embodiments. To learn more, we are joined by Hongyu Li and Jiahui Fu from RAI. Watch Episode #63 of RoboPapers with Chris Paxton and Michael Cho - Rbt/Acc now to learn more!

The holy grail of robotics is to be able to perform previously-unseen, out-of-distribution manipulation tasks “zero shot” in a new environment. NovaFlow proposes an approach which (1) generates a video, (2) computes predicted flow — how points move through the scene — and (3) uses this flow as an objective to generate a motion. Using this procedure, NovaFlow generates motions in unseen scenes, for unseen tasks, and can transfer across embodiments. To learn more, we are joined by Hongyu Li and Jiahui Fu from RAI. Watch Episode #63 of RoboPapers with Chris Paxton and Michael Cho - Rbt/Acc now to learn more!

RoboPapers

21,295 views • 4 months ago

Meet Microsoft Scout. An always-on agent that keeps work moving, taking action without needing to be prompted each time. As Microsoft’s first Autopilot agent, Microsoft Scout works across Teams, Outlook, OneDrive, and more—taking action within the controls your organization sets. Learn more:

Meet Microsoft Scout. An always-on agent that keeps work moving, taking action without needing to be prompted each time. As Microsoft’s first Autopilot agent, Microsoft Scout works across Teams, Outlook, OneDrive, and more—taking action within the controls your organization sets. Learn more:

Microsoft 365

445,995 views • 1 month ago

Can robots self-improve by collecting data autonomously🤖? Introducing SOAR: a system for large-scale autonomous data collection 🚀 and autonomous improvement📈of a multi-task language-conditioned policy in diverse scenes without human interventions .

Can robots self-improve by collecting data autonomously🤖? Introducing SOAR: a system for large-scale autonomous data collection 🚀 and autonomous improvement📈of a multi-task language-conditioned policy in diverse scenes without human interventions .

Paul Zhou

47,667 views • 1 year ago