Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

Introducing an approach to directly ground video generation models to policy execution without needing any action labels! Our approach uses a generic goal-conditioned exploration procedure to learn a policy that works across robots / embodiments!

du_yilun's profile picture

Yilun Du

12,410 subscribers

21,409 Aufrufe • vor 1 Jahr •via X (Twitter)

BildungWissenschaft & TechnologieGesundheit & Wellness
LIVE

Anya Rossi• Live Now

Private livecam show

2 Kommentare

Profilbild von sonu
sonuvor 1 Jahr

Great work 👍

Profilbild von Yilun Du
Yilun Duvor 1 Jahr

Ohh haven't created an account there yet -- didn't realize lots of people were moving over.

Ähnliche Videos

The most frustrating part of imitation learning is collecting huge amounts of teleop data. But why teleop robots when robots can learn by watching us? Introducing Point Policy, a novel framework that enables robots to learn from human videos without any teleop, sim2real, or RL.
0:40

The most frustrating part of imitation learning is collecting huge amounts of teleop data. But why teleop robots when robots can learn by watching us? Introducing Point Policy, a novel framework that enables robots to learn from human videos without any teleop, sim2real, or RL.

Siddhant Haldar

69,056 Aufrufe • vor 1 Jahr

By unifying prediction and control, NavWAM acts as a closed-loop policy out of the box. In evaluations, it outperforms planning-based world models without needing test-time action search, matches a much larger 7B VLA policy, and transfers successfully to real mobile robots.
0:27

By unifying prediction and control, NavWAM acts as a closed-loop policy out of the box. In evaluations, it outperforms planning-based world models without needing test-time action search, matches a much larger 7B VLA policy, and transfers successfully to real mobile robots.

Robots Digest 🤖

11,076 Aufrufe • vor 3 Monaten

Respect the call. Introducing a stronger policy to protect our referees. Learn More >
1:53

Respect the call. Introducing a stronger policy to protect our referees. Learn More >

U.S. Soccer

15,977 Aufrufe • vor 1 Jahr

Vision-language models perform diverse tasks via in-context learning. Time for robots to do the same! Introducing In-Context Robot Transformer (ICRT): a robot policy that learns new tasks by prompting with robot trajectories, without any fine-tuning. [1/N]
0:25

Vision-language models perform diverse tasks via in-context learning. Time for robots to do the same! Introducing In-Context Robot Transformer (ICRT): a robot policy that learns new tasks by prompting with robot trajectories, without any fine-tuning. [1/N]

Max Fu

40,466 Aufrufe • vor 2 Jahren

How to teach robots to perform long-horizon tasks efficiently and robustly🦾? Introducing MimicPlay - an imitation learning algorithm that uses "cheap human play data". Our approach unlocks both real-time planning through raw perception and strong robustness to disturbances!🧵👇
1:15

How to teach robots to perform long-horizon tasks efficiently and robustly🦾? Introducing MimicPlay - an imitation learning algorithm that uses "cheap human play data". Our approach unlocks both real-time planning through raw perception and strong robustness to disturbances!🧵👇

Chen Wang

288,817 Aufrufe • vor 3 Jahren

Long-horizon visual goals remain surprisingly hard for robot manipulation. We introduce Act2Goal, a goal-conditioned policy that uses a visual world model to reason about progress toward a goal, and practice it autonomously in the real world.
0:20

Long-horizon visual goals remain surprisingly hard for robot manipulation. We introduce Act2Goal, a goal-conditioned policy that uses a visual world model to reason about progress toward a goal, and practice it autonomously in the real world.

Jianlan Luo

95,100 Aufrufe • vor 8 Monaten

A credible, clear and effective development policy is key to delivering a world free from poverty on a liveable planet. Today we’re launching a Development Review to help in that mission. With an approach based on respect, genuine partnership and evidence of what works.
1:30

A credible, clear and effective development policy is key to delivering a world free from poverty on a liveable planet. Today we’re launching a Development Review to help in that mission. With an approach based on respect, genuine partnership and evidence of what works.

Anneliese Dodds

35,622 Aufrufe • vor 2 Jahren

Presenting DemoDiffusion: An extremely simple approach enabling a pre-trained 'generalist' diffusion policy to follow a human-demonstration for a novel task during inference One-shot human imitation *without* requiring any paired human-robot data or online RL 🙂 1/n
0:30

Presenting DemoDiffusion: An extremely simple approach enabling a pre-trained 'generalist' diffusion policy to follow a human-demonstration for a novel task during inference One-shot human imitation *without* requiring any paired human-robot data or online RL 🙂 1/n

Homanga Bharadhwaj

32,919 Aufrufe • vor 1 Jahr

We’re bringing powerful AI directly onto robots with Gemini Robotics On-Device. 🤖 It’s our first vision-language-action model to help make robots faster, highly efficient, and adaptable to new tasks and environments - without needing a constant internet connection. 🧵
0:35

We’re bringing powerful AI directly onto robots with Gemini Robotics On-Device. 🤖 It’s our first vision-language-action model to help make robots faster, highly efficient, and adaptable to new tasks and environments - without needing a constant internet connection. 🧵

Google DeepMind

819,985 Aufrufe • vor 1 Jahr

🤔 Can we train one VLA policy to control multi-robot teams without any explicit communication? ✨ Introducing CHORUS: a single policy for decentralized, multi-embodiment collaboration 🧵⬇️
0:32

🤔 Can we train one VLA policy to control multi-robot teams without any explicit communication? ✨ Introducing CHORUS: a single policy for decentralized, multi-embodiment collaboration 🧵⬇️

Ria Doshi

60,546 Aufrufe • vor 3 Monaten

#ICC Office of the Prosecutor launched its Policy on Gender-based Crimes. #ASP22 Prosecutor Karim A. A. Khan KC: I have emphasised that we need to do better in relation to gender crimes & crimes against children. This Policy is a fundamental revision of our approach to gender crimes.
1:56

#ICC Office of the Prosecutor launched its Policy on Gender-based Crimes. #ASP22 Prosecutor Karim A. A. Khan KC: I have emphasised that we need to do better in relation to gender crimes & crimes against children. This Policy is a fundamental revision of our approach to gender crimes.

Int'l Criminal Court

28,449 Aufrufe • vor 2 Jahren

Industrial policy is back and greener than ever. Green industrial policy is how countries can continue to grow, while tackling the climate crisis head on. Open Society’s Luis Godoy explains how it is also an opportunity to build a fairer, cleaner, and stronger economy that benefits everyone—especially countries in the Global South. Learn more about our approach to economic and climate prosperity:
1:05

Industrial policy is back and greener than ever. Green industrial policy is how countries can continue to grow, while tackling the climate crisis head on. Open Society’s Luis Godoy explains how it is also an opportunity to build a fairer, cleaner, and stronger economy that benefits everyone—especially countries in the Global South. Learn more about our approach to economic and climate prosperity:

Open Society Foundations

14,699 Aufrufe • vor 9 Monaten

Testified at the Senate of Canada this morning, explaining to Senators why our current carbon pricing approach is flawed policy — and why the Grocery Code of Conduct is sound policy. Have a listen.
3:31

Testified at the Senate of Canada this morning, explaining to Senators why our current carbon pricing approach is flawed policy — and why the Grocery Code of Conduct is sound policy. Have a listen.

The Food Professor

21,806 Aufrufe • vor 7 Monaten

The holy grail of robotics is to be able to perform previously-unseen, out-of-distribution manipulation tasks “zero shot” in a new environment. NovaFlow proposes an approach which (1) generates a video, (2) computes predicted flow — how points move through the scene — and (3) uses this flow as an objective to generate a motion. Using this procedure, NovaFlow generates motions in unseen scenes, for unseen tasks, and can transfer across embodiments. To learn more, we are joined by Hongyu Li and Jiahui Fu from RAI. Watch Episode #63 of RoboPapers with Chris Paxton and Michael Cho - Rbt/Acc now to learn more!
1:08:49

The holy grail of robotics is to be able to perform previously-unseen, out-of-distribution manipulation tasks “zero shot” in a new environment. NovaFlow proposes an approach which (1) generates a video, (2) computes predicted flow — how points move through the scene — and (3) uses this flow as an objective to generate a motion. Using this procedure, NovaFlow generates motions in unseen scenes, for unseen tasks, and can transfer across embodiments. To learn more, we are joined by Hongyu Li and Jiahui Fu from RAI. Watch Episode #63 of RoboPapers with Chris Paxton and Michael Cho - Rbt/Acc now to learn more!

RoboPapers

21,434 Aufrufe • vor 6 Monaten

We open-source Action Images — a new representation that translates 7-DoF robot actions into interpretable images. Video models are emerging as powerful robotic foundation models, but a key challenge remains: how can we seamlessly integrate robot policies into video models? Instead of representing actions as low-dimensional control tokens, Action Images provide a pixel-grounded action representation, reframing policy learning as a visual tracking problem! By unifying observations and actions in the same video space, Action Images enable a unified robotics world model that supports video-action joint generation, action-conditioned video generation, and action labeling! Code: Paper:
0:51

We open-source Action Images — a new representation that translates 7-DoF robot actions into interpretable images. Video models are emerging as powerful robotic foundation models, but a key challenge remains: how can we seamlessly integrate robot policies into video models? Instead of representing actions as low-dimensional control tokens, Action Images provide a pixel-grounded action representation, reframing policy learning as a visual tracking problem! By unifying observations and actions in the same video space, Action Images enable a unified robotics world model that supports video-action joint generation, action-conditioned video generation, and action labeling! Code: Paper:

Chuang Gan

20,525 Aufrufe • vor 2 Monaten

Meet Microsoft Scout. An always-on agent that keeps work moving, taking action without needing to be prompted each time. As Microsoft’s first Autopilot agent, Microsoft Scout works across Teams, Outlook, OneDrive, and more—taking action within the controls your organization sets. Learn more:
0:58

Meet Microsoft Scout. An always-on agent that keeps work moving, taking action without needing to be prompted each time. As Microsoft’s first Autopilot agent, Microsoft Scout works across Teams, Outlook, OneDrive, and more—taking action within the controls your organization sets. Learn more:

Microsoft 365

448,818 Aufrufe • vor 3 Monaten