Video yükleniyor...

Video Yüklenemedi

Ana Sayfaya Dön

Can we leverage VLMs for robot manipulation in the open world? Checkout our new work MOKA, a simple and effective visual prompting method!

fangchenliu_'s profile picture

Fangchen Liu

1,489 subscribers

81,346 görüntüleme • 2 yıl önce •via X (Twitter)

EğitimBilim & Teknoloji
LIVE

Anya Rossi• Live Now

Private livecam show

0 Yorum

Yorum bulunmuyor

Orijinal gönderinin yorumları burada görünecek

Benzer Videolar

Long-horizon visual goals remain surprisingly hard for robot manipulation. We introduce Act2Goal, a goal-conditioned policy that uses a visual world model to reason about progress toward a goal, and practice it autonomously in the real world.
0:20

Long-horizon visual goals remain surprisingly hard for robot manipulation. We introduce Act2Goal, a goal-conditioned policy that uses a visual world model to reason about progress toward a goal, and practice it autonomously in the real world.

Jianlan Luo

95,100 görüntüleme • 9 ay önce

How can we more effectively leverage robot data from different embodiments for skill transfer? Excited to share that our new work, RoVi-Aug, has been accepted to Conference on Robot Learning as an oral paper！ WIth RoVi-Aug, you can augment an existing robot dataset into a different robot and different viewpoints. A policy trained on the augmented dataset can zero-shot deploy on the unseen target robot with significantly different camera angles! 🧵👇 🔗 Check out our paper:
0:18

How can we more effectively leverage robot data from different embodiments for skill transfer? Excited to share that our new work, RoVi-Aug, has been accepted to Conference on Robot Learning as an oral paper！ WIth RoVi-Aug, you can augment an existing robot dataset into a different robot and different viewpoints. A policy trained on the augmented dataset can zero-shot deploy on the unseen target robot with significantly different camera angles! 🧵👇 🔗 Check out our paper:

Chenfeng_X

28,329 görüntüleme • 2 yıl önce

Evaluating policies on a real robot can be painful. Can we use a world model to get a rough estimate of how good a policy is? Checkout "Evaluating Robot Policies in a World Model". Paper: Demo: Code:
0:10

Evaluating policies on a real robot can be painful. Can we use a world model to get a rough estimate of how good a policy is? Checkout "Evaluating Robot Policies in a World Model". Paper: Demo: Code:

Sherry Yang

37,028 görüntüleme • 1 yıl önce

🤖Why is robot manipulation still an open challenge? This video shows a kitting task -- packing multiple items into a single product. No robot today can do this autonomously. Big challenges = big opportunities for research and industry. #robotics #manipulation #automation
0:19

🤖Why is robot manipulation still an open challenge? This video shows a kitting task -- packing multiple items into a single product. No robot today can do this autonomously. Big challenges = big opportunities for research and industry. #robotics #manipulation #automation

Yu Xiang

21,236 görüntüleme • 1 yıl önce

How can we leverage diverse human videos to improve robot manipulation? Excited to introduce EgoVLA — a Vision-Language-Action model trained on egocentric human videos by explicitly modeling wrist & hand motion. We build a shared action space between humans and robots, enabling seamless transfer. With some robot demos, EgoVLA becomes a powerful, generalizable robot policy.
0:14

How can we leverage diverse human videos to improve robot manipulation? Excited to introduce EgoVLA — a Vision-Language-Action model trained on egocentric human videos by explicitly modeling wrist & hand motion. We build a shared action space between humans and robots, enabling seamless transfer. With some robot demos, EgoVLA becomes a powerful, generalizable robot policy.

Ruihan Yang

58,941 görüntüleme • 1 yıl önce

Made for the details and the demands of real work. Built for craft and mastery. A first look at our new robot. Coming soon.
0:14

Made for the details and the demands of real work. Built for craft and mastery. A first look at our new robot. Coming soon.

Dyna Robotics

207,707 görüntüleme • 12 gün önce

This is how a simple self-defence technique can help you escape danger — effective moves focus on leverage, balance, and timing rather than brute force.
0:13

This is how a simple self-defence technique can help you escape danger — effective moves focus on leverage, balance, and timing rather than brute force.

Fight Club

39,741 görüntüleme • 3 ay önce

Turns out you can train humanoid hands without any robot data. The idea in HUG is quite simple: (a) collect human data with smart glasses, (b) train a human manipulation model, (c) retarget to multi-fingered robot hands.
0:18

Turns out you can train humanoid hands without any robot data. The idea in HUG is quite simple: (a) collect human data with smart glasses, (b) train a human manipulation model, (c) retarget to multi-fingered robot hands.

Lerrel Pinto

34,942 görüntüleme • 3 ay önce

🚀 Imitating shortest paths in simulation enables effective navigation and manipulation in the real world. Our findings fly in the face of conventional wisdom! This is a big joint effort from PRIOR Ai2 (6 first authors!).
0:15

🚀 Imitating shortest paths in simulation enables effective navigation and manipulation in the real world. Our findings fly in the face of conventional wisdom! This is a big joint effort from PRIOR Ai2 (6 first authors!).

Kiana Ehsani

61,032 görüntüleme • 2 yıl önce

We started using super-b trailers for loading all our drills. Just trying a new method of filling in the fields.
0:10

We started using super-b trailers for loading all our drills. Just trying a new method of filling in the fields.

Tyler Burns

22,253 görüntüleme • 1 yıl önce

Spent the past few weeks building a fully automated pipeline for egocentric manipulation data with Kyle Jiang. Any new manipulation task captured using our own wearable sensing platform can now be dropped into the same pipeline with minimal manual fix. Instead of raw videos and annotations, we transform human demonstrations into robot-ready training data. Next, we're bringing touch into the loop with tactile gloves.
0:11

Spent the past few weeks building a fully automated pipeline for egocentric manipulation data with Kyle Jiang. Any new manipulation task captured using our own wearable sensing platform can now be dropped into the same pipeline with minimal manual fix. Instead of raw videos and annotations, we transform human demonstrations into robot-ready training data. Next, we're bringing touch into the loop with tactile gloves.

Grace Zhang

23,160 görüntüleme • 10 ay önce

A Gaussian Splat can become a world where Robots and AI agents can act. In our latest OVER Research experiment, we placed a robot inside a real-world 3D capture, with a VLM making decisions based on what it sees. At every step, the robot holds a pose in the reconstruction, gets a newly rendered view of the environment, takes an action, moves, and sees the world again from its new position. Why does this matter? Because 3D captures can become more than reconstructions to explore. They can become environments where embodied AI and robots can navigate, act, be evaluated and eventually train across real-world spaces at scale. Capture a place once. Then turn it into a world where AI and Robots can act. The full experiment, including what we discovered once we actually put the loop to the test:
0:14

A Gaussian Splat can become a world where Robots and AI agents can act. In our latest OVER Research experiment, we placed a robot inside a real-world 3D capture, with a VLM making decisions based on what it sees. At every step, the robot holds a pose in the reconstruction, gets a newly rendered view of the environment, takes an action, moves, and sees the world again from its new position. Why does this matter? Because 3D captures can become more than reconstructions to explore. They can become environments where embodied AI and robots can navigate, act, be evaluated and eventually train across real-world spaces at scale. Capture a place once. Then turn it into a world where AI and Robots can act. The full experiment, including what we discovered once we actually put the loop to the test:

Over the Reality 🌐

15,316 görüntüleme • 1 ay önce

There are many things going on in the world, many evil things, the most important is the manipulation of our the air we breathe and the manipulation of the weather. Stay focused.
0:10

There are many things going on in the world, many evil things, the most important is the manipulation of our the air we breathe and the manipulation of the weather. Stay focused.

The GeoFight

58,572 görüntüleme • 7 ay önce

Simple yet extremely effective. Non contact mirror dodge drill. One of the biggest training focuses in this drill for us is their eyes. We give our lineman a specific visual target ( position 1,2, or 3) on the defender. His reactionary movement is based on that target moving.
0:10

Simple yet extremely effective. Non contact mirror dodge drill. One of the biggest training focuses in this drill for us is their eyes. We give our lineman a specific visual target ( position 1,2, or 3) on the defender. His reactionary movement is based on that target moving.

Angus Reid

10,556 görüntüleme • 3 ay önce

We always open the windows for the evening birdsong and sunset at our farm here in New Hampshire.
0:14

We always open the windows for the evening birdsong and sunset at our farm here in New Hampshire.

Shaun Terhune

19,498 görüntüleme • 4 ay önce

We’ve seen humanoid robots walk around for a while, but when will they actually help with useful tasks in daily life? The challenge here is the diversity and complexity of real-world scenes. Our new work tackles this problem via 3D visuomotor policy learning. Using data from only 1 scene, our Improved 3D Diffusion Policy (iDP3) enables a full-sized humanoid robot to autonomously pick&place objects, pour water, and wipe tables, in the wild open world. (and all these skills are useful, right?) Web: Fully open-sourced code:
0:18

We’ve seen humanoid robots walk around for a while, but when will they actually help with useful tasks in daily life? The challenge here is the diversity and complexity of real-world scenes. Our new work tackles this problem via 3D visuomotor policy learning. Using data from only 1 scene, our Improved 3D Diffusion Policy (iDP3) enables a full-sized humanoid robot to autonomously pick&place objects, pour water, and wipe tables, in the wild open world. (and all these skills are useful, right?) Web: Fully open-sourced code:

Yanjie Ze

75,420 görüntüleme • 2 yıl önce

Learning bimanual, contact-rich robot manipulation policies that generalize over diverse objects has long been a challenge. Excited to share our work: Planning-Guided Diffusion Policy Learning for Generalizable Contact-Rich Bimanual Manipulation! 🧵1/n
0:15

Learning bimanual, contact-rich robot manipulation policies that generalize over diverse objects has long been a challenge. Excited to share our work: Planning-Guided Diffusion Policy Learning for Generalizable Contact-Rich Bimanual Manipulation! 🧵1/n

Xuanlin Li (Simon)

20,970 görüntüleme • 1 yıl önce

NVIDIA announces the first open humanoid robot reference design built for robotics research. The NVIDIA Isaac GR00T Reference Humanoid Robot combines the Unitree H2 humanoid robot, Sharpa Wave five-fingered hands for dexterous manipulation, Jetson Thor onboard compute, and Isaac GR00T open software and models, giving researchers a full-stack platform from data capture to model deployment. Read the #NVIDIAGTC Taipei announcement:
0:20

NVIDIA announces the first open humanoid robot reference design built for robotics research. The NVIDIA Isaac GR00T Reference Humanoid Robot combines the Unitree H2 humanoid robot, Sharpa Wave five-fingered hands for dexterous manipulation, Jetson Thor onboard compute, and Isaac GR00T open software and models, giving researchers a full-stack platform from data capture to model deployment. Read the #NVIDIAGTC Taipei announcement:

NVIDIA Robotics

174,660 görüntüleme • 4 ay önce

World Models are the path for some AI Models in the future. But how can we efficiently train these models to not only see the world the way humans do but to see the world in a new and unique way. By visualizing, what is normally sequenced audio patterns, we can derive much more insights. Here we see Paganini in a visual form that can than be described and transcribed into a World Model. We can observe connections in a manner that may not have been clear prior to the digitalization of music and sound in this way. The company with the most valuable potential in building a World Model is Tesla. Not that this type of visualization is being used, but that the mechanisms are in place, and the technology is in place for the company to thrive in this new form of AI.
0:19

World Models are the path for some AI Models in the future. But how can we efficiently train these models to not only see the world the way humans do but to see the world in a new and unique way. By visualizing, what is normally sequenced audio patterns, we can derive much more insights. Here we see Paganini in a visual form that can than be described and transcribed into a World Model. We can observe connections in a manner that may not have been clear prior to the digitalization of music and sound in this way. The company with the most valuable potential in building a World Model is Tesla. Not that this type of visualization is being used, but that the mechanisms are in place, and the technology is in place for the company to thrive in this new form of AI.

Brian Roemmele

57,454 görüntüleme • 10 ay önce