Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

How can we help any image-input policy generalize better? 👉 Meet PEEK 🤖 — a framework that uses VLMs to decide where to look and what to do, so downstream policies — from ACT, 3D-DA, or even π₀ — generalize more effectively! 🧵

Jesse Zhang

1,677 subscribers

13,700 views • 8 months ago •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

Current robot policies often face a tradeoff: they're either precise (but brittle) or generalizable (but imprecise). We present ViTaL, a framework that lets robots generalize precise, contact-rich manipulation skills across unseen environments with millimeter-level precision. 🧵

Current robot policies often face a tradeoff: they're either precise (but brittle) or generalizable (but imprecise). We present ViTaL, a framework that lets robots generalize precise, contact-rich manipulation skills across unseen environments with millimeter-level precision. 🧵

Siddhant Haldar

145,963 views • 11 months ago

UniHair can create 3D hair models from single-view portraits, handling both braided and un-braided styles! It uses a large dataset and advanced techniques to accurately capture complex hairstyles and generalize well to real images. Links ⬇️

UniHair can create 3D hair models from single-view portraits, handling both braided and un-braided styles! It uses a large dataset and advanced techniques to accurately capture complex hairstyles and generalize well to real images. Links ⬇️

Dreaming Tulpa 🥓👑

25,200 views • 1 year ago

🤖 How can robot policies zero-shot generalize to any new environment and any new object? Introducing our new project: 🚀Data Scaling Laws in Imitation Learning for Robotic Manipulation🚀—bringing us closer to the dream of having robots work as waiters in hot pot restaurants! 🍲

🤖 How can robot policies zero-shot generalize to any new environment and any new object? Introducing our new project: 🚀Data Scaling Laws in Imitation Learning for Robotic Manipulation🚀—bringing us closer to the dream of having robots work as waiters in hot pot restaurants! 🍲

Yang Gao

124,995 views • 1 year ago

Say goodbye to green screens! DiffusionMat is an AI framework that uses a diffusion model to create refined alpha mattes by correcting the output at every denoising step to look more like the original input image.

Say goodbye to green screens! DiffusionMat is an AI framework that uses a diffusion model to create refined alpha mattes by correcting the output at every denoising step to look more like the original input image.

Dreaming Tulpa 🥓👑

146,568 views • 2 years ago

New research from BlackForestLabsAI - Unofficial 🥳 Meet Self-Flow: our self-supervised framework for image, audio, video & world models 🤖 Do generative models really need DINO to learn strong representations? We propose teaching them directly via a joint framework instead 🧵

New research from BlackForestLabsAI - Unofficial 🥳 Meet Self-Flow: our self-supervised framework for image, audio, video & world models 🤖 Do generative models really need DINO to learn strong representations? We propose teaching them directly via a joint framework instead 🧵

Hila Chefer

67,589 views • 3 months ago

She broke DNA down. I keep telling you that Euro Americans created this narrative. Grok uses a study from 2015 to generalize African.

She broke DNA down. I keep telling you that Euro Americans created this narrative. Grok uses a study from 2015 to generalize African.

Arthur Watkins Jr. 🇺🇸

20,772 views • 7 months ago

Joe Mazzulla on how to bounce back from this loss: “Take a look at it. Figure out where you have to better, and put a full 48 minutes together. Have an understanding that it’s going to be difficult. That’s why we do what we do. Figure out where we have to get better and do that”

Joe Mazzulla on how to bounce back from this loss: “Take a look at it. Figure out where you have to better, and put a full 48 minutes together. Have an understanding that it’s going to be difficult. That’s why we do what we do. Figure out where we have to get better and do that”

Justin Turpin

24,265 views • 1 year ago

🚨 Without Any Motion Priors, how to make humanoids do versatile parkour jumping🦘, clapping dance🤸, cliff traversal🧗, and box pick-and-move📦 with a unified RL framework? Introduce WoCoCo: 🧗 Whole-body humanoid Control with sequential Contacts 🎯Unified designs for minimal tuning across tasks 🤖Generalize to various high-DoF robots Website:

🚨 Without Any Motion Priors, how to make humanoids do versatile parkour jumping🦘, clapping dance🤸, cliff traversal🧗, and box pick-and-move📦 with a unified RL framework? Introduce WoCoCo: 🧗 Whole-body humanoid Control with sequential Contacts 🎯Unified designs for minimal tuning across tasks 🤖Generalize to various high-DoF robots Website:

Wenli Xiao

70,321 views • 2 years ago

📢 A Recipe for Generating 3D Worlds From a Single Image 📢 Our recipe explains how existing generative models can be adapted with minimal training effort to generate 3D worlds from a single input image.

📢 A Recipe for Generating 3D Worlds From a Single Image 📢 Our recipe explains how existing generative models can be adapted with minimal training effort to generate 3D worlds from a single input image.

Katja Schwarz

13,970 views • 1 year ago

New work with Alec Radford and David Duvenaud: Have you ever dreamed of talking to someone from the past? Introducing talkie, a 13B model trained only on pre-1931 text. Vintage models should help us to understand how LMs generalize (e.g., can we teach talkie to code?). Thread:

New work with Alec Radford and David Duvenaud: Have you ever dreamed of talking to someone from the past? Introducing talkie, a 13B model trained only on pre-1931 text. Vintage models should help us to understand how LMs generalize (e.g., can we teach talkie to code?). Thread:

Nick Levine

1,191,200 views • 1 month ago

Are black-box VLAs really the endgame for generalist robots? With SAGA, we explore an alternative path to efficiently generalize across environments, behaviors, and user specifications by explicitly grounding visual affordances in 3D space. 🧵

Are black-box VLAs really the endgame for generalist robots? With SAGA, we explore an alternative path to efficiently generalize across environments, behaviors, and user specifications by explicitly grounding visual affordances in 3D space. 🧵

Kuan Fang

197,834 views • 6 months ago

S/O dark Kenny niggas !! We don’t generalize ! And no nigga who too much a coward to show his face can tell me shit !

S/O dark Kenny niggas !! We don’t generalize ! And no nigga who too much a coward to show his face can tell me shit !

Jalopy Bungus

23,570 views • 1 year ago

Robotics fundamentally involves understanding the dynamics of how things change in the world in response to action and force. This is impossible to learn from static images; instead, it’s far more effective and more data-efficient to learn from video. Elvis Nava joins us to talk about Mimic Robotic. One of the key findings from mimic-video is that pretraining on webscale video allows robots to learn physics priors; as a result, policies train faster, generalize better, and are capable of more impressive dexterity, versus training on static images or image-language pairs as per a VLM. Watch Episode #81 of RoboPapers with Michael Cho - Rbt/Acc and Chris Paxton to learn more!

Robotics fundamentally involves understanding the dynamics of how things change in the world in response to action and force. This is impossible to learn from static images; instead, it’s far more effective and more data-efficient to learn from video. Elvis Nava joins us to talk about Mimic Robotic. One of the key findings from mimic-video is that pretraining on webscale video allows robots to learn physics priors; as a result, policies train faster, generalize better, and are capable of more impressive dexterity, versus training on static images or image-language pairs as per a VLM. Watch Episode #81 of RoboPapers with Michael Cho - Rbt/Acc and Chris Paxton to learn more!

RoboPapers

46,190 views • 1 month ago

Hindi lahat ng Carmelle supporters toxic. We’ve been RESPECTFUL, SUPPORTIVE, and KIND, not just to Carmelle, but even to Krystal. So don’t generalize us just to justify a questionable move. You KNOW the impact of votes. And yet you chose to do that and had the audacity+

Hindi lahat ng Carmelle supporters toxic. We’ve been RESPECTFUL, SUPPORTIVE, and KIND, not just to Carmelle, but even to Krystal. So don’t generalize us just to justify a questionable move. You KNOW the impact of votes. And yet you chose to do that and had the audacity+

everdies I DM LIMIT

557,632 views • 5 months ago

Children learn to manipulate the world by playing with toys — can robots do the same? 🧸🤖 We show that robots trained on 250 "toys" made of 4 shape primitives (🔵,🔶,🧱,💍) can generalize grasping to real objects. Jitendra MALIK trevordarrell Shankar Sastry Berkeley AI Research😊

Children learn to manipulate the world by playing with toys — can robots do the same? 🧸🤖 We show that robots trained on 250 "toys" made of 4 shape primitives (🔵,🔶,🧱,💍) can generalize grasping to real objects. Jitendra MALIK trevordarrell Shankar Sastry Berkeley AI Research😊

Roei Herzig

27,455 views • 8 months ago

I built software to cut docker image sizes by more than 80%. It uses dynamic analysis and and has a LOT more room for improvement. The demo here shows an ETL workflow docker image, cut down from 1.28GB to 193MB (-85%)! If large docker image sizes are becoming a production problem, I'd be happy to explore how we can help.

I built software to cut docker image sizes by more than 80%. It uses dynamic analysis and and has a LOT more room for improvement. The demo here shows an ETL workflow docker image, cut down from 1.28GB to 193MB (-85%)! If large docker image sizes are becoming a production problem, I'd be happy to explore how we can help.

omkaar

353,153 views • 1 year ago

How can we get VLMs to move their eyes—and reason step-by-step in visually grounded ways? 👀 We introduce ViGoRL, a RL method that anchors reasoning to image regions. 🎯 It outperforms vanilla GRPO and SFT across grounding, spatial tasks, and visual search (86.4% on V*). 👇🧵

How can we get VLMs to move their eyes—and reason step-by-step in visually grounded ways? 👀 We introduce ViGoRL, a RL method that anchors reasoning to image regions. 🎯 It outperforms vanilla GRPO and SFT across grounding, spatial tasks, and visual search (86.4% on V*). 👇🧵

Gabriel Sarch

76,548 views • 1 year ago

If you let VLMs experiment on their own, they can do surprising things! From an image, we let a VLM code a 3D scene from scratch in Blender, and then render to verify/refine in a loop. Even when each step is imperfect, it gets results like this, with zero training.

If you let VLMs experiment on their own, they can do surprising things! From an image, we let a VLM code a 3D scene from scratch in Blender, and then render to verify/refine in a loop. Even when each step is imperfect, it gets results like this, with zero training.

Angjoo Kanazawa

51,845 views • 4 months ago

Can a single neural network policy generalize over poses, objects, obstacles, backgrounds, scene arrangements, in-hand objects, and start/goal states? Introducing Neural MP: A generalist policy for solving motion planning tasks in the real world 🤖 1/N

Can a single neural network policy generalize over poses, objects, obstacles, backgrounds, scene arrangements, in-hand objects, and start/goal states? Introducing Neural MP: A generalist policy for solving motion planning tasks in the real world 🤖 1/N

Murtaza Dalal

114,538 views • 1 year ago