正在加载视频...

视频加载失败

加载此视频时出现问题。这可能是由于临时网络问题，或视频可能不可用。

How can we help any image-input policy generalize better? 👉 Meet PEEK 🤖 — a framework that uses VLMs to decide where to look and what to do, so downstream policies — from ACT, 3D-DA, or even π₀ — generalize more effectively! 🧵

Jesse Zhang

1,604 subscribers

13,295 次观看 • 8 个月前 •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

0 条评论

暂无评论

原始帖子的评论将显示在这里

相关视频

Current robot policies often face a tradeoff: they're either precise (but brittle) or generalizable (but imprecise). We present ViTaL, a framework that lets robots generalize precise, contact-rich manipulation skills across unseen environments with millimeter-level precision. 🧵

Current robot policies often face a tradeoff: they're either precise (but brittle) or generalizable (but imprecise). We present ViTaL, a framework that lets robots generalize precise, contact-rich manipulation skills across unseen environments with millimeter-level precision. 🧵

Siddhant Haldar

145,945 次观看 • 11 个月前

UniHair can create 3D hair models from single-view portraits, handling both braided and un-braided styles! It uses a large dataset and advanced techniques to accurately capture complex hairstyles and generalize well to real images. Links ⬇️

UniHair can create 3D hair models from single-view portraits, handling both braided and un-braided styles! It uses a large dataset and advanced techniques to accurately capture complex hairstyles and generalize well to real images. Links ⬇️

Dreaming Tulpa 🥓👑

25,200 次观看 • 1 年前

🤖 How can robot policies zero-shot generalize to any new environment and any new object? Introducing our new project: 🚀Data Scaling Laws in Imitation Learning for Robotic Manipulation🚀—bringing us closer to the dream of having robots work as waiters in hot pot restaurants! 🍲

🤖 How can robot policies zero-shot generalize to any new environment and any new object? Introducing our new project: 🚀Data Scaling Laws in Imitation Learning for Robotic Manipulation🚀—bringing us closer to the dream of having robots work as waiters in hot pot restaurants! 🍲

Yang Gao

124,983 次观看 • 1 年前

Say goodbye to green screens! DiffusionMat is an AI framework that uses a diffusion model to create refined alpha mattes by correcting the output at every denoising step to look more like the original input image.

Say goodbye to green screens! DiffusionMat is an AI framework that uses a diffusion model to create refined alpha mattes by correcting the output at every denoising step to look more like the original input image.

Dreaming Tulpa 🥓👑

146,566 次观看 • 2 年前

New research from BlackForestLabsAI - Unofficial 🥳 Meet Self-Flow: our self-supervised framework for image, audio, video & world models 🤖 Do generative models really need DINO to learn strong representations? We propose teaching them directly via a joint framework instead 🧵

New research from BlackForestLabsAI - Unofficial 🥳 Meet Self-Flow: our self-supervised framework for image, audio, video & world models 🤖 Do generative models really need DINO to learn strong representations? We propose teaching them directly via a joint framework instead 🧵

Hila Chefer

67,358 次观看 • 3 个月前

She broke DNA down. I keep telling you that Euro Americans created this narrative. Grok uses a study from 2015 to generalize African.

She broke DNA down. I keep telling you that Euro Americans created this narrative. Grok uses a study from 2015 to generalize African.

Arthur Watkins Jr. 🇺🇸

20,772 次观看 • 7 个月前

Joe Mazzulla on how to bounce back from this loss: “Take a look at it. Figure out where you have to better, and put a full 48 minutes together. Have an understanding that it’s going to be difficult. That’s why we do what we do. Figure out where we have to get better and do that”

Joe Mazzulla on how to bounce back from this loss: “Take a look at it. Figure out where you have to better, and put a full 48 minutes together. Have an understanding that it’s going to be difficult. That’s why we do what we do. Figure out where we have to get better and do that”

Justin Turpin

24,265 次观看 • 1 年前

🚨 Without Any Motion Priors, how to make humanoids do versatile parkour jumping🦘, clapping dance🤸, cliff traversal🧗, and box pick-and-move📦 with a unified RL framework? Introduce WoCoCo: 🧗 Whole-body humanoid Control with sequential Contacts 🎯Unified designs for minimal tuning across tasks 🤖Generalize to various high-DoF robots Website:

🚨 Without Any Motion Priors, how to make humanoids do versatile parkour jumping🦘, clapping dance🤸, cliff traversal🧗, and box pick-and-move📦 with a unified RL framework? Introduce WoCoCo: 🧗 Whole-body humanoid Control with sequential Contacts 🎯Unified designs for minimal tuning across tasks 🤖Generalize to various high-DoF robots Website:

Wenli Xiao

70,296 次观看 • 2 年前

📢 A Recipe for Generating 3D Worlds From a Single Image 📢 Our recipe explains how existing generative models can be adapted with minimal training effort to generate 3D worlds from a single input image.

📢 A Recipe for Generating 3D Worlds From a Single Image 📢 Our recipe explains how existing generative models can be adapted with minimal training effort to generate 3D worlds from a single input image.

Katja Schwarz

13,970 次观看 • 1 年前

New work with Alec Radford and David Duvenaud: Have you ever dreamed of talking to someone from the past? Introducing talkie, a 13B model trained only on pre-1931 text. Vintage models should help us to understand how LMs generalize (e.g., can we teach talkie to code?). Thread:

New work with Alec Radford and David Duvenaud: Have you ever dreamed of talking to someone from the past? Introducing talkie, a 13B model trained only on pre-1931 text. Vintage models should help us to understand how LMs generalize (e.g., can we teach talkie to code?). Thread:

Nick Levine

1,175,842 次观看 • 1 个月前

Are black-box VLAs really the endgame for generalist robots? With SAGA, we explore an alternative path to efficiently generalize across environments, behaviors, and user specifications by explicitly grounding visual affordances in 3D space. 🧵

Are black-box VLAs really the endgame for generalist robots? With SAGA, we explore an alternative path to efficiently generalize across environments, behaviors, and user specifications by explicitly grounding visual affordances in 3D space. 🧵

Kuan Fang

197,753 次观看 • 5 个月前

S/O dark Kenny niggas !! We don’t generalize ! And no nigga who too much a coward to show his face can tell me shit !

S/O dark Kenny niggas !! We don’t generalize ! And no nigga who too much a coward to show his face can tell me shit !

Jalopy Bungus

23,570 次观看 • 1 年前

Robotics fundamentally involves understanding the dynamics of how things change in the world in response to action and force. This is impossible to learn from static images; instead, it’s far more effective and more data-efficient to learn from video. Elvis Nava joins us to talk about Mimic Robotic. One of the key findings from mimic-video is that pretraining on webscale video allows robots to learn physics priors; as a result, policies train faster, generalize better, and are capable of more impressive dexterity, versus training on static images or image-language pairs as per a VLM. Watch Episode #81 of RoboPapers with Michael Cho - Rbt/Acc and Chris Paxton to learn more!

Robotics fundamentally involves understanding the dynamics of how things change in the world in response to action and force. This is impossible to learn from static images; instead, it’s far more effective and more data-efficient to learn from video. Elvis Nava joins us to talk about Mimic Robotic. One of the key findings from mimic-video is that pretraining on webscale video allows robots to learn physics priors; as a result, policies train faster, generalize better, and are capable of more impressive dexterity, versus training on static images or image-language pairs as per a VLM. Watch Episode #81 of RoboPapers with Michael Cho - Rbt/Acc and Chris Paxton to learn more!

RoboPapers

46,020 次观看 • 21 天前

Hindi lahat ng Carmelle supporters toxic. We’ve been RESPECTFUL, SUPPORTIVE, and KIND, not just to Carmelle, but even to Krystal. So don’t generalize us just to justify a questionable move. You KNOW the impact of votes. And yet you chose to do that and had the audacity+

Hindi lahat ng Carmelle supporters toxic. We’ve been RESPECTFUL, SUPPORTIVE, and KIND, not just to Carmelle, but even to Krystal. So don’t generalize us just to justify a questionable move. You KNOW the impact of votes. And yet you chose to do that and had the audacity+

everdies I DM LIMIT

557,435 次观看 • 5 个月前

Children learn to manipulate the world by playing with toys — can robots do the same? 🧸🤖 We show that robots trained on 250 "toys" made of 4 shape primitives (🔵,🔶,🧱,💍) can generalize grasping to real objects. Jitendra MALIK trevordarrell Shankar Sastry Berkeley AI Research😊

Children learn to manipulate the world by playing with toys — can robots do the same? 🧸🤖 We show that robots trained on 250 "toys" made of 4 shape primitives (🔵,🔶,🧱,💍) can generalize grasping to real objects. Jitendra MALIK trevordarrell Shankar Sastry Berkeley AI Research😊

Roei Herzig

27,262 次观看 • 7 个月前

I built software to cut docker image sizes by more than 80%. It uses dynamic analysis and and has a LOT more room for improvement. The demo here shows an ETL workflow docker image, cut down from 1.28GB to 193MB (-85%)! If large docker image sizes are becoming a production problem, I'd be happy to explore how we can help.

I built software to cut docker image sizes by more than 80%. It uses dynamic analysis and and has a LOT more room for improvement. The demo here shows an ETL workflow docker image, cut down from 1.28GB to 193MB (-85%)! If large docker image sizes are becoming a production problem, I'd be happy to explore how we can help.

omkaar

353,139 次观看 • 1 年前

How can we get VLMs to move their eyes—and reason step-by-step in visually grounded ways? 👀 We introduce ViGoRL, a RL method that anchors reasoning to image regions. 🎯 It outperforms vanilla GRPO and SFT across grounding, spatial tasks, and visual search (86.4% on V*). 👇🧵

How can we get VLMs to move their eyes—and reason step-by-step in visually grounded ways? 👀 We introduce ViGoRL, a RL method that anchors reasoning to image regions. 🎯 It outperforms vanilla GRPO and SFT across grounding, spatial tasks, and visual search (86.4% on V*). 👇🧵

Gabriel Sarch

76,548 次观看 • 1 年前

If you let VLMs experiment on their own, they can do surprising things! From an image, we let a VLM code a 3D scene from scratch in Blender, and then render to verify/refine in a loop. Even when each step is imperfect, it gets results like this, with zero training.

If you let VLMs experiment on their own, they can do surprising things! From an image, we let a VLM code a 3D scene from scratch in Blender, and then render to verify/refine in a loop. Even when each step is imperfect, it gets results like this, with zero training.

Angjoo Kanazawa

51,769 次观看 • 4 个月前

Can a single neural network policy generalize over poses, objects, obstacles, backgrounds, scene arrangements, in-hand objects, and start/goal states? Introducing Neural MP: A generalist policy for solving motion planning tasks in the real world 🤖 1/N

Can a single neural network policy generalize over poses, objects, obstacles, backgrounds, scene arrangements, in-hand objects, and start/goal states? Introducing Neural MP: A generalist policy for solving motion planning tasks in the real world 🤖 1/N

Murtaza Dalal

114,538 次观看 • 1 年前