Загрузка видео...

Не удалось загрузить видео

На главную

How can we help *any* image-input policy generalize better? 👉 Meet PEEK 🤖 — a framework that uses VLMs to decide *where* to look and *what* to do, so downstream policies — from ACT, 3D-DA, or even π₀ — generalize more effectively! 🧵

Jesse_Y_Zhang's profile picture

Jesse Zhang

1,677 subscribers

13,700 просмотров • 8 месяцев назад •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

Комментарии: 0

Нет доступных комментариев

Здесь появятся комментарии из оригинального поста

Похожие видео

Current robot policies often face a tradeoff: they're either precise (but brittle) or generalizable (but imprecise). We present ViTaL, a framework that lets robots generalize precise, contact-rich manipulation skills across unseen environments with millimeter-level precision. 🧵
0:35

Current robot policies often face a tradeoff: they're either precise (but brittle) or generalizable (but imprecise). We present ViTaL, a framework that lets robots generalize precise, contact-rich manipulation skills across unseen environments with millimeter-level precision. 🧵

Siddhant Haldar

145,945 просмотров • 11 месяцев назад

UniHair can create 3D hair models from single-view portraits, handling both braided and un-braided styles! It uses a large dataset and advanced techniques to accurately capture complex hairstyles and generalize well to real images. Links ⬇️
1:04

UniHair can create 3D hair models from single-view portraits, handling both braided and un-braided styles! It uses a large dataset and advanced techniques to accurately capture complex hairstyles and generalize well to real images. Links ⬇️

Dreaming Tulpa 🥓👑

25,200 просмотров • 1 год назад

🤖 How can robot policies zero-shot generalize to any new environment and any new object? Introducing our new project: 🚀Data Scaling Laws in Imitation Learning for Robotic Manipulation🚀—bringing us closer to the dream of having robots work as waiters in hot pot restaurants! 🍲
0:29

🤖 How can robot policies zero-shot generalize to any new environment and any new object? Introducing our new project: 🚀Data Scaling Laws in Imitation Learning for Robotic Manipulation🚀—bringing us closer to the dream of having robots work as waiters in hot pot restaurants! 🍲

Yang Gao

124,995 просмотров • 1 год назад

Say goodbye to green screens! DiffusionMat is an AI framework that uses a diffusion model to create refined alpha mattes by correcting the output at every denoising step to look more like the original input image.
0:31

Say goodbye to green screens! DiffusionMat is an AI framework that uses a diffusion model to create refined alpha mattes by correcting the output at every denoising step to look more like the original input image.

Dreaming Tulpa 🥓👑

146,568 просмотров • 2 лет назад

New research from BlackForestLabsAI - Unofficial 🥳 Meet Self-Flow: our self-supervised framework for image, audio, video & world models 🤖 Do generative models really need DINO to learn strong representations? We propose teaching them directly via a joint framework instead 🧵
2:39

New research from BlackForestLabsAI - Unofficial 🥳 Meet Self-Flow: our self-supervised framework for image, audio, video & world models 🤖 Do generative models really need DINO to learn strong representations? We propose teaching them directly via a joint framework instead 🧵

Hila Chefer

67,523 просмотров • 3 месяцев назад

She broke DNA down. I keep telling you that Euro Americans created this narrative. Grok uses a study from 2015 to generalize African.
6:26

She broke DNA down. I keep telling you that Euro Americans created this narrative. Grok uses a study from 2015 to generalize African.

Arthur Watkins Jr. 🇺🇸

20,772 просмотров • 7 месяцев назад

Joe Mazzulla on how to bounce back from this loss: “Take a look at it. Figure out where you have to better, and put a full 48 minutes together. Have an understanding that it’s going to be difficult. That’s why we do what we do. Figure out where we have to get better and do that”
0:21

Joe Mazzulla on how to bounce back from this loss: “Take a look at it. Figure out where you have to better, and put a full 48 minutes together. Have an understanding that it’s going to be difficult. That’s why we do what we do. Figure out where we have to get better and do that”

Justin Turpin

24,265 просмотров • 1 год назад

🚨 Without Any Motion Priors, how to make humanoids do versatile parkour jumping🦘, clapping dance🤸, cliff traversal🧗, and box pick-and-move📦 with a unified RL framework? Introduce WoCoCo: 🧗 Whole-body humanoid Control with sequential Contacts 🎯Unified designs for minimal tuning across tasks 🤖Generalize to various high-DoF robots Website:
0:54

🚨 Without Any Motion Priors, how to make humanoids do versatile parkour jumping🦘, clapping dance🤸, cliff traversal🧗, and box pick-and-move📦 with a unified RL framework? Introduce WoCoCo: 🧗 Whole-body humanoid Control with sequential Contacts 🎯Unified designs for minimal tuning across tasks 🤖Generalize to various high-DoF robots Website:

Wenli Xiao

70,321 просмотров • 2 лет назад

📢 A Recipe for Generating 3D Worlds From a Single Image 📢 Our recipe explains how existing generative models can be adapted with minimal training effort to generate 3D worlds from a single input image.
0:28

📢 A Recipe for Generating 3D Worlds From a Single Image 📢 Our recipe explains how existing generative models can be adapted with minimal training effort to generate 3D worlds from a single input image.

Katja Schwarz

13,970 просмотров • 1 год назад

New work with Alec Radford and David Duvenaud: Have you ever dreamed of talking to someone from the past? Introducing talkie, a 13B model trained only on pre-1931 text. Vintage models should help us to understand how LMs generalize (e.g., can we teach talkie to code?). Thread:
7:58

New work with Alec Radford and David Duvenaud: Have you ever dreamed of talking to someone from the past? Introducing talkie, a 13B model trained only on pre-1931 text. Vintage models should help us to understand how LMs generalize (e.g., can we teach talkie to code?). Thread:

Nick Levine

1,189,007 просмотров • 1 месяц назад

Are black-box VLAs really the endgame for generalist robots? With SAGA, we explore an alternative path to efficiently generalize across environments, behaviors, and user specifications by explicitly grounding visual affordances in 3D space. 🧵
0:36

Are black-box VLAs really the endgame for generalist robots? With SAGA, we explore an alternative path to efficiently generalize across environments, behaviors, and user specifications by explicitly grounding visual affordances in 3D space. 🧵

Kuan Fang

197,834 просмотров • 6 месяцев назад

S/O dark Kenny niggas !! We don’t generalize ! And no nigga who too much a coward to show his face can tell me shit !
0:32

S/O dark Kenny niggas !! We don’t generalize ! And no nigga who too much a coward to show his face can tell me shit !

Jalopy Bungus

23,570 просмотров • 1 год назад

Robotics fundamentally involves understanding the dynamics of how things change in the world in response to action and force. This is impossible to learn from static images; instead, it’s far more effective and more data-efficient to learn from video. Elvis Nava joins us to talk about Mimic Robotic. One of the key findings from mimic-video is that pretraining on webscale video allows robots to learn physics priors; as a result, policies train faster, generalize better, and are capable of more impressive dexterity, versus training on static images or image-language pairs as per a VLM. Watch Episode #81 of RoboPapers with Michael Cho - Rbt/Acc and Chris Paxton to learn more!
1:08:43

Robotics fundamentally involves understanding the dynamics of how things change in the world in response to action and force. This is impossible to learn from static images; instead, it’s far more effective and more data-efficient to learn from video. Elvis Nava joins us to talk about Mimic Robotic. One of the key findings from mimic-video is that pretraining on webscale video allows robots to learn physics priors; as a result, policies train faster, generalize better, and are capable of more impressive dexterity, versus training on static images or image-language pairs as per a VLM. Watch Episode #81 of RoboPapers with Michael Cho - Rbt/Acc and Chris Paxton to learn more!

RoboPapers

46,190 просмотров • 1 месяц назад

Hindi lahat ng Carmelle supporters toxic. We’ve been RESPECTFUL, SUPPORTIVE, and KIND, not just to Carmelle, but even to Krystal. So don’t generalize us just to justify a questionable move. You KNOW the impact of votes. And yet you chose to do that and had the audacity+
1:23

Hindi lahat ng Carmelle supporters toxic. We’ve been RESPECTFUL, SUPPORTIVE, and KIND, not just to Carmelle, but even to Krystal. So don’t generalize us just to justify a questionable move. You KNOW the impact of votes. And yet you chose to do that and had the audacity+

everdies I DM LIMIT

557,632 просмотров • 5 месяцев назад

Children learn to manipulate the world by playing with toys — can robots do the same? 🧸🤖 We show that robots trained on 250 "toys" made of 4 shape primitives (🔵,🔶,🧱,💍) can generalize grasping to real objects. Jitendra MALIK trevordarrell Shankar Sastry Berkeley AI Research😊
0:42

Children learn to manipulate the world by playing with toys — can robots do the same? 🧸🤖 We show that robots trained on 250 "toys" made of 4 shape primitives (🔵,🔶,🧱,💍) can generalize grasping to real objects. Jitendra MALIK trevordarrell Shankar Sastry Berkeley AI Research😊

Roei Herzig

27,455 просмотров • 8 месяцев назад

I built software to cut docker image sizes by more than 80%. It uses dynamic analysis and and has a LOT more room for improvement. The demo here shows an ETL workflow docker image, cut down from 1.28GB to 193MB (-85%)! If large docker image sizes are becoming a production problem, I'd be happy to explore how we can help.
1:21

I built software to cut docker image sizes by more than 80%. It uses dynamic analysis and and has a LOT more room for improvement. The demo here shows an ETL workflow docker image, cut down from 1.28GB to 193MB (-85%)! If large docker image sizes are becoming a production problem, I'd be happy to explore how we can help.

omkaar

353,153 просмотров • 1 год назад

How can we get VLMs to move their eyes—and reason step-by-step in visually grounded ways? 👀 We introduce ViGoRL, a RL method that anchors reasoning to image regions. 🎯 It outperforms vanilla GRPO and SFT across grounding, spatial tasks, and visual search (86.4% on V*). 👇🧵
0:39

How can we get VLMs to move their eyes—and reason step-by-step in visually grounded ways? 👀 We introduce ViGoRL, a RL method that anchors reasoning to image regions. 🎯 It outperforms vanilla GRPO and SFT across grounding, spatial tasks, and visual search (86.4% on V*). 👇🧵

Gabriel Sarch

76,548 просмотров • 1 год назад

If you let VLMs experiment on their own, they can do surprising things! From an image, we let a VLM code a 3D scene from scratch in Blender, and then render to verify/refine in a loop. Even when each step is imperfect, it gets results like this, with zero training.
0:33

If you let VLMs experiment on their own, they can do surprising things! From an image, we let a VLM code a 3D scene from scratch in Blender, and then render to verify/refine in a loop. Even when each step is imperfect, it gets results like this, with zero training.

Angjoo Kanazawa

51,841 просмотров • 4 месяцев назад

Can a single neural network policy generalize over poses, objects, obstacles, backgrounds, scene arrangements, in-hand objects, and start/goal states? Introducing Neural MP: A generalist policy for solving motion planning tasks in the real world 🤖 1/N
1:15

Can a single neural network policy generalize over poses, objects, obstacles, backgrounds, scene arrangements, in-hand objects, and start/goal states? Introducing Neural MP: A generalist policy for solving motion planning tasks in the real world 🤖 1/N

Murtaza Dalal

114,538 просмотров • 1 год назад