Video yükleniyor...

Video Yüklenemedi

Ana Sayfaya Dön

One core bottleneck in VLA models is action representation. Discrete tokens scale beautifully with VLM pretraining—but lose precision. Continuous actions are precise—but often break VLM reasoning. In our new work, we resolve this tension at the representation level. 🧵

jianlanluo's profile picture

Jianlan Luo

1,950 subscribers

47,993 görüntüleme • 6 ay önce •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

0 Yorum

Yorum bulunmuyor

Orijinal gönderinin yorumları burada görünecek

Benzer Videolar

Gemini Robotics 1.5 features a separate reasoning engine (ER), but its VLA model is also capable of thinking due to interleaved reasoning tokens. The VLA is able to independently operate long autonomous sequences (15+ minutes) without aid from the ER/VLM.
1:58

Gemini Robotics 1.5 features a separate reasoning engine (ER), but its VLA model is also capable of thinking due to interleaved reasoning tokens. The VLA is able to independently operate long autonomous sequences (15+ minutes) without aid from the ER/VLM.

The Humanoid Hub

34,214 görüntüleme • 7 ay önce

Excited to introduce 𝐋𝐀𝐏𝐀: the first unsupervised pretraining method for Vision-Language-Action models. Outperforms SOTA models trained with ground-truth actions 30x more efficient than conventional VLA pretraining 📝: 🧵 1/9
0:58

Excited to introduce 𝐋𝐀𝐏𝐀: the first unsupervised pretraining method for Vision-Language-Action models. Outperforms SOTA models trained with ground-truth actions 30x more efficient than conventional VLA pretraining 📝: 🧵 1/9

Joel Jang

46,018 görüntüleme • 1 yıl önce

Flow reversal steering allows "steering" diffusion-based VLAs with high-level actions, for example from VLM reasoning. This also lets us run RL in the diffusion noise space with exploration guided by high-level reasoning: think through a task, then practice it! 👇
0:55

Flow reversal steering allows "steering" diffusion-based VLAs with high-level actions, for example from VLM reasoning. This also lets us run RL in the diffusion noise space with exploration guided by high-level reasoning: think through a task, then practice it! 👇

Sergey Levine

74,382 görüntüleme • 29 gün önce

What structural task representation enables multi-stage, in-the-wild, bimanual, reactive manipulation? Introducing ReKep: LVM to label keypoints & VLM to write keypoint-based constraints, solve w/ optimization for diverse tasks, w/o task-specific training or env models. 🧵👇
2:10

What structural task representation enables multi-stage, in-the-wild, bimanual, reactive manipulation? Introducing ReKep: LVM to label keypoints & VLM to write keypoint-based constraints, solve w/ optimization for diverse tasks, w/o task-specific training or env models. 🧵👇

Wenlong Huang

190,914 görüntüleme • 1 yıl önce

✨New preprint: Dual-Process Image Generation! We distill *feedback from a VLM* into *feed-forward image generation*, at inference time. The result is flexible control: parameterize tasks as multimodal inputs, visually inspect the images with the VLM, and update the generator.🧵
0:23

✨New preprint: Dual-Process Image Generation! We distill *feedback from a VLM* into *feed-forward image generation*, at inference time. The result is flexible control: parameterize tasks as multimodal inputs, visually inspect the images with the VLM, and update the generator.🧵

Grace Luo @ ICML 2026

133,395 görüntüleme • 1 yıl önce

Combinatorial complexity is often the bane of imitation learning - including VLA models! Jesse Zhang and Marius Memmel proposed a way around this, using VLMs to perform problem reduction for imitation. The insight is simple - 1) High-level VLM takes a complex scene/task and reducing it a minimal representation (via masking and path prediction) that is needed to act in the world. 2) A low-level policy then takes this reduced representation and generates actions to be executed in the world. The high-level policy absorbs all the combinatorial complexity of the problem, leaving the low-level to focus on dexterity and geometric reasoning. Super simple, works really well across policy classes and problem settings! - 41.4× sim2real improvement (3DDA) and 2–3.5× boosts for π₀ and ACT in the real world. Paper: Website: Demo: Fun collaboration led by Jesse Zhang Marius Memmel with lots of collaborators! Let us know what you think 😀
0:50

Combinatorial complexity is often the bane of imitation learning - including VLA models! Jesse Zhang and Marius Memmel proposed a way around this, using VLMs to perform problem reduction for imitation. The insight is simple - 1) High-level VLM takes a complex scene/task and reducing it a minimal representation (via masking and path prediction) that is needed to act in the world. 2) A low-level policy then takes this reduced representation and generates actions to be executed in the world. The high-level policy absorbs all the combinatorial complexity of the problem, leaving the low-level to focus on dexterity and geometric reasoning. Super simple, works really well across policy classes and problem settings! - 41.4× sim2real improvement (3DDA) and 2–3.5× boosts for π₀ and ACT in the real world. Paper: Website: Demo: Fun collaboration led by Jesse Zhang Marius Memmel with lots of collaborators! Let us know what you think 😀

Abhishek Gupta

22,354 görüntüleme • 9 ay önce

Current robot policies often face a tradeoff: they're either precise (but brittle) or generalizable (but imprecise). We present ViTaL, a framework that lets robots generalize precise, contact-rich manipulation skills across unseen environments with millimeter-level precision. 🧵
0:35

Current robot policies often face a tradeoff: they're either precise (but brittle) or generalizable (but imprecise). We present ViTaL, a framework that lets robots generalize precise, contact-rich manipulation skills across unseen environments with millimeter-level precision. 🧵

Siddhant Haldar

145,976 görüntüleme • 1 yıl önce

Chain-of-thought reasoning is a powerful tool to enable language models to work through complex problems. Can we use this with robots? With embodied chain-of-thought, vision-language-action (VLA) models can think through perception and planning! A 🧵👇
1:39

Chain-of-thought reasoning is a powerful tool to enable language models to work through complex problems. Can we use this with robots? With embodied chain-of-thought, vision-language-action (VLA) models can think through perception and planning! A 🧵👇

Sergey Levine

30,388 görüntüleme • 2 yıl önce

Last night’s performance made New Jersey proud. The representation we saw on the field is the same representation we see reflected in our state. We are stronger when we come together and celebrate that.
0:34

Last night’s performance made New Jersey proud. The representation we saw on the field is the same representation we see reflected in our state. We are stronger when we come together and celebrate that.

Senator Andy Kim

32,749 görüntüleme • 5 ay önce

What happens when VLMs meet 3D foundation models? See VLM-3R (CVPR 2026). VLM-3R links a vision-language model (e.g., Qwen) with 3D geometric foundation models (e.g., CUT3R) at metric scale. Given an uncalibrated video, it moves beyond pixels to perceive and reason in 3D space. Code (open source):
0:20

What happens when VLMs meet 3D foundation models? See VLM-3R (CVPR 2026). VLM-3R links a vision-language model (e.g., Qwen) with 3D geometric foundation models (e.g., CUT3R) at metric scale. Given an uncalibrated video, it moves beyond pixels to perceive and reason in 3D space. Code (open source):

Zhiwen(Aaron) Fan

10,641 görüntüleme • 4 ay önce

Turns out that vision-language models can control robots too. The secret is to just finetune them to print out the actions (literally, as text). Really excited about our new result, the successor to RT-1. RT-2 is a pre-trained VLM: Short 🧵👇
1:52

Turns out that vision-language models can control robots too. The secret is to just finetune them to print out the actions (literally, as text). Really excited about our new result, the successor to RT-1. RT-2 is a pre-trained VLM: Short 🧵👇

Sergey Levine

165,052 görüntüleme • 2 yıl önce

PhysX-Anything: creates simulation-ready, articulated 3D assets from a single image; VLM-based model using a new 3D representation that reduces token count by 193x
1:01

PhysX-Anything: creates simulation-ready, articulated 3D assets from a single image; VLM-based model using a new 3D representation that reduces token count by 193x

Wildminder

49,884 görüntüleme • 7 ay önce

Why don’t VLAs generalize as well as their VLM counterparts? One culprit: catastrophic forgetting during fine-tuning. 🧠 We introduce VLM2VLA: a training paradigm that preserves the VLM capabilities while teaching robotic control. 🧵
0:38

Why don’t VLAs generalize as well as their VLM counterparts? One culprit: catastrophic forgetting during fine-tuning. 🧠 We introduce VLM2VLA: a training paradigm that preserves the VLM capabilities while teaching robotic control. 🧵

Anirudha Majumdar

60,529 görüntüleme • 9 ay önce

We are thrilled to welcome six newly elected Seanadóirí to our team! With this, our Sinn Féin representation in the Oireachtas now stands at 45—our largest team ever. We’re also proud to have strong female representation, with 19 women among our members!💚👏🏼 #sinnféin
0:28

We are thrilled to welcome six newly elected Seanadóirí to our team! With this, our Sinn Féin representation in the Oireachtas now stands at 45—our largest team ever. We’re also proud to have strong female representation, with 19 women among our members!💚👏🏼 #sinnféin

Mary Lou McDonald

29,886 görüntüleme • 1 yıl önce

New Hampshire is totally Red at the state level but ⁦New Hampshire Republican Party⁩ has zero representation in DC. The recent poll shows that I am tied to ⁦Congresswoman Maggie Goodlander⁩ in NH02. Please chip in to help me finish Q4 with strength.
0:42

New Hampshire is totally Red at the state level but ⁦New Hampshire Republican Party⁩ has zero representation in DC. The recent poll shows that I am tied to ⁦Congresswoman Maggie Goodlander⁩ in NH02. Please chip in to help me finish Q4 with strength.

Lily Tang Williams

35,235 görüntüleme • 6 ay önce

Google DeepMind introduces Gemini Robotics 1.5, enabling robots to perceive, plan, think, use tools, and act on complex tasks. The agentic framework comprises: ⦿ Gemini Robotics-ER 1.5 (VLM): Orchestrates high-level embodied reasoning and planning. ⦿ Gemini Robotics 1.5 (VLA): Converts visuals and instructions provided by ER 1.5 into actions.
1:42

Google DeepMind introduces Gemini Robotics 1.5, enabling robots to perceive, plan, think, use tools, and act on complex tasks. The agentic framework comprises: ⦿ Gemini Robotics-ER 1.5 (VLM): Orchestrates high-level embodied reasoning and planning. ⦿ Gemini Robotics 1.5 (VLA): Converts visuals and instructions provided by ER 1.5 into actions.

The Humanoid Hub

65,928 görüntüleme • 9 ay önce

That’s our lone representation in the TN assembly for now but look at the quality !!! That’s BJP for you .
5:06

That’s our lone representation in the TN assembly for now but look at the quality !!! That’s BJP for you .

karthik gopinath

17,778 görüntüleme • 17 gün önce

Nigel Farage, "We need an electoral system where we get some proportional representation in parliament.. A new type of politics, we might get some real change" It doesn't happen very often that I agree with NF, but I do support the call for proportional representation - we might get more Greens and LibDems into parliament
0:33

Nigel Farage, "We need an electoral system where we get some proportional representation in parliament.. A new type of politics, we might get some real change" It doesn't happen very often that I agree with NF, but I do support the call for proportional representation - we might get more Greens and LibDems into parliament

Farrukh

118,577 görüntüleme • 2 yıl önce

America is under siege, Christianity is under siege, and we are at war with satanic cult worshiping pedophiles who have taken over our government. We the people have zero representation in our government.
3:01

America is under siege, Christianity is under siege, and we are at war with satanic cult worshiping pedophiles who have taken over our government. We the people have zero representation in our government.

RyanMatta 🇺🇸 🦅

47,190 görüntüleme • 4 ay önce