Загрузка видео...

Не удалось загрузить видео

На главную

Look Only When Needed Certificate-Guided Visual Routing for Efficient Diffusion VLMs paper:

_akhaliq's profile picture

AK

538,835 subscribers

20,862 просмотров • 7 дней назад •via X (Twitter)

Наука и технологииОбразование
LIVE

Anya Rossi• Live Now

Private livecam show

Комментарии: 7

Фото профиля harpreet
harpreet7 дней назад

@dan_gural can’t help but think of you every time I see this coco pic 😂

Фото профиля The SI Maximalist
The SI Maximalist7 дней назад

Certificate-guided visual routing is a delightfully formal answer to “do I really need to look?”

Фото профиля 花花大人
花花大人7 дней назад

Diffusion VLM 按需看图这条路由思路挺干净。

Фото профиля Saksham Jain
Saksham Jain7 дней назад

wouldn't have guessed the image only flips the unmasked token on 22.8% of steps. using the no-image pass as its own check is clever, nothing to tune

Фото профиля Shinka - AI
Shinka - AI7 дней назад

Certificate-guided routing is how diffusion VLMs stop wasting compute on background tokens.

Фото профиля Ethan Satoshi
Ethan Satoshi7 дней назад

Smarter visual routing could make VLMs much more efficient.

Фото профиля Matthew Broadstreet
Matthew Broadstreet7 дней назад

⏫

Похожие видео

FP4 Explore, BF16 Train Diffusion Reinforcement Learning via Efficient Rollout Scaling paper:
0:35

FP4 Explore, BF16 Train Diffusion Reinforcement Learning via Efficient Rollout Scaling paper:

AK

12,817 просмотров • 7 месяцев назад

MolmoPoint Better Pointing for VLMs with Grounding Tokens paper: models: app:
1:22

MolmoPoint Better Pointing for VLMs with Grounding Tokens paper: models: app:

AK

20,057 просмотров • 7 месяцев назад

👀Humans compare images by looking back and forth. Many open-weight VLMs encode each image independently, and defer comparison to the LM. We introduce SVE: Stateful Visual Encoders for Vision-Language Models, where the visual encoder itself becomes change-aware. 🌐Project: 📰Paper: 💻Code: 1/n
1:11

👀Humans compare images by looking back and forth. Many open-weight VLMs encode each image independently, and defer comparison to the LM. We introduce SVE: Stateful Visual Encoders for Vision-Language Models, where the visual encoder itself becomes change-aware. 🌐Project: 📰Paper: 💻Code: 1/n

Zirui "Colin" Wang ✈️EMNLP

63,924 просмотров • 5 месяцев назад

Diffusion Policies and VLAs with Formal Safety Guarantees🛡️ We just released our new paper: “From Demonstrations to Safe Deployment: Path-Consistent Safety Filtering for Diffusion Policies” 📑Paper: 🌐Website:
0:34

Diffusion Policies and VLAs with Formal Safety Guarantees🛡️ We just released our new paper: “From Demonstrations to Safe Deployment: Path-Consistent Safety Filtering for Diffusion Policies” 📑Paper: 🌐Website:

Learning Systems and Robotics Lab (is hiring!)

20,534 просмотров • 1 год назад

(1/n) Time to unify your favorite visual generative models, VLMs, and simulators for controllable visual generation—Introducing a Product of Experts (PoE) framework for inference-time knowledge composition from heterogeneous models.
0:55

(1/n) Time to unify your favorite visual generative models, VLMs, and simulators for controllable visual generation—Introducing a Product of Experts (PoE) framework for inference-time knowledge composition from heterogeneous models.

Yunzhi Zhang

49,225 просмотров • 1 год назад

What makes a maze look like a maze? Humans can reason about infinitely many instantiations of mazes—made of candy canes, sticks, icing, yarn, etc. But VLMs often struggle to make sense of such visual abstractions. We improve VLMs' ability to interpret these abstract concepts.
1:26

What makes a maze look like a maze? Humans can reason about infinitely many instantiations of mazes—made of candy canes, sticks, icing, yarn, etc. But VLMs often struggle to make sense of such visual abstractions. We improve VLMs' ability to interpret these abstract concepts.

Joy Hsu

43,528 просмотров • 2 лет назад

Excited to share our #CVPR2023 on synthesizing new views along a camera trajectory from a **single image**! How? 💡 The good old epipolar constraints in a pose-guided diffusion model! Paper: Project:
0:51

Excited to share our #CVPR2023 on synthesizing new views along a camera trajectory from a **single image**! How? 💡 The good old epipolar constraints in a pose-guided diffusion model! Paper: Project:

Jia-Bin Huang

94,199 просмотров • 3 лет назад

🚀 New Paper: Pixel Reasoner 🧠🖼️ How can Vision-Language Models (VLMs) perform chain-of-thought reasoning within the image itself? We introduce Pixel Reasoner, the first open-source framework that enables VLMs to “think in pixel space” through curiosity-driven reinforcement learning. Current VLMs reason only in text — even when grounded in rich images or videos, their logical steps are verbalized in natural language. This restricts their ability to interrogate visual evidence and demonstrate how conclusions are drawn. 🔍 So we ask: What if we could make VLMs "show their work" by reasoning directly in the pixel space? Inspired by GPT-o3’s "think-in-image" ability, we propose a framework where VLMs use interactive visual operations — zoom, select-frame, highlight — to reason through complex visual inputs. To do this, we design a two-stage training process: Instruction tuning with synthesized visual reasoning traces. Reinforcement learning with curiosity-driven reward to balance exploration between pixel and text reasoning ✨ With this, Pixel Reasoner achieves near-SoTA performance on many information-rich multimodal benchmarks: 📊 84% on InfographicsVQA 🧠 84% on V* benchmark 🧩 74% on TallyQA-Complex It also achieves strong accuracy of 68% on MVBench (a video benchmark). Website: Paper: Code: Demo: (coming soon)
0:43

🚀 New Paper: Pixel Reasoner 🧠🖼️ How can Vision-Language Models (VLMs) perform chain-of-thought reasoning within the image itself? We introduce Pixel Reasoner, the first open-source framework that enables VLMs to “think in pixel space” through curiosity-driven reinforcement learning. Current VLMs reason only in text — even when grounded in rich images or videos, their logical steps are verbalized in natural language. This restricts their ability to interrogate visual evidence and demonstrate how conclusions are drawn. 🔍 So we ask: What if we could make VLMs "show their work" by reasoning directly in the pixel space? Inspired by GPT-o3’s "think-in-image" ability, we propose a framework where VLMs use interactive visual operations — zoom, select-frame, highlight — to reason through complex visual inputs. To do this, we design a two-stage training process: Instruction tuning with synthesized visual reasoning traces. Reinforcement learning with curiosity-driven reward to balance exploration between pixel and text reasoning ✨ With this, Pixel Reasoner achieves near-SoTA performance on many information-rich multimodal benchmarks: 📊 84% on InfographicsVQA 🧠 84% on V* benchmark 🧩 74% on TallyQA-Complex It also achieves strong accuracy of 68% on MVBench (a video benchmark). Website: Paper: Code: Demo: (coming soon)

Wenhu Chen

82,829 просмотров • 1 год назад

Excited to share Norman Müller's DiffRF: Rendering-guided 3D Radiance Field Diffusion #CVPR2023 highlight! 2D diffusion is great, but what about 3D? We show radiance field diffusion with rendering guidance for consistent and editable 3D synthesis. Vid:
0:27

Excited to share Norman Müller's DiffRF: Rendering-guided 3D Radiance Field Diffusion #CVPR2023 highlight! 2D diffusion is great, but what about 3D? We show radiance field diffusion with rendering guidance for consistent and editable 3D synthesis. Vid:

Matthias Niessner

118,156 просмотров • 3 лет назад

Introducing Wave Link 3.0 (full public release). Audio mixing software built for all creators that delivers broadcast-level routing and effects. 🆕 Now free for Windows and Mac 🫵 Use your own mic 💡 Reimagined interface 👀 Visual routing table 🎙️ Support for multiple mics + ∞ software inputs 🔊 5 independent output mixes Now free for all, no Elgato hardware needed:
0:42

Introducing Wave Link 3.0 (full public release). Audio mixing software built for all creators that delivers broadcast-level routing and effects. 🆕 Now free for Windows and Mac 🫵 Use your own mic 💡 Reimagined interface 👀 Visual routing table 🎙️ Support for multiple mics + ∞ software inputs 🔊 5 independent output mixes Now free for all, no Elgato hardware needed:

Elgato

44,009 просмотров • 8 месяцев назад

When appliances run unnecessarily, your electricity bill keeps growing.Use them only when needed and adopt smart habits for a comfortable and efficient summer. #UPPCL #SmartUsage #EnergyAwareness #SaveElectricity CM Office, GoUP Energy Minister Office UP A K Sharma MDUPPCL
0:32

When appliances run unnecessarily, your electricity bill keeps growing.Use them only when needed and adopt smart habits for a comfortable and efficient summer. #UPPCL #SmartUsage #EnergyAwareness #SaveElectricity CM Office, GoUP Energy Minister Office UP A K Sharma MDUPPCL

UPPCL

28,266 просмотров • 4 месяцев назад

Making paper airplanes with efficient flight
0:32

Making paper airplanes with efficient flight

Today I Learned

241,866 просмотров • 1 год назад

EasyControl just dropped on Hugging Face Adding Efficient and Flexible Control for Diffusion Transformer Free chatgpt style ghibli image generation with easy control
0:34

EasyControl just dropped on Hugging Face Adding Efficient and Flexible Control for Diffusion Transformer Free chatgpt style ghibli image generation with easy control

AK

222,239 просмотров • 1 год назад

Vision-language models (VLMs) can see well, but they struggle to reason. In this episode, Antonia Wüst (PhD researcher, TU Darmstadt) explains how combining VLMs with program synthesis yields more reliable visual reasoning, with fewer tokens than chain-of-thought.
0:46

Vision-language models (VLMs) can see well, but they struggle to reason. In this episode, Antonia Wüst (PhD researcher, TU Darmstadt) explains how combining VLMs with program synthesis yields more reliable visual reasoning, with fewer tokens than chain-of-thought.

Ndea

22,130 просмотров • 9 месяцев назад

Continuous diffusion had a good run—now it’s time for Discrete diffusion! Introducing Anchored Posterior Sampling (APS) APS outperforms discrete and continuous baselines in terms of performance & scaling on inverse problems, stylization, and text-guided editing.
0:33

Continuous diffusion had a good run—now it’s time for Discrete diffusion! Introducing Anchored Posterior Sampling (APS) APS outperforms discrete and continuous baselines in terms of performance & scaling on inverse problems, stylization, and text-guided editing.

Litu Rout

40,315 просмотров • 1 год назад

I built a RAG Routing Agent that exactly knows where to look. It automatically route queries to the right knowledge base and falls back to web search when needed. 100% Opensource Code with step-by-step tutorial.
2:33

I built a RAG Routing Agent that exactly knows where to look. It automatically route queries to the right knowledge base and falls back to web search when needed. 100% Opensource Code with step-by-step tutorial.

Shubham Saboo

80,579 просмотров • 1 год назад

Introducing 🔥BLIP-Diffusion🔥, a novel method for enabling Text-to-image Diffusion models with multimodal controllable generation/editing, powered by BLIP-2 pre-trained text-aligned subject representation. Paper: Project: (1/n)
2:03

Introducing 🔥BLIP-Diffusion🔥, a novel method for enabling Text-to-image Diffusion models with multimodal controllable generation/editing, powered by BLIP-2 pre-trained text-aligned subject representation. Paper: Project: (1/n)

Steven Hoi

40,615 просмотров • 3 лет назад