Video yükleniyor...

Video Yüklenemedi

Ana Sayfaya Dön

Look Only When Needed Certificate-Guided Visual Routing for Efficient Diffusion VLMs paper:

_akhaliq's profile picture

AK

538,835 subscribers

20,862 görüntüleme • 7 gün önce •via X (Twitter)

Bilim & TeknolojiEğitim
LIVE

Anya Rossi• Live Now

Private livecam show

7 Yorum

harpreet profil fotoğrafı
harpreet7 gün önce

@dan_gural can’t help but think of you every time I see this coco pic 😂

The SI Maximalist profil fotoğrafı
The SI Maximalist7 gün önce

Certificate-guided visual routing is a delightfully formal answer to “do I really need to look?”

花花大人 profil fotoğrafı
花花大人7 gün önce

Diffusion VLM 按需看图这条路由思路挺干净。

Saksham Jain profil fotoğrafı
Saksham Jain7 gün önce

wouldn't have guessed the image only flips the unmasked token on 22.8% of steps. using the no-image pass as its own check is clever, nothing to tune

Shinka - AI profil fotoğrafı
Shinka - AI7 gün önce

Certificate-guided routing is how diffusion VLMs stop wasting compute on background tokens.

Ethan Satoshi profil fotoğrafı
Ethan Satoshi7 gün önce

Smarter visual routing could make VLMs much more efficient.

Matthew Broadstreet profil fotoğrafı
Matthew Broadstreet7 gün önce

⏫

Benzer Videolar

FP4 Explore, BF16 Train Diffusion Reinforcement Learning via Efficient Rollout Scaling paper:
0:35

FP4 Explore, BF16 Train Diffusion Reinforcement Learning via Efficient Rollout Scaling paper:

AK

12,817 görüntüleme • 7 ay önce

MolmoPoint Better Pointing for VLMs with Grounding Tokens paper: models: app:
1:22

MolmoPoint Better Pointing for VLMs with Grounding Tokens paper: models: app:

AK

20,057 görüntüleme • 7 ay önce

👀Humans compare images by looking back and forth. Many open-weight VLMs encode each image independently, and defer comparison to the LM. We introduce SVE: Stateful Visual Encoders for Vision-Language Models, where the visual encoder itself becomes change-aware. 🌐Project: 📰Paper: 💻Code: 1/n
1:11

👀Humans compare images by looking back and forth. Many open-weight VLMs encode each image independently, and defer comparison to the LM. We introduce SVE: Stateful Visual Encoders for Vision-Language Models, where the visual encoder itself becomes change-aware. 🌐Project: 📰Paper: 💻Code: 1/n

Zirui "Colin" Wang ✈️EMNLP

63,924 görüntüleme • 5 ay önce

Diffusion Policies and VLAs with Formal Safety Guarantees🛡️ We just released our new paper: “From Demonstrations to Safe Deployment: Path-Consistent Safety Filtering for Diffusion Policies” 📑Paper: 🌐Website:
0:34

Diffusion Policies and VLAs with Formal Safety Guarantees🛡️ We just released our new paper: “From Demonstrations to Safe Deployment: Path-Consistent Safety Filtering for Diffusion Policies” 📑Paper: 🌐Website:

Learning Systems and Robotics Lab (is hiring!)

20,534 görüntüleme • 1 yıl önce

(1/n) Time to unify your favorite visual generative models, VLMs, and simulators for controllable visual generation—Introducing a Product of Experts (PoE) framework for inference-time knowledge composition from heterogeneous models.
0:55

(1/n) Time to unify your favorite visual generative models, VLMs, and simulators for controllable visual generation—Introducing a Product of Experts (PoE) framework for inference-time knowledge composition from heterogeneous models.

Yunzhi Zhang

49,225 görüntüleme • 1 yıl önce

What makes a maze look like a maze? Humans can reason about infinitely many instantiations of mazes—made of candy canes, sticks, icing, yarn, etc. But VLMs often struggle to make sense of such visual abstractions. We improve VLMs' ability to interpret these abstract concepts.
1:26

What makes a maze look like a maze? Humans can reason about infinitely many instantiations of mazes—made of candy canes, sticks, icing, yarn, etc. But VLMs often struggle to make sense of such visual abstractions. We improve VLMs' ability to interpret these abstract concepts.

Joy Hsu

43,528 görüntüleme • 2 yıl önce

Excited to share our #CVPR2023 on synthesizing new views along a camera trajectory from a **single image**! How? 💡 The good old epipolar constraints in a pose-guided diffusion model! Paper: Project:
0:51

Excited to share our #CVPR2023 on synthesizing new views along a camera trajectory from a **single image**! How? 💡 The good old epipolar constraints in a pose-guided diffusion model! Paper: Project:

Jia-Bin Huang

94,199 görüntüleme • 3 yıl önce

🚀 New Paper: Pixel Reasoner 🧠🖼️ How can Vision-Language Models (VLMs) perform chain-of-thought reasoning within the image itself? We introduce Pixel Reasoner, the first open-source framework that enables VLMs to “think in pixel space” through curiosity-driven reinforcement learning. Current VLMs reason only in text — even when grounded in rich images or videos, their logical steps are verbalized in natural language. This restricts their ability to interrogate visual evidence and demonstrate how conclusions are drawn. 🔍 So we ask: What if we could make VLMs "show their work" by reasoning directly in the pixel space? Inspired by GPT-o3’s "think-in-image" ability, we propose a framework where VLMs use interactive visual operations — zoom, select-frame, highlight — to reason through complex visual inputs. To do this, we design a two-stage training process: Instruction tuning with synthesized visual reasoning traces. Reinforcement learning with curiosity-driven reward to balance exploration between pixel and text reasoning ✨ With this, Pixel Reasoner achieves near-SoTA performance on many information-rich multimodal benchmarks: 📊 84% on InfographicsVQA 🧠 84% on V* benchmark 🧩 74% on TallyQA-Complex It also achieves strong accuracy of 68% on MVBench (a video benchmark). Website: Paper: Code: Demo: (coming soon)
0:43

🚀 New Paper: Pixel Reasoner 🧠🖼️ How can Vision-Language Models (VLMs) perform chain-of-thought reasoning within the image itself? We introduce Pixel Reasoner, the first open-source framework that enables VLMs to “think in pixel space” through curiosity-driven reinforcement learning. Current VLMs reason only in text — even when grounded in rich images or videos, their logical steps are verbalized in natural language. This restricts their ability to interrogate visual evidence and demonstrate how conclusions are drawn. 🔍 So we ask: What if we could make VLMs "show their work" by reasoning directly in the pixel space? Inspired by GPT-o3’s "think-in-image" ability, we propose a framework where VLMs use interactive visual operations — zoom, select-frame, highlight — to reason through complex visual inputs. To do this, we design a two-stage training process: Instruction tuning with synthesized visual reasoning traces. Reinforcement learning with curiosity-driven reward to balance exploration between pixel and text reasoning ✨ With this, Pixel Reasoner achieves near-SoTA performance on many information-rich multimodal benchmarks: 📊 84% on InfographicsVQA 🧠 84% on V* benchmark 🧩 74% on TallyQA-Complex It also achieves strong accuracy of 68% on MVBench (a video benchmark). Website: Paper: Code: Demo: (coming soon)

Wenhu Chen

82,829 görüntüleme • 1 yıl önce

Excited to share Norman Müller's DiffRF: Rendering-guided 3D Radiance Field Diffusion #CVPR2023 highlight! 2D diffusion is great, but what about 3D? We show radiance field diffusion with rendering guidance for consistent and editable 3D synthesis. Vid:
0:27

Excited to share Norman Müller's DiffRF: Rendering-guided 3D Radiance Field Diffusion #CVPR2023 highlight! 2D diffusion is great, but what about 3D? We show radiance field diffusion with rendering guidance for consistent and editable 3D synthesis. Vid:

Matthias Niessner

118,156 görüntüleme • 3 yıl önce

Introducing Wave Link 3.0 (full public release). Audio mixing software built for all creators that delivers broadcast-level routing and effects. 🆕 Now free for Windows and Mac 🫵 Use your own mic 💡 Reimagined interface 👀 Visual routing table 🎙️ Support for multiple mics + ∞ software inputs 🔊 5 independent output mixes Now free for all, no Elgato hardware needed:
0:42

Introducing Wave Link 3.0 (full public release). Audio mixing software built for all creators that delivers broadcast-level routing and effects. 🆕 Now free for Windows and Mac 🫵 Use your own mic 💡 Reimagined interface 👀 Visual routing table 🎙️ Support for multiple mics + ∞ software inputs 🔊 5 independent output mixes Now free for all, no Elgato hardware needed:

Elgato

44,009 görüntüleme • 8 ay önce

When appliances run unnecessarily, your electricity bill keeps growing.Use them only when needed and adopt smart habits for a comfortable and efficient summer. #UPPCL #SmartUsage #EnergyAwareness #SaveElectricity CM Office, GoUP Energy Minister Office UP A K Sharma MDUPPCL
0:32

When appliances run unnecessarily, your electricity bill keeps growing.Use them only when needed and adopt smart habits for a comfortable and efficient summer. #UPPCL #SmartUsage #EnergyAwareness #SaveElectricity CM Office, GoUP Energy Minister Office UP A K Sharma MDUPPCL

UPPCL

28,266 görüntüleme • 4 ay önce

Making paper airplanes with efficient flight
0:32

Making paper airplanes with efficient flight

Today I Learned

241,866 görüntüleme • 1 yıl önce

EasyControl just dropped on Hugging Face Adding Efficient and Flexible Control for Diffusion Transformer Free chatgpt style ghibli image generation with easy control
0:34

EasyControl just dropped on Hugging Face Adding Efficient and Flexible Control for Diffusion Transformer Free chatgpt style ghibli image generation with easy control

AK

222,239 görüntüleme • 1 yıl önce

Vision-language models (VLMs) can see well, but they struggle to reason. In this episode, Antonia Wüst (PhD researcher, TU Darmstadt) explains how combining VLMs with program synthesis yields more reliable visual reasoning, with fewer tokens than chain-of-thought.
0:46

Vision-language models (VLMs) can see well, but they struggle to reason. In this episode, Antonia Wüst (PhD researcher, TU Darmstadt) explains how combining VLMs with program synthesis yields more reliable visual reasoning, with fewer tokens than chain-of-thought.

Ndea

22,130 görüntüleme • 9 ay önce

Continuous diffusion had a good run—now it’s time for Discrete diffusion! Introducing Anchored Posterior Sampling (APS) APS outperforms discrete and continuous baselines in terms of performance & scaling on inverse problems, stylization, and text-guided editing.
0:33

Continuous diffusion had a good run—now it’s time for Discrete diffusion! Introducing Anchored Posterior Sampling (APS) APS outperforms discrete and continuous baselines in terms of performance & scaling on inverse problems, stylization, and text-guided editing.

Litu Rout

40,315 görüntüleme • 1 yıl önce

I built a RAG Routing Agent that exactly knows where to look. It automatically route queries to the right knowledge base and falls back to web search when needed. 100% Opensource Code with step-by-step tutorial.
2:33

I built a RAG Routing Agent that exactly knows where to look. It automatically route queries to the right knowledge base and falls back to web search when needed. 100% Opensource Code with step-by-step tutorial.

Shubham Saboo

80,579 görüntüleme • 1 yıl önce

Introducing 🔥BLIP-Diffusion🔥, a novel method for enabling Text-to-image Diffusion models with multimodal controllable generation/editing, powered by BLIP-2 pre-trained text-aligned subject representation. Paper: Project: (1/n)
2:03

Introducing 🔥BLIP-Diffusion🔥, a novel method for enabling Text-to-image Diffusion models with multimodal controllable generation/editing, powered by BLIP-2 pre-trained text-aligned subject representation. Paper: Project: (1/n)

Steven Hoi

40,615 görüntüleme • 3 yıl önce