Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

Look Only When Needed Certificate-Guided Visual Routing for Efficient Diffusion VLMs paper:

_akhaliq's profile picture

AK

538,835 subscribers

20,862 Aufrufe • vor 7 Tagen •via X (Twitter)

Wissenschaft & TechnologieBildung
LIVE

Anya Rossi• Live Now

Private livecam show

7 Kommentare

Profilbild von harpreet
harpreetvor 7 Tagen

@dan_gural can’t help but think of you every time I see this coco pic 😂

Profilbild von The SI Maximalist
The SI Maximalistvor 7 Tagen

Certificate-guided visual routing is a delightfully formal answer to “do I really need to look?”

Profilbild von 花花大人
花花大人vor 7 Tagen

Diffusion VLM 按需看图这条路由思路挺干净。

Profilbild von Saksham Jain
Saksham Jainvor 7 Tagen

wouldn't have guessed the image only flips the unmasked token on 22.8% of steps. using the no-image pass as its own check is clever, nothing to tune

Profilbild von Shinka - AI
Shinka - AIvor 7 Tagen

Certificate-guided routing is how diffusion VLMs stop wasting compute on background tokens.

Profilbild von Ethan Satoshi
Ethan Satoshivor 7 Tagen

Smarter visual routing could make VLMs much more efficient.

Profilbild von Matthew Broadstreet
Matthew Broadstreetvor 7 Tagen

⏫

Ähnliche Videos

FP4 Explore, BF16 Train Diffusion Reinforcement Learning via Efficient Rollout Scaling paper:
0:35

FP4 Explore, BF16 Train Diffusion Reinforcement Learning via Efficient Rollout Scaling paper:

AK

12,817 Aufrufe • vor 7 Monaten

MolmoPoint Better Pointing for VLMs with Grounding Tokens paper: models: app:
1:22

MolmoPoint Better Pointing for VLMs with Grounding Tokens paper: models: app:

AK

20,057 Aufrufe • vor 7 Monaten

👀Humans compare images by looking back and forth. Many open-weight VLMs encode each image independently, and defer comparison to the LM. We introduce SVE: Stateful Visual Encoders for Vision-Language Models, where the visual encoder itself becomes change-aware. 🌐Project: 📰Paper: 💻Code: 1/n
1:11

👀Humans compare images by looking back and forth. Many open-weight VLMs encode each image independently, and defer comparison to the LM. We introduce SVE: Stateful Visual Encoders for Vision-Language Models, where the visual encoder itself becomes change-aware. 🌐Project: 📰Paper: 💻Code: 1/n

Zirui "Colin" Wang ✈️EMNLP

63,924 Aufrufe • vor 5 Monaten

Diffusion Policies and VLAs with Formal Safety Guarantees🛡️ We just released our new paper: “From Demonstrations to Safe Deployment: Path-Consistent Safety Filtering for Diffusion Policies” 📑Paper: 🌐Website:
0:34

Diffusion Policies and VLAs with Formal Safety Guarantees🛡️ We just released our new paper: “From Demonstrations to Safe Deployment: Path-Consistent Safety Filtering for Diffusion Policies” 📑Paper: 🌐Website:

Learning Systems and Robotics Lab (is hiring!)

20,534 Aufrufe • vor 1 Jahr

(1/n) Time to unify your favorite visual generative models, VLMs, and simulators for controllable visual generation—Introducing a Product of Experts (PoE) framework for inference-time knowledge composition from heterogeneous models.
0:55

(1/n) Time to unify your favorite visual generative models, VLMs, and simulators for controllable visual generation—Introducing a Product of Experts (PoE) framework for inference-time knowledge composition from heterogeneous models.

Yunzhi Zhang

49,225 Aufrufe • vor 1 Jahr

What makes a maze look like a maze? Humans can reason about infinitely many instantiations of mazes—made of candy canes, sticks, icing, yarn, etc. But VLMs often struggle to make sense of such visual abstractions. We improve VLMs' ability to interpret these abstract concepts.
1:26

What makes a maze look like a maze? Humans can reason about infinitely many instantiations of mazes—made of candy canes, sticks, icing, yarn, etc. But VLMs often struggle to make sense of such visual abstractions. We improve VLMs' ability to interpret these abstract concepts.

Joy Hsu

43,528 Aufrufe • vor 2 Jahren

Excited to share our #CVPR2023 on synthesizing new views along a camera trajectory from a **single image**! How? 💡 The good old epipolar constraints in a pose-guided diffusion model! Paper: Project:
0:51

Excited to share our #CVPR2023 on synthesizing new views along a camera trajectory from a **single image**! How? 💡 The good old epipolar constraints in a pose-guided diffusion model! Paper: Project:

Jia-Bin Huang

94,199 Aufrufe • vor 3 Jahren

🚀 New Paper: Pixel Reasoner 🧠🖼️ How can Vision-Language Models (VLMs) perform chain-of-thought reasoning within the image itself? We introduce Pixel Reasoner, the first open-source framework that enables VLMs to “think in pixel space” through curiosity-driven reinforcement learning. Current VLMs reason only in text — even when grounded in rich images or videos, their logical steps are verbalized in natural language. This restricts their ability to interrogate visual evidence and demonstrate how conclusions are drawn. 🔍 So we ask: What if we could make VLMs "show their work" by reasoning directly in the pixel space? Inspired by GPT-o3’s "think-in-image" ability, we propose a framework where VLMs use interactive visual operations — zoom, select-frame, highlight — to reason through complex visual inputs. To do this, we design a two-stage training process: Instruction tuning with synthesized visual reasoning traces. Reinforcement learning with curiosity-driven reward to balance exploration between pixel and text reasoning ✨ With this, Pixel Reasoner achieves near-SoTA performance on many information-rich multimodal benchmarks: 📊 84% on InfographicsVQA 🧠 84% on V* benchmark 🧩 74% on TallyQA-Complex It also achieves strong accuracy of 68% on MVBench (a video benchmark). Website: Paper: Code: Demo: (coming soon)
0:43

🚀 New Paper: Pixel Reasoner 🧠🖼️ How can Vision-Language Models (VLMs) perform chain-of-thought reasoning within the image itself? We introduce Pixel Reasoner, the first open-source framework that enables VLMs to “think in pixel space” through curiosity-driven reinforcement learning. Current VLMs reason only in text — even when grounded in rich images or videos, their logical steps are verbalized in natural language. This restricts their ability to interrogate visual evidence and demonstrate how conclusions are drawn. 🔍 So we ask: What if we could make VLMs "show their work" by reasoning directly in the pixel space? Inspired by GPT-o3’s "think-in-image" ability, we propose a framework where VLMs use interactive visual operations — zoom, select-frame, highlight — to reason through complex visual inputs. To do this, we design a two-stage training process: Instruction tuning with synthesized visual reasoning traces. Reinforcement learning with curiosity-driven reward to balance exploration between pixel and text reasoning ✨ With this, Pixel Reasoner achieves near-SoTA performance on many information-rich multimodal benchmarks: 📊 84% on InfographicsVQA 🧠 84% on V* benchmark 🧩 74% on TallyQA-Complex It also achieves strong accuracy of 68% on MVBench (a video benchmark). Website: Paper: Code: Demo: (coming soon)

Wenhu Chen

82,829 Aufrufe • vor 1 Jahr

Excited to share Norman Müller's DiffRF: Rendering-guided 3D Radiance Field Diffusion #CVPR2023 highlight! 2D diffusion is great, but what about 3D? We show radiance field diffusion with rendering guidance for consistent and editable 3D synthesis. Vid:
0:27

Excited to share Norman Müller's DiffRF: Rendering-guided 3D Radiance Field Diffusion #CVPR2023 highlight! 2D diffusion is great, but what about 3D? We show radiance field diffusion with rendering guidance for consistent and editable 3D synthesis. Vid:

Matthias Niessner

118,156 Aufrufe • vor 3 Jahren

Introducing Wave Link 3.0 (full public release). Audio mixing software built for all creators that delivers broadcast-level routing and effects. 🆕 Now free for Windows and Mac 🫵 Use your own mic 💡 Reimagined interface 👀 Visual routing table 🎙️ Support for multiple mics + ∞ software inputs 🔊 5 independent output mixes Now free for all, no Elgato hardware needed:
0:42

Introducing Wave Link 3.0 (full public release). Audio mixing software built for all creators that delivers broadcast-level routing and effects. 🆕 Now free for Windows and Mac 🫵 Use your own mic 💡 Reimagined interface 👀 Visual routing table 🎙️ Support for multiple mics + ∞ software inputs 🔊 5 independent output mixes Now free for all, no Elgato hardware needed:

Elgato

44,009 Aufrufe • vor 8 Monaten

When appliances run unnecessarily, your electricity bill keeps growing.Use them only when needed and adopt smart habits for a comfortable and efficient summer. #UPPCL #SmartUsage #EnergyAwareness #SaveElectricity CM Office, GoUP Energy Minister Office UP A K Sharma MDUPPCL
0:32

When appliances run unnecessarily, your electricity bill keeps growing.Use them only when needed and adopt smart habits for a comfortable and efficient summer. #UPPCL #SmartUsage #EnergyAwareness #SaveElectricity CM Office, GoUP Energy Minister Office UP A K Sharma MDUPPCL

UPPCL

28,266 Aufrufe • vor 4 Monaten

Making paper airplanes with efficient flight
0:32

Making paper airplanes with efficient flight

Today I Learned

241,866 Aufrufe • vor 1 Jahr

EasyControl just dropped on Hugging Face Adding Efficient and Flexible Control for Diffusion Transformer Free chatgpt style ghibli image generation with easy control
0:34

EasyControl just dropped on Hugging Face Adding Efficient and Flexible Control for Diffusion Transformer Free chatgpt style ghibli image generation with easy control

AK

222,239 Aufrufe • vor 1 Jahr

Vision-language models (VLMs) can see well, but they struggle to reason. In this episode, Antonia Wüst (PhD researcher, TU Darmstadt) explains how combining VLMs with program synthesis yields more reliable visual reasoning, with fewer tokens than chain-of-thought.
0:46

Vision-language models (VLMs) can see well, but they struggle to reason. In this episode, Antonia Wüst (PhD researcher, TU Darmstadt) explains how combining VLMs with program synthesis yields more reliable visual reasoning, with fewer tokens than chain-of-thought.

Ndea

22,130 Aufrufe • vor 9 Monaten

Continuous diffusion had a good run—now it’s time for Discrete diffusion! Introducing Anchored Posterior Sampling (APS) APS outperforms discrete and continuous baselines in terms of performance & scaling on inverse problems, stylization, and text-guided editing.
0:33

Continuous diffusion had a good run—now it’s time for Discrete diffusion! Introducing Anchored Posterior Sampling (APS) APS outperforms discrete and continuous baselines in terms of performance & scaling on inverse problems, stylization, and text-guided editing.

Litu Rout

40,315 Aufrufe • vor 1 Jahr

I built a RAG Routing Agent that exactly knows where to look. It automatically route queries to the right knowledge base and falls back to web search when needed. 100% Opensource Code with step-by-step tutorial.
2:33

I built a RAG Routing Agent that exactly knows where to look. It automatically route queries to the right knowledge base and falls back to web search when needed. 100% Opensource Code with step-by-step tutorial.

Shubham Saboo

80,579 Aufrufe • vor 1 Jahr

Introducing 🔥BLIP-Diffusion🔥, a novel method for enabling Text-to-image Diffusion models with multimodal controllable generation/editing, powered by BLIP-2 pre-trained text-aligned subject representation. Paper: Project: (1/n)
2:03

Introducing 🔥BLIP-Diffusion🔥, a novel method for enabling Text-to-image Diffusion models with multimodal controllable generation/editing, powered by BLIP-2 pre-trained text-aligned subject representation. Paper: Project: (1/n)

Steven Hoi

40,615 Aufrufe • vor 3 Jahren