正在加载视频...

视频加载失败

加载此视频时出现问题。这可能是由于临时网络问题，或视频可能不可用。

Look Only When Needed Certificate-Guided Visual Routing for Efficient Diffusion VLMs paper:

AK

538,835 subscribers

20,862 次观看 • 7 天前 •via X (Twitter)

科学技术教育

Anya Rossi• Live Now

Private livecam show

7 条评论

harpreet 的头像

harpreet7 天前

@dan_gural can’t help but think of you every time I see this coco pic 😂

The SI Maximalist 的头像

The SI Maximalist7 天前

Certificate-guided visual routing is a delightfully formal answer to “do I really need to look?”

花花大人的头像

花花大人6 天前

Diffusion VLM 按需看图这条路由思路挺干净。

Saksham Jain 的头像

Saksham Jain7 天前

wouldn't have guessed the image only flips the unmasked token on 22.8% of steps. using the no-image pass as its own check is clever, nothing to tune

Shinka - AI 的头像

Shinka - AI7 天前

Certificate-guided routing is how diffusion VLMs stop wasting compute on background tokens.

Ethan Satoshi 的头像

Ethan Satoshi7 天前

Smarter visual routing could make VLMs much more efficient.

Matthew Broadstreet 的头像

Matthew Broadstreet7 天前

⏫

相关视频

FP4 Explore, BF16 Train Diffusion Reinforcement Learning via Efficient Rollout Scaling paper:

FP4 Explore, BF16 Train Diffusion Reinforcement Learning via Efficient Rollout Scaling paper:

AK

12,817 次观看 • 7 个月前

MolmoPoint Better Pointing for VLMs with Grounding Tokens paper: models: app:

MolmoPoint Better Pointing for VLMs with Grounding Tokens paper: models: app:

AK

20,057 次观看 • 7 个月前

👀Humans compare images by looking back and forth. Many open-weight VLMs encode each image independently, and defer comparison to the LM. We introduce SVE: Stateful Visual Encoders for Vision-Language Models, where the visual encoder itself becomes change-aware. 🌐Project: 📰Paper: 💻Code: 1/n

👀Humans compare images by looking back and forth. Many open-weight VLMs encode each image independently, and defer comparison to the LM. We introduce SVE: Stateful Visual Encoders for Vision-Language Models, where the visual encoder itself becomes change-aware. 🌐Project: 📰Paper: 💻Code: 1/n

Zirui "Colin" Wang ✈️EMNLP

63,924 次观看 • 5 个月前

Diffusion Policies and VLAs with Formal Safety Guarantees🛡️ We just released our new paper: “From Demonstrations to Safe Deployment: Path-Consistent Safety Filtering for Diffusion Policies” 📑Paper: 🌐Website:

Diffusion Policies and VLAs with Formal Safety Guarantees🛡️ We just released our new paper: “From Demonstrations to Safe Deployment: Path-Consistent Safety Filtering for Diffusion Policies” 📑Paper: 🌐Website:

Learning Systems and Robotics Lab (is hiring!)

20,534 次观看 • 1 年前

(1/n) Time to unify your favorite visual generative models, VLMs, and simulators for controllable visual generation—Introducing a Product of Experts (PoE) framework for inference-time knowledge composition from heterogeneous models.

(1/n) Time to unify your favorite visual generative models, VLMs, and simulators for controllable visual generation—Introducing a Product of Experts (PoE) framework for inference-time knowledge composition from heterogeneous models.

Yunzhi Zhang

49,225 次观看 • 1 年前

What makes a maze look like a maze? Humans can reason about infinitely many instantiations of mazes—made of candy canes, sticks, icing, yarn, etc. But VLMs often struggle to make sense of such visual abstractions. We improve VLMs' ability to interpret these abstract concepts.

What makes a maze look like a maze? Humans can reason about infinitely many instantiations of mazes—made of candy canes, sticks, icing, yarn, etc. But VLMs often struggle to make sense of such visual abstractions. We improve VLMs' ability to interpret these abstract concepts.

Joy Hsu

43,528 次观看 • 2 年前

Excited to share our #CVPR2023 on synthesizing new views along a camera trajectory from a **single image**! How? 💡 The good old epipolar constraints in a pose-guided diffusion model! Paper: Project:

Excited to share our #CVPR2023 on synthesizing new views along a camera trajectory from a single image! How? 💡 The good old epipolar constraints in a pose-guided diffusion model! Paper: Project:

Jia-Bin Huang

94,199 次观看 • 3 年前

🚀 New Paper: Pixel Reasoner 🧠🖼️ How can Vision-Language Models (VLMs) perform chain-of-thought reasoning within the image itself? We introduce Pixel Reasoner, the first open-source framework that enables VLMs to “think in pixel space” through curiosity-driven reinforcement learning. Current VLMs reason only in text — even when grounded in rich images or videos, their logical steps are verbalized in natural language. This restricts their ability to interrogate visual evidence and demonstrate how conclusions are drawn. 🔍 So we ask: What if we could make VLMs "show their work" by reasoning directly in the pixel space? Inspired by GPT-o3’s "think-in-image" ability, we propose a framework where VLMs use interactive visual operations — zoom, select-frame, highlight — to reason through complex visual inputs. To do this, we design a two-stage training process: Instruction tuning with synthesized visual reasoning traces. Reinforcement learning with curiosity-driven reward to balance exploration between pixel and text reasoning ✨ With this, Pixel Reasoner achieves near-SoTA performance on many information-rich multimodal benchmarks: 📊 84% on InfographicsVQA 🧠 84% on V* benchmark 🧩 74% on TallyQA-Complex It also achieves strong accuracy of 68% on MVBench (a video benchmark). Website: Paper: Code: Demo: (coming soon)

🚀 New Paper: Pixel Reasoner 🧠🖼️ How can Vision-Language Models (VLMs) perform chain-of-thought reasoning within the image itself? We introduce Pixel Reasoner, the first open-source framework that enables VLMs to “think in pixel space” through curiosity-driven reinforcement learning. Current VLMs reason only in text — even when grounded in rich images or videos, their logical steps are verbalized in natural language. This restricts their ability to interrogate visual evidence and demonstrate how conclusions are drawn. 🔍 So we ask: What if we could make VLMs "show their work" by reasoning directly in the pixel space? Inspired by GPT-o3’s "think-in-image" ability, we propose a framework where VLMs use interactive visual operations — zoom, select-frame, highlight — to reason through complex visual inputs. To do this, we design a two-stage training process: Instruction tuning with synthesized visual reasoning traces. Reinforcement learning with curiosity-driven reward to balance exploration between pixel and text reasoning ✨ With this, Pixel Reasoner achieves near-SoTA performance on many information-rich multimodal benchmarks: 📊 84% on InfographicsVQA 🧠 84% on V* benchmark 🧩 74% on TallyQA-Complex It also achieves strong accuracy of 68% on MVBench (a video benchmark). Website: Paper: Code: Demo: (coming soon)

Wenhu Chen

82,829 次观看 • 1 年前

Excited to share Norman Müller's DiffRF: Rendering-guided 3D Radiance Field Diffusion #CVPR2023 highlight! 2D diffusion is great, but what about 3D? We show radiance field diffusion with rendering guidance for consistent and editable 3D synthesis. Vid:

Excited to share Norman Müller's DiffRF: Rendering-guided 3D Radiance Field Diffusion #CVPR2023 highlight! 2D diffusion is great, but what about 3D? We show radiance field diffusion with rendering guidance for consistent and editable 3D synthesis. Vid:

Matthias Niessner

118,156 次观看 • 3 年前

Introducing Wave Link 3.0 (full public release). Audio mixing software built for all creators that delivers broadcast-level routing and effects. 🆕 Now free for Windows and Mac 🫵 Use your own mic 💡 Reimagined interface 👀 Visual routing table 🎙️ Support for multiple mics + ∞ software inputs 🔊 5 independent output mixes Now free for all, no Elgato hardware needed:

Introducing Wave Link 3.0 (full public release). Audio mixing software built for all creators that delivers broadcast-level routing and effects. 🆕 Now free for Windows and Mac 🫵 Use your own mic 💡 Reimagined interface 👀 Visual routing table 🎙️ Support for multiple mics + ∞ software inputs 🔊 5 independent output mixes Now free for all, no Elgato hardware needed:

Elgato

44,009 次观看 • 8 个月前

When appliances run unnecessarily, your electricity bill keeps growing.Use them only when needed and adopt smart habits for a comfortable and efficient summer. #UPPCL #SmartUsage #EnergyAwareness #SaveElectricity CM Office, GoUP Energy Minister Office UP A K Sharma MDUPPCL

When appliances run unnecessarily, your electricity bill keeps growing.Use them only when needed and adopt smart habits for a comfortable and efficient summer. #UPPCL #SmartUsage #EnergyAwareness #SaveElectricity CM Office, GoUP Energy Minister Office UP A K Sharma MDUPPCL

UPPCL

28,266 次观看 • 4 个月前

Making paper airplanes with efficient flight

Making paper airplanes with efficient flight

Today I Learned

241,866 次观看 • 1 年前

EasyControl just dropped on Hugging Face Adding Efficient and Flexible Control for Diffusion Transformer Free chatgpt style ghibli image generation with easy control

EasyControl just dropped on Hugging Face Adding Efficient and Flexible Control for Diffusion Transformer Free chatgpt style ghibli image generation with easy control

AK

222,239 次观看 • 1 年前

Vision-language models (VLMs) can see well, but they struggle to reason. In this episode, Antonia Wüst (PhD researcher, TU Darmstadt) explains how combining VLMs with program synthesis yields more reliable visual reasoning, with fewer tokens than chain-of-thought.

Vision-language models (VLMs) can see well, but they struggle to reason. In this episode, Antonia Wüst (PhD researcher, TU Darmstadt) explains how combining VLMs with program synthesis yields more reliable visual reasoning, with fewer tokens than chain-of-thought.

Ndea

22,130 次观看 • 9 个月前

Continuous diffusion had a good run—now it’s time for Discrete diffusion! Introducing Anchored Posterior Sampling (APS) APS outperforms discrete and continuous baselines in terms of performance & scaling on inverse problems, stylization, and text-guided editing.

Continuous diffusion had a good run—now it’s time for Discrete diffusion! Introducing Anchored Posterior Sampling (APS) APS outperforms discrete and continuous baselines in terms of performance & scaling on inverse problems, stylization, and text-guided editing.

Litu Rout

40,315 次观看 • 1 年前

I built a RAG Routing Agent that exactly knows where to look. It automatically route queries to the right knowledge base and falls back to web search when needed. 100% Opensource Code with step-by-step tutorial.

I built a RAG Routing Agent that exactly knows where to look. It automatically route queries to the right knowledge base and falls back to web search when needed. 100% Opensource Code with step-by-step tutorial.

Shubham Saboo

80,579 次观看 • 1 年前

Introducing 🔥BLIP-Diffusion🔥, a novel method for enabling Text-to-image Diffusion models with multimodal controllable generation/editing, powered by BLIP-2 pre-trained text-aligned subject representation. Paper: Project: (1/n)

Introducing 🔥BLIP-Diffusion🔥, a novel method for enabling Text-to-image Diffusion models with multimodal controllable generation/editing, powered by BLIP-2 pre-trained text-aligned subject representation. Paper: Project: (1/n)

Steven Hoi

40,615 次观看 • 3 年前