Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

Listen to Samar Khanna explain why parallel generation, rather than sequential, raises the performance ceiling for language models. Learn more about diffusion LLMs. → We're hiring:

Inception

19,458 subscribers

18,582 views • 8 months ago •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

2 Comments

Shreyansh8 months ago

In the parallel / diffusion-style generation, does the model start from fully random tokens and iteratively replace them with new tokens at each refinement step? If so, doesn’t that effectively mean multiple token generations per position, increasing token compute compared to autoregressive decoding? Or are the correct tokens already implicitly represented in the initial state and just being refined without “new” token sampling each step? Just curious 👀.

Gareth Jax,,,8 months ago

@samar_a_khanna is it multimodal ?

Related Videos

Reasoning = step-by-step thinking. Usually. Diffusion reasons in parallel and refines natively. Better quality without the latency tax. Samar Khanna on how diffusion reasoning works. We're hiring →

Reasoning = step-by-step thinking. Usually. Diffusion reasons in parallel and refines natively. Better quality without the latency tax. Samar Khanna on how diffusion reasoning works. We're hiring →

Inception

16,490 views • 7 months ago

This video explains how diffusion models are overtaking Large Language Models for generation tasks like: 1. Code Generation 2. Image Generation 3. Video Generation 00:00 Agenda 00:20 How are they different from LLMs? 05:09 Internal Mechanism 10:09 How are vectors generated? 12:08 Conclusion 13:02 Opinion Piece AI Engineering Course: #Diffusion #AI #LLMs

This video explains how diffusion models are overtaking Large Language Models for generation tasks like: 1. Code Generation 2. Image Generation 3. Video Generation 00:00 Agenda 00:20 How are they different from LLMs? 05:09 Internal Mechanism 10:09 How are vectors generated? 12:08 Conclusion 13:02 Opinion Piece AI Engineering Course: #Diffusion #AI #LLMs

Gaurav Sen

28,027 views • 1 year ago

Just found the dLLM library to create Diffusion Language Models It's still early but it's insanely fun to experiment with diffusion (training, inference, eval) dLLM has the potential of becoming the main library for diffusion LLMs

Just found the dLLM library to create Diffusion Language Models It's still early but it's insanely fun to experiment with diffusion (training, inference, eval) dLLM has the potential of becoming the main library for diffusion LLMs

Maxime Labonne

75,593 views • 11 months ago

Excited to introduce SketchAgent!👩‍🎨 We leverage the prior of pretrained multimodal LLMs for language-driven, sequential sketch generation and human-agent collaborative sketching! ✨ Try our fun interface here:

Excited to introduce SketchAgent!👩‍🎨 We leverage the prior of pretrained multimodal LLMs for language-driven, sequential sketch generation and human-agent collaborative sketching! ✨ Try our fun interface here:

Yael Vinker

32,094 views • 1 year ago

Diffusion language models are SO FAST!! A new startup, Inception Labs, has released Mercury Coder, "the first commercial-scale diffusion large language model" It's 5-10x faster than current gen LLMs, providing high-quality responses at low costs. And you can try it now!

Diffusion language models are SO FAST!! A new startup, Inception Labs, has released Mercury Coder, "the first commercial-scale diffusion large language model" It's 5-10x faster than current gen LLMs, providing high-quality responses at low costs. And you can try it now!

Tanishq Mathew Abraham, Ph.D.

354,404 views • 1 year ago

We're moving beyond autoregressive LLMs! Autoregressive LLMs generate text word-by-word, which can be slow and affect quality, while diffusion models refine noise step-by-step, allowing for faster iterations and error correction. Here's Gemini Diffusion running at 857 tokens/s:

We're moving beyond autoregressive LLMs! Autoregressive LLMs generate text word-by-word, which can be slow and affect quality, while diffusion models refine noise step-by-step, allowing for faster iterations and error correction. Here's Gemini Diffusion running at 857 tokens/s:

Akshay 🚀

34,524 views • 1 year ago

Excited to share our interview with Sander Dieleman! This is Part 3 in our History of Diffusion series. We talk about diffusion as spectral autoregression, diffusion language models, flow matching, and much more. Enjoy!

Excited to share our interview with Sander Dieleman! This is Part 3 in our History of Diffusion series. We talk about diffusion as spectral autoregression, diffusion language models, flow matching, and much more. Enjoy!

Slater Stich

28,721 views • 1 year ago

Yann LeCun says we're fooled by LLMs because they manipulate language well, and we associate that with intelligence But language fluency doesn't mean underlying intelligence Every generation since the 1950s claimed its technique was the ticket to human-level AI All were wrong. "this generation with LLMs is also wrong"

Yann LeCun says we're fooled by LLMs because they manipulate language well, and we associate that with intelligence But language fluency doesn't mean underlying intelligence Every generation since the 1950s claimed its technique was the ticket to human-level AI All were wrong. "this generation with LLMs is also wrong"

Haider.

626,144 views • 10 months ago

LLM-grounded Diffusion: Enhancing Prompt Understanding of Text-to-Image Diffusion Models with Large Language Models paper page: github: Recent advancements in text-to-image generation with diffusion models have yielded remarkable results synthesizing highly realistic and diverse images. However, these models still encounter difficulties when generating images from prompts that demand spatial or common sense reasoning. We propose to equip diffusion models with enhanced reasoning capabilities by using off-the-shelf pretrained large language models (LLMs) in a novel two-stage generation process. First, we adapt an LLM to be a text-guided layout generator through in-context learning. When provided with an image prompt, an LLM outputs a scene layout in the form of bounding boxes along with corresponding individual descriptions. Second, we steer a diffusion model with a novel controller to generate images conditioned on the layout. Both stages utilize frozen pretrained models without any LLM or diffusion model parameter optimization. We validate the superiority of our design by demonstrating its ability to outperform the base diffusion model in accurately generating images according to prompts that necessitate both language and spatial reasoning. Additionally, our method naturally allows dialog-based scene specification and is able to handle prompts in a language that is not well-supported by the underlying diffusion model.

LLM-grounded Diffusion: Enhancing Prompt Understanding of Text-to-Image Diffusion Models with Large Language Models paper page: github: Recent advancements in text-to-image generation with diffusion models have yielded remarkable results synthesizing highly realistic and diverse images. However, these models still encounter difficulties when generating images from prompts that demand spatial or common sense reasoning. We propose to equip diffusion models with enhanced reasoning capabilities by using off-the-shelf pretrained large language models (LLMs) in a novel two-stage generation process. First, we adapt an LLM to be a text-guided layout generator through in-context learning. When provided with an image prompt, an LLM outputs a scene layout in the form of bounding boxes along with corresponding individual descriptions. Second, we steer a diffusion model with a novel controller to generate images conditioned on the layout. Both stages utilize frozen pretrained models without any LLM or diffusion model parameter optimization. We validate the superiority of our design by demonstrating its ability to outperform the base diffusion model in accurately generating images according to prompts that necessitate both language and spatial reasoning. Additionally, our method naturally allows dialog-based scene specification and is able to handle prompts in a language that is not well-supported by the underlying diffusion model.

AK

83,681 views • 3 years ago

We're hiring research scientists in diffusion / generative models at World Labs! Check out the role here: Feel free to DM me if you would like to chat!👋

We're hiring research scientists in diffusion / generative models at World Labs! Check out the role here: Feel free to DM me if you would like to chat!👋

Chao-Yuan Wu

190,939 views • 1 year ago

Today's autoregressive models generate one token at a time. Mercury 2 generates tokens in parallel. Over 1,000 tok/sec on standard GPUs, at comparable quality to speed-optimized models. Since launch, the community has been showing what diffusion LLMs can unlock. Thanks to the team at Clyep for the breakdown.

Today's autoregressive models generate one token at a time. Mercury 2 generates tokens in parallel. Over 1,000 tok/sec on standard GPUs, at comparable quality to speed-optimized models. Since launch, the community has been showing what diffusion LLMs can unlock. Thanks to the team at Clyep for the breakdown.

Inception

21,231 views • 5 months ago

Mercury 2 is live 🚀🚀 The world’s first reasoning diffusion LLM, delivering 5x faster performance than leading speed-optimized LLMs. Watching the team turn years of research into a real product never gets old, and I’m incredibly proud of what we’ve built. We’re just getting started on what diffusion can do for language.

Mercury 2 is live 🚀🚀 The world’s first reasoning diffusion LLM, delivering 5x faster performance than leading speed-optimized LLMs. Watching the team turn years of research into a real product never gets old, and I’m incredibly proud of what we’ve built. We’re just getting started on what diffusion can do for language.

Stefano Ermon

1,052,119 views • 8 months ago

Block Diffusion Interpolating Between Autoregressive and Diffusion Language Models

Block Diffusion Interpolating Between Autoregressive and Diffusion Language Models

AK

160,698 views • 1 year ago

I asked 6 agents in two conditions (parallel v/s sequential) to create the same tower defence game 👇 LLMs are known to have mode collapse, i.e. if you have multiple parallel runs on the same prompt, you get very similar responses. Is there a way to increase diversity? Yes, it turns out that if you do such multiple runs **sequentially**, instead of in parallel. (Plus for each sequential run, append results from previous runs and instruct the LLM to create something different from before.) I did this quick experiment to explore parallel v/s sequential runs for diversity. Given the same prompt to build a tower defence game, sequential runs had a much higher diversity and innovation v/s parallel runs. Check out this (AI-generated) video for gameplay clips. The sequential cohort created interesting new dynamics and visual style, while parallel ones all collapsed to similar colors and orbital style dynamics.

I asked 6 agents in two conditions (parallel v/s sequential) to create the same tower defence game 👇 LLMs are known to have mode collapse, i.e. if you have multiple parallel runs on the same prompt, you get very similar responses. Is there a way to increase diversity? Yes, it turns out that if you do such multiple runs sequentially, instead of in parallel. (Plus for each sequential run, append results from previous runs and instruct the LLM to create something different from before.) I did this quick experiment to explore parallel v/s sequential runs for diversity. Given the same prompt to build a tower defence game, sequential runs had a much higher diversity and innovation v/s parallel runs. Check out this (AI-generated) video for gameplay clips. The sequential cohort created interesting new dynamics and visual style, while parallel ones all collapsed to similar colors and orbital style dynamics.

Paras Chopra

11,592 views • 2 months ago

try out the Gradio Demo for AudioLDM: Text-to-Audio Generation with Latent Diffusion Models on Hugging Face demo:

try out the Gradio Demo for AudioLDM: Text-to-Audio Generation with Latent Diffusion Models on Hugging Face demo:

AK

82,138 views • 3 years ago

From the inside out, it’s built different. JB explains how the new EXPS3-HD raises the bar for durability, speed, and control—delivering next-generation holographic performance in a heavy-duty design built to withstand real-world abuse. Learn more here: #EOTECH

From the inside out, it’s built different. JB explains how the new EXPS3-HD raises the bar for durability, speed, and control—delivering next-generation holographic performance in a heavy-duty design built to withstand real-world abuse. Learn more here: #EOTECH

EOTECH

21,242 views • 9 months ago

“Next-token prediction” just got a serious rival 🤯 Ant Group just dropped LLaDA 2.1, and it challenges the dominant paradigm of LLMs. Unlike most models that generate one token at a time, LLaDA 2.1 uses diffusion to generate blocks of text in parallel. Why this changes everything: → Global Planning: It effectively sees the "future" while writing the "past" → Parallel Generation: It generates chunks in parallel, not sequentially, being much faster → Massive Efficiency: 16B MoE architecture that only uses ~1.4B active parameters per step. 100% Open Source.

“Next-token prediction” just got a serious rival 🤯 Ant Group just dropped LLaDA 2.1, and it challenges the dominant paradigm of LLMs. Unlike most models that generate one token at a time, LLaDA 2.1 uses diffusion to generate blocks of text in parallel. Why this changes everything: → Global Planning: It effectively sees the "future" while writing the "past" → Parallel Generation: It generates chunks in parallel, not sequentially, being much faster → Massive Efficiency: 16B MoE architecture that only uses ~1.4B active parameters per step. 100% Open Source.

Alvaro Cintas

31,277 views • 8 months ago