Loading video...

Video Failed to Load

Go Home

We just brought flow maps to language modeling for one-step sequence generation 💥 Discrete diffusion is not necessary -- continuous flows over one-hot encodings achieve SoTA performance and ≥8.3× faster generation 🔥 We believe this is a major step forward for discrete generative modeling and language modeling alike. 🚀...

nmboffi's profile picture

Nicholas Boffi

1,918 subscribers

43,721 views • 3 months ago •via X (Twitter)

EducationScience & Technology
LIVE

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

Speed and quality can finally coexist in diffusion-based language generation. Introducing DiDi-Instruct, a Discrete Diffusion Divergence Instruct method that distills a pre-trained discrete diffusion language model (dLLM) into a few-step student for ultra-fast generation. Built on integral KL-divergence minimization, DiDi-Instruct achieves up to 64× faster decoding, surpasses both its teacher and GPT-2, and cuts training time by 20×. Ultra-Fast Language Generation via Discrete Diffusion Divergence Instruct Paper: Code: Project: Our report: 📬 #PapersAccepted by Jiqizhixin
0:40

Speed and quality can finally coexist in diffusion-based language generation. Introducing DiDi-Instruct, a Discrete Diffusion Divergence Instruct method that distills a pre-trained discrete diffusion language model (dLLM) into a few-step student for ultra-fast generation. Built on integral KL-divergence minimization, DiDi-Instruct achieves up to 64× faster decoding, surpasses both its teacher and GPT-2, and cuts training time by 20×. Ultra-Fast Language Generation via Discrete Diffusion Divergence Instruct Paper: Code: Project: Our report: 📬 #PapersAccepted by Jiqizhixin

机器之心 JIQIZHIXIN

18,126 views • 7 months ago

New Kling, Runway, Luma competitor? Open text and image video generation model Pyramidal Flow Matching for Efficient Video Generative Modeling
2:23

New Kling, Runway, Luma competitor? Open text and image video generation model Pyramidal Flow Matching for Efficient Video Generative Modeling

AK

90,965 views • 1 year ago

We present Neural Riemannian Motion Fields (NRMF) as a promising paradigm for generative modeling of articulated motion. Unlike various diffusion based models, NRMF is useful both in generation and in deployment as a prior across tasks: #CVPR #AI #CVML👇
3:59

We present Neural Riemannian Motion Fields (NRMF) as a promising paradigm for generative modeling of articulated motion. Unlike various diffusion based models, NRMF is useful both in generation and in deployment as a prior across tasks: #CVPR #AI #CVML👇

Tolga Birdal

16,624 views • 8 months ago

(1/2) LightIt: Illumination Modeling and Control for Diffusion Models! #CVPR2024 We facilitate lighting control for novel image generation from text prompts. We can also edit lighting for a given input image. Video: Project:
0:31

(1/2) LightIt: Illumination Modeling and Control for Diffusion Models! #CVPR2024 We facilitate lighting control for novel image generation from text prompts. We can also edit lighting for a given input image. Video: Project:

Matthias Niessner

19,835 views • 2 years ago

🚀 Introducing Emu3.5 — a large-scale multimodal world model that natively predicts the next vision-language state. 🔥 Trained on over 10T interleaved vision-language tokens and enhanced with reinforcement learning, Emu3.5 achieves powerful multimodal reasoning and generation. ⚡ Powered by our new Discrete Diffusion Adaptation (DiDA) for 20× faster inference. 🔥 Emu3.5 outperforms Nano Banana across image generation, editing, interleaved tasks and more. 🌍 Explore Emu3.5: Github: #Emu3 #MultimodalAI #WorldModel #NextTokenPrediction
2:10

🚀 Introducing Emu3.5 — a large-scale multimodal world model that natively predicts the next vision-language state. 🔥 Trained on over 10T interleaved vision-language tokens and enhanced with reinforcement learning, Emu3.5 achieves powerful multimodal reasoning and generation. ⚡ Powered by our new Discrete Diffusion Adaptation (DiDA) for 20× faster inference. 🔥 Emu3.5 outperforms Nano Banana across image generation, editing, interleaved tasks and more. 🌍 Explore Emu3.5: Github: #Emu3 #MultimodalAI #WorldModel #NextTokenPrediction

BAAI

51,683 views • 7 months ago

Continuous diffusion had a good run—now it’s time for Discrete diffusion! Introducing Anchored Posterior Sampling (APS) APS outperforms discrete and continuous baselines in terms of performance & scaling on inverse problems, stylization, and text-guided editing.
0:33

Continuous diffusion had a good run—now it’s time for Discrete diffusion! Introducing Anchored Posterior Sampling (APS) APS outperforms discrete and continuous baselines in terms of performance & scaling on inverse problems, stylization, and text-guided editing.

Litu Rout

39,896 views • 8 months ago

Excited to share what my team has been working on lately - Gemini diffusion! We bring diffusion to language modeling, yielding more power and blazing speeds! 🚀🚀🚀 Gemini diffusion is especially strong at coding. In this example the model generates at 2000 tokens/sec, including overheads like tokenization, prefill, safety filters etc.
0:24

Excited to share what my team has been working on lately - Gemini diffusion! We bring diffusion to language modeling, yielding more power and blazing speeds! 🚀🚀🚀 Gemini diffusion is especially strong at coding. In this example the model generates at 2000 tokens/sec, including overheads like tokenization, prefill, safety filters etc.

Brendan O'Donoghue

577,386 views • 1 year ago

Can we simplify video generation by decomposing it into interleaved text-video co-generation? Would explicit, repeated thinking in language improve generation in pixels? We introduce TV2TV: a unified model that jointly learns - language modeling (next-token prediction) - video flow matching (next-frame prediction) At inference, TV2TV dynamically alternates between textual thinking and video generation. Model generations below: interleaved text plans and video slices (~1–2s) are co-generated over time, conditioned on a single frame per sport. 📖
0:36

Can we simplify video generation by decomposing it into interleaved text-video co-generation? Would explicit, repeated thinking in language improve generation in pixels? We introduce TV2TV: a unified model that jointly learns - language modeling (next-token prediction) - video flow matching (next-frame prediction) At inference, TV2TV dynamically alternates between textual thinking and video generation. Model generations below: interleaved text plans and video slices (~1–2s) are co-generated over time, conditioned on a single frame per sport. 📖

Xiaochuang Han

28,925 views • 6 months ago

Was it disrespectful of Sen. John Fetterman to wear shorts to the inauguration? @PeggyNoonanNYC: "I believe that he's in a grown-up business and he should be modeling for young men how we go forward in life." @BillMaher: "He is, he's modeling shorts." #RTOvertime
1:29

Was it disrespectful of Sen. John Fetterman to wear shorts to the inauguration? @PeggyNoonanNYC: "I believe that he's in a grown-up business and he should be modeling for young men how we go forward in life." @BillMaher: "He is, he's modeling shorts." #RTOvertime

Real Time with Bill Maher

127,946 views • 1 year ago

Generative AI is cool and all, but SDF modeling just hits different. Non-destructive shaping, blending and boolean modeling on an iPad using Uniform 1.4
0:55

Generative AI is cool and all, but SDF modeling just hits different. Non-destructive shaping, blending and boolean modeling on an iPad using Uniform 1.4

Bilawal Sidhu

83,479 views • 1 year ago

From being a pure water hawker to modeling for yhemo lee. This is so beautiful 🥹❤️
7:23

From being a pure water hawker to modeling for yhemo lee. This is so beautiful 🥹❤️

𝔸b𝕒𝕫𝕫

32,524 views • 3 months ago

Seed-Music A Unified Framework for High Quality and Controlled Music Generation discuss: We introduce Seed-Music, a suite of music generation systems capable of producing high-quality music with fine-grained style control. Our unified framework leverages both auto-regressive language modeling and diffusion approaches to support two key music creation workflows: controlled music generation and post-production editing. For controlled music generation, our system enables vocal music generation with performance controls from multi-modal inputs, including style descriptions, audio references, musical scores, and voice prompts. For post-production editing, it offers interactive tools for editing lyrics and vocal melodies directly in the generated audio.
2:51

Seed-Music A Unified Framework for High Quality and Controlled Music Generation discuss: We introduce Seed-Music, a suite of music generation systems capable of producing high-quality music with fine-grained style control. Our unified framework leverages both auto-regressive language modeling and diffusion approaches to support two key music creation workflows: controlled music generation and post-production editing. For controlled music generation, our system enables vocal music generation with performance controls from multi-modal inputs, including style descriptions, audio references, musical scores, and voice prompts. For post-production editing, it offers interactive tools for editing lyrics and vocal melodies directly in the generated audio.

AK

32,967 views • 1 year ago

We found a hack to make Whisper transcribe into *any* language (not just English)! ⚡️ This is an EXPERIMENTAL one-step feature: enter your audio as input, and get a transcription in another language. 🔥 Check it out here:
0:32

We found a hack to make Whisper transcribe into *any* language (not just English)! ⚡️ This is an EXPERIMENTAL one-step feature: enter your audio as input, and get a transcription in another language. 🔥 Check it out here:

Vaibhav (VB) Srivastav

74,516 views • 3 years ago

What's the best first step for getting into modeling? Rufus and Captain Jack Andrews converge on creativity and curiosity.
2:41

What's the best first step for getting into modeling? Rufus and Captain Jack Andrews converge on creativity and curiosity.

Unabated

15,814 views • 5 months ago

Introducing Omni, one unified model can support any-to-any multimodal modeling, including multimodal understanding, image/video generation and editing, world modeling and 3D reconstruction. All in one that adopts standard mixture-of-experts arch with only 3B activations.
1:20

Introducing Omni, one unified model can support any-to-any multimodal modeling, including multimodal understanding, image/video generation and editing, world modeling and 3D reconstruction. All in one that adopts standard mixture-of-experts arch with only 3B activations.

Ceyuan Yang

30,956 views • 1 month ago

StreamVLN Streaming Vision-and-Language Navigation via SlowFast Context Modeling
3:27

StreamVLN Streaming Vision-and-Language Navigation via SlowFast Context Modeling

AK

26,650 views • 11 months ago

Are We The Last Generation To Speak Kashmiri? What happens when a generation stops speaking its own language? The Kashmiri language is fading from homes and classrooms. This video highlights the urgent need to preserve Kashmiri before it becomes just a memory #kashmir #LANGUAGE
9:00

Are We The Last Generation To Speak Kashmiri? What happens when a generation stops speaking its own language? The Kashmiri language is fading from homes and classrooms. This video highlights the urgent need to preserve Kashmiri before it becomes just a memory #kashmir #LANGUAGE

Kashmir Observer®

86,393 views • 3 months ago

How to Use Automator on Google Flow for Image Generation — New 2026 Update. 🔥 A step-by-step guide on setting up automated image generation workflows smoothly and efficiently on Google Flow. RT if you gained value. 🚀
5:34

How to Use Automator on Google Flow for Image Generation — New 2026 Update. 🔥 A step-by-step guide on setting up automated image generation workflows smoothly and efficiently on Google Flow. RT if you gained value. 🚀

Mayowa

23,513 views • 1 month ago

New research from Meta FAIR: Large Concept Models (LCM) is a fundamentally different paradigm for language modeling that decouples reasoning from language representation, inspired by how humans can plan high-level thoughts to communicate.
0:36

New research from Meta FAIR: Large Concept Models (LCM) is a fundamentally different paradigm for language modeling that decouples reasoning from language representation, inspired by how humans can plan high-level thoughts to communicate.

AI at Meta

531,432 views • 1 year ago

MotionGPT: Human Motion as a Foreign Language paper page: Though the advancement of pre-trained large language models unfolds, the exploration of building a unified model for language and other multi-modal data, such as motion, remains challenging and untouched so far. Fortunately, human motion displays a semantic coupling akin to human language, often perceived as a form of body language. By fusing language data with large-scale motion models, motion-language pre-training that can enhance the performance of motion-related tasks becomes feasible. Driven by this insight, we propose MotionGPT, a unified, versatile, and user-friendly motion-language model to handle multiple motion-relevant tasks. Specifically, we employ the discrete vector quantization for human motion and transfer 3D motion into motion tokens, similar to the generation process of word tokens. Building upon this "motion vocabulary", we perform language modeling on both motion and text in a unified manner, treating human motion as a specific language. Moreover, inspired by prompt learning, we pre-train MotionGPT with a mixture of motion-language data and fine-tune it on prompt-based question-and-answer tasks. Extensive experiments demonstrate that MotionGPT achieves state-of-the-art performances on multiple motion tasks including text-driven motion generation, motion captioning, motion prediction, and motion in-between.
0:48

MotionGPT: Human Motion as a Foreign Language paper page: Though the advancement of pre-trained large language models unfolds, the exploration of building a unified model for language and other multi-modal data, such as motion, remains challenging and untouched so far. Fortunately, human motion displays a semantic coupling akin to human language, often perceived as a form of body language. By fusing language data with large-scale motion models, motion-language pre-training that can enhance the performance of motion-related tasks becomes feasible. Driven by this insight, we propose MotionGPT, a unified, versatile, and user-friendly motion-language model to handle multiple motion-relevant tasks. Specifically, we employ the discrete vector quantization for human motion and transfer 3D motion into motion tokens, similar to the generation process of word tokens. Building upon this "motion vocabulary", we perform language modeling on both motion and text in a unified manner, treating human motion as a specific language. Moreover, inspired by prompt learning, we pre-train MotionGPT with a mixture of motion-language data and fine-tune it on prompt-based question-and-answer tasks. Extensive experiments demonstrate that MotionGPT achieves state-of-the-art performances on multiple motion tasks including text-driven motion generation, motion captioning, motion prediction, and motion in-between.

AK

125,311 views • 3 years ago