Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

We just brought flow maps to language modeling for one-step sequence generation 💥 Discrete diffusion is not necessary -- continuous flows over one-hot encodings achieve SoTA performance and ≥8.3× faster generation 🔥 We believe this is a major step forward for discrete generative modeling and language modeling alike. 🚀...

nmboffi's profile picture

Nicholas Boffi

2,332 subscribers

44,714 Aufrufe • vor 8 Monaten •via X (Twitter)

BildungWissenschaft & Technologie
LIVE

Anya Rossi• Live Now

Private livecam show

8 Kommentare

Profilbild von Nicholas Boffi
Nicholas Boffivor 8 Monaten

Beyond our core empirical results, we map out the complete design space of flow maps for discrete data, opening the door to a suite of new algorithms: 📐 Progressive / Shortcut-style approaches 🌊 Eulerian / Mean Flow-style approaches 🎯 Lagrangian / Terminal Velocity Matching approaches

Profilbild von Nicholas Boffi
Nicholas Boffivor 8 Monaten

Tagging some folks who might be interested 👇 @sedielem @_albertgu @zicokolter @RickyTQChen @max_simchowitz @StefanoErmon @DrYangSong @msalbergo @FrankNoeBerlin @electronickale @DaniloJRezende @karsten_kreis @bose_joey @lipmanya @ZhengyangGeng @baaadas @jaschasd @poolio @ValentinDeBort1 @ArashVahdat @du_yilun @emiel_hoogeboom @dpkingma @robrombach @DavidDuvenaud

Profilbild von Nicholas Boffi
Nicholas Boffivor 8 Monaten

@_albertgu @zicokolter @RickyTQChen @max_simchowitz @StefanoErmon @DrYangSong @msalbergo @FrankNoeBerlin @electronickale @DaniloJRezende @karsten_kreis @bose_joey @lipmanya @ZhengyangGeng @baaadas @jaschasd @poolio further info below: 📰arXiv: 📷project page: 📷code:

Profilbild von Gregor
Gregorvor 8 Monaten

Don't these continuous flows introduce a whole new layer of complexity in terms of understanding and debugging the generated sequences? Are there any efforts being made to provide more interpretability of these continuous flows, or is that a trade-off for the increased speed and performance?

Profilbild von Nicholas Boffi
Nicholas Boffivor 8 Monaten

can you explain what you mean a bit further? it's actually quite a lot simpler than the standard CTMC formulation of a discrete diffusion!

Profilbild von Weijian Luo @ CVPR26 Denver
Weijian Luo @ CVPR26 Denvervor 7 Monaten

Congrats, Nicholas! I read the paper and found it pretty constructive, particularly the boundary condition under the CE loss part.

Profilbild von Laurence Wu
Laurence Wuvor 8 Monaten

I thought the results should be evaluated on benchmark datasets like gsm8k minerva_math right

Profilbild von Nicholas Boffi
Nicholas Boffivor 8 Monaten

LM1B and OWT are pretty standard here -- those are math datasets.

Ähnliche Videos

Images, video, audio, actions — generative modeling has converged on one recipe: compress into continuous latents, generate in latent space, decode. Language is the lone exception: still generated token by token, as a long discrete stream. Latent Thought Flows: We compress 256 text tokens into 8 continuous latents, generate them with a one-step flow model, and read them out as text using an autoregressive decoder. Result: a better inference-compute vs. generation-quality Pareto frontier than a tuned autoregressive baseline — thanks to compression and one-step generation. Co-led w/ Zhengyang Geng 🧵
1:02

Images, video, audio, actions — generative modeling has converged on one recipe: compress into continuous latents, generate in latent space, decode. Language is the lone exception: still generated token by token, as a long discrete stream. Latent Thought Flows: We compress 256 text tokens into 8 continuous latents, generate them with a one-step flow model, and read them out as text using an autoregressive decoder. Result: a better inference-compute vs. generation-quality Pareto frontier than a tuned autoregressive baseline — thanks to compression and one-step generation. Co-led w/ Zhengyang Geng 🧵

Mihir Prabhudesai

83,120 Aufrufe • vor 3 Monaten

Speed and quality can finally coexist in diffusion-based language generation. Introducing DiDi-Instruct, a Discrete Diffusion Divergence Instruct method that distills a pre-trained discrete diffusion language model (dLLM) into a few-step student for ultra-fast generation. Built on integral KL-divergence minimization, DiDi-Instruct achieves up to 64× faster decoding, surpasses both its teacher and GPT-2, and cuts training time by 20×. Ultra-Fast Language Generation via Discrete Diffusion Divergence Instruct Paper: Code: Project: Our report: 📬 #PapersAccepted by Jiqizhixin
0:40

Speed and quality can finally coexist in diffusion-based language generation. Introducing DiDi-Instruct, a Discrete Diffusion Divergence Instruct method that distills a pre-trained discrete diffusion language model (dLLM) into a few-step student for ultra-fast generation. Built on integral KL-divergence minimization, DiDi-Instruct achieves up to 64× faster decoding, surpasses both its teacher and GPT-2, and cuts training time by 20×. Ultra-Fast Language Generation via Discrete Diffusion Divergence Instruct Paper: Code: Project: Our report: 📬 #PapersAccepted by Jiqizhixin

机器之心 JIQIZHIXIN

18,196 Aufrufe • vor 11 Monaten

New Kling, Runway, Luma competitor? Open text and image video generation model Pyramidal Flow Matching for Efficient Video Generative Modeling
2:23

New Kling, Runway, Luma competitor? Open text and image video generation model Pyramidal Flow Matching for Efficient Video Generative Modeling

AK

91,250 Aufrufe • vor 2 Jahren

We present Neural Riemannian Motion Fields (NRMF) as a promising paradigm for generative modeling of articulated motion. Unlike various diffusion based models, NRMF is useful both in generation and in deployment as a prior across tasks: #CVPR #AI #CVML👇
3:59

We present Neural Riemannian Motion Fields (NRMF) as a promising paradigm for generative modeling of articulated motion. Unlike various diffusion based models, NRMF is useful both in generation and in deployment as a prior across tasks: #CVPR #AI #CVML👇

Tolga Birdal

16,795 Aufrufe • vor 1 Jahr

(1/2) LightIt: Illumination Modeling and Control for Diffusion Models! #CVPR2024 We facilitate lighting control for novel image generation from text prompts. We can also edit lighting for a given input image. Video: Project:
0:31

(1/2) LightIt: Illumination Modeling and Control for Diffusion Models! #CVPR2024 We facilitate lighting control for novel image generation from text prompts. We can also edit lighting for a given input image. Video: Project:

Matthias Niessner

19,849 Aufrufe • vor 2 Jahren

3Blue1Brown (grant sanderson) asked a jane street researcher whether people modelling markets think about time as continuous rather than discrete. "when you're modeling real world time series, they're not always so cleanly discrete, or as regular." "i do also think when you're modeling real world physical phenomena it is actually quite nice to say time is continuous. we just sample it at some points that we care for." in language the index is the whole object. token 1, token 2, token 3. there is no duration between tokens unless you add one. market data comes with a clock attached. quotes and trades arrive when they arrive, a 1m or daily bar is a grid you laid over them afterwards, and the waiting time between two events carries information: intensity, liquidity, whether anything is happening at all.
1:08

3Blue1Brown (grant sanderson) asked a jane street researcher whether people modelling markets think about time as continuous rather than discrete. "when you're modeling real world time series, they're not always so cleanly discrete, or as regular." "i do also think when you're modeling real world physical phenomena it is actually quite nice to say time is continuous. we just sample it at some points that we care for." in language the index is the whole object. token 1, token 2, token 3. there is no duration between tokens unless you add one. market data comes with a clock attached. quotes and trades arrive when they arrive, a 1m or daily bar is a grid you laid over them afterwards, and the waiting time between two events carries information: intensity, liquidity, whether anything is happening at all.

Goshawk Trades

176,670 Aufrufe • vor 2 Monaten

🚀 Introducing Emu3.5 — a large-scale multimodal world model that natively predicts the next vision-language state. 🔥 Trained on over 10T interleaved vision-language tokens and enhanced with reinforcement learning, Emu3.5 achieves powerful multimodal reasoning and generation. ⚡ Powered by our new Discrete Diffusion Adaptation (DiDA) for 20× faster inference. 🔥 Emu3.5 outperforms Nano Banana across image generation, editing, interleaved tasks and more. 🌍 Explore Emu3.5: Github: #Emu3 #MultimodalAI #WorldModel #NextTokenPrediction
2:10

🚀 Introducing Emu3.5 — a large-scale multimodal world model that natively predicts the next vision-language state. 🔥 Trained on over 10T interleaved vision-language tokens and enhanced with reinforcement learning, Emu3.5 achieves powerful multimodal reasoning and generation. ⚡ Powered by our new Discrete Diffusion Adaptation (DiDA) for 20× faster inference. 🔥 Emu3.5 outperforms Nano Banana across image generation, editing, interleaved tasks and more. 🌍 Explore Emu3.5: Github: #Emu3 #MultimodalAI #WorldModel #NextTokenPrediction

BAAI

51,880 Aufrufe • vor 1 Jahr

Continuous diffusion had a good run—now it’s time for Discrete diffusion! Introducing Anchored Posterior Sampling (APS) APS outperforms discrete and continuous baselines in terms of performance & scaling on inverse problems, stylization, and text-guided editing.
0:33

Continuous diffusion had a good run—now it’s time for Discrete diffusion! Introducing Anchored Posterior Sampling (APS) APS outperforms discrete and continuous baselines in terms of performance & scaling on inverse problems, stylization, and text-guided editing.

Litu Rout

40,244 Aufrufe • vor 1 Jahr

🚀 Introducing Next-Concept Modeling (NCM): our new paradigm for fast, parallel, coarse-to-fine text generation. Compared to strong diffusion language models (DLMs) at similar parameter size, NCM achieves: ⚡ Up to 31× faster generation on downstream tasks like summarization and machine translation with comparable or better quality. 🏎️ 10–21× faster unconditional generation across TinyStories, LM1B, and OpenWebText, with comparable or better generative perplexity and using 3–5× fewer training tokens. 🧩 Better performance on reasoning tasks like Sudoku, outperforming both AR and diffusion baselines with fewer sampling steps. With Ishan Khare, Iddah Mlauzi, Kelly Buchanan, Simran Arora, Azalia Mirhoseini, and Hazy Research: Strip Mall AI Research Club. More details in the thread 🧵👇
0:21

🚀 Introducing Next-Concept Modeling (NCM): our new paradigm for fast, parallel, coarse-to-fine text generation. Compared to strong diffusion language models (DLMs) at similar parameter size, NCM achieves: ⚡ Up to 31× faster generation on downstream tasks like summarization and machine translation with comparable or better quality. 🏎️ 10–21× faster unconditional generation across TinyStories, LM1B, and OpenWebText, with comparable or better generative perplexity and using 3–5× fewer training tokens. 🧩 Better performance on reasoning tasks like Sudoku, outperforming both AR and diffusion baselines with fewer sampling steps. With Ishan Khare, Iddah Mlauzi, Kelly Buchanan, Simran Arora, Azalia Mirhoseini, and Hazy Research: Strip Mall AI Research Club. More details in the thread 🧵👇

Hermann Kumbong

493,602 Aufrufe • vor 12 Tagen

Excited to share what my team has been working on lately - Gemini diffusion! We bring diffusion to language modeling, yielding more power and blazing speeds! 🚀🚀🚀 Gemini diffusion is especially strong at coding. In this example the model generates at 2000 tokens/sec, including overheads like tokenization, prefill, safety filters etc.
0:24

Excited to share what my team has been working on lately - Gemini diffusion! We bring diffusion to language modeling, yielding more power and blazing speeds! 🚀🚀🚀 Gemini diffusion is especially strong at coding. In this example the model generates at 2000 tokens/sec, including overheads like tokenization, prefill, safety filters etc.

Brendan O'Donoghue

578,099 Aufrufe • vor 1 Jahr

Can we simplify video generation by decomposing it into interleaved text-video co-generation? Would explicit, repeated thinking in language improve generation in pixels? We introduce TV2TV: a unified model that jointly learns - language modeling (next-token prediction) - video flow matching (next-frame prediction) At inference, TV2TV dynamically alternates between textual thinking and video generation. Model generations below: interleaved text plans and video slices (~1–2s) are co-generated over time, conditioned on a single frame per sport. 📖
0:36

Can we simplify video generation by decomposing it into interleaved text-video co-generation? Would explicit, repeated thinking in language improve generation in pixels? We introduce TV2TV: a unified model that jointly learns - language modeling (next-token prediction) - video flow matching (next-frame prediction) At inference, TV2TV dynamically alternates between textual thinking and video generation. Model generations below: interleaved text plans and video slices (~1–2s) are co-generated over time, conditioned on a single frame per sport. 📖

Xiaochuang Han

32,818 Aufrufe • vor 10 Monaten

Introducing 💫 Gumbel Straight Flow 💫: Distilling Autoregressive Models into One-step Flow Maps 🚀 Can autoregressive (language) models be used as teachers for flow map parallel samplers?
0:32

Introducing 💫 Gumbel Straight Flow 💫: Distilling Autoregressive Models into One-step Flow Maps 🚀 Can autoregressive (language) models be used as teachers for flow map parallel samplers?

David Ruhe

24,518 Aufrufe • vor 24 Tagen

Generative AI is cool and all, but SDF modeling just hits different. Non-destructive shaping, blending and boolean modeling on an iPad using Uniform 1.4
0:55

Generative AI is cool and all, but SDF modeling just hits different. Non-destructive shaping, blending and boolean modeling on an iPad using Uniform 1.4

Bilawal Sidhu

83,493 Aufrufe • vor 1 Jahr

Seed-Music A Unified Framework for High Quality and Controlled Music Generation discuss: We introduce Seed-Music, a suite of music generation systems capable of producing high-quality music with fine-grained style control. Our unified framework leverages both auto-regressive language modeling and diffusion approaches to support two key music creation workflows: controlled music generation and post-production editing. For controlled music generation, our system enables vocal music generation with performance controls from multi-modal inputs, including style descriptions, audio references, musical scores, and voice prompts. For post-production editing, it offers interactive tools for editing lyrics and vocal melodies directly in the generated audio.
2:51

Seed-Music A Unified Framework for High Quality and Controlled Music Generation discuss: We introduce Seed-Music, a suite of music generation systems capable of producing high-quality music with fine-grained style control. Our unified framework leverages both auto-regressive language modeling and diffusion approaches to support two key music creation workflows: controlled music generation and post-production editing. For controlled music generation, our system enables vocal music generation with performance controls from multi-modal inputs, including style descriptions, audio references, musical scores, and voice prompts. For post-production editing, it offers interactive tools for editing lyrics and vocal melodies directly in the generated audio.

AK

32,967 Aufrufe • vor 2 Jahren

From being a pure water hawker to modeling for yhemo lee. This is so beautiful 🥹❤️
7:23

From being a pure water hawker to modeling for yhemo lee. This is so beautiful 🥹❤️

𝔸b𝕒𝕫𝕫

32,615 Aufrufe • vor 8 Monaten

We found a hack to make Whisper transcribe into *any* language (not just English)! ⚡️ This is an EXPERIMENTAL one-step feature: enter your audio as input, and get a transcription in another language. 🔥 Check it out here:
0:32

We found a hack to make Whisper transcribe into *any* language (not just English)! ⚡️ This is an EXPERIMENTAL one-step feature: enter your audio as input, and get a transcription in another language. 🔥 Check it out here:

Vaibhav (VB) Srivastav

74,525 Aufrufe • vor 3 Jahren

What's the best first step for getting into modeling? Rufus and Captain Jack Andrews converge on creativity and curiosity.
2:41

What's the best first step for getting into modeling? Rufus and Captain Jack Andrews converge on creativity and curiosity.

Unabated

15,817 Aufrufe • vor 9 Monaten

Introducing Omni, one unified model can support any-to-any multimodal modeling, including multimodal understanding, image/video generation and editing, world modeling and 3D reconstruction. All in one that adopts standard mixture-of-experts arch with only 3B activations.
1:20

Introducing Omni, one unified model can support any-to-any multimodal modeling, including multimodal understanding, image/video generation and editing, world modeling and 3D reconstruction. All in one that adopts standard mixture-of-experts arch with only 3B activations.

Ceyuan Yang

32,971 Aufrufe • vor 6 Monaten

Are We The Last Generation To Speak Kashmiri? What happens when a generation stops speaking its own language? The Kashmiri language is fading from homes and classrooms. This video highlights the urgent need to preserve Kashmiri before it becomes just a memory #kashmir #LANGUAGE
9:00

Are We The Last Generation To Speak Kashmiri? What happens when a generation stops speaking its own language? The Kashmiri language is fading from homes and classrooms. This video highlights the urgent need to preserve Kashmiri before it becomes just a memory #kashmir #LANGUAGE

Kashmir Observer®

87,370 Aufrufe • vor 8 Monaten