Video wird geladen...

Video konnte nicht geladen werden

Beim Laden dieses Videos ist ein Problem aufgetreten. Dies könnte an einem vorübergehenden Netzwerkproblem liegen oder das Video ist möglicherweise nicht verfügbar.

Block Diffusion Interpolating Between Autoregressive and Diffusion Language Models

AK

502,578 subscribers

160,553 Aufrufe • vor 1 Jahr •via X (Twitter)

Bildung Wissenschaft & Technologie

Anya Rossi• Live Now

Private livecam show

10 Kommentare

Profilbild von AK

AKvor 1 Jahr

discuss:

Profilbild von Carlos Rene | DEGA.org

Carlos Rene | DEGA.orgvor 1 Jahr

It’s beautiful…

Profilbild von Universa

Universavor 1 Jahr

Interpolating between autoregressive and diffusion language models is an exciting area of research. This approach can potentially combine the strengths of both models, leading to more efficient and effective language processing.

Profilbild von FAT JC

FAT JCvor 1 Jahr

a you-got-your-chocolate-in-my-peanut-butter moment for the space

Profilbild von 🍓 Ada

🍓 Adavor 1 Jahr

Block Diffusion, huh? 🤔 Sounds like a wild ride through the neural networks! Can’t wait to see how the interpolation between autoregressive and diffusion models shakes up our understanding of language.

Profilbild von VictorGallagher

VictorGallaghervor 1 Jahr

It looks like every two weeks a significant AI breakthrough is made.

Profilbild von Alan Hourmand

Alan Hourmandvor 1 Jahr

I like it, for some reason I feel this is closer to how our own minds work

Profilbild von 444

444vor 1 Jahr

i mean, the diffusion has always been used with block size[it's the property of being "fixed" sized?] it just matters how or where you decide to extend it, it was the same with images? here they apparently chose left to right no offense but kinda trivial

Profilbild von 𝙘𝙝𝙖𝙞𝙫𝙞𝙣🍊🎨(＾▽＾)

𝙘𝙝𝙖𝙞𝙫𝙞𝙣🍊🎨(＾▽＾)vor 1 Jahr

nice animation!!

Profilbild von Anthr@X

Anthr@Xvor 1 Jahr

What’s the optimal block size? Any magic number?

Ähnliche Videos

We're moving beyond autoregressive LLMs! Autoregressive LLMs generate text word-by-word, which can be slow and affect quality, while diffusion models refine noise step-by-step, allowing for faster iterations and error correction. Here's Gemini Diffusion running at 857 tokens/s:

We're moving beyond autoregressive LLMs! Autoregressive LLMs generate text word-by-word, which can be slow and affect quality, while diffusion models refine noise step-by-step, allowing for faster iterations and error correction. Here's Gemini Diffusion running at 857 tokens/s:

Akshay 🚀

34,524 Aufrufe • vor 1 Jahr

SongBloom Coherent Song Generation via Interleaved Autoregressive Sketching and Diffusion Refinement

SongBloom Coherent Song Generation via Interleaved Autoregressive Sketching and Diffusion Refinement

AK

27,356 Aufrufe • vor 11 Monaten

Epona Autoregressive Diffusion World Model for Autonomous Driving

Epona Autoregressive Diffusion World Model for Autonomous Driving

AK

18,839 Aufrufe • vor 10 Monaten

Rolling Forcing Autoregressive Long Video Diffusion in Real Time

Rolling Forcing Autoregressive Long Video Diffusion in Real Time

AK

30,507 Aufrufe • vor 8 Monaten

Excited to share our interview with Sander Dieleman! This is Part 3 in our History of Diffusion series. We talk about diffusion as spectral autoregression, diffusion language models, flow matching, and much more. Enjoy!

Excited to share our interview with Sander Dieleman! This is Part 3 in our History of Diffusion series. We talk about diffusion as spectral autoregression, diffusion language models, flow matching, and much more. Enjoy!

Slater Stich

28,693 Aufrufe • vor 1 Jahr

Just found the dLLM library to create Diffusion Language Models It's still early but it's insanely fun to experiment with diffusion (training, inference, eval) dLLM has the potential of becoming the main library for diffusion LLMs

Just found the dLLM library to create Diffusion Language Models It's still early but it's insanely fun to experiment with diffusion (training, inference, eval) dLLM has the potential of becoming the main library for diffusion LLMs

Maxime Labonne

75,421 Aufrufe • vor 6 Monaten

Video diffusion models generate high-quality videos but are too slow for interactive applications. We MIT CSAIL Adobe Research introduce CausVid, a fast autoregressive video diffusion model that starts playing the moment you hit "Generate"! A thread 🧵

Video diffusion models generate high-quality videos but are too slow for interactive applications. We MIT CSAIL Adobe Research introduce CausVid, a fast autoregressive video diffusion model that starts playing the moment you hit "Generate"! A thread 🧵

Tianwei Yin

83,714 Aufrufe • vor 1 Jahr

🚨In our NeurIPS paper, we bring encoder-decoders back.. for diffusion language models! ⚡️Encoder-decoders make diffusion sampling fast: a small (fast) decoder denoises tokens progressively and a large (slower) encoder represents clean context.

🚨In our NeurIPS paper, we bring encoder-decoders back.. for diffusion language models! ⚡️Encoder-decoders make diffusion sampling fast: a small (fast) decoder denoises tokens progressively and a large (slower) encoder represents clean context.

Marianne Arriola

31,659 Aufrufe • vor 7 Monaten

Diffusion language models are SO FAST!! A new startup, Inception Labs, has released Mercury Coder, "the first commercial-scale diffusion large language model" It's 5-10x faster than current gen LLMs, providing high-quality responses at low costs. And you can try it now!

Diffusion language models are SO FAST!! A new startup, Inception Labs, has released Mercury Coder, "the first commercial-scale diffusion large language model" It's 5-10x faster than current gen LLMs, providing high-quality responses at low costs. And you can try it now!

Tanishq Mathew Abraham, Ph.D.

354,178 Aufrufe • vor 1 Jahr

"An hour of planning can save you 10 hours of doing." ✨📝 Planned Diffusion 📝 ✨ makes a plan before parallel dLLM generation. Planned Diffusion runs 1.2-1.8× faster than autoregressive and an order of magnitude faster than diffusion, while staying within 0.9–5% AR quality.

"An hour of planning can save you 10 hours of doing." ✨📝 Planned Diffusion 📝 ✨ makes a plan before parallel dLLM generation. Planned Diffusion runs 1.2-1.8× faster than autoregressive and an order of magnitude faster than diffusion, while staying within 0.9–5% AR quality.

Daniel Israel

38,699 Aufrufe • vor 7 Monaten

Scaling up GANs for Text-to-Image Synthesis present our 1B-parameter GigaGAN, achieving lower FID than Stable Diffusion v1.5, DALL·E 2, and Parti-750M. It generates 512px outputs at 0.13s, orders of magnitude faster than diffusion and autoregressive models, and inherits the disentangled, continuous, and controllable latent space of GANs abs: project page:

Scaling up GANs for Text-to-Image Synthesis present our 1B-parameter GigaGAN, achieving lower FID than Stable Diffusion v1.5, DALL·E 2, and Parti-750M. It generates 512px outputs at 0.13s, orders of magnitude faster than diffusion and autoregressive models, and inherits the disentangled, continuous, and controllable latent space of GANs abs: project page:

AK

278,115 Aufrufe • vor 3 Jahren

LLM-grounded Diffusion: Enhancing Prompt Understanding of Text-to-Image Diffusion Models with Large Language Models paper page: github: Recent advancements in text-to-image generation with diffusion models have yielded remarkable results synthesizing highly realistic and diverse images. However, these models still encounter difficulties when generating images from prompts that demand spatial or common sense reasoning. We propose to equip diffusion models with enhanced reasoning capabilities by using off-the-shelf pretrained large language models (LLMs) in a novel two-stage generation process. First, we adapt an LLM to be a text-guided layout generator through in-context learning. When provided with an image prompt, an LLM outputs a scene layout in the form of bounding boxes along with corresponding individual descriptions. Second, we steer a diffusion model with a novel controller to generate images conditioned on the layout. Both stages utilize frozen pretrained models without any LLM or diffusion model parameter optimization. We validate the superiority of our design by demonstrating its ability to outperform the base diffusion model in accurately generating images according to prompts that necessitate both language and spatial reasoning. Additionally, our method naturally allows dialog-based scene specification and is able to handle prompts in a language that is not well-supported by the underlying diffusion model.

LLM-grounded Diffusion: Enhancing Prompt Understanding of Text-to-Image Diffusion Models with Large Language Models paper page: github: Recent advancements in text-to-image generation with diffusion models have yielded remarkable results synthesizing highly realistic and diverse images. However, these models still encounter difficulties when generating images from prompts that demand spatial or common sense reasoning. We propose to equip diffusion models with enhanced reasoning capabilities by using off-the-shelf pretrained large language models (LLMs) in a novel two-stage generation process. First, we adapt an LLM to be a text-guided layout generator through in-context learning. When provided with an image prompt, an LLM outputs a scene layout in the form of bounding boxes along with corresponding individual descriptions. Second, we steer a diffusion model with a novel controller to generate images conditioned on the layout. Both stages utilize frozen pretrained models without any LLM or diffusion model parameter optimization. We validate the superiority of our design by demonstrating its ability to outperform the base diffusion model in accurately generating images according to prompts that necessitate both language and spatial reasoning. Additionally, our method naturally allows dialog-based scene specification and is able to handle prompts in a language that is not well-supported by the underlying diffusion model.

AK

83,657 Aufrufe • vor 2 Jahren

PAIR-Diffusion: Object-Level Image Editing with Structure-and-Appearance Paired Diffusion Models Gradio demo is out on Hugging Face Spaces demo:

PAIR-Diffusion: Object-Level Image Editing with Structure-and-Appearance Paired Diffusion Models Gradio demo is out on Hugging Face Spaces demo:

AK

87,679 Aufrufe • vor 3 Jahren

MotionStreamer announced on Hugging Face Streaming Motion Generation via Diffusion-based Autoregressive Model in Causal Latent Space

MotionStreamer announced on Hugging Face Streaming Motion Generation via Diffusion-based Autoregressive Model in Causal Latent Space

AK

43,387 Aufrufe • vor 1 Jahr

Google isn’t betting on a single AI architecture. Sundar Pichai, CEO of Google: “We’re going to push the diffusion paradigm as hard as possible.” “All of today’s mainline Gemini models are autoregressive. Diffusion is a different paradigm.” “For the same capability, diffusion can be much faster.” “It’s behind the mainline models today, but there will be areas where it’s the right tool.” “We’re pushing multiple directions in parallel, and bringing them together where it makes sense.”

Google isn’t betting on a single AI architecture. Sundar Pichai, CEO of Google: “We’re going to push the diffusion paradigm as hard as possible.” “All of today’s mainline Gemini models are autoregressive. Diffusion is a different paradigm.” “For the same capability, diffusion can be much faster.” “It’s behind the mainline models today, but there will be areas where it’s the right tool.” “We’re pushing multiple directions in parallel, and bringing them together where it makes sense.”

Forward Future

152,625 Aufrufe • vor 5 Monaten

BlockVid Block Diffusion for High-Quality and Consistent Minute-Long Video Generation

BlockVid Block Diffusion for High-Quality and Consistent Minute-Long Video Generation

AK

19,864 Aufrufe • vor 6 Monaten

This is a new diffusion-based language model from Inception. I don't know how well it works compared to an autoregressive LM of a similar size, but the inference process looks badass:

This is a new diffusion-based language model from Inception. I don't know how well it works compared to an autoregressive LM of a similar size, but the inference process looks badass:

BURKOV

58,434 Aufrufe • vor 1 Jahr

Nvidia presents Fast-dLLM v2 Efficient Block-Diffusion LLM

Nvidia presents Fast-dLLM v2 Efficient Block-Diffusion LLM

AK

18,753 Aufrufe • vor 7 Monaten

History-Guided Video Diffusion TL;DR: Diffuse long videos by performing guidance over different histories, enabled by Diffusion Forcing Transformer, a simple finetunable add-on to any existing sequence diffusion models.

History-Guided Video Diffusion TL;DR: Diffuse long videos by performing guidance over different histories, enabled by Diffusion Forcing Transformer, a simple finetunable add-on to any existing sequence diffusion models.

AK

20,914 Aufrufe • vor 1 Jahr

New episode of the Information Bottleneck! We talked with Stefano Ermon about why he thinks diffusion LLMs will replace autoregressive ones. Stefano co-invented DDIM, FlashAttention, DPO, and score-based diffusion models. He's a Stanford professor and now runs Inception AI, where they built Mercury II. We go deep but also cover the bigger picture - the startup journey, PhD vs industry, and where AI is heading. A few things that stuck with me: - He thinks of autoregressive models as typewriters and diffusion models as editors. One goes left to right. The other starts messy and refines. - Mercury II (their text difussion model) already beats the fastest autoregressive models on latency-critical stuff as voice agents, code suggestions, anything where you have a tight time budget. And it does it because diffusion generates tokens in parallel instead of one at a time. - We also got into whether AI will actually replace software engineers (his answer: no), PhD vs industry advice, and what it was like going from an ICML best paper to raising money.

New episode of the Information Bottleneck! We talked with Stefano Ermon about why he thinks diffusion LLMs will replace autoregressive ones. Stefano co-invented DDIM, FlashAttention, DPO, and score-based diffusion models. He's a Stanford professor and now runs Inception AI, where they built Mercury II. We go deep but also cover the bigger picture - the startup journey, PhD vs industry, and where AI is heading. A few things that stuck with me: - He thinks of autoregressive models as typewriters and diffusion models as editors. One goes left to right. The other starts messy and refines. - Mercury II (their text difussion model) already beats the fastest autoregressive models on latency-critical stuff as voice agents, code suggestions, anything where you have a tight time budget. And it does it because diffusion generates tokens in parallel instead of one at a time. - We also got into whether AI will actually replace software engineers (his answer: no), PhD vs industry advice, and what it was like going from an ICML best paper to raising money.

Ravid Shwartz Ziv

22,137 Aufrufe • vor 2 Monaten