Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

MMaDA-Parallel Multimodal Large Diffusion Language Models for Thinking-Aware Editing and Generation

_akhaliq's profile picture

AK

516,961 subscribers

66,158 Aufrufe • vor 11 Monaten •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Models "Block diffusion sequentially generates blocks of tokens by performing diffusion within each block and conditioning on previous blocks. By combining strength from autoregressive and diffusion models, block diffusion overcomes the limitations of both approaches by supporting variable-length, higher-quality generation and improving inference efficiency with KV caching and parallel sampling."
0:19

Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Models "Block diffusion sequentially generates blocks of tokens by performing diffusion within each block and conditioning on previous blocks. By combining strength from autoregressive and diffusion models, block diffusion overcomes the limitations of both approaches by supporting variable-length, higher-quality generation and improving inference efficiency with KV caching and parallel sampling."

Tanishq Mathew Abraham, Ph.D.

21,813 Aufrufe • vor 1 Jahr

We are excited to introduce Mercury, the first commercial-grade diffusion large language model (dLLM)! dLLMs push the frontier of intelligence and speed with parallel, coarse-to-fine text generation.
0:19

We are excited to introduce Mercury, the first commercial-grade diffusion large language model (dLLM)! dLLMs push the frontier of intelligence and speed with parallel, coarse-to-fine text generation.

Inception

1,918,919 Aufrufe • vor 1 Jahr

🚨Announcing our #ICLR2025 Oral! 🔥Diffusion LMs are on the rise for parallel text generation! But unlike autoregressive LMs, they struggle with quality, fixed-length constraints & lack of KV caching. 🚀Introducing Block Diffusion—combining autoregressive and diffusion models for the best of both worlds! 👇1/7
0:17

🚨Announcing our #ICLR2025 Oral! 🔥Diffusion LMs are on the rise for parallel text generation! But unlike autoregressive LMs, they struggle with quality, fixed-length constraints & lack of KV caching. 🚀Introducing Block Diffusion—combining autoregressive and diffusion models for the best of both worlds! 👇1/7

Marianne Arriola

150,803 Aufrufe • vor 1 Jahr

Major AI breakthrough: Diffusion Large Language Models are here! They're 10x faster and 10x cheaper than traditional LLMs. Here's everything you need to know:
0:19

Major AI breakthrough: Diffusion Large Language Models are here! They're 10x faster and 10x cheaper than traditional LLMs. Here's everything you need to know:

Matthew Berman

429,249 Aufrufe • vor 1 Jahr

Show-o One Single Transformer to Unify Multimodal Understanding and Generation discuss: We present a unified transformer, i.e., Show-o, that unifies multimodal understanding and generation. Unlike fully autoregressive models, Show-o unifies autoregressive and (discrete) diffusion modeling to adaptively handle inputs and outputs of various and mixed modalities. The unified model flexibly supports a wide range of vision-language tasks including visual question-answering, text-to-image generation, text-guided inpainting/extrapolation, and mixed-modality generation. Across various benchmarks, it demonstrates comparable or superior performance to existing individual models with an equivalent or larger number of parameters tailored for understanding or generation. This significantly highlights its potential as a next-generation foundation model.
0:13

Show-o One Single Transformer to Unify Multimodal Understanding and Generation discuss: We present a unified transformer, i.e., Show-o, that unifies multimodal understanding and generation. Unlike fully autoregressive models, Show-o unifies autoregressive and (discrete) diffusion modeling to adaptively handle inputs and outputs of various and mixed modalities. The unified model flexibly supports a wide range of vision-language tasks including visual question-answering, text-to-image generation, text-guided inpainting/extrapolation, and mixed-modality generation. Across various benchmarks, it demonstrates comparable or superior performance to existing individual models with an equivalent or larger number of parameters tailored for understanding or generation. This significantly highlights its potential as a next-generation foundation model.

AK

124,085 Aufrufe • vor 2 Jahren

MiniMax H3 is more than a video generation model. Its open-weight release enables developers to build on a powerful multimodal system with generation, editing, and consistency features. Supported by ComfyUI, Diffusers, SGLang Diffusion, and Hugging Face, it simplifies integration and could inspire new AI video applications.
0:15

MiniMax H3 is more than a video generation model. Its open-weight release enables developers to build on a powerful multimodal system with generation, editing, and consistency features. Supported by ComfyUI, Diffusers, SGLang Diffusion, and Hugging Face, it simplifies integration and could inspire new AI video applications.

Sean Wallace

51,777 Aufrufe • vor 2 Monaten

1/ Happy to share UniDisc - Unified Multimodal Discrete Diffusion – We train a 1.5 billion parameter transformer model from scratch on 250 million image/caption pairs using a **discrete diffusion objective**. Our model has all the benefits of diffusion models but now in multimodal space! - flexible compute-quality tradeoff, zero-shot inpainting and editing, better control via classifier-free guidance and lower latency! We open source everything - our code, weights and the training dataset.
0:10

1/ Happy to share UniDisc - Unified Multimodal Discrete Diffusion – We train a 1.5 billion parameter transformer model from scratch on 250 million image/caption pairs using a **discrete diffusion objective**. Our model has all the benefits of diffusion models but now in multimodal space! - flexible compute-quality tradeoff, zero-shot inpainting and editing, better control via classifier-free guidance and lower latency! We open source everything - our code, weights and the training dataset.

Mihir Prabhudesai

105,076 Aufrufe • vor 1 Jahr

NCMs are pretty fast! See the language generation speed vs. autoregressive and diffusion baselines below.
0:11

NCMs are pretty fast! See the language generation speed vs. autoregressive and diffusion baselines below.

Azalia Mirhoseini

10,276 Aufrufe • vor 8 Tagen

gemini omni flash is here: our high-quality, cost-efficient model for video generation and conversational editing designed to support multimodal workflows, it enables you to refine videos using natural language and simple prompting start building with it today via ai studio and the gemini api
0:19

gemini omni flash is here: our high-quality, cost-efficient model for video generation and conversational editing designed to support multimodal workflows, it enables you to refine videos using natural language and simple prompting start building with it today via ai studio and the gemini api

Google AI Studio

976,814 Aufrufe • vor 3 Monaten

DimensionX: Create Any 3D and 4D Scenes from a Single Image with Controllable Video Diffusion TL;DR: Create 3/4DGS from Video Diffusion Note: Some first inference code released (not all yet). Contributions (cited): • We present DimensionX, a novel framework for generating photorealistic 3D and 4D scenes from only a single image using controllable video diffusion. • We propose ST-Director, which decouples the spatial and temporal priors in video diffusion models by learning (spatial and temporal) dimension-aware modules with our curated datasets. We further enhance the hybriddimension control with a training-free composition approach according to the essence of video diffusion denoising process. • To bridge the gap between video diffusion and real-world scenes, we design a trajectory-aware mechanism for 3D generation and an identity-preserving denoising approach for 4D generation, enabling more realistic and controllable scene synthesis. • Extensive experiments manifest that our DimensionX delivers superior performance in video, 3D, and 4D generation compared with baseline methods.
0:18

DimensionX: Create Any 3D and 4D Scenes from a Single Image with Controllable Video Diffusion TL;DR: Create 3/4DGS from Video Diffusion Note: Some first inference code released (not all yet). Contributions (cited): • We present DimensionX, a novel framework for generating photorealistic 3D and 4D scenes from only a single image using controllable video diffusion. • We propose ST-Director, which decouples the spatial and temporal priors in video diffusion models by learning (spatial and temporal) dimension-aware modules with our curated datasets. We further enhance the hybriddimension control with a training-free composition approach according to the essence of video diffusion denoising process. • To bridge the gap between video diffusion and real-world scenes, we design a trajectory-aware mechanism for 3D generation and an identity-preserving denoising approach for 4D generation, enabling more realistic and controllable scene synthesis. • Extensive experiments manifest that our DimensionX delivers superior performance in video, 3D, and 4D generation compared with baseline methods.

MrNeRF

17,062 Aufrufe • vor 1 Jahr

Artificial Intelligence (#AI) models that process and interpret visual data are called large vision models. These models, like large language models, are trained on large amounts of data. #midjourney #dalle #stablediffusion (1/8) 🧵 ⤵️
0:10

Artificial Intelligence (#AI) models that process and interpret visual data are called large vision models. These models, like large language models, are trained on large amounts of data. #midjourney #dalle #stablediffusion (1/8) 🧵 ⤵️

Nuklai

13,614 Aufrufe • vor 2 Jahren

I think my large language models are malfunctioning
0:15

I think my large language models are malfunctioning

Pump.fun

283,487 Aufrufe • vor 1 Jahr

Introducing MiniMax H3, our next-gen open-weight multimodal video model, built for general intelligence beyond single-task generation. H3 understands text, images, video, and audio together, interpreting motion, sound, emotion, and cinematography as one unified creative language, delivering generation with fine-grained precision and cinematic quality. → Native Multimodal Understanding & Generation → Precision Editing & Control → Commercial-Grade Output: film, ads, MVs, UI, game CG → Cinematic Audio-Visual Quality: native stereo audio, up to 2K/24FPS Multi-asset reference for characters, motion, camera movement + voice cloning. Hailuo AI (MiniMax)'s H3 takes ideas from concept to production in one flow. Try it now: #MiniMaxH3
0:15

Introducing MiniMax H3, our next-gen open-weight multimodal video model, built for general intelligence beyond single-task generation. H3 understands text, images, video, and audio together, interpreting motion, sound, emotion, and cinematography as one unified creative language, delivering generation with fine-grained precision and cinematic quality. → Native Multimodal Understanding & Generation → Precision Editing & Control → Commercial-Grade Output: film, ads, MVs, UI, game CG → Cinematic Audio-Visual Quality: native stereo audio, up to 2K/24FPS Multi-asset reference for characters, motion, camera movement + voice cloning. Hailuo AI (MiniMax)'s H3 takes ideas from concept to production in one flow. Try it now: #MiniMaxH3

SARAH

97,618 Aufrufe • vor 2 Monaten

⚡ Video diffusion is slow. PDD changes that. 🚀 Introducing Parallel Decoding Distillation, our latest work on accelerating video diffusion models. Paper: Page: By Neta Shaul, Julius Berner, and Chao Liu. Below PDD on LTX-2.3 🔊
0:10

⚡ Video diffusion is slow. PDD changes that. 🚀 Introducing Parallel Decoding Distillation, our latest work on accelerating video diffusion models. Paper: Page: By Neta Shaul, Julius Berner, and Chao Liu. Below PDD on LTX-2.3 🔊

Arash Vahdat

21,669 Aufrufe • vor 2 Monaten

Massive update for AI Engineers! Training diffusion models just got a lot easier. dLLM is an open-source library that does for diffusion models what Hugging Face did for transformers. Here's why this matters: Traditional autoregressive models generate text left-to-right, one token at a time. Diffusion models work differently - they refine the entire sequence iteratively, giving you better control over generation quality and more flexible editing capabilities. The problem? Building and training these models required stitching together scattered tools and reimplementing research papers from scratch. Most of the tooling has been scattered and is hard to reproduce. dLLM changes this: It unifies everything you need to train, evaluate, and deploy diffusion language models: ↳ Scalable training with LoRA, DeepSpeed, and FSDP support ↳ Unified evaluation that abstracts away inference complexity ↳ Ready-to-use recipes for pretraining, finetuning, and evaluation The library includes implementations of models like LLaDA and Dream, plus training algorithms like Edit Flows that enable insertion, deletion, and substitution operations. The team just released ModernBERT-Chat models showing you can turn BERT into lightweight chatbots through masked instruction tuning. This is practical and worth exploring. The setup is straightforward. Run locally with Accelerate or scale to multi-node clusters with Slurm. If you're working with language models and want to explore diffusion-based approaches without rebuilding infrastructure, dLLM gives you a production-ready starting point. Link to the repo in the next tweet.
0:14

Massive update for AI Engineers! Training diffusion models just got a lot easier. dLLM is an open-source library that does for diffusion models what Hugging Face did for transformers. Here's why this matters: Traditional autoregressive models generate text left-to-right, one token at a time. Diffusion models work differently - they refine the entire sequence iteratively, giving you better control over generation quality and more flexible editing capabilities. The problem? Building and training these models required stitching together scattered tools and reimplementing research papers from scratch. Most of the tooling has been scattered and is hard to reproduce. dLLM changes this: It unifies everything you need to train, evaluate, and deploy diffusion language models: ↳ Scalable training with LoRA, DeepSpeed, and FSDP support ↳ Unified evaluation that abstracts away inference complexity ↳ Ready-to-use recipes for pretraining, finetuning, and evaluation The library includes implementations of models like LLaDA and Dream, plus training algorithms like Edit Flows that enable insertion, deletion, and substitution operations. The team just released ModernBERT-Chat models showing you can turn BERT into lightweight chatbots through masked instruction tuning. This is practical and worth exploring. The setup is straightforward. Run locally with Accelerate or scale to multi-node clusters with Slurm. If you're working with language models and want to explore diffusion-based approaches without rebuilding infrastructure, dLLM gives you a production-ready starting point. Link to the repo in the next tweet.

Akshay 🚀

64,011 Aufrufe • vor 11 Monaten

LLaDA (the first Large Language Diffusion Model) is *just* out 💥 and I've built a demo, try out now 👨‍💻 It's mesmerizing to watch the diffusion process 🌀, and it being a diffusion model gives you superpowers like "the 4th word has to be pineapple" 🦸 Demo and weights 👇
0:17

LLaDA (the first Large Language Diffusion Model) is *just* out 💥 and I've built a demo, try out now 👨‍💻 It's mesmerizing to watch the diffusion process 🌀, and it being a diffusion model gives you superpowers like "the 4th word has to be pineapple" 🦸 Demo and weights 👇

apolinario (poli)

82,599 Aufrufe • vor 1 Jahr

All the big language models under one roof for the very first time 🤯 Compare the output of OpenAI ChatGPT, Anthropic's Claude and Cohere's language model in a single playground!! Check out this amazing tool to get the best of large language models 👇
0:10

All the big language models under one roof for the very first time 🤯 Compare the output of OpenAI ChatGPT, Anthropic's Claude and Cohere's language model in a single playground!! Check out this amazing tool to get the best of large language models 👇

Shubham Saboo

299,259 Aufrufe • vor 3 Jahren

Google is making progress on their diffusion models... It's now as good as a Gemini 2.0 Flash Lite. The writing is on the wall, a majority of language AI use in the future will be diffusion models.
0:14

Google is making progress on their diffusion models... It's now as good as a Gemini 2.0 Flash Lite. The writing is on the wall, a majority of language AI use in the future will be diffusion models.

Carlos E. Perez

193,952 Aufrufe • vor 9 Monaten