Video yükleniyor...
Video Yüklenemedi
Seedance 2.0 is impressive. But it's closed-source! Introducing our daVinci-MagiHuman — a single-stream 15B Transformer trained from scratch that jointly generates video + audio. No cross-attention. No multi-stream branches. Just self-attention. ⚡ 5s 1080p video in 38s on a single H100 🏆 80% win rate vs Ovi 1.1 |... show more
301,178 görüntüleme • 6 ay önce •via X (Twitter)
39 Yorum

🏗️ Architecture deep dive: A single 15B-param, 40-layer Transformer processes text, video, and audio tokens through self-attention only. 🥪 "Sandwich" design: first & last 4 layers use modality-specific projections, middle 32 layers share params across all modalities. 🚫 No timestep embeddings — the model infers denoising state directly from input latents.

🏎️ Why it's SO fast: ⚡ Latent-space super-res — upscale in latent space, and skip the extra VAE decode-encode round trip 🔄 Turbo VAE Decoder — lightweight retrained decoder slashes decoding overhead 🔧 MagiCompiler — full-graph compilation fuses ops across layers (~1.2x speedup) 🎯 DMD-2 distillation → only 8 denoising steps, no CFG 256p in 2s ⚡ 540p in 8s ⚡ 1080p in 38s All on a single H100 🤯

🧑 Human-centric quality that hits different: 😊 Expressive facial performance 🗣️ Natural speech-expression coordination 🕺 Realistic body motion 🎵 Accurate audio-video sync 🌍 Multilingual: Chinese (Mandarin + Cantonese), English, Japanese, Korean, German, French 📉 WER of just 14.6% — lowest among all competitors

📊 Benchmarks — 2,000 pairwise human comparisons: 🥊 vs Ovi 1.1 → 80.0% win rate 💥 🥊 vs LTX 2.3 → 60.9% win rate 🔥 🎨 Visual Quality: 4.80 (highest) 📝 Text Alignment: 4.18 (highest) 🎙️ WER: 14.60% (lowest) Opensource SOTA across the board 👑

📦 Fully open-source under Apache 2.0 — the FULL stack: ✅ Base model (256p) ✅ Distilled model ✅ Super-resolution model (540p & 1080p) ✅ Inference code 🤗 Models: 🎮 Demo: 💻 Code: 📄 Paper: More videos below! Go build something amazing 🚀

Awesome! Censored or Uncensored?

The american video gen models are so bad that the competition now is between Chinese OSS vs Chinese closed source

Wow what’s going on here

Looks interesting. When are you dropping the weights?

Fully open source. Thank you. This is what is needed to push even further. Seedance 2.0 performs super well with animation styles such as Anime. How does daVinci-MagiHuman perform with different animation styles?

Simplicity is simply better! Lfg to the team👁🥰

Ain't no way. This better be base model because if SR model 1080p then its trash

My seed dance clone attempt based off the paper

Disney, Marvel suing for a reason

This looks really impressive. I plan to give this a spin on my inference rig later today. Nice work, and thank you.

Instant follow, Holy fucking shit, this is amazing.

Is this open weight? @jfischoff can you please look into this?

closed-source is a tough sell. open-source encourages more collaboration and innovation. long-term, it wins.

incredible..

Cool, but it does more than close ups of people talking?

This is amazing. I'm going to try and get it up and running in my Mac mini today

Amazing work! Any plans to build a harness around this?

@llmgram

Single-stream joint video+audio is the right call. Cross-attention between modalities always creates a synchronization bottleneck. Curious how the 14.6% WER holds up with heavy motion sequences though.

Hope quant and open weight release, @UnslothAI new quest added to you

video quality is amazing. For game content creators, having an open alternative matters a lot. The joint video+audio generation is especially interesting for cutscene workflows where dialogue sync is everything

hmm very cool.

才15b?

lfg!! thanks for this guys

this looks really cool! I'd love to try it out

How to run this on @modal

@elevendr69 Can you make a version with a cameo function just like Sora AI? Having to manually search for reference images is SO inconvenient…

这个效果很强

curious about the implications of not using cross-attention or multi-stream branches. does this simplify training significantly, or are there trade-offs in terms of output quality?

Wen at ComfyUi with gguf's? 😅

@ArRENCEAI

The shift just happened

Closeups and limited motion is relatively easy these days. What about hands and high motion?

Interesting — a single-stream 15B Transformer sounds like a unique approach! How do you think this impacts the quality of the generated content compared to models that use multi-stream architectures? Curious if anyone has insights on that.
