Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

StreamDiT Real-Time Streaming Text-to-Video Generation StreamDiT enables real-time text-to-video generation at 16 FPS on a single GPU (H100)

_akhaliq's profile picture

AK

529,501 subscribers

43,694 Aufrufe • vor 1 Jahr •via X (Twitter)

Wissenschaft & Technologie
LIVE

Anya Rossi• Live Now

Private livecam show

5 Kommentare

Profilbild von AK
AKvor 1 Jahr

discuss with author:

Profilbild von Hidir Yesiltepe
Hidir Yesiltepevor 1 Jahr

It is very promising to see high freq detail loss specific to long-form video generation is tackled with this work. On the other hand, in most of the video examples the “raster pattern” still persists: every ~1s the visual details change. Framepack also had the same problem.

Profilbild von KANDREJ LARPATHY -- e/acc
KANDREJ LARPATHY -- e/accvor 1 Jahr

slowly, then suddenly

Profilbild von Darryl Mason
Darryl Masonvor 1 Jahr

I've tried real-time text-to-music on Google's Music FX DJ and it's an extraordinary experience, adding new prompts as fast as possible, or just leaving the last prompt to see how what unfolds from it. To be able to do this with video now, or soon, will be next level.

Profilbild von GG
GGvor 1 Jahr

speech2text2StreamDiT = Dream Machine

Ähnliche Videos

🚀 We open-sourced LongLive — interactive, real-time long-video generation. 👥Generates video in real time as users enter text prompts. ⚡️20.7 FPS on a single H100,⏱️up to 240s per clip. 🎬Fine-tunes SOTA short-video models (e.g., Wan) into long-video generators. 🌍One step closer to World Models. All code for training & inference, model weights, demo page, and videos released! Paper: Code: Model: Demo Page: Introduction Video:
3:18

🚀 We open-sourced LongLive — interactive, real-time long-video generation. 👥Generates video in real time as users enter text prompts. ⚡️20.7 FPS on a single H100,⏱️up to 240s per clip. 🎬Fine-tunes SOTA short-video models (e.g., Wan) into long-video generators. 🌍One step closer to World Models. All code for training & inference, model weights, demo page, and videos released! Paper: Code: Model: Demo Page: Introduction Video:

Yukang Chen

11,943 Aufrufe • vor 1 Jahr

We present MotionStream — real-time, long-duration video generation that you can interactively control just by dragging your mouse. All videos here are raw, real-time screen captures without any post-processing. Model runs on a single H100 at 29 FPS and 0.4s latency.
0:42

We present MotionStream — real-time, long-duration video generation that you can interactively control just by dragging your mouse. All videos here are raw, real-time screen captures without any post-processing. Model runs on a single H100 at 29 FPS and 0.4s latency.

Xun Huang

99,242 Aufrufe • vor 11 Monaten

🤯Real-time video generation just got HUGE. Introducing Helios: A 14B parameter model running at 19.5 FPS on a single H100. Cheaper, Faster, and Stronger than 1.3B models, generating minute-long videos without the usual tricks (even no quantization).
0:50

🤯Real-time video generation just got HUGE. Introducing Helios: A 14B parameter model running at 19.5 FPS on a single H100. Cheaper, Faster, and Stronger than 1.3B models, generating minute-long videos without the usual tricks (even no quantization).

Bin Lin

27,792 Aufrufe • vor 7 Monaten

Vidu S1, the World’s Most Advanced Model for Real-Time AI Interaction Vidu S1 introduces a new era of real-time interactive video. Voice-Driven Character Control: Go beyond voice-driven lip sync with real-time behavior generation. Characters understand, react, and respond naturally. Unlimited Real-Time Generation: Experience seamless real-time video generation with unlimited interaction duration. 540P at 25 FPS, built for live interaction: Generate video in real time at 540P / 25 FPS, with support for up to 42 FPS for even smoother experiences. Create Custom Characters from Images & Voices: Customize both appearance and voice. Bring humans, anime characters, pets, and more to life with personalized AI characters. Use code VIDUS1 to get started for free.
0:54

Vidu S1, the World’s Most Advanced Model for Real-Time AI Interaction Vidu S1 introduces a new era of real-time interactive video. Voice-Driven Character Control: Go beyond voice-driven lip sync with real-time behavior generation. Characters understand, react, and respond naturally. Unlimited Real-Time Generation: Experience seamless real-time video generation with unlimited interaction duration. 540P at 25 FPS, built for live interaction: Generate video in real time at 540P / 25 FPS, with support for up to 42 FPS for even smoother experiences. Create Custom Characters from Images & Voices: Customize both appearance and voice. Bring humans, anime characters, pets, and more to life with personalized AI characters. Use code VIDUS1 to get started for free.

Vidu AI

1,537,934 Aufrufe • vor 3 Monaten

Ovi is out on Hugging Face Twin Backbone Cross-Modal Fusion for Audio-Video Generation Ovi is a veo-3 like, video+audio generation model that simultaneously generates both video and audio content from text or text+image inputs. Video+Audio Generation: Generate synchronized video and audio content simultaneously Flexible Input: Supports text-only or text+image conditioning 5-second Videos: Generates 5-second videos at 24 FPS, area of 720×720, at various aspect ratios (9:16, 16:9, 1:1, etc)
2:08

Ovi is out on Hugging Face Twin Backbone Cross-Modal Fusion for Audio-Video Generation Ovi is a veo-3 like, video+audio generation model that simultaneously generates both video and audio content from text or text+image inputs. Video+Audio Generation: Generate synchronized video and audio content simultaneously Flexible Input: Supports text-only or text+image conditioning 5-second Videos: Generates 5-second videos at 24 FPS, area of 720×720, at various aspect ratios (9:16, 16:9, 1:1, etc)

AK

23,082 Aufrufe • vor 1 Jahr

LTX-Video new text-to-video model generates stunning, high-quality videos faster than real-time—5 seconds of 24fps video at 768x512 in just 4 seconds on an Nvidia H100 open code & weights
0:26

LTX-Video new text-to-video model generates stunning, high-quality videos faster than real-time—5 seconds of 24fps video at 768x512 in just 4 seconds on an Nvidia H100 open code & weights

AK

51,498 Aufrufe • vor 1 Jahr

VideoComposer brings ControlNet guidance to text and video-to-video. The model enables to combine multiple modalities like text, sketch, style and even motion to drive video generation. The results look extremely good.
2:40

VideoComposer brings ControlNet guidance to text and video-to-video. The model enables to combine multiple modalities like text, sketch, style and even motion to drive video generation. The results look extremely good.

Dreaming Tulpa 🥓👑

18,045 Aufrufe • vor 3 Jahren

Introducing SDXL Turbo: A real-time text-to-image generation model. SDXL Turbo achieves state-of-the-art performance with a new distillation technology, enabling single-step image generation with unprecedented quality, reducing the required step count from 50 to just one. The code, research paper, and weights for non-commercial use are now available on our website. You can test SDXL Turbo on Stability AI’s image editing platform Clipdrop, with a beta demonstration of the real-time text-to-image generation capabilities. Learn more:
0:23

Introducing SDXL Turbo: A real-time text-to-image generation model. SDXL Turbo achieves state-of-the-art performance with a new distillation technology, enabling single-step image generation with unprecedented quality, reducing the required step count from 50 to just one. The code, research paper, and weights for non-commercial use are now available on our website. You can test SDXL Turbo on Stability AI’s image editing platform Clipdrop, with a beta demonstration of the real-time text-to-image generation capabilities. Learn more:

Stability AI

977,000 Aufrufe • vor 2 Jahren

Helios Real Real-Time Long Video Generation Model paper:
1:13

Helios Real Real-Time Long Video Generation Model paper:

AK

25,951 Aufrufe • vor 7 Monaten

🚀Generate a 30-second 1080p video in just 7 seconds! We’re open-sourcing FastVideo Dreamverse: real-time vibe directing for video generation on a single NVIDIA B200 GPU with LTX-2 model LTX Repo: Blog:
0:38

🚀Generate a 30-second 1080p video in just 7 seconds! We’re open-sourcing FastVideo Dreamverse: real-time vibe directing for video generation on a single NVIDIA B200 GPU with LTX-2 model LTX Repo: Blog:

Hao AI Lab

227,318 Aufrufe • vor 4 Monaten

Can we simplify video generation by decomposing it into interleaved text-video co-generation? Would explicit, repeated thinking in language improve generation in pixels? We introduce TV2TV: a unified model that jointly learns - language modeling (next-token prediction) - video flow matching (next-frame prediction) At inference, TV2TV dynamically alternates between textual thinking and video generation. Model generations below: interleaved text plans and video slices (~1–2s) are co-generated over time, conditioned on a single frame per sport. 📖
0:36

Can we simplify video generation by decomposing it into interleaved text-video co-generation? Would explicit, repeated thinking in language improve generation in pixels? We introduce TV2TV: a unified model that jointly learns - language modeling (next-token prediction) - video flow matching (next-frame prediction) At inference, TV2TV dynamically alternates between textual thinking and video generation. Model generations below: interleaved text plans and video slices (~1–2s) are co-generated over time, conditioned on a single frame per sport. 📖

Xiaochuang Han

32,709 Aufrufe • vor 10 Monaten

Apparate Labs launched PROTEUS, a new real-time AI video generation model. It creates realistic avatars and lip-syncs from a single reference image, similar to VASA-1, but it's completely real-time.
1:05

Apparate Labs launched PROTEUS, a new real-time AI video generation model. It creates realistic avatars and lip-syncs from a single reference image, similar to VASA-1, but it's completely real-time.

Rowan Cheung

13,253 Aufrufe • vor 2 Jahren

CineMaster A 3D-Aware and Controllable Framework for Cinematic Text-to-Video Generation
1:43

CineMaster A 3D-Aware and Controllable Framework for Cinematic Text-to-Video Generation

AK

18,843 Aufrufe • vor 1 Jahr

⚡️Generating 3DGS scenes in 5 seconds on a single GPU⚡️ #FlashWorld enables ⚡️*fast*⚡️ (10~100x faster than previous methods) and 🔥*high-quality*🔥 3D world generation, from a single image or text prompt. Code: Page:
0:56

⚡️Generating 3DGS scenes in 5 seconds on a single GPU⚡️ #FlashWorld enables ⚡️*fast*⚡️ (10~100x faster than previous methods) and 🔥*high-quality*🔥 3D world generation, from a single image or text prompt. Code: Page:

Tengfei Wang

119,877 Aufrufe • vor 1 Jahr

Collaborative Video Diffusion Consistent Multi-video Generation with Camera Control Research on video generation has recently made tremendous progress, enabling high-quality videos to be generated from text prompts or images. Adding control to the video generation
2:16

Collaborative Video Diffusion Consistent Multi-video Generation with Camera Control Research on video generation has recently made tremendous progress, enabling high-quality videos to be generated from text prompts or images. Adding control to the video generation

AK

29,278 Aufrufe • vor 2 Jahren

Excited to announce LTX-Video! Our new text-to-video model generates stunning, high-quality videos faster than real-time—5 seconds of 24fps video at 768x512 in just 4 seconds on an Nvidia H100! ⚡ We’re open-sourcing the code & weights. Check out the results 🎥👇
0:26

Excited to announce LTX-Video! Our new text-to-video model generates stunning, high-quality videos faster than real-time—5 seconds of 24fps video at 768x512 in just 4 seconds on an Nvidia H100! ⚡ We’re open-sourcing the code & weights. Check out the results 🎥👇

Yoav HaCohen

50,427 Aufrufe • vor 1 Jahr

Introducing RTFM (Real-Time Frame Model): a highly efficient World Model that generates video frames in real time as you interact with it, powered by a single H100 GPU. RTFM renders persistent and 3D consistent worlds, both real and imaginary. Try our demo of RTFM today!
2:21

Introducing RTFM (Real-Time Frame Model): a highly efficient World Model that generates video frames in real time as you interact with it, powered by a single H100 GPU. RTFM renders persistent and 3D consistent worlds, both real and imaginary. Try our demo of RTFM today!

World Labs

341,993 Aufrufe • vor 1 Jahr

🤩Excited to share SANA-WM: a 2.6B open-source world model for minute-scale 720p video generation. Given one image + text + a 6-DoF camera trajectory, it synthesizes action-controllable 60s worlds on a single GPU. Project: Paper:
1:14

🤩Excited to share SANA-WM: a 2.6B open-source world model for minute-scale 720p video generation. Given one image + text + a 6-DoF camera trajectory, it synthesizes action-controllable 60s worlds on a single GPU. Project: Paper:

Haoyi Zhu

124,608 Aufrufe • vor 5 Monaten