Most video models 🤯forget the past 🐌slow down over... time 🔁rely on bidirectional (not causal) attention Our state-space video world models (SSM) 🧠remember across hundreds of frames ⚡️generate at constant speed ⏩is fully causal, enabling real-time rollout 1/3show more

The context size of video world models is only... a few frames. Like a human with severe memory loss! We design a long-term memory for world models based on explicit 3D representations inspired by the human mind. This enables long-term consistency. 1/3show more

(1/n) 🚀 With FastVideo, you can now generate a... 5-second video in 5 seconds on a single H200 GPU! Introducing FastWan series, a family of fast video generation models trained via a new recipe we term as “sparse distillation”, to speed up video denoising time by 70X! 🖥️ Live demo: (Thanks to @gmicloud for the support!) 🔗 Blog: 🔓 We fully open-source our models, code, and data with Apache-2.0 licensesshow more

NVIDIA just released a very impressive text-to-video paper. Video... Latent Diffusion Models (Video LDMs) use a diffusion model in a compressed latent space to generate high-resolution videos. Here's a brief overview of how it works: 1. Pre-train image LDM on a dataset of images. 2. Turn the image LDM into a Video LDM by adding temporal layers to model video frames. 3. Fine-tune the Video LDM on encoded video sequences to create a video generator. 4. Temporally align diffusion model upsamplers to generate high-resolution videos. 5. Validate Video LDM on real driving videos of 512x1024 resolution, achieving state-of-the-art performance. 6. Apply the approach in creative content creation with text-to-video modeling. Paper: Project:show more

Video generation is powerful but too slow for real-world... robotic tasks. How can we enable both video and action generation while ensuring real-time policy inference? Check out our work on the Unified Video Action Model (UVA) to find out! (1/7)show more

Introducing Find the best local models based on real... data 1. People run and submit benchmarks 2. Stats aggregated over models / devices 3. Find the best model for you `npx whatcanirun`, fully open-sourceshow more

🤩Video world models are cool, but it is cooler... if they can simulate any 3D physical actions in real time! Introducing RealWonder⚡️: Now you can simulate 3D physical action (robot actions, 3D forces, force fields, etc.) consequences from a single image in real time! 🧵1/6show more

New AI video model MotionStream lets you control motion... in real time. This is a glimpse at what all video models will allow you to do in the future.show more

Matrix-Game 2.0 — The FIRST open-source, real-time, long-sequence interactive... world model Last week, DeepMind's Genie 3 shook the AI world with real-time interactive world models. But... it wasn't open-sourced. Today, Matrix-Game 2.0 changed the game. 🚀 25FPS. Minutes-long interaction. Fully open-source.show more

The team at Runway is pushing the frontier of... what's possible in world models. We're proud to work with them to power real-time inference for Runway Characters.show more

Every search, recommendation, simulation and AI-generated response depends on... rapid access to #data.​​ From training large AI models to enabling real-time inference, #DRAM helps keep today's most demanding workloads moving at the speed innovation requires.​​ This National DRAM day, join us in celebrating the technology that helps bridge the gap between processors and data, enabling everything from smartphones and PCs to #AI infrastructure.show more

Video understanding is the next frontier, but not all... videos are alike. Models now reason over youtube clips and feature films, but what about the everyday spaces we—and our future AI assistants—navigate and experience? Introducing Thinking in Space, our latest study exploring how multimodal LLMs see, remember and recall spaces. 🧵[1/n]show more

This video is in real time 🤯

🤔Applying Depth Estimation models directly to videos can result... in inconsistency between frames. 💪Well, not anymore. 🔥ChronoDepth is a new approach to video depth estimation that focuses on achieving both accuracy within each frame and consistency across frames. 📜More👇show more

Rerender A Video: Zero-Shot Text-Guided Video-to-Video Translation paper page:... Large text-to-image diffusion models have exhibited impressive proficiency in generating high-quality images. However, when applying these models to video domain, ensuring temporal consistency across video frames remains a formidable challenge. This paper proposes a novel zero-shot text-guided video-to-video translation framework to adapt image models to videos. The framework includes two parts: key frame translation and full video translation. The first part uses an adapted diffusion model to generate key frames, with hierarchical cross-frame constraints applied to enforce coherence in shapes, textures and colors. The second part propagates the key frames to other frames with temporal-aware patch matching and frame blending. Our framework achieves global style and local texture temporal consistency at a low cost (without re-training or optimization). The adaptation is compatible with existing image diffusion techniques, allowing our framework to take advantage of them, such as customizing a specific subject with LoRA, and introducing extra spatial guidance with ControlNet. Extensive experimental results demonstrate the effectiveness of our proposed framework over existing methods in rendering high-quality and temporally-coherent videos.show more

We are open-sourcing TTSKit! Run state-of-the-art text-to-speech models on... your Mac and iPhone. The launch version supports Qwen Qwen3-TTS and generates audio faster than real-time playback with sub-200 ms time-to-first-byte. Voice cloning and advanced speed optimizations will be in the next version. Link to the GitHub repo and models on Hugging Face in comments.show more

Grok ranks #1 across many major AI coding platforms,... with multi‑trillion token usage and the biggest lead over rival models in real‑world developer trafficshow more

Our models run real time on a laptop 5090... at 100+ FPS. We're so excited for people to start playing with it.show more

🚀 Self-speculation brings 6.75x real speedup for LLM generation... with SGLang inference! Same model drafts future tokens in Diffusion mode → then verifies them in AR (causal) mode. One model and one KV cache. Just different attention masks. Thanks to perfect alignment, we get 2× longer acceptance lengths than MTP techniques (Eagle-3, MTP, dFlash). We run 2 forward passes… but the 2× higher acceptance means we break even - and with zero overhead from extra drafter, KV cache, or LM head that comes with MTP - those are not free. Last week we released Nemotron-Labs-Diffusion + Tri-mode LLMs! We did continued pre-training on Ministral-3 models by switching attention patterns (block causal bidirectional). Result: one model that runs AR mode, Diffusion mode, and Self-Speculation. Diffusion mode already shows high benchmark accuracy - excited to see what happens when someone beats left-to-right acceptance! 🔥 Github: Paper: SGLang inference: Try the models on HF:show more

Predicting how the environment changes over time is a... key task in robotics, enabling all kinds of downstream tasks. Trace Anything is a very interesting look at how you can train models which predict the trajectory each point will take in the future -- you can use this for robot manipulation, video prediction, and more. Learn more ->show more