Video yükleniyor...

Video Yüklenemedi

Bu video yüklenirken bir sorun oluştu. Bu geçici bir ağ sorunundan kaynaklanıyor olabilir veya video kullanılamıyor olabilir.

Ana Sayfaya Dön

🚀 Excited to introduce StreamRender-H3 — bringing real-time interactive worlds to life with diffusion models! Built on Sol-Engine, we post-trained MiniMax (official) H3 Ref2VA into a real-time streaming neural renderer. 🎮 Code drives the world. A game engine handles physics, object states, and player interactions, while H3 generates the... visuals. ⚡ Real-time streaming: 2 denoising steps, 24 FPS, and ~273 ms per chunk on 8×GB200 GPUs. 🎨 One world, endless appearances: Change the reference image and prompt to render the same interactive world in entirely different styles. 🔥 Powered by streaming post-training, distillation, a fast causal H3 VAE, and full-stack inference acceleration. Check out our demo and code! 👇 demo: code:show more

Enze Xie

3,338 subscribers

68,819 görüntüleme • 3 gün önce •via X (Twitter)

Oyun Bilim & Teknoloji

Anya Rossi• Live Now

Private livecam show

35 Yorum

Nilin profil fotoğrafı

Nilin3 gün önce

@MiniMax_AI Is 8xGB200 a good starter setup for gaming? jk impressive results

Enze Xie profil fotoğrafı

Enze Xie3 gün önce

@MiniMax_AI ideally should run this on RTX series GPU~

Tyler Palmer profil fotoğrafı

Tyler Palmer3 gün önce

@Scobleizer @MiniMax_AI Build your own gta 6 before gta 6

NI4NI profil fotoğrafı

NI4NI3 gün önce

@MiniMax_AI Impressive, real time on >$500.000 of GPU though🫤 DLSS5 is promising I can run GTA V in 4K50 with 2x DLSS5 on my RTX 4070Ti Super, maybe the future is low quality 3D with AI filters.

Liam Liang Ding profil fotoğrafı

Liam Liang Ding3 gün önce

@MiniMax_AI Cool step towards programmable world👏

Simon profil fotoğrafı

Simon2 gün önce

@MiniMax_AI Sol-Engine and 2 denoising steps and you still need 8x GB200?

Enze Xie profil fotoğrafı

Enze Xie2 gün önce

@MiniMax_AI yes, because H3 is large(33B) and latency should be as low as possible

Simon profil fotoğrafı

Simon2 gün önce

@MiniMax_AI Can't you get a similar result with for example 8x 5090s if latency is the main issue?

Baduz - Action Adventure Game Maker profil fotoğrafı

Baduz - Action Adventure Game Maker2 gün önce

@MiniMax_AI Great work - we're working on a similar tech, will soon be released on @BaduzGame and work in the browser!

Junsong_Chen profil fotoğrafı

Junsong_Chen3 gün önce

@MiniMax_AI

Oliver Mackenzie profil fotoğrafı

Oliver Mackenzie2 gün önce

@MiniMax_AI Amazing work!

Gavin Guo profil fotoğrafı

Gavin Guo3 gün önce

@MiniMax_AI super cool!

Enze Xie profil fotoğrafı

Enze Xie3 gün önce

@MiniMax_AI Will Muse have streaming version？🤔

Gavin Guo profil fotoğrafı

Gavin Guo3 gün önce

@MiniMax_AI stay tuned!

Deep Bhalerao profil fotoğrafı

Deep Bhalerao3 gün önce

@MiniMax_AI sounds very very cool

Andreas Bryant profil fotoğrafı

Andreas Bryant3 gün önce

@MiniMax_AI Well done. Over time, this may be superior to traditional game rendering, especially alongside controller models.

Fastest Ghoda profil fotoğrafı

Fastest Ghoda3 gün önce

@MiniMax_AI this is amazing Will love to read on it

Adria Blancafort profil fotoğrafı

Adria Blancafort2 gün önce

@MiniMax_AI This makes a lot of sense

stride profil fotoğrafı

stride3 gün önce

@MiniMax_AI amazing!

Anthony K Nairn, PhD profil fotoğrafı

Anthony K Nairn, PhD2 gün önce

@MiniMax_AI Neat

安叫兽|Bird🕊️ 🔶 BNB profil fotoğrafı

安叫兽|Bird🕊️ 🔶 BNB3 gün önce

@MiniMax_AI 快速转视角的时候，画面能跟得上吗？

Manu_TechAndGames profil fotoğrafı

Manu_TechAndGames2 gün önce

@MiniMax_AI How do you handle coherency? Like will everything be the same for the second round of the race, even if coming from a different angle, and point of view ? Or that all the other cars stay exactly the same for the whole experience?

Amani Kiruga profil fotoğrafı

Amani Kiruga2 gün önce

@MiniMax_AI How do you get around MiniMax strict usage license in the US?

Johnny Núñez profil fotoğrafı

Johnny Núñez2 gün önce

@MiniMax_AI amazing!

Keshav profil fotoğrafı

Keshav2 gün önce

@MiniMax_AI 👏👏👏

Huge Brain Labs profil fotoğrafı

Huge Brain Labs2 gün önce

@MiniMax_AI Doesn’t it take 8xGb200 GPUs? 💰

Subsidy Bro profil fotoğrafı

Subsidy Bro2 gün önce

@MiniMax_AI @MoshiMoshiMoan cant wait to buy these skins on the in-game store 😂

Zishan Shao profil fotoğrafı

Zishan Shao2 gün önce

@MiniMax_AI Amazing Work!

Udayan Maurya profil fotoğrafı

Udayan Maurya2 gün önce

@MiniMax_AI super cool! I run @glitchlive_ AI livestreaming platform. We are looking for a faster and cheaper way to render realtime streams. Can I use this model as a service?

Lank profil fotoğrafı

Lank2 gün önce

@MiniMax_AI Letting the game engine own physics and H3 only paint the frames is the right split. 273 ms per chunk on 8×GB200 is the number I'll be watching shrink.

Evia AI profil fotoğrafı

Evia AI2 gün önce

@MiniMax_AI This real-time neural renderer concept is insane

FrankStain profil fotoğrafı

FrankStain2 gün önce

@MiniMax_AI 8×GB200 GPUs. easier to generate the code that then renders on a more normal GPU

Agentsfy profil fotoğrafı

Agentsfy2 gün önce

@MiniMax_AI This is sick.

埃迪Eddie profil fotoğrafı

埃迪Eddie3 gün önce

@MiniMax_AI 请问有做 inverse-cfg 吗？还是直接训？

AcOrP profil fotoğrafı

AcOrP2 gün önce

@MiniMax_AI Giga Impressive, sadly the HW it requires make it not very practical

Benzer Videolar

🚀 We open-sourced LongLive — interactive, real-time long-video generation. 👥Generates video in real time as users enter text prompts. ⚡️20.7 FPS on a single H100,⏱️up to 240s per clip. 🎬Fine-tunes SOTA short-video models (e.g., Wan) into long-video generators. 🌍One step closer to World Models. All code for training & inference, model weights, demo page, and videos released! Paper: Code: Model: Demo Page: Introduction Video:

🚀 We open-sourced LongLive — interactive, real-time long-video generation. 👥Generates video in real time as users enter text prompts. ⚡️20.7 FPS on a single H100,⏱️up to 240s per clip. 🎬Fine-tunes SOTA short-video models (e.g., Wan) into long-video generators. 🌍One step closer to World Models. All code for training & inference, model weights, demo page, and videos released! Paper: Code: Model: Demo Page: Introduction Video:

Yukang Chen

11,943 görüntüleme • 1 yıl önce

🚀 Sol-H3: MiniMax (official) H3 Video Generation Faster Than Playback 🤩 Five seconds of world. 1.653 seconds to infer. We’re releasing Sol-H3, our fastest end-to-end MiniMax-H3 inference stack yet. On one 8× NVIDIA B300 Blackwell system, it generates five seconds of 1344×768 video with stereo audio in 1.653 seconds. Across 1×, 4×, and 8× B300, Sol-H3 reaches up to a 15.54× speedup versus Base H3. Compared with 50-step Base H3 Dense on the same 8× B300 system, the four-step Sol-H3 profile delivers: • 5s: 18.250s → 1.653s (11.04×) • 10s: 50.660s → 3.732s (13.57×) • 15s: 99.513s → 6.612s (15.05×) Sol-H3 also scales across GPU counts: • 4× B300: 2.918s / 6.993s / 12.542s for 5s / 10s / 15s (12.11–15.54×) • 1× B300: 13.745s / 37.813s / 52.260s for 5s / 10s / 15s (9.45–14.29×) All figures are medians of three measured runs after one warmup at 1344×768 and 24 FPS with stereo audio. Base H3 uses 50 scheduler points (49 DiT forwards); Sol-H3 uses four DiT forwards, so this is a full-profile comparison—not an attention-only runtime change. Sol-H3 uses Dense attention on 1× B300 and SOL with INT8 QKV / FP8 output transport on 4× / 8×. Timing includes text encoding, DiT denoising, and video/audio VAE decoding; model loading, compilation warmup, and final MP4 encoding are excluded. Sol-H3 brings Sol-Engine × Sol-Attn into one full-stack runtime: • dynamic sparse attention with no retraining • fused norm, RoPE, MLP, and sparse-attention setup • fused INT8 QKV / FP8 output communication across 8 GPUs • parallel, batched VAE decoding • precomputed AdaLN caching Inside the stack: • sparse-attention setup: 1.206 → 0.285 ms (−76.4%) • VAE decode: 7.55 → 0.602 s • ~24 GB memory freed per GPU Any MiniMax-H3 few-step LoRA can plug into the same engine, and the code is deployment-friendly under Apache 2.0. For us, the bigger milestone is crossing from “fast generation” into “faster than playback.” That opens the path toward continuous 24 FPS generation and truly interactive video systems. We’re excited to partner with reactor to release Sol-H3 and make it available as an API day-0. Try it now on Reactor: 🔗 Amazing team effort—full credits in the blog. LoveSy Junsong_Chen yitong li Haopeng Li Haocheng Xi 🚗 COLM Song Han

Enze Xie

208,148 görüntüleme • 1 ay önce

We are thrilled to announce "Layered Diffusion Brushes": a real-time training-free image editor powered by diffusion models. 🎨✨ This is new work from my PhD student Peyman Gholami Peyman Gholami. Explore the interactive demo and check out more videos at:

We are thrilled to announce "Layered Diffusion Brushes": a real-time training-free image editor powered by diffusion models. 🎨✨ This is new work from my PhD student Peyman Gholami Peyman Gholami. Explore the interactive demo and check out more videos at:

Robert Xiao

14,334 görüntüleme • 2 yıl önce

🚀 Excited to release LongLive 2.0! 🎬 An end-to-end infrastructure for long video generation, with FP4 and parallelism at the core of both training and inference. ⚡45.7 FPS generation speed on 5B model⚡ ✨ LongLive 2.0 supports real-video training, few-step distillation, multi-shot training/inference, sequence-parallel acceleration, NVFP4 KV cache, and async VAE decoding deployment. 🧩 To our knowledge, this is the first open-source 4-bit long video generation infra that covers both training and inference. 🙌 Welcome to check it out, try it, and share feedback! 🔗 Code: 📰 Paper: 🎥 Demo: #LongVideoGeneration #VideoGeneration #Realtime #AIInfra #EfficientAI #FP4 #Parallel #NVIDIA

🚀 Excited to release LongLive 2.0! 🎬 An end-to-end infrastructure for long video generation, with FP4 and parallelism at the core of both training and inference. ⚡45.7 FPS generation speed on 5B model⚡ ✨ LongLive 2.0 supports real-video training, few-step distillation, multi-shot training/inference, sequence-parallel acceleration, NVFP4 KV cache, and async VAE decoding deployment. 🧩 To our knowledge, this is the first open-source 4-bit long video generation infra that covers both training and inference. 🙌 Welcome to check it out, try it, and share feedback! 🔗 Code: 📰 Paper: 🎥 Demo: #LongVideoGeneration #VideoGeneration #Realtime #AIInfra #EfficientAI #FP4 #Parallel #NVIDIA

Yukang Chen

60,010 görüntüleme • 4 ay önce

🚀🚀🚀Introducing HY World 1.5 (WorldPlay)! We have now open-sourced the most systemized, comprehensive real-time world model framework in the industry. In HY World 1.5, we develop WorldPlay, a streaming video diffusion model that enables real-time, interactive world modeling with long-term geometric consistency, resolving the trade-off between speed and memory that limits current methods. You can generate and explore 3D worlds simply by inputting text or images. Walk, look around, and interact like you're playing a game. Highlights: 🔹Real-Time: Generates long-horizon streaming video at 24 FPS with superior consistency. 🔹Geometric Consistency: Achieved using a Reconstituted Context Memory mechanism to dynamically rebuild context from past frames to alleviate memory attenuation 🔹Robust Control: Uses a Dual Action Representation for robust response to user keyboard and mouse inputs. 🔹Versatile Applications: Supports both first-person and third-person perspectives, enabling applications like promptable events and infinite world extension. 👉🏻Try it now: 🌐Project Page: 🔗Github: 🤗Hugging Face: 📄Technical Report:

🚀🚀🚀Introducing HY World 1.5 (WorldPlay)! We have now open-sourced the most systemized, comprehensive real-time world model framework in the industry. In HY World 1.5, we develop WorldPlay, a streaming video diffusion model that enables real-time, interactive world modeling with long-term geometric consistency, resolving the trade-off between speed and memory that limits current methods. You can generate and explore 3D worlds simply by inputting text or images. Walk, look around, and interact like you're playing a game. Highlights: 🔹Real-Time: Generates long-horizon streaming video at 24 FPS with superior consistency. 🔹Geometric Consistency: Achieved using a Reconstituted Context Memory mechanism to dynamically rebuild context from past frames to alleviate memory attenuation 🔹Robust Control: Uses a Dual Action Representation for robust response to user keyboard and mouse inputs. 🔹Versatile Applications: Supports both first-person and third-person perspectives, enabling applications like promptable events and infinite world extension. 👉🏻Try it now: 🌐Project Page: 🔗Github: 🤗Hugging Face: 📄Technical Report:

Tencent Hy

189,802 görüntüleme • 10 ay önce

Meet LingBot-World 2.0.🌍A 1.3B world model, running continuous interactive generation in real time on a single consumer GPU. 🤖 ⚡ It responds to movement, camera control, combat, and text-driven events in real time, while the scene keeps unfolding instead of ending as a fixed video clip. 🌌 Causal pretraining and MoBA help preserve textures, geometry, and scene consistency over long autoregressive rollouts. 🧠 Consistency and distribution-matching distillation compress multi-step generation and train the model on its own rollouts, reducing accumulated drift. 🛠️ The 14B Causal Pretrain backbone and Bidirectional teacher are also open, giving developers a path to post-train, distill, compress, and adapt new world models. 🔒 CC BY-NC-SA 4.0. Non-commercial use only.

Meet LingBot-World 2.0.🌍A 1.3B world model, running continuous interactive generation in real time on a single consumer GPU. 🤖 ⚡ It responds to movement, camera control, combat, and text-driven events in real time, while the scene keeps unfolding instead of ending as a fixed video clip. 🌌 Causal pretraining and MoBA help preserve textures, geometry, and scene consistency over long autoregressive rollouts. 🧠 Consistency and distribution-matching distillation compress multi-step generation and train the model on its own rollouts, reducing accumulated drift. 🛠️ The 14B Causal Pretrain backbone and Bidirectional teacher are also open, giving developers a path to post-train, distill, compress, and adapt new world models. 🔒 CC BY-NC-SA 4.0. Non-commercial use only.

ModelScope

33,630 görüntüleme • 1 ay önce

Introducing MIRA. A playable, multiplayer world model. A dream of Rocket League. Trained on 10k hours of data collected with publicly available bots, MIRA learns the dynamics of a four-player game. The model runs in real time at 20 fps, based on the keys you and the other players press. Built by General Intuition and kyutai, in collaboration with Epic Games. Not used to develop Rocket League. ▶️ Play the demo, read the technical report, and explore the open-source code at At ICML? Find us at Booth 111 to try it yourself and dig into the results with the team.

Introducing MIRA. A playable, multiplayer world model. A dream of Rocket League. Trained on 10k hours of data collected with publicly available bots, MIRA learns the dynamics of a four-player game. The model runs in real time at 20 fps, based on the keys you and the other players press. Built by General Intuition and kyutai, in collaboration with Epic Games. Not used to develop Rocket League. ▶️ Play the demo, read the technical report, and explore the open-source code at At ICML? Find us at Booth 111 to try it yourself and dig into the results with the team.

General Intuition

507,926 görüntüleme • 3 ay önce

At Avalon we are building "Real-time creating" - the ability to generate gameplay ready persistent worlds prompted from text. While others are building real-time video world models, Avalon is building real-time world generation inside a fully playable, persistent multiplayer engine. Internally running at 3840×2180 at 60 FPS. Built on Unreal Engine. Multiplayer by default. Persistent by default. Gameplay-ready by default. This is not a video latent replay. Not a simulation of interaction. It is a real 3D world with physics, logic, and authoritative multiplayer state. Avalon is trained on proprietary Avalon interaction data and powered by a hybrid system that combines language understanding, 3D model generation, procedural systems, and structured gameplay logic synthesis. Players can walk through a live world and generate environments, assets, mechanics, and entirely new gameplay modes using natural language. We accomplish this through a combination of 3D model generation, game logic generation based on our proprietary systems, and AI driven world creation. While other players are inside it. Changes persist instantly. State is synchronized in real time. Creation happens inside the world, not outside of it. Describe a biome. Spawn a civilization. Create a survival mode. Build a dungeon crawler. Launch a new game inside the world. Avalon interprets intent and integrates it directly into the live multiplayer environment. This is not a world model predicting video. This is a gameplay engine that understands language. If you can describe it, you can build it. And others can walk into it instantly.

At Avalon we are building "Real-time creating" - the ability to generate gameplay ready persistent worlds prompted from text. While others are building real-time video world models, Avalon is building real-time world generation inside a fully playable, persistent multiplayer engine. Internally running at 3840×2180 at 60 FPS. Built on Unreal Engine. Multiplayer by default. Persistent by default. Gameplay-ready by default. This is not a video latent replay. Not a simulation of interaction. It is a real 3D world with physics, logic, and authoritative multiplayer state. Avalon is trained on proprietary Avalon interaction data and powered by a hybrid system that combines language understanding, 3D model generation, procedural systems, and structured gameplay logic synthesis. Players can walk through a live world and generate environments, assets, mechanics, and entirely new gameplay modes using natural language. We accomplish this through a combination of 3D model generation, game logic generation based on our proprietary systems, and AI driven world creation. While other players are inside it. Changes persist instantly. State is synchronized in real time. Creation happens inside the world, not outside of it. Describe a biome. Spawn a civilization. Create a survival mode. Build a dungeon crawler. Launch a new game inside the world. Avalon interprets intent and integrates it directly into the live multiplayer environment. This is not a world model predicting video. This is a gameplay engine that understands language. If you can describe it, you can build it. And others can walk into it instantly.

AVALON

66,204 görüntüleme • 8 ay önce

World models are moving beyond offline generation towards interactive, real-time experiences. Introducing ⚡FlashDreams⚡: an open-source high-performance inference and serving library built for autoregressive world models: 🔥 Up to 3.10× faster LingBot-World inference 🔥 Up to 2.12× faster Self-Forcing inference 🔥 Up to 1.40× faster Wan2.1 inference 🔥 8 integrated models 🔥 Multi-GPU, streaming, low-latency serving 🔥 Agentic skills that teach you how to use it FlashDreams is designed for a new generation of AI systems that continuously evolve over time while responding to user interactions. It powers applications across robotics, autonomous vehicle simulation, gaming, and virtual worlds. Github: Docs: Research page: Join the #flashdreams Discord channel at FlashDreams is also the runtime backbone behind NVIDIA OmniDreams ( 1/n #AI #WorldModels #FastInference #PhysicalAI #OpenSource #NVIDIA

World models are moving beyond offline generation towards interactive, real-time experiences. Introducing ⚡FlashDreams⚡: an open-source high-performance inference and serving library built for autoregressive world models: 🔥 Up to 3.10× faster LingBot-World inference 🔥 Up to 2.12× faster Self-Forcing inference 🔥 Up to 1.40× faster Wan2.1 inference 🔥 8 integrated models 🔥 Multi-GPU, streaming, low-latency serving 🔥 Agentic skills that teach you how to use it FlashDreams is designed for a new generation of AI systems that continuously evolve over time while responding to user interactions. It powers applications across robotics, autonomous vehicle simulation, gaming, and virtual worlds. Github: Docs: Research page: Join the #flashdreams Discord channel at FlashDreams is also the runtime backbone behind NVIDIA OmniDreams ( 1/n #AI #WorldModels #FastInference #PhysicalAI #OpenSource #NVIDIA

Ruilong Li

100,075 görüntüleme • 4 ay önce

A real-time world simulator, now in 0.5B. ABot-World-0.5B-LF brings action-conditioned interactive rollout to ModelScope. 🎮License: Apache 2.0. 🤖 ⚡ Desktop-ready rollout: reported at 720p, 16 FPS, 1.2s latency, and 19GB GPU memory on a single-desktop setup. 🌍 Infinite interaction: responds to user actions continuously and keeps generating beyond fixed video-length limits. 🎬 LongForcing training helps the model expand new scenes and dynamics during rollout, without prompt switching or scene lock-in. 🛠️ Built on Wan2.2-TI2V-5B as a causal student model for lower-latency interactive world rollout.

A real-time world simulator, now in 0.5B. ABot-World-0.5B-LF brings action-conditioned interactive rollout to ModelScope. 🎮License: Apache 2.0. 🤖 ⚡ Desktop-ready rollout: reported at 720p, 16 FPS, 1.2s latency, and 19GB GPU memory on a single-desktop setup. 🌍 Infinite interaction: responds to user actions continuously and keeps generating beyond fixed video-length limits. 🎬 LongForcing training helps the model expand new scenes and dynamics during rollout, without prompt switching or scene lock-in. 🛠️ Built on Wan2.2-TI2V-5B as a causal student model for lower-latency interactive world rollout.

ModelScope

19,642 görüntüleme • 3 ay önce

World Engine Turns Memes Into Living Worlds🚀🌎 What if memes didn’t just go viral… but launched entire AI-powered worlds? 🐸 $PEPE: Frog-filled lands with community-built mini-games 🐶 $DOGE: Meme missions on lunar outposts 🐕 $BONK: Doggo challenges, hangout zones, co-op creativity Now imagine: 🎮 Weekly game drops powered by no-code tools 🏗 Real-time co-building sessions with 128+ users 📺 Built-in streaming tools for meme-native events 🫂 Meme-powered economies with live events and social hubs ✏️ Natural language prompts generating full 3D scenes Thanks to our World Engine, this isn't wishful thinking. It’s collaborative, AI-native, stream-ready, and cross-platform. The next evolution isn’t just memecoins It’s AI meme worlds you can play, build, share, and own.

World Engine Turns Memes Into Living Worlds🚀🌎 What if memes didn’t just go viral… but launched entire AI-powered worlds? 🐸 $PEPE: Frog-filled lands with community-built mini-games 🐶 $DOGE: Meme missions on lunar outposts 🐕 $BONK: Doggo challenges, hangout zones, co-op creativity Now imagine: 🎮 Weekly game drops powered by no-code tools 🏗 Real-time co-building sessions with 128+ users 📺 Built-in streaming tools for meme-native events 🫂 Meme-powered economies with live events and social hubs ✏️ Natural language prompts generating full 3D scenes Thanks to our World Engine, this isn't wishful thinking. It’s collaborative, AI-native, stream-ready, and cross-platform. The next evolution isn’t just memecoins It’s AI meme worlds you can play, build, share, and own.

RYFT CORP

49,808 görüntüleme • 1 yıl önce

Introducing Code Video Model 🎬 Amazing work led by Ziyu Wan and the team! Agentic coding as the engine for controllable video generation. Scene layout, motion, timing, camera trajectories, and physical dynamics, all in editable code. A coding agent turns prompts into executable programs that render proxy videos. Our training-free early-step conditioning lets a pretrained video model follow the proxy’s structure while adopting a reference image’s appearance. From physics-grounded motion and bullet time to 3D/4D reconstruction and robotics simulation, it all starts with code. Watch the demo and explore more visual results 👇 Project page: Paper: Code: Data: Youtube:

Introducing Code Video Model 🎬 Amazing work led by Ziyu Wan and the team! Agentic coding as the engine for controllable video generation. Scene layout, motion, timing, camera trajectories, and physical dynamics, all in editable code. A coding agent turns prompts into executable programs that render proxy videos. Our training-free early-step conditioning lets a pretrained video model follow the proxy’s structure while adopting a reference image’s appearance. From physics-grounded motion and bullet time to 3D/4D reconstruction and robotics simulation, it all starts with code. Watch the demo and explore more visual results 👇 Project page: Paper: Code: Data: Youtube:

Gordon (Guocheng) Qian

36,650 görüntüleme • 3 gün önce

Everything you love about generative models — now powered by real physics! Announcing the Genesis project — after a 24-month large-scale research collaboration involving over 20 research labs — a generative physics engine able to generate 4D dynamical worlds powered by a physics simulation platform designed for general-purpose robotics and physical AI applications. Genesis's physics engine is developed in pure Python, while being 10-80x faster than existing GPU-accelerated stacks like Isaac Gym and MJX. It delivers a simulation speed ~430,000 faster than in real-time, and takes only 26 seconds to train a robotic locomotion policy transferrable to the real world on a single RTX4090 (see tutorial: The Genesis physics engine and simulation platform is fully open source at We'll gradually roll out access to our generative framework in the near future. Genesis implements a unified simulation framework all from scratch, integrating a wide spectrum of state-of-the-art physics solvers, allowing simulation of the whole physical world in a virtual realm with the highest realism. We aim to build a universal data engine that leverages an upper-level generative framework to autonomously create physical worlds, together with various modes of data, including environments, camera motions, robotic task proposals, reward functions, robot policies, character motions, fully interactive 3D scenes, open-world articulated assets, and more, aiming towards fully automated data generation for robotics, physical AI and other applications. Open Source Code: Project webpage: Documentation: 1/n

Everything you love about generative models — now powered by real physics! Announcing the Genesis project — after a 24-month large-scale research collaboration involving over 20 research labs — a generative physics engine able to generate 4D dynamical worlds powered by a physics simulation platform designed for general-purpose robotics and physical AI applications. Genesis's physics engine is developed in pure Python, while being 10-80x faster than existing GPU-accelerated stacks like Isaac Gym and MJX. It delivers a simulation speed ~430,000 faster than in real-time, and takes only 26 seconds to train a robotic locomotion policy transferrable to the real world on a single RTX4090 (see tutorial: The Genesis physics engine and simulation platform is fully open source at We'll gradually roll out access to our generative framework in the near future. Genesis implements a unified simulation framework all from scratch, integrating a wide spectrum of state-of-the-art physics solvers, allowing simulation of the whole physical world in a virtual realm with the highest realism. We aim to build a universal data engine that leverages an upper-level generative framework to autonomously create physical worlds, together with various modes of data, including environments, camera motions, robotic task proposals, reward functions, robot policies, character motions, fully interactive 3D scenes, open-world articulated assets, and more, aiming towards fully automated data generation for robotics, physical AI and other applications. Open Source Code: Project webpage: Documentation: 1/n

Zhou Xian

3,823,385 görüntüleme • 1 yıl önce

Today we are announcing Genie 3, a general purpose world model by Google DeepMind that can generate dynamic, interactive environments with a single text prompt. World models are AI that understand facets of the world (like Veo's knowledge of intuitive physics or Genie's mastery of new environments), and serve as a key stepping stone on the path to AGI. Genie 3 is our first world model to allow interaction in real-time, while also improving consistency and realism compared to Genie 2. Learn more ➡️

Today we are announcing Genie 3, a general purpose world model by Google DeepMind that can generate dynamic, interactive environments with a single text prompt. World models are AI that understand facets of the world (like Veo's knowledge of intuitive physics or Genie's mastery of new environments), and serve as a key stepping stone on the path to AGI. Genie 3 is our first world model to allow interaction in real-time, while also improving consistency and realism compared to Genie 2. Learn more ➡️

Google AI

102,788 görüntüleme • 1 yıl önce

🔥🔥🔥We’ve been listening to your feedback! Our latest world model HY-World 1.5 just got a major upgrade to make world generation more accessible than ever: 🛠️ Open Training Code: Fully customizable code for building and training your own models. ⚡ Accelerated Inference: Turbocharged speed and optimized VRAM for real-time interaction. 📉 Lite 5B Model: A new lightweight model that fits into small-VRAM GPUs. 🙌 Zero Waitlist: Our online app is now fully open to everyone—no application required. This is just the beginning. HY-World is building the future of spatial intelligence—open, accessible, and community-driven. 🕹️ Play now: ⭐ GitHub:

🔥🔥🔥We’ve been listening to your feedback! Our latest world model HY-World 1.5 just got a major upgrade to make world generation more accessible than ever: 🛠️ Open Training Code: Fully customizable code for building and training your own models. ⚡ Accelerated Inference: Turbocharged speed and optimized VRAM for real-time interaction. 📉 Lite 5B Model: A new lightweight model that fits into small-VRAM GPUs. 🙌 Zero Waitlist: Our online app is now fully open to everyone—no application required. This is just the beginning. HY-World is building the future of spatial intelligence—open, accessible, and community-driven. 🕹️ Play now: ⭐ GitHub:

Tencent Hy

20,581 görüntüleme • 9 ay önce

🚀 Sol-H3 on DGX Spark: 768p in Under a Minute 🤩 Monday: 8×B300, 5s 768p in 1.65s — faster than playback. Today: the same stack on one desktop Spark — about 56s hot E2E. Five seconds of 1344×768 video at 24 FPS with stereo audio, on a single NVIDIA DGX Spark (GB10). Two-stage, not the datacenter profile: 384p H3 draft → latent ×2 → H3-to-LTX VAE adapter → 768p LTX refine → VAE decode No decode/re-encode between stages. Stage 2 is conditioned on the draft latent, so Gemma stays off the box. Quantized weights stay resident; sparse attention cuts the rest. Stage 1 takes any MiniMax-H3 few-step LoRA. Timing is hot E2E (encode → both stages → video/audio VAE); cold start and MP4 mux are separate. Apache 2.0. Server was realtime. Edge is one box, under a minute. 🔗 Amazing team effort—full credits in the blog. Haopeng Li Junsong_Chen yitong li LoveSy ,Jingyu Xin, Haocheng Xi 🚗 COLM Song Han

Enze Xie

127,356 görüntüleme • 1 ay önce

An interactive world model developed by NVIDIA in collaboration with academic partners. - DreamDojo turns egocentric human video data into physical intelligence. - Human data is more scalable than robotics data but lacks action labels. - To solve this, a dedicated action model extracts latent actions by identifying physics and motion deltas between frames. Training - A massive 44k hours of video data are used for pre-training. - Post-training on small-scale robot datasets maps human physics to specific robot embodiments. - An additional distillation stage converts the model into an autoregressive, few-step diffusion model, enabling real-time, action-controllable simulation. Primary Use Cases - Live Teleoperation: Controlling a robot inside a world simulation in real-time. - Model-based Planning: Previewing and curating the best actions for improved success. - Policy Evaluation: Testing robot policies in realistic, out-of-distribution scenarios. Everything that's open-sourced: weights, code, post-training dataset, eval set, and details to reproduce.

An interactive world model developed by NVIDIA in collaboration with academic partners. - DreamDojo turns egocentric human video data into physical intelligence. - Human data is more scalable than robotics data but lacks action labels. - To solve this, a dedicated action model extracts latent actions by identifying physics and motion deltas between frames. Training - A massive 44k hours of video data are used for pre-training. - Post-training on small-scale robot datasets maps human physics to specific robot embodiments. - An additional distillation stage converts the model into an autoregressive, few-step diffusion model, enabling real-time, action-controllable simulation. Primary Use Cases - Live Teleoperation: Controlling a robot inside a world simulation in real-time. - Model-based Planning: Previewing and curating the best actions for improved success. - Policy Evaluation: Testing robot policies in realistic, out-of-distribution scenarios. Everything that's open-sourced: weights, code, post-training dataset, eval set, and details to reproduce.

The Humanoid Hub

11,575 görüntüleme • 7 ay önce

We’re on the verge of interactive, real-time, photorealistic video generation with what are called World Models. These require a fair amount of expensive compute, but costs will come down over time. Today, these interactions are essentially real-time dreams. They lack the persistence and shared logic that turns a video into a multiplayer game. Or concert. Or classroom. Or the holodeck. We believe the ultimate architecture for gaming (and the holodeck) is Roblox Reality. It’s a hybrid architecture that marries the structured data and logic of the Roblox Engine and Roblox Cloud with the generative power of Video World Models. The Roblox Engine provides the underlying synchronized ground truth—the score, physics, multiplayer sync, etc.—while our video model acts as a Super Upsampler to layer on photorealistic detail. We believe this will ultimately remove barriers to high-fidelity creation, allowing a team of three people to build a narrative-driven, photorealistic masterpiece in a single week. This is an early look at turning solitary AI dreams into a social, playable reality.

We’re on the verge of interactive, real-time, photorealistic video generation with what are called World Models. These require a fair amount of expensive compute, but costs will come down over time. Today, these interactions are essentially real-time dreams. They lack the persistence and shared logic that turns a video into a multiplayer game. Or concert. Or classroom. Or the holodeck. We believe the ultimate architecture for gaming (and the holodeck) is Roblox Reality. It’s a hybrid architecture that marries the structured data and logic of the Roblox Engine and Roblox Cloud with the generative power of Video World Models. The Roblox Engine provides the underlying synchronized ground truth—the score, physics, multiplayer sync, etc.—while our video model acts as a Super Upsampler to layer on photorealistic detail. We believe this will ultimately remove barriers to high-fidelity creation, allowing a team of three people to build a narrative-driven, photorealistic masterpiece in a single week. This is an early look at turning solitary AI dreams into a social, playable reality.

David Baszucki

791,897 görüntüleme • 5 ay önce

VideoRF: Rendering Dynamic Radiance Fields as 2D Feature Video Streams paper page: Neural Radiance Fields (NeRFs) excel in photorealistically rendering static scenes. However, rendering dynamic, long-duration radiance fields on ubiquitous devices remains challenging, due to data storage and computational constraints. In this paper, we introduce VideoRF, the first approach to enable real-time streaming and rendering of dynamic radiance fields on mobile platforms. At the core is a serialized 2D feature image stream representing the 4D radiance field all in one. We introduce a tailored training scheme directly applied to this 2D domain to impose the temporal and spatial redundancy of the feature image stream. By leveraging the redundancy, we show that the feature image stream can be efficiently compressed by 2D video codecs, which allows us to exploit video hardware accelerators to achieve real-time decoding. On the other hand, based on the feature image stream, we propose a novel rendering pipeline for VideoRF, which has specialized space mappings to query radiance properties efficiently. Paired with a deferred shading model, VideoRF has the capability of real-time rendering on mobile devices thanks to its efficiency. We have developed a real-time interactive player that enables online streaming and rendering of dynamic scenes, offering a seamless and immersive free-viewpoint experience across a range of devices, from desktops to mobile phones.

VideoRF: Rendering Dynamic Radiance Fields as 2D Feature Video Streams paper page: Neural Radiance Fields (NeRFs) excel in photorealistically rendering static scenes. However, rendering dynamic, long-duration radiance fields on ubiquitous devices remains challenging, due to data storage and computational constraints. In this paper, we introduce VideoRF, the first approach to enable real-time streaming and rendering of dynamic radiance fields on mobile platforms. At the core is a serialized 2D feature image stream representing the 4D radiance field all in one. We introduce a tailored training scheme directly applied to this 2D domain to impose the temporal and spatial redundancy of the feature image stream. By leveraging the redundancy, we show that the feature image stream can be efficiently compressed by 2D video codecs, which allows us to exploit video hardware accelerators to achieve real-time decoding. On the other hand, based on the feature image stream, we propose a novel rendering pipeline for VideoRF, which has specialized space mappings to query radiance properties efficiently. Paired with a deferred shading model, VideoRF has the capability of real-time rendering on mobile devices thanks to its efficiency. We have developed a real-time interactive player that enables online streaming and rendering of dynamic scenes, offering a seamless and immersive free-viewpoint experience across a range of devices, from desktops to mobile phones.

AK

38,686 görüntüleme • 2 yıl önce

Forget lidar. One single camera. Runs in real time & is open source: A streaming 3D model that reconstructs scenes live, at ~20 FPS, over long sequences. End-to-end. Optimization tricks, cleanup steps? Nope. And it beats both streaming and even some offline methods. Perception is becoming software-first. Closer to machines that see and understand the world as it unfolds. Thanks for sharing, Yinghao Xu 📍Models: Project page: Code: Paper: —— Weekly robotics and AI insights. Subscribe free:

Forget lidar. One single camera. Runs in real time & is open source: A streaming 3D model that reconstructs scenes live, at ~20 FPS, over long sequences. End-to-end. Optimization tricks, cleanup steps? Nope. And it beats both streaming and even some offline methods. Perception is becoming software-first. Closer to machines that see and understand the world as it unfolds. Thanks for sharing, Yinghao Xu 📍Models: Project page: Code: Paper: —— Weekly robotics and AI insights. Subscribe free:

Ilir Aliu

112,718 görüntüleme • 7 gün önce