正在加载视频...

视频加载失败

加载此视频时出现问题。这可能是由于临时网络问题，或视频可能不可用。

Today we announced RTFM (Real-Time Frame Model) — a world model that generates frames in real time from any camera viewpoint. Unlike standard video models, RTFM understands 3D geometry. You can literally move the camera through the generated world. 🎥👇

Keunhong Park

3,561 subscribers

16,114 次观看 • 1 年前 •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

16 条评论

Keunhong Park 的头像

Keunhong Park1 年前

🎬 First up: the dolly zoom effect. RTFM generates new views as the camera moves. All learned, no explicit geometry.

Keunhong Park 的头像

Keunhong Park1 年前

🚶 You can walk around the scene freely. The model keeps the world consistent and coherent.

Keunhong Park 的头像

Keunhong Park1 年前

🌀 If you need a perfect camera orbit, you can just sample viewpoints on a circle. RTFM generates smooth, consistent motion.

Keunhong Park 的头像

Keunhong Park1 年前

🐟 It can reproduce lens effects like fisheye distortion as an emergent capability of the model.

Keunhong Park 的头像

Keunhong Park1 年前

Want to learn more? 📖 Read the manual: 🕹️ Play the demo:

Keunhong Park 的头像

Keunhong Park1 年前

If building World Models excites you, we’re hiring:

Central Processing Unit 🐯 的头像

Central Processing Unit 🐯1 年前

Finally, someone has deciphered the acronym RTFM.

Xu Cheng 的头像

Xu Cheng1 年前

Really impressive!

Noodles 的头像

Noodles1 年前

@TimSweeneyEpic ue should be worried ?🤔

mmsdaus 的头像

mmsdaus1 年前

what would it take to make the resolution 4k?

Amanda Sherry 的头像

Amanda Sherry11 个月前

Cool👍

Adam Goldberg 的头像

Adam Goldberg1 年前

RTFM - READ THE FUCKING MANUAL

VR GUY 的头像

VR GUY1 年前

How long until we can use this in VR (with stereoscopic rendering)?

Keunhong Park 的头像

Keunhong Park1 年前

@LoganPeterJones You can do it now. The resolution and frame rates to make it appealing will take some work but it's only a matter of time.

𝚖𝚟𝚛𝚌𝚔𝚑𝚌𝚔𝚛 的头像

𝚖𝚟𝚛𝚌𝚔𝚑𝚌𝚔𝚛1 年前

@StraughterG Very cool

Flutterwhat 的头像

Flutterwhat1 年前

Extremely limited demo. Very little room to move. Just a room model world model.

相关视频

Introducing Atlas: The world's first multimodal world model that generates image and video frames with pixel-perfect camera control and reconstructs them in 3D. Model the world, move the camera, and simulate space & time.

Introducing Atlas: The world's first multimodal world model that generates image and video frames with pixel-perfect camera control and reconstructs them in 3D. Model the world, move the camera, and simulate space & time.

World Labs

10,894,327 次观看 • 2 个月前

Introducing H3 Max Camera Controls, the world's fastest 3D video model. Turn a single viewpoint into a navigable 3D scene in less than 3 seconds. Set the horizontal and vertical camera angle in degrees, and H3 Max Camera Controls moves through the scene while preserving geometry, position, and materials across perspectives. Fast enough to control the camera in real time.

Introducing H3 Max Camera Controls, the world's fastest 3D video model. Turn a single viewpoint into a navigable 3D scene in less than 3 seconds. Set the horizontal and vertical camera angle in degrees, and H3 Max Camera Controls moves through the scene while preserving geometry, position, and materials across perspectives. Fast enough to control the camera in real time.

fal

240,319 次观看 • 1 个月前

Our third announcement is GWM Worlds. A world model for real-time environment simulation. You give the model a static scene, and it generates an immersive, infinite, explorable space as you move through it, with geometry, lighting and physics. All in real-time. You can travel to any place, real or imagined. You can become any agent, a person walking through a city, a drone flying over a snowy mountain or a robot navigating a warehouse.

Our third announcement is GWM Worlds. A world model for real-time environment simulation. You give the model a static scene, and it generates an immersive, infinite, explorable space as you move through it, with geometry, lighting and physics. All in real-time. You can travel to any place, real or imagined. You can become any agent, a person walking through a city, a drone flying over a snowy mountain or a robot navigating a warehouse.

Runway

80,709 次观看 • 10 个月前

World Reconstruction Model for Sports Free-viewpoint replays, true to what actually happened: • 8 cameras as input • Near-real-time rendering • Any camera trajectory you can imagine, even impossible ones

World Reconstruction Model for Sports Free-viewpoint replays, true to what actually happened: • 8 cameras as input • Near-real-time rendering • Any camera trajectory you can imagine, even impossible ones

Gracia

18,812 次观看 • 17 天前

Spatial Intelligence just went Open Source. 🌐 Introducing InSpatio-WorldFM — the world’s first open-source real-time interactive 3D world model. Last October, World Labs's RTFM showed the vision. Today, we’re bringing that idea into the open for everyone. ✅Efficient, multi-view consistent spatial reasoning ✅Real-time interactive exploration ✅Open-source weights & code Stop watching demos. Start building now. 👇

Spatial Intelligence just went Open Source. 🌐 Introducing InSpatio-WorldFM — the world’s first open-source real-time interactive 3D world model. Last October, World Labs's RTFM showed the vision. Today, we’re bringing that idea into the open for everyone. ✅Efficient, multi-view consistent spatial reasoning ✅Real-time interactive exploration ✅Open-source weights & code Stop watching demos. Start building now. 👇

InSpatio｜影溯

37,488 次观看 • 8 个月前

Holy shit... OpenArt just launched a full 3D world builder that runs entirely in your browser. One prompt or one image generates a navigable environment with real spatial geometry you can move through from any direction. Here is the full breakdown: ↓

Holy shit... OpenArt just launched a full 3D world builder that runs entirely in your browser. One prompt or one image generates a navigable environment with real spatial geometry you can move through from any direction. Here is the full breakdown: ↓

The AI Colony

54,650 次观看 • 7 个月前

China open-sourced a model that reconstructs any scene in 3D from a regular video, in real-time. one camera. no LiDAR. 10,000+ frames without falling apart. just walk around with your camera and watch the entire world get rebuilt in 3D at 20 fps. → runs at ~20 FPS on a single GPU → Stable over 10,000+ frames → Beats optimization-based methods on benchmarks → Works on drone footage, driving videos, indoor walkthroughs 100% open source.

China open-sourced a model that reconstructs any scene in 3D from a regular video, in real-time. one camera. no LiDAR. 10,000+ frames without falling apart. just walk around with your camera and watch the entire world get rebuilt in 3D at 20 fps. → runs at ~20 FPS on a single GPU → Stable over 10,000+ frames → Beats optimization-based methods on benchmarks → Works on drone footage, driving videos, indoor walkthroughs 100% open source.

Yasir Ai

255,088 次观看 • 3 个月前

China open-sourced a model that reconstructs any scene in 3D from a regular video, in real-time. one camera. no LiDAR. 10,000+ frames without falling apart. just walk around with your camera and watch the entire world get rebuilt in 3D at 20 fps. → runs at ~20 FPS on a single GPU → Stable over 10,000+ frames → Beats optimization-based methods on benchmarks → Works on drone footage, driving videos, indoor walkthroughs 100% open source.

China open-sourced a model that reconstructs any scene in 3D from a regular video, in real-time. one camera. no LiDAR. 10,000+ frames without falling apart. just walk around with your camera and watch the entire world get rebuilt in 3D at 20 fps. → runs at ~20 FPS on a single GPU → Stable over 10,000+ frames → Beats optimization-based methods on benchmarks → Works on drone footage, driving videos, indoor walkthroughs 100% open source.

Superman

1,472,679 次观看 • 3 个月前

Today we are releasing JST-2, the world's first full-stack physics-native world foundation model. Many world models learn in pixel/video space. We went back to first principles and built the entire model stack in 3D. Generate a 3D scene in seconds and expand it in real time. Create characters with motion built in, all through one general architecture.

Today we are releasing JST-2, the world's first full-stack physics-native world foundation model. Many world models learn in pixel/video space. We went back to first principles and built the entire model stack in 3D. Generate a 3D scene in seconds and expand it in real time. Create characters with motion built in, all through one general architecture.

ViggleAI

1,736,764 次观看 • 17 天前

Introducing Starchild-1 from Odyssey, the first ever real-time multimodal world model. This a model that can generate interactive simulations of the world that you can—for the first time ever—hear. Starchild-1 represents a big step towards a general-purpose world simulator.

Introducing Starchild-1 from Odyssey, the first ever real-time multimodal world model. This a model that can generate interactive simulations of the world that you can—for the first time ever—hear. Starchild-1 represents a big step towards a general-purpose world simulator.

Oliver Cameron

247,268 次观看 • 5 个月前

Introducing MirageLSD: The First Live-Stream Diffusion (LSD) AI Model Input any video stream, from a camera or video chat to a computer screen or game, and transform it into any world you desire, in real-time (<40ms latency). Here’s how it works (w/ demo you can use!):

Introducing MirageLSD: The First Live-Stream Diffusion (LSD) AI Model Input any video stream, from a camera or video chat to a computer screen or game, and transform it into any world you desire, in real-time (<40ms latency). Here’s how it works (w/ demo you can use!):

Decart

1,017,876 次观看 • 1 年前

Genie3 generates videos. We generate 𝟯𝗗 𝘄𝗼𝗿𝗹𝗱𝘀 you can actually use. Launching tomorrow — Tencent #HYWorld 2.0, an engine-ready World Model🚀 This isn't a video. It's a real 3D scene, all generated & editable. One image in. A whole 3D world out. 🔥Open-source tomorrow

Genie3 generates videos. We generate 𝟯𝗗 𝘄𝗼𝗿𝗹𝗱𝘀 you can actually use. Launching tomorrow — Tencent #HYWorld 2.0, an engine-ready World Model🚀 This isn't a video. It's a real 3D scene, all generated & editable. One image in. A whole 3D world out. 🔥Open-source tomorrow

Tengfei Wang

553,274 次观看 • 6 个月前

SITUATION EXPLAINED: A new generation of AI labs are betting that world models are the path to general intelligence that can interact with the real world. We had General Intuition CEO Pim de Witte on MTS to explain what world models are, and why intuitively world models offer a clearer path towards general intelligence than LLMs. For humans, text is a technology for compressing information about the unfolding, three-dimensional world around us and communicating it to other people. When LLMs reason about the world, they do so by reconstructing the world through text. World models work by modeling the three-dimensional world at the architecture level and then allowing agents to interact with that world. Unlike an LLM, which understands space and time through text, world models perform spatial and temporal reasoning natively. General Intuition hopes to use world models to create embodied AI—machines that can reason and act in the real world. But other companies are also pushing the frontier in this space. Odyssey ML today announced a multi-agent world model, Agora-1. Their launch video shows four participants interacting with a real-time world model simulation of GoldenEye, a first-person shooter for the N64. A playable preview of the GoldenEye simulation is available on their site.

SITUATION EXPLAINED: A new generation of AI labs are betting that world models are the path to general intelligence that can interact with the real world. We had General Intuition CEO Pim de Witte on MTS to explain what world models are, and why intuitively world models offer a clearer path towards general intelligence than LLMs. For humans, text is a technology for compressing information about the unfolding, three-dimensional world around us and communicating it to other people. When LLMs reason about the world, they do so by reconstructing the world through text. World models work by modeling the three-dimensional world at the architecture level and then allowing agents to interact with that world. Unlike an LLM, which understands space and time through text, world models perform spatial and temporal reasoning natively. General Intuition hopes to use world models to create embodied AI—machines that can reason and act in the real world. But other companies are also pushing the frontier in this space. Odyssey ML today announced a multi-agent world model, Agora-1. Their launch video shows four participants interacting with a real-time world model simulation of GoldenEye, a first-person shooter for the N64. A playable preview of the GoldenEye simulation is available on their site.

MTS

11,693 次观看 • 5 个月前

Say hi to Odyssey-2 Pro—our most intelligent, interactive, and general-purpose world model yet. Every pixel is generated in real-time, and frame-by-frame the model simulates the world forward. Today for minutes, soon for hours. Available to developers and our API very soon.

Say hi to Odyssey-2 Pro—our most intelligent, interactive, and general-purpose world model yet. Every pixel is generated in real-time, and frame-by-frame the model simulates the world forward. Today for minutes, soon for hours. Available to developers and our API very soon.

Odyssey

285,653 次观看 • 10 个月前

What if we can simulate an *interactive 3D world*, from a single image, in the wild, in real time? Introducing PointWorld-1B: a large pre-trained 3D world model that predicts env dynamics given RGB-D capture and robot actions. 🌐 from Stanford University NVIDIA

What if we can simulate an interactive 3D world, from a single image, in the wild, in real time? Introducing PointWorld-1B: a large pre-trained 3D world model that predicts env dynamics given RGB-D capture and robot actions. 🌐 from Stanford University NVIDIA

Wenlong Huang

280,325 次观看 • 10 个月前

🌍 What if an AI-generated world didn’t end after a few seconds—but kept running for hours, responded to every move, let you attack, cast spells, shoot, or summon storms, and continued evolving through AI agents? That’s LingBot-World 2.0: our open-source real-time interactive world model, scaling to 720p/60fps in our full setup. Today, we’re open-sourcing 3 more models: 🔹 LingBot-World 2.0 Small (1.3B) 🔹 LingBot-World 2.0 Bidirectional 🔹 LingBot-World 2.0 Causal Pretrain And now, with our 1.3B Small Model, that world can run in real time on a single consumer GPU. 🧵

🌍 What if an AI-generated world didn’t end after a few seconds—but kept running for hours, responded to every move, let you attack, cast spells, shoot, or summon storms, and continued evolving through AI agents? That’s LingBot-World 2.0: our open-source real-time interactive world model, scaling to 720p/60fps in our full setup. Today, we’re open-sourcing 3 more models: 🔹 LingBot-World 2.0 Small (1.3B) 🔹 LingBot-World 2.0 Bidirectional 🔹 LingBot-World 2.0 Causal Pretrain And now, with our 1.3B Small Model, that world can run in real time on a single consumer GPU. 🧵

Robbyant

1,999,698 次观看 • 1 个月前

Here’s how it works: 🔵 Design your world and character using text and visual prompts. 🔵 Nano Banana Pro makes an image preview that you can adjust. 🔵 Our Genie 3 world model generates the environment in real-time as you move through. 🔵 Remix existing worlds or discover new ones in the gallery.

Here’s how it works: 🔵 Design your world and character using text and visual prompts. 🔵 Nano Banana Pro makes an image preview that you can adjust. 🔵 Our Genie 3 world model generates the environment in real-time as you move through. 🔵 Remix existing worlds or discover new ones in the gallery.

Google DeepMind

374,638 次观看 • 9 个月前

🚀Echo-2 is here - our new world model! These aren’t videos. These are 𝟑𝐃 𝐬𝐜𝐞𝐧𝐞𝐬. Generated from a single image. - Stunning visual quality. - Real-time rendering. - Interactive camera control. - Physically grounded. 🧵More details👇

🚀Echo-2 is here - our new world model! These aren’t videos. These are 𝟑𝐃 𝐬𝐜𝐞𝐧𝐞𝐬. Generated from a single image. - Stunning visual quality. - Real-time rendering. - Interactive camera control. - Physically grounded. 🧵More details👇

SpAItial AI

104,242 次观看 • 6 个月前

📢 Lyra: Generative 3D Scene Reconstruction via Video Diffusion Model Self-Distillation Got only one or a few images and wondering if recovering the 3D environment is a reconstruction or generation problem? Why not do it with a generative reconstruction model! We show that a camera-conditioned video diffusion model can be transformed into a generative reconstruction model that directly outputs a high-quality 3D Gaussian Splatting representation through self-distillation, without requiring real-world training data. Check out our results in the video (wait for dynamic scenes in the second half!) : Project Page: Code and Models: Paper:

📢 Lyra: Generative 3D Scene Reconstruction via Video Diffusion Model Self-Distillation Got only one or a few images and wondering if recovering the 3D environment is a reconstruction or generation problem? Why not do it with a generative reconstruction model! We show that a camera-conditioned video diffusion model can be transformed into a generative reconstruction model that directly outputs a high-quality 3D Gaussian Splatting representation through self-distillation, without requiring real-world training data. Check out our results in the video (wait for dynamic scenes in the second half!) : Project Page: Code and Models: Paper:

Sherwin Bahmani

66,960 次观看 • 1 年前

We’re on a path to video of any kind being imagined by a model in real-time. Because it’s real-time generated, each video becomes fully interactive & personalized. Type or speak into the model, and the video will respond in real-time. A revolutionary form of media is emerging!

We’re on a path to video of any kind being imagined by a model in real-time. Because it’s real-time generated, each video becomes fully interactive & personalized. Type or speak into the model, and the video will respond in real-time. A revolutionary form of media is emerging!

Oliver Cameron

67,924 次观看 • 1 年前