Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

Whoa, this is Temporal Gaussian Hierarchy (TGH). This AI can real-time render long volumetric videos in 1080p at 450 fps on a single RTX 4090 GPU. And 18,000 frames (around 10 minutes) long 🤯 7 wild demos: 1. Real-time demos on MobileStage dataset

Min Choi

375,017 subscribers

464,043 views • 1 year ago •via X (Twitter)

Education Science & Technology

Anya Rossi• Live Now

Private livecam show

10 Comments

Min Choi1 year ago

2. Real-time rendering demo on the SelfCap dataset

Min Choi1 year ago

3. Real-time demos on the Sports dataset

Min Choi1 year ago

4. Real-time VR demos on Apple Vision Pro

Min Choi1 year ago

5. Real-time demos on the CMU-Panoptic dataset

Min Choi1 year ago

6. Real-time demos on the ENeRF-Outdoor dataset

Min Choi1 year ago

7. Real-time demos on the Neural3DV dataset

Min Choi1 year ago

Read the paper here:

Min Choi1 year ago

Project page:

Min Choi1 year ago

If you enjoyed this thread, Follow me @minchoi and please Bookmark, Like, Comment & Repost the first Post below to share with your friends:

Min Choi1 year ago

If you want to keep up with the latest AI developments and tools, subscribe to The Rundown it's FREE. And you'll never miss a thing in AI again:

Related Videos

[SIGGRAPH Asia '24 (TOG)] Representing Long Volumetric Video with Temporal Gaussian Hierarchy Contributions: • We introduce a novel, efficient, and expressive Temporal Gaussian Hierarchy representation for long volumetric video. To our knowledge, our method is the first approach capable of handling minutes of volumetric video data. • We propose a Compact Appearance Model and a new rasterization implementation to facilitate real-time, high-quality dynamic view synthesis while maintaining a compact size. • We propose a system to efficiently model long volumetric videos for the first time and demonstrate state-of-the-art dynamic view synthesis quality on the Neural3DV [Li et al. 2022], ENeRF-Outdoor [Lin et al. 2022], and MobileStage [Xu et al. 2024b] datasets, while also achieving the best rendering speed with reduced training cost and memory usage.

[SIGGRAPH Asia '24 (TOG)] Representing Long Volumetric Video with Temporal Gaussian Hierarchy Contributions: • We introduce a novel, efficient, and expressive Temporal Gaussian Hierarchy representation for long volumetric video. To our knowledge, our method is the first approach capable of handling minutes of volumetric video data. • We propose a Compact Appearance Model and a new rasterization implementation to facilitate real-time, high-quality dynamic view synthesis while maintaining a compact size. • We propose a system to efficiently model long volumetric videos for the first time and demonstrate state-of-the-art dynamic view synthesis quality on the Neural3DV [Li et al. 2022], ENeRF-Outdoor [Lin et al. 2022], and MobileStage [Xu et al. 2024b] datasets, while also achieving the best rendering speed with reduced training cost and memory usage.

MrNeRF

79,379 views • 1 year ago

CLM: Removing the GPU Memory Barrier for 3D Gaussian Splatting CLM trains a 102-million Gaussian model on the MatrixCity BigCity Aerial Dataset under 4 hours using a single RTX 4090, achieving a PSNR of 25.

CLM: Removing the GPU Memory Barrier for 3D Gaussian Splatting CLM trains a 102-million Gaussian model on the MatrixCity BigCity Aerial Dataset under 4 hours using a single RTX 4090, achieving a PSNR of 25.

Ryohei Sasaki@engineer

34,570 views • 6 months ago

3D Gaussian Splats are cool, but they're static (Part 12). Spacetime Gaussian Feature Splatting is able to capture dynamic, and transient content within a scene and render them at 8K resolution and 60 FPS on an RTX 4090.

3D Gaussian Splats are cool, but they're static (Part 12). Spacetime Gaussian Feature Splatting is able to capture dynamic, and transient content within a scene and render them at 8K resolution and 60 FPS on an RTX 4090.

Dreaming Tulpa 🥓👑

37,319 views • 2 years ago

StreamDiT Real-Time Streaming Text-to-Video Generation StreamDiT enables real-time text-to-video generation at 16 FPS on a single GPU (H100)

StreamDiT Real-Time Streaming Text-to-Video Generation StreamDiT enables real-time text-to-video generation at 16 FPS on a single GPU (H100)

AK

43,659 views • 1 year ago

[SIGGRAPH '26] Anchored Temporal Gaussian Splatting for Long Volumetric Video Representation TL;DR: We present ATGS, a novel framework for volumetric video reconstruction that effectively handles long sequences and complex motions. By utilizing time-conditioned anchors and a temporal windowing strategy, ATGS enhances temporal coherence and scalability. Abstract (excerpt): Key insight is that explicitly tracking long term complex motion with individual Gaussian primitives is inherently unstable. Instead, we organize Gaussians around time conditioned anchors that localize their spatial and temporal support, thereby reducing long range motion complexity. We further introduce a temporal windowing strategy to activate only anchors relevant to the queried time, which improves scalability and temporal coherence. In addition, to ensure spatial and temporal stability, we design a compact set of multi level anchor features that encode global features, local spatial features, and local temporal features, jointly constraining Gaussian generation. Extensive experiments demonstrate that ATGS consistently outperforms prior methods on long sequence volumetric videos with complex motions.

[SIGGRAPH '26] Anchored Temporal Gaussian Splatting for Long Volumetric Video Representation TL;DR: We present ATGS, a novel framework for volumetric video reconstruction that effectively handles long sequences and complex motions. By utilizing time-conditioned anchors and a temporal windowing strategy, ATGS enhances temporal coherence and scalability. Abstract (excerpt): Key insight is that explicitly tracking long term complex motion with individual Gaussian primitives is inherently unstable. Instead, we organize Gaussians around time conditioned anchors that localize their spatial and temporal support, thereby reducing long range motion complexity. We further introduce a temporal windowing strategy to activate only anchors relevant to the queried time, which improves scalability and temporal coherence. In addition, to ensure spatial and temporal stability, we design a compact set of multi level anchor features that encode global features, local spatial features, and local temporal features, jointly constraining Gaussian generation. Extensive experiments demonstrate that ATGS consistently outperforms prior methods on long sequence volumetric videos with complex motions.

MrNeRF

26,905 views • 2 months ago

Huge if true: A neural net learned every Atari game simultaneously, from scratch, in real time, one hour total on a single 4090. That’s roughly 56 seconds of GPU time per game. No pauses, no resets, no memory peeking, just raw 60 fps color frames.

Huge if true: A neural net learned every Atari game simultaneously, from scratch, in real time, one hour total on a single 4090. That’s roughly 56 seconds of GPU time per game. No pauses, no resets, no memory peeking, just raw 60 fps color frames.

Mark Kretschmann

134,832 views • 9 months ago

We present MotionStream — real-time, long-duration video generation that you can interactively control just by dragging your mouse. All videos here are raw, real-time screen captures without any post-processing. Model runs on a single H100 at 29 FPS and 0.4s latency.

We present MotionStream — real-time, long-duration video generation that you can interactively control just by dragging your mouse. All videos here are raw, real-time screen captures without any post-processing. Model runs on a single H100 at 29 FPS and 0.4s latency.

Xun Huang

98,346 views • 8 months ago

This is wild. China's Alibaba just dropped Live Avatar. This AI turns any voice into a realtime, talking avatar with infinite length at 20 FPS. 10 wild demos:👇 1. Ilya interview that never happened

Min Choi

220,656 views • 7 months ago

Volumetrics are amazing but historically heavy, dense, and painful to art direct. 3D Gaussian Splatting changes that by turning volumetric simulations into real-time, lightweight assets. With sufficient temporal data, these can evolve into true 4D Gaussian Splats. Even from a single static 3DGS, artists can create pseudo-4D behavior using simple deformers and procedural effects - bending, twisting, reshaping, and re-timing volumetric forms at will. This is a massive shift in how volumetrics can be captured, manipulated, and directed and also redefines them as swift real-time modular assets.

Volumetrics are amazing but historically heavy, dense, and painful to art direct. 3D Gaussian Splatting changes that by turning volumetric simulations into real-time, lightweight assets. With sufficient temporal data, these can evolve into true 4D Gaussian Splats. Even from a single static 3DGS, artists can create pseudo-4D behavior using simple deformers and procedural effects - bending, twisting, reshaping, and re-timing volumetric forms at will. This is a massive shift in how volumetrics can be captured, manipulated, and directed and also redefines them as swift real-time modular assets.

Spenser Dickerson

12,962 views • 6 months ago

🤯Real-time video generation just got HUGE. Introducing Helios: A 14B parameter model running at 19.5 FPS on a single H100. Cheaper, Faster, and Stronger than 1.3B models, generating minute-long videos without the usual tricks (even no quantization).

🤯Real-time video generation just got HUGE. Introducing Helios: A 14B parameter model running at 19.5 FPS on a single H100. Cheaper, Faster, and Stronger than 1.3B models, generating minute-long videos without the usual tricks (even no quantization).

Bin Lin

27,737 views • 4 months ago

MoVerse turns an image into a navigable 3D world. Runs at 8 FPS on a single RTX 4090.

MoVerse turns an image into a navigable 3D world. Runs at 8 FPS on a single RTX 4090.

⚡AI Search⚡

18,555 views • 24 days ago

Gaussian Splatting is pretty wild. Being able to take dense collections of assets and package them into fast, real-time splats opens up a ton of new possibilities for look-dev, previs, and interactive scenes. For this test, I built a pile of 400 skulls - roughly 40+ million polygons (and over 400 8K texture instances) - rendered out a 600 frame dataset on The Render Network with OTOY OctaneRender and then converted the result into a single splat asset. Now it runs in real-time as one compact Gaussian Splat.

Gaussian Splatting is pretty wild. Being able to take dense collections of assets and package them into fast, real-time splats opens up a ton of new possibilities for look-dev, previs, and interactive scenes. For this test, I built a pile of 400 skulls - roughly 40+ million polygons (and over 400 8K texture instances) - rendered out a 600 frame dataset on The Render Network with OTOY OctaneRender and then converted the result into a single splat asset. Now it runs in real-time as one compact Gaussian Splat.

Spenser Dickerson

46,547 views • 8 months ago

Things you can do in VR with an RTX 4090 (5 free incredible Unreal Engine 5 demos you can play in VR with UEVR) To celebrate Flat2VR’s Trombone Champ: Unflattened launching Nov 26, we’re giving away an RTX 4090 OC GPU + tons of other Flat2VR merch! Sign up here:

Things you can do in VR with an RTX 4090 (5 free incredible Unreal Engine 5 demos you can play in VR with UEVR) To celebrate Flat2VR’s Trombone Champ: Unflattened launching Nov 26, we’re giving away an RTX 4090 OC GPU + tons of other Flat2VR merch! Sign up here:

Flat2VR

22,320 views • 1 year ago

This is Genesis. Revolutionary new AI physics engine. You can turn text into 3D worlds; Simulate robots, humans, & objects; Runs at 43M FPS (yes, millions!) 8 wild demos:

This is Genesis. Revolutionary new AI physics engine. You can turn text into 3D worlds; Simulate robots, humans, & objects; Runs at 43M FPS (yes, millions!) 8 wild demos:

Min Choi

786,032 views • 1 year ago

3D Gaussian Splats are cool, but they're static (Part 33). GGHead can generate and render 3D heads at 1K resolution in real-time.

3D Gaussian Splats are cool, but they're static (Part 33). GGHead can generate and render 3D heads at 1K resolution in real-time.

Dreaming Tulpa 🥓👑

104,538 views • 2 years ago

🍓 Ichigo-llama3.1: Local Real-Time Voice AI We bring 2 key improvements to Ichigo: - It can talk back (Yes!) - It recognizes when it can't comprehend input You can now run this little strawberry on your device! Demo on a single NVIDIA 3090 GPU. 1/10

🍓 Ichigo-llama3.1: Local Real-Time Voice AI We bring 2 key improvements to Ichigo: - It can talk back (Yes!) - It recognizes when it can't comprehend input You can now run this little strawberry on your device! Demo on a single NVIDIA 3090 GPU. 1/10

Menlo Research

67,175 views • 1 year ago

This is wild. OpenAI just dropped ChatGPT-4o and it will completely change the AI assistant game. 10 wild examples: 1. Visual assistant in real-time

This is wild. OpenAI just dropped ChatGPT-4o and it will completely change the AI assistant game. 10 wild examples: 1. Visual assistant in real-time

Min Choi

12,580,215 views • 2 years ago

3d gaussian splatting is fucking cool. And now you can capture real world spaces just by walking around in your meta quest headset. The real-time feedback ensures you don’t miss a spot. Apple needs to get on this asap:

3d gaussian splatting is fucking cool. And now you can capture real world spaces just by walking around in your meta quest headset. The real-time feedback ensures you don’t miss a spot. Apple needs to get on this asap:

Bilawal Sidhu

162,888 views • 9 months ago

No, rally crowds do not equal winning. But this is Milwaukee a few minutes ago. I’ve been in and around political events for a long time, and believe me - this is WILD.

No, rally crowds do not equal winning. But this is Milwaukee a few minutes ago. I’ve been in and around political events for a long time, and believe me - this is WILD.

Matt Bennett

1,622,427 views • 1 year ago

Generative AI is cool and all, but I continue to be blown away by the strides in real-time visual effects. Case in point — ripping through immaculate fluid simulations on an RTX 4090 using EmberGen:

Generative AI is cool and all, but I continue to be blown away by the strides in real-time visual effects. Case in point — ripping through immaculate fluid simulations on an RTX 4090 using EmberGen:

Bilawal Sidhu

552,643 views • 2 years ago