Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

SpaceTimePilot Generative Rendering of Dynamic Scenes Across Space and Time

AK

513,443 subscribers

22,363 views • 8 months ago •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

Check out our CVPR 2023 Award Candidate paper, DynIBaR! DynIBaR takes monocular videos of dynamic scenes and renders novel views in space and time. It addresses limitations of prior dynamic NeRF methods, rendering much higher quality views.

Check out our CVPR 2023 Award Candidate paper, DynIBaR! DynIBaR takes monocular videos of dynamic scenes and renders novel views in space and time. It addresses limitations of prior dynamic NeRF methods, rendering much higher quality views.

Zhengqi Li

86,219 views • 3 years ago

VideoRF: Rendering Dynamic Radiance Fields as 2D Feature Video Streams paper page: Neural Radiance Fields (NeRFs) excel in photorealistically rendering static scenes. However, rendering dynamic, long-duration radiance fields on ubiquitous devices remains challenging, due to data storage and computational constraints. In this paper, we introduce VideoRF, the first approach to enable real-time streaming and rendering of dynamic radiance fields on mobile platforms. At the core is a serialized 2D feature image stream representing the 4D radiance field all in one. We introduce a tailored training scheme directly applied to this 2D domain to impose the temporal and spatial redundancy of the feature image stream. By leveraging the redundancy, we show that the feature image stream can be efficiently compressed by 2D video codecs, which allows us to exploit video hardware accelerators to achieve real-time decoding. On the other hand, based on the feature image stream, we propose a novel rendering pipeline for VideoRF, which has specialized space mappings to query radiance properties efficiently. Paired with a deferred shading model, VideoRF has the capability of real-time rendering on mobile devices thanks to its efficiency. We have developed a real-time interactive player that enables online streaming and rendering of dynamic scenes, offering a seamless and immersive free-viewpoint experience across a range of devices, from desktops to mobile phones.

VideoRF: Rendering Dynamic Radiance Fields as 2D Feature Video Streams paper page: Neural Radiance Fields (NeRFs) excel in photorealistically rendering static scenes. However, rendering dynamic, long-duration radiance fields on ubiquitous devices remains challenging, due to data storage and computational constraints. In this paper, we introduce VideoRF, the first approach to enable real-time streaming and rendering of dynamic radiance fields on mobile platforms. At the core is a serialized 2D feature image stream representing the 4D radiance field all in one. We introduce a tailored training scheme directly applied to this 2D domain to impose the temporal and spatial redundancy of the feature image stream. By leveraging the redundancy, we show that the feature image stream can be efficiently compressed by 2D video codecs, which allows us to exploit video hardware accelerators to achieve real-time decoding. On the other hand, based on the feature image stream, we propose a novel rendering pipeline for VideoRF, which has specialized space mappings to query radiance properties efficiently. Paired with a deferred shading model, VideoRF has the capability of real-time rendering on mobile devices thanks to its efficiency. We have developed a real-time interactive player that enables online streaming and rendering of dynamic scenes, offering a seamless and immersive free-viewpoint experience across a range of devices, from desktops to mobile phones.

AK

38,686 views • 2 years ago

🔥Unbounded 4D City Generation🔥 #CityDreamer4D is a generative model of unbounded 4D cities that decouples static and dynamic scenes with realism and controllability - Project: - Code: - CityTopia Dataset:

🔥Unbounded 4D City Generation🔥 #CityDreamer4D is a generative model of unbounded 4D cities that decouples static and dynamic scenes with realism and controllability - Project: - Code: - CityTopia Dataset:

Ziwei Liu

24,326 views • 1 year ago

Light-X Generative 4D Video Rendering with Camera and Illumination Control

Light-X Generative 4D Video Rendering with Camera and Illumination Control

AK

31,126 views • 9 months ago

City-on-Web: Real-time Neural Rendering of Large-scale Scenes on the Web paper page: enables real-time neural rendering of large-scale scenes over web using laptop GPUs. The result is tested on the NVIDIA RTX 3060 Laptop GPU with 1920x1080 resolution

City-on-Web: Real-time Neural Rendering of Large-scale Scenes on the Web paper page: enables real-time neural rendering of large-scale scenes over web using laptop GPUs. The result is tested on the NVIDIA RTX 3060 Laptop GPU with 1920x1080 resolution

AK

79,439 views • 2 years ago

behind the scenes of generative art a flower...

behind the scenes of generative art a flower...

Adam genlight

21,060 views • 3 years ago

2024 was a year of contrasts across Europe, with extreme heatwaves and large wildfires, alongside flooding and drought. @EUMETSAT’s video captures this dynamic year from space! Full #ESOTC2024 report:

2024 was a year of contrasts across Europe, with extreme heatwaves and large wildfires, alongside flooding and drought. @EUMETSAT’s video captures this dynamic year from space! Full #ESOTC2024 report:

World Meteorological Organization

12,655 views • 1 year ago

WIP V-Tuber program in Unreal Engine 🎥 Developing a custom post-processing shader for the character! Focused on dynamic rim lighting and shadow rendering that preserves the model’s sense of depth and form even in dark scenes.

WIP V-Tuber program in Unreal Engine 🎥 Developing a custom post-processing shader for the character! Focused on dynamic rim lighting and shadow rendering that preserves the model’s sense of depth and form even in dark scenes.

Raming ( ´∀｀ )b

37,222 views • 10 months ago

Introducing VGGT-Ω: scaling feed-forward reconstruction across static and dynamic scenes, and studying whether the learned geometric representations transfer beyond reconstruction.

Introducing VGGT-Ω: scaling feed-forward reconstruction across static and dynamic scenes, and studying whether the learned geometric representations transfer beyond reconstruction.

Jianyuan

783,167 views • 3 months ago

Have too little time for working on the project lately. Just updated the calculation and rendering of housing regions. Next steps will be dynamic calculation during building process and show/hide them when needed

Have too little time for working on the project lately. Just updated the calculation and rendering of housing regions. Next steps will be dynamic calculation during building process and show/hide them when needed

Pavel Valakh | Lands of Koastalia 🐏

17,500 views • 1 year ago

Introducing LTX-2.5, the world model the world builds on. One of the biggest upgrades yet to the model already powering film, robotics, and real-time workflows. Higher pixel fidelity, multishot scenes that hold together across cuts, and a pretrained foundation built to be finetuned across domains. Plus: Diffusion Fidelity Rendering, a new rendering approach built for pixel quality that holds up frame by frame, even on a cinema screen. This isn't a tool you rent. It's a foundation you build on, open and yours.

Introducing LTX-2.5, the world model the world builds on. One of the biggest upgrades yet to the model already powering film, robotics, and real-time workflows. Higher pixel fidelity, multishot scenes that hold together across cuts, and a pretrained foundation built to be finetuned across domains. Plus: Diffusion Fidelity Rendering, a new rendering approach built for pixel quality that holds up frame by frame, even on a cinema screen. This isn't a tool you rent. It's a foundation you build on, open and yours.

LTX.io

13,727,215 views • 1 month ago

Fight across time, space, and the streets of Toronto when Scott Pilgrim EX, a 2D action-adventure brawler, arrives next year. #IGNFanFest

Fight across time, space, and the streets of Toronto when Scott Pilgrim EX, a 2D action-adventure brawler, arrives next year. #IGNFanFest

IGN

37,307 views • 11 months ago

Today we're launching OpenUI Cloud We've spent months arguing AI deserves more than text. The uncomfortable truth is Generative UI in production needs much more than just rendering. The boring parts - validation, error correction, fallbacks, versioning, audit trails - eat up 80% of the time and become the critical factor in deciding whether your Generative UI agent reaches production or dies trying. OpenUI (open source) handles the rendering. OpenUI Cloud handles that 80% nobody should have to rebuild. > npx @openuidev/cli@latest create

Today we're launching OpenUI Cloud We've spent months arguing AI deserves more than text. The uncomfortable truth is Generative UI in production needs much more than just rendering. The boring parts - validation, error correction, fallbacks, versioning, audit trails - eat up 80% of the time and become the critical factor in deciding whether your Generative UI agent reaches production or dies trying. OpenUI (open source) handles the rendering. OpenUI Cloud handles that 80% nobody should have to rebuild. > npx @openuidev/cli@latest create

Rabi Shanker Guha

49,498 views • 2 months ago

Here is the recording ( page load ) NextJs on left ( 1.38 s = 1380ms ) Tanstack Start on the right ( 209ms ... ) ~7x - Playwright is now less flaky, 100% faster - Rendering and navigation is at least 3x faster across all dynamic and non-dynamic pages - file structure is more sane and customizable - just works client side and server side...... - Fast, easy to understand...

Here is the recording ( page load ) NextJs on left ( 1.38 s = 1380ms ) Tanstack Start on the right ( 209ms ... ) ~7x - Playwright is now less flaky, 100% faster - Rendering and navigation is at least 3x faster across all dynamic and non-dynamic pages - file structure is more sane and customizable - just works client side and server side...... - Fast, easy to understand...

Meabed

43,693 views • 8 months ago

GPT-5.5 aka 'Spud 🥔' will be released this week and its impact will echo across the fabric of time & space for eternity.

GPT-5.5 aka 'Spud 🥔' will be released this week and its impact will echo across the fabric of time & space for eternity.

Dan McAteer

21,583 views • 4 months ago

SharedNeRF: Leveraging Photorealistic and View-dependent Rendering for Real-time and Remote Collaboration Abstract (excerpt): When collaborators are remote, coordinating the sharing of views of their physical environment becomes challenging. Video-conferencing tools often do not provide the desired viewpoints for a remote viewer. While RGB-D cameras offer 3D views, they lack the necessary fidelity. We introduce SharedNeRF, designed to enhance synchronous remote collaboration by leveraging the photorealistic and view-dependent nature of Neural Radiance Field (NeRF). The system complements the higher visual quality of the NeRF rendering with the instantaneity of a point cloud and combines them through carefully accommodating the dynamic elements within the shared space, such as hand gestures and moving objects. The system employs a head-mounted camera for data collection, creating a volumetric task space on the fly and updating it as the task space changes.

SharedNeRF: Leveraging Photorealistic and View-dependent Rendering for Real-time and Remote Collaboration Abstract (excerpt): When collaborators are remote, coordinating the sharing of views of their physical environment becomes challenging. Video-conferencing tools often do not provide the desired viewpoints for a remote viewer. While RGB-D cameras offer 3D views, they lack the necessary fidelity. We introduce SharedNeRF, designed to enhance synchronous remote collaboration by leveraging the photorealistic and view-dependent nature of Neural Radiance Field (NeRF). The system complements the higher visual quality of the NeRF rendering with the instantaneity of a point cloud and combines them through carefully accommodating the dynamic elements within the shared space, such as hand gestures and moving objects. The system employs a head-mounted camera for data collection, creating a volumetric task space on the fly and updating it as the task space changes.

MrNeRF

10,615 views • 1 year ago