Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

3D Gaussian Splatting is really impressive 🙂 Pushing some test here with 25 photos as input and only 10000 iterations. Hair, eyelashes are incomparable to photogrammetry. I'll bench against NeRF a bit more in detail soon!

AlexandreDevaux's profile picture

Alexandre Devaux

7,792 subscribers

671,605 Aufrufe • vor 2 Jahren •via X (Twitter)

Wissenschaft & Technologie
LIVE

Anya Rossi• Live Now

Private livecam show

10 Kommentare

Profilbild von Alexandre Devaux
Alexandre Devauxvor 2 Jahren

By the way if you're curious about the Photogrammetric version (processed through @RealityCapture_ ) I had a demo in webgl (@threejs) here:

Profilbild von George Saoulidis ⚡
George Saoulidis ⚡vor 2 Jahren

How can I play with that without an iPhone?

Profilbild von Alexandre Devaux
Alexandre Devauxvor 2 Jahren

Some Webgl viewer are going to appear very soon! (Some already exist like

Profilbild von Justin: Immersive AR 🥽 Creator & bird app tourist
Justin: Immersive AR 🥽 Creator & bird app touristvor 2 Jahren

Absolutely incredible. This is such a great technical milestone that will allow better 3D to fall into more consumer applications the next few years (3D shopping, location previews, their own 3D photos, etc). 🎉🙌🏻

Profilbild von Julien Blanchon
Julien Blanchonvor 2 Jahren

Did you pload the .ply file somewhere. I'm writing a custom viewer, looking for datasets to test.

Profilbild von Alexandre Devaux
Alexandre Devauxvor 2 Jahren

I generated the ply using the original code also for the viewer I used their SIBR_viewer. But I dream to make it run in WebGL like ! I could sent it to you if you want to play

Profilbild von Chuck Webster MD, MS AI, MS Systems Engineering
Chuck Webster MD, MS AI, MS Systems Engineeringvor 2 Jahren

Amazing. Never heard of it. Presume this is good overview. Is there a way to view or import into a VR headset?

Profilbild von Tiddy McGee
Tiddy McGeevor 2 Jahren

VR porn will be wild in the future...

Profilbild von Michael Figured - Founder of lucido
Michael Figured - Founder of lucidovor 2 Jahren

Is it possible to export this result as an obj or fbx file?

Profilbild von Alexandre Devaux
Alexandre Devauxvor 2 Jahren

I wonder if some people already created some converters. You can easily access the point cloud but to get a textured mesh would need some work

Ähnliche Videos

holy crap! apple just beat google to the punch -- 3d gaussian splatting is coming to apple maps. these 3d scenes are made from oblique aerial imagery. but unlike blobby photogrammetry -- no more broccoli trees, no more melted powerlines -- ground level detail that actually holds up. here's hoping google maps/earth follows suite soon -- they have a significantly larger corpus of sensor data to work with. time to splat the world!
0:25

holy crap! apple just beat google to the punch -- 3d gaussian splatting is coming to apple maps. these 3d scenes are made from oblique aerial imagery. but unlike blobby photogrammetry -- no more broccoli trees, no more melted powerlines -- ground level detail that actually holds up. here's hoping google maps/earth follows suite soon -- they have a significantly larger corpus of sensor data to work with. time to splat the world!

Bilawal Sidhu

1,091,900 Aufrufe • vor 2 Monaten

Gaussian splat videos are getting really good. The detail here, especially in the fabric and hair, is seriously impressive. Well done Gracia Thanks for letting me check out the TestFlight on Apple Vision Pro.
0:59

Gaussian splat videos are getting really good. The detail here, especially in the fabric and hair, is seriously impressive. Well done Gracia Thanks for letting me check out the TestFlight on Apple Vision Pro.

Spatial Insider

75,866 Aufrufe • vor 2 Monaten

Porting over some HLSL point cloud shaders over to spark, a gaussian splatting renderer for Three.js. Some of the effects look a bit messy with 3DGS, but fun to play around with! Also using their new depth of field here which looks really nice. Demo coming soon!
1:21

Porting over some HLSL point cloud shaders over to spark, a gaussian splatting renderer for Three.js. Some of the effects look a bit messy with 3DGS, but fun to play around with! Also using their new depth of field here which looks really nice. Demo coming soon!

grade eterna

10,946 Aufrufe • vor 1 Jahr

Meta AI strikes again, with Relightable Gaussian Codec Avatars This is an update to the Meta Codec Avatars 2.0, building on 3D Gaussian Splatting. As a result, we get fully relightable real-time avatars, accurate at the hair strand level 🤯 More details below ⬇️⬇️
0:27

Meta AI strikes again, with Relightable Gaussian Codec Avatars This is an update to the Meta Codec Avatars 2.0, building on 3D Gaussian Splatting. As a result, we get fully relightable real-time avatars, accurate at the hair strand level 🤯 More details below ⬇️⬇️

Alex Carlier

445,764 Aufrufe • vor 2 Jahren

More and more users are turning their 360 footage into fully explorable 3D Gaussian Splats with View it here: This reconstruction started from a simple capture and is now a fully explorable 3D Gaussian Splat rendered in the FreeGaussian viewer, complete with point cloud reconstruction and high-fidelity spatial detail. More and more creators are discovering that their existing footage can become immersive 3D scenes. And they’re doing it for free with FreeGaussian
0:58

More and more users are turning their 360 footage into fully explorable 3D Gaussian Splats with View it here: This reconstruction started from a simple capture and is now a fully explorable 3D Gaussian Splat rendered in the FreeGaussian viewer, complete with point cloud reconstruction and high-fidelity spatial detail. More and more creators are discovering that their existing footage can become immersive 3D scenes. And they’re doing it for free with FreeGaussian

Over the Reality 🌐

14,759 Aufrufe • vor 2 Monaten

Glad to have the opportunity to collaborate with the World Labs team on their new tool Marble! It’s currently the best tool for generating 3D worlds with Gaussian Splatting. Here are some of the results I’ve been getting and putting together in a video.
1:12

Glad to have the opportunity to collaborate with the World Labs team on their new tool Marble! It’s currently the best tool for generating 3D worlds with Gaussian Splatting. Here are some of the results I’ve been getting and putting together in a video.

Cristian Peñas ░░░░░░░░

42,022 Aufrufe • vor 10 Monaten

📸 A few photos. ⚡ A few seconds. A high-quality 3D scene！🎬 Meet Topos-Lite, our new 3D Gaussian Splatting model. With just a few sparse multi-view images, it can generate a high-quality, renderable 3D scene in few seconds We’re excited to share more soon. The technical report is coming shortly — stay tuned and stay in touch. 👀 #ToposLite #3DGS #3DGeneration #SpatialAI #AIGC #photograph #3DArtwork
1:03

📸 A few photos. ⚡ A few seconds. A high-quality 3D scene！🎬 Meet Topos-Lite, our new 3D Gaussian Splatting model. With just a few sparse multi-view images, it can generate a high-quality, renderable 3D scene in few seconds We’re excited to share more soon. The technical report is coming shortly — stay tuned and stay in touch. 👀 #ToposLite #3DGS #3DGeneration #SpatialAI #AIGC #photograph #3DArtwork

InSpatio｜影溯

22,386 Aufrufe • vor 22 Tagen

3D was either pretty, or fast. Now it’s BOTH! Meet Interactive Scenes built with Gaussian Splatting: 🔥Browser & Phone-Friendly: Hyperefficient and fast rendering everywhere 👌Embed Anywhere: 8-20MB streaming files (even smaller soon!) ✨Ultra High Quality offline NeRF renders & mesh exports 🍕Creating is as easy as capturing a video on your phone Available TODAY in Luma iOS App, Luma Web, as well as the Luma API. And fully commercially usable. Get Started for Free → #lumaai #3d #ai #gaussainsplatting #nerf
0:28

3D was either pretty, or fast. Now it’s BOTH! Meet Interactive Scenes built with Gaussian Splatting: 🔥Browser & Phone-Friendly: Hyperefficient and fast rendering everywhere 👌Embed Anywhere: 8-20MB streaming files (even smaller soon!) ✨Ultra High Quality offline NeRF renders & mesh exports 🍕Creating is as easy as capturing a video on your phone Available TODAY in Luma iOS App, Luma Web, as well as the Luma API. And fully commercially usable. Get Started for Free → #lumaai #3d #ai #gaussainsplatting #nerf

Luma

528,804 Aufrufe • vor 2 Jahren

Nvidia announces GAvatar: Animatable 3D Gaussian Avatars with Implicit Mesh Learning paper page: Gaussian splatting has emerged as a powerful 3D representation that harnesses the advantages of both explicit (mesh) and implicit (NeRF) 3D representations. In this paper, we seek to leverage Gaussian splatting to generate realistic animatable avatars from textual descriptions, addressing the limitations (e.g., flexibility and efficiency) imposed by mesh or NeRF-based representations. However, a naive application of Gaussian splatting cannot generate high-quality animatable avatars and suffers from learning instability; it also cannot capture fine avatar geometries and often leads to degenerate body parts. To tackle these problems, we first propose a primitive-based 3D Gaussian representation where Gaussians are defined inside pose-driven primitives to facilitate animation. Second, to stabilize and amortize the learning of millions of Gaussians, we propose to use neural implicit fields to predict the Gaussian attributes (e.g., colors). Finally, to capture fine avatar geometries and extract detailed meshes, we propose a novel SDF-based implicit mesh learning approach for 3D Gaussians that regularizes the underlying geometries and extracts highly detailed textured meshes. Our proposed method, GAvatar, enables the large-scale generation of diverse animatable avatars using only text prompts. GAvatar significantly surpasses existing methods in terms of both appearance and geometry quality, and achieves extremely fast rendering (100 fps) at 1K resolution.
5:36

Nvidia announces GAvatar: Animatable 3D Gaussian Avatars with Implicit Mesh Learning paper page: Gaussian splatting has emerged as a powerful 3D representation that harnesses the advantages of both explicit (mesh) and implicit (NeRF) 3D representations. In this paper, we seek to leverage Gaussian splatting to generate realistic animatable avatars from textual descriptions, addressing the limitations (e.g., flexibility and efficiency) imposed by mesh or NeRF-based representations. However, a naive application of Gaussian splatting cannot generate high-quality animatable avatars and suffers from learning instability; it also cannot capture fine avatar geometries and often leads to degenerate body parts. To tackle these problems, we first propose a primitive-based 3D Gaussian representation where Gaussians are defined inside pose-driven primitives to facilitate animation. Second, to stabilize and amortize the learning of millions of Gaussians, we propose to use neural implicit fields to predict the Gaussian attributes (e.g., colors). Finally, to capture fine avatar geometries and extract detailed meshes, we propose a novel SDF-based implicit mesh learning approach for 3D Gaussians that regularizes the underlying geometries and extracts highly detailed textured meshes. Our proposed method, GAvatar, enables the large-scale generation of diverse animatable avatars using only text prompts. GAvatar significantly surpasses existing methods in terms of both appearance and geometry quality, and achieves extremely fast rendering (100 fps) at 1K resolution.

AK

140,992 Aufrufe • vor 2 Jahren

Drivable 3D Gaussian Avatars paper page: present Drivable 3D Gaussian Avatars (D3GA), the first 3D controllable model for human bodies rendered with Gaussian splats. Current photorealistic drivable avatars require either accurate 3D registrations during training, dense input images during testing, or both. The ones based on neural radiance fields also tend to be prohibitively slow for telepresence applications. This work uses the recently presented 3D Gaussian Splatting (3DGS) technique to render realistic humans at real-time framerates, using dense calibrated multi-view videos as input. To deform those primitives, we depart from the commonly used point deformation method of linear blend skinning (LBS) and use a classic volumetric deformation method: cage deformations. Given their smaller size, we drive these deformations with joint angles and keypoints, which are more suitable for communication applications. Our experiments on nine subjects with varied body shapes, clothes, and motions obtain higher-quality results than state-of-the-art methods when using the same training and test data.
6:20

Drivable 3D Gaussian Avatars paper page: present Drivable 3D Gaussian Avatars (D3GA), the first 3D controllable model for human bodies rendered with Gaussian splats. Current photorealistic drivable avatars require either accurate 3D registrations during training, dense input images during testing, or both. The ones based on neural radiance fields also tend to be prohibitively slow for telepresence applications. This work uses the recently presented 3D Gaussian Splatting (3DGS) technique to render realistic humans at real-time framerates, using dense calibrated multi-view videos as input. To deform those primitives, we depart from the commonly used point deformation method of linear blend skinning (LBS) and use a classic volumetric deformation method: cage deformations. Given their smaller size, we drive these deformations with joint angles and keypoints, which are more suitable for communication applications. Our experiments on nine subjects with varied body shapes, clothes, and motions obtain higher-quality results than state-of-the-art methods when using the same training and test data.

AK

327,105 Aufrufe • vor 2 Jahren

A comparison between my script, which converts a single image into Gaussian Splatting, and Apple’s Sharp. I’m getting better and more consistent results, with no deformation, and I can generate as many viewing angles as I want. The only downside is that it takes more time to compute.
0:44

A comparison between my script, which converts a single image into Gaussian Splatting, and Apple’s Sharp. I’m getting better and more consistent results, with no deformation, and I can generate as many viewing angles as I want. The only downside is that it takes more time to compute.

Alex

14,355 Aufrufe • vor 2 Monaten

Xenia Netplay recently released the 5.0 version and the stability improvements are amazing. We are able to play GoldenEye XBLA against players in Australia while I'm here in Spain with no desync or any input delay. The setup is also much simpler now, I'll have a tutorial soon!
1:23

Xenia Netplay recently released the 5.0 version and the stability improvements are amazing. We are able to play GoldenEye XBLA against players in Australia while I'm here in Spain with no desync or any input delay. The setup is also much simpler now, I'll have a tutorial soon!

Graslu00

20,667 Aufrufe • vor 1 Jahr

Here some overall test footage, WIP bot npcs and performance. Saw some Death Stranding 2 on the Internet and decided to go back and update the lighting for a bit. I'm gonna work on characters and 'cargo mechanic' next month. (I'll also turn 2D UI into 3D UI again soon)
0:52

Here some overall test footage, WIP bot npcs and performance. Saw some Death Stranding 2 on the Internet and decided to go back and update the lighting for a bit. I'm gonna work on characters and 'cargo mechanic' next month. (I'll also turn 2D UI into 3D UI again soon)

MATTO

70,995 Aufrufe • vor 1 Jahr

Finally got #gaussiansplatting working! I’m really impressed by how well it captured the fine details of the sneaker fabric and shoelaces, as well as the text. This was generated from 75 DSLR photos I shot on a 24mm lens, took about 30 minutes to process on an RTX 3090, and is running at a smooth 60fps on the SIBR Viewer. Going to run more tests to see how this compares to some of my experiments in photogrammetry. I’m hoping a plugin for #UnrealEngine5 comes out soon! #nerf
0:21

Finally got #gaussiansplatting working! I’m really impressed by how well it captured the fine details of the sneaker fabric and shoelaces, as well as the text. This was generated from 75 DSLR photos I shot on a 24mm lens, took about 30 minutes to process on an RTX 3090, and is running at a smooth 60fps on the SIBR Viewer. Going to run more tests to see how this compares to some of my experiments in photogrammetry. I’m hoping a plugin for #UnrealEngine5 comes out soon! #nerf

CoffeeVectors

183,486 Aufrufe • vor 2 Jahren

WOW. 😳 Apple just quietly won the 3D maps war at WWDC. Gaussian Splatting is coming to Apple Maps Flyover this fall. Apple Maps Flyover covers 300+ cities. Until yesterday, every single one was built on standard drone photogrammetry. The technology captures photos from the air and reconstructs 3D geometry from them. Gaussian Splatting does not reconstruct geometry. It represents the scene as millions of tiny 3D ellipsoids, each one carrying its own color and opacity information based on how light actually behaves in that location. The output is not a mesh model. It is a field of light. When you move through it, it does not crumble at the edges. The detail holds because it was never geometry to begin with. Apple has been hiring for this for years. Their SHARP model, published in research last year, generates photorealistic 3D scenes from a single image in under a second. Google has more sensor data than anyone. More Street View cars, more satellites, more capture history. On navigation accuracy and geodata depth, Google Maps is still ahead by most measures. But fidelity in 3D city rendering is a different competition, and Apple just set a bar in that. Most people will experience this in the fall without knowing the name of the technology. They will open Flyover, look at a city they know, and notice it looks different. Real, not rendered. That is the moment Gaussian Splatting stops being a research term and becomes something a billion people use. Bookmark this. It will look prescient by October.
0:24

WOW. 😳 Apple just quietly won the 3D maps war at WWDC. Gaussian Splatting is coming to Apple Maps Flyover this fall. Apple Maps Flyover covers 300+ cities. Until yesterday, every single one was built on standard drone photogrammetry. The technology captures photos from the air and reconstructs 3D geometry from them. Gaussian Splatting does not reconstruct geometry. It represents the scene as millions of tiny 3D ellipsoids, each one carrying its own color and opacity information based on how light actually behaves in that location. The output is not a mesh model. It is a field of light. When you move through it, it does not crumble at the edges. The detail holds because it was never geometry to begin with. Apple has been hiring for this for years. Their SHARP model, published in research last year, generates photorealistic 3D scenes from a single image in under a second. Google has more sensor data than anyone. More Street View cars, more satellites, more capture history. On navigation accuracy and geodata depth, Google Maps is still ahead by most measures. But fidelity in 3D city rendering is a different competition, and Apple just set a bar in that. Most people will experience this in the fall without knowing the name of the technology. They will open Flyover, look at a city they know, and notice it looks different. Real, not rendered. That is the moment Gaussian Splatting stops being a research term and becomes something a billion people use. Bookmark this. It will look prescient by October.

Shruti

19,832 Aufrufe • vor 2 Monaten

Enter the SPLATRIX! We digitized Henry Pearce and converted him over to 4DGS to be played back in real-time. 3D Gaussians Splats in Motion. 4DGS. Our goal was to improve the quality of our AeonX & 4DGS pipeline, with automated background removal to retain fine hair details as well as to improve overall surface consistency on a full torso capture. We're currently using a post process to temporally filter the 2D frames. We still need to solve 4DGS temporal stability. We're hoping to test with Jonathon Luiten dynamic gaussian code soon... 🙏 for that script! We captured this performance under global illumination. This is an easy task compared to RGB+W lighting, which is going to be much harder! This is our next challenge. We're posting these snippet videos to show our progress with 3D/4DGS. We plan to write a full blog post on our 3D/4DGS journey soon. 3D Gaussian Splatting #GaussianSplatting #inria #radiancefields #ir #aeonx #rgbw #idatronic #ximea #sibr #hdri #60fps More info here - Higher Quality Videos:
1:14

Enter the SPLATRIX! We digitized Henry Pearce and converted him over to 4DGS to be played back in real-time. 3D Gaussians Splats in Motion. 4DGS. Our goal was to improve the quality of our AeonX & 4DGS pipeline, with automated background removal to retain fine hair details as well as to improve overall surface consistency on a full torso capture. We're currently using a post process to temporally filter the 2D frames. We still need to solve 4DGS temporal stability. We're hoping to test with Jonathon Luiten dynamic gaussian code soon... 🙏 for that script! We captured this performance under global illumination. This is an easy task compared to RGB+W lighting, which is going to be much harder! This is our next challenge. We're posting these snippet videos to show our progress with 3D/4DGS. We plan to write a full blog post on our 3D/4DGS journey soon. 3D Gaussian Splatting #GaussianSplatting #inria #radiancefields #ir #aeonx #rgbw #idatronic #ximea #sibr #hdri #60fps More info here - Higher Quality Videos:

Infinite-Realities

90,662 Aufrufe • vor 2 Jahren

I went a little crazy with my first 360-drone captured gaussian splat. Of a public park I always bring my daughter to It really is so amazing how much detail a person could capture in 3D with just hobbyist hardware and software
0:57

I went a little crazy with my first 360-drone captured gaussian splat. Of a public park I always bring my daughter to It really is so amazing how much detail a person could capture in 3D with just hobbyist hardware and software

Brad Lynch

35,155 Aufrufe • vor 4 Monaten

3D scanning and rendering is moving so fast - got my splats up and running and I'm mind blown getting ~100fps for this complex 3D scene ⬇️ 🤯 1. WAY faster than NeRF: For comparison, NeRFs would takes around 10 seconds per frame (!) Instead I'm zipping around with FPV controls without breaking a sweat - though I do crash a few times towards the end of the video lol 2. Old Meets New: Gaussian Splatting is cool in that it fuses classical graphics and deep learning techniques. Like NeRFs, this is still a radiance field - just without the slower (ne)ural rendering part. 3. Explicit Representation: Instead you represent a 3D scene as a collection of ellipsoidal "splats" called gaussians. Each gaussian has a position, size, and color. Rendering in real-time is done by projecting into the image plane and alpha blending. 4. Photorealistic Effects: Gaussian splatting use spherical harmonics to represent the view-dependent effects and lighting - allowing surfaces to change color when viewed from different angles, enabling greater photorealism. It doesn't use a neural network, but the training loop is similar to deep learning. 5. Enables Direct Editing: But it's not just speed - with Gaussian Splatting you also get 3D editing support! So you can select, move, and delete stuff, even relight stuff. This type of editing has been more tedious to do with NeRFs and their implicit black box representations. 📲 More tests cooking! Much more to unpack here including simpler explanations. If you enjoyed this post, you might enjoy my feed: Bilawal Sidhu
0:35

3D scanning and rendering is moving so fast - got my splats up and running and I'm mind blown getting ~100fps for this complex 3D scene ⬇️ 🤯 1. WAY faster than NeRF: For comparison, NeRFs would takes around 10 seconds per frame (!) Instead I'm zipping around with FPV controls without breaking a sweat - though I do crash a few times towards the end of the video lol 2. Old Meets New: Gaussian Splatting is cool in that it fuses classical graphics and deep learning techniques. Like NeRFs, this is still a radiance field - just without the slower (ne)ural rendering part. 3. Explicit Representation: Instead you represent a 3D scene as a collection of ellipsoidal "splats" called gaussians. Each gaussian has a position, size, and color. Rendering in real-time is done by projecting into the image plane and alpha blending. 4. Photorealistic Effects: Gaussian splatting use spherical harmonics to represent the view-dependent effects and lighting - allowing surfaces to change color when viewed from different angles, enabling greater photorealism. It doesn't use a neural network, but the training loop is similar to deep learning. 5. Enables Direct Editing: But it's not just speed - with Gaussian Splatting you also get 3D editing support! So you can select, move, and delete stuff, even relight stuff. This type of editing has been more tedious to do with NeRFs and their implicit black box representations. 📲 More tests cooking! Much more to unpack here including simpler explanations. If you enjoyed this post, you might enjoy my feed: Bilawal Sidhu

Bilawal Sidhu

337,090 Aufrufe • vor 2 Jahren

✨ Today we are ready to take AI Graphics to the next level: full volumetric photorealistic NeRF rendering on the web. All in realtime! What you see is what you get for photorealistic 3D is finally here! Starting with the the Fields Editor soon you’ll be able to reshoot in NeRF, not a proxy. Following that, we will bring Luma's NeRF-RT renderer to other tools 🚀
0:26

✨ Today we are ready to take AI Graphics to the next level: full volumetric photorealistic NeRF rendering on the web. All in realtime! What you see is what you get for photorealistic 3D is finally here! Starting with the the Fields Editor soon you’ll be able to reshoot in NeRF, not a proxy. Following that, we will bring Luma's NeRF-RT renderer to other tools 🚀

Luma

842,032 Aufrufe • vor 3 Jahren

GS^3: Efficient Relighting with Triple Gaussian Splatting Abstract: We present a spatial and angular Gaussian based representation and a triple splatting process, for real-time, high-quality novel lighting-and-view synthesis from multi-view point-lit input images. To describe complex ap pearance, we employ a Lambertian plus a mixture of angular Gaussians as an effective reflectance function for each spatial Gaussian. To generate self-shadow, we splat all spatial Gaussians towards the light source to obtain shadow values, which are further refined by a small multi-layer perceptron. To compensate for other effects like global illumination, another network is trained to compute and add a per-spatial-Gaussian RGB tuple. The effectiveness of our representation is demonstrated on 30 samples with a wide variation in geometry (from solid to fluffy) and appearance (from translucent to anisotropic), as well as using different forms of input data, including rendered images of synthetic/reconstructed objects, photographs captured with a handheld camera and a flash, or from a professional lightstage. We achieve a training time of 40-70 minutes and a rendering speed of 90 fps on a single commodity GPU. Our results compare favorably with state-of-the-art techniques in terms of quality/performance.
0:40

GS^3: Efficient Relighting with Triple Gaussian Splatting Abstract: We present a spatial and angular Gaussian based representation and a triple splatting process, for real-time, high-quality novel lighting-and-view synthesis from multi-view point-lit input images. To describe complex ap pearance, we employ a Lambertian plus a mixture of angular Gaussians as an effective reflectance function for each spatial Gaussian. To generate self-shadow, we splat all spatial Gaussians towards the light source to obtain shadow values, which are further refined by a small multi-layer perceptron. To compensate for other effects like global illumination, another network is trained to compute and add a per-spatial-Gaussian RGB tuple. The effectiveness of our representation is demonstrated on 30 samples with a wide variation in geometry (from solid to fluffy) and appearance (from translucent to anisotropic), as well as using different forms of input data, including rendered images of synthetic/reconstructed objects, photographs captured with a handheld camera and a flash, or from a professional lightstage. We achieve a training time of 40-70 minutes and a rendering speed of 90 fps on a single commodity GPU. Our results compare favorably with state-of-the-art techniques in terms of quality/performance.

MrNeRF

17,786 Aufrufe • vor 1 Jahr