Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

First fully ML-framework-free 3D Gaussian Splatting implementation in LichtFeld Studio. I’ve completed the migration of the full training pipeline to a custom CUDA-based tensor library. No PyTorch, no LibTorch, no autograd. Every gradient is implemented by hand, either through CUDA kernels or minimal abstractions on top. This makes it...

janusch_patas's profile picture

MrNeRF

16,847 subscribers

50,571 Aufrufe • vor 1 Jahr •via X (Twitter)

Wissenschaft & Technologie
LIVE

Anya Rossi• Live Now

Private livecam show

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

Apple just trained a 3D Gaussian head reconstruction model on 10,000+ subjects. Feed-forward. No test-time optimization. New identity in, reconstructed Gaussian head out. The UV-parameterized Gaussian representation decouples the number of Gaussians from the number and resolution of input images, making it practical to train with many high resolution views. And the heads are not just static either: text-conditioned identity generation, plus blendshape-driven latent animation across identities. We've been building in the 3D Gaussian Splatting space for a while. The gap between "research demo" and "works on real people at scale" is closing fast.
0:12

Apple just trained a 3D Gaussian head reconstruction model on 10,000+ subjects. Feed-forward. No test-time optimization. New identity in, reconstructed Gaussian head out. The UV-parameterized Gaussian representation decouples the number of Gaussians from the number and resolution of input images, making it practical to train with many high resolution views. And the heads are not just static either: text-conditioned identity generation, plus blendshape-driven latent animation across identities. We've been building in the 3D Gaussian Splatting space for a while. The gap between "research demo" and "works on real people at scale" is closing fast.

KIRI Engine - 3D Scanner App

12,266 Aufrufe • vor 5 Monaten

In the actual account of meeting Brig ML Khetarpal and Brig Naseer, there is no cowardice, there is no repentance, there is no anti-war sentiment. Only Soldierly pride and mutual respect for professionalism. Brig ML Khetarpal was a third generation soldier. He could have never said things like, why his son did not yield (in the face of enemy fire). This fictionalized account is a disgrace to the memory of the paramveer Arun Khterpal and a disservice to his father Brig ML Khetarpal. Today, Brig Khetarpal is not there to refute this mockery, but are we all dead to accept this farce?
0:10

In the actual account of meeting Brig ML Khetarpal and Brig Naseer, there is no cowardice, there is no repentance, there is no anti-war sentiment. Only Soldierly pride and mutual respect for professionalism. Brig ML Khetarpal was a third generation soldier. He could have never said things like, why his son did not yield (in the face of enemy fire). This fictionalized account is a disgrace to the memory of the paramveer Arun Khterpal and a disservice to his father Brig ML Khetarpal. Today, Brig Khetarpal is not there to refute this mockery, but are we all dead to accept this farce?

We, the people of India

82,415 Aufrufe • vor 10 Monaten

F3D-Gaus: Feed-forward 3D-aware Generation on ImageNet with Cycle-Consistent Gaussian Splatting Contributions: • We pioneer 3D-aware generation using generalizable feed-forward Gaussian Splatting representation, achieving significant efficiency and favorable rendering quality on monocular datasets. • We significantly advance the capability of pixel-aligned Gaussian Splatting representations by designing a self-supervised cycle training strategy specifically tailored for monocular datasets. • We further mitigate the artifacts of 3D-aware representations caused by large viewpoint shifts by introducing geometry-aware video priors.
0:16

F3D-Gaus: Feed-forward 3D-aware Generation on ImageNet with Cycle-Consistent Gaussian Splatting Contributions: • We pioneer 3D-aware generation using generalizable feed-forward Gaussian Splatting representation, achieving significant efficiency and favorable rendering quality on monocular datasets. • We significantly advance the capability of pixel-aligned Gaussian Splatting representations by designing a self-supervised cycle training strategy specifically tailored for monocular datasets. • We further mitigate the artifacts of 3D-aware representations caused by large viewpoint shifts by introducing geometry-aware video priors.

MrNeRF

14,229 Aufrufe • vor 1 Jahr

MoE-GS: Mixture of Experts for Dynamic Gaussian Splatting Contributions: • MoE-GS: the first dynamic Gaussian splatting framework employing a Mixture-of-Experts architecture, enabling robust and adaptive reconstruction across diverse dynamic scenes. • A novel Volume-aware Pixel Router integrates expert outputs through differentiable weight splatting, achieving spatially and temporally coherent adaptive blending. • Efficiency of MoE-GS is improved through single-pass multi-expert rendering and gate-aware Gaussian pruning. A separate knowledge distillation strategy trains individual experts with pseudo-labels from the MoE model, enhancing quality without modifying the architecture.
0:10

MoE-GS: Mixture of Experts for Dynamic Gaussian Splatting Contributions: • MoE-GS: the first dynamic Gaussian splatting framework employing a Mixture-of-Experts architecture, enabling robust and adaptive reconstruction across diverse dynamic scenes. • A novel Volume-aware Pixel Router integrates expert outputs through differentiable weight splatting, achieving spatially and temporally coherent adaptive blending. • Efficiency of MoE-GS is improved through single-pass multi-expert rendering and gate-aware Gaussian pruning. A separate knowledge distillation strategy trains individual experts with pseudo-labels from the MoE model, enhancing quality without modifying the architecture.

MrNeRF

10,408 Aufrufe • vor 1 Jahr

Two weeks ago I fixed one of my teeth with algorithms I wrote a couple of years ago! I got hooked by 3D scanning when I started to work for a software shop in Zurich that was programming 3D computational geometry algorithms for denture scanning to produce crowns (and more). Back then, a typical reconstruction pipeline was like: scan the patient’s teeth using an intraoral scanner, reconstruct the surface mesh, design the restoration digitally, and finally mill the crown out of ceramic. We were working mostly with point clouds and meshes, but it wasn’t just math, it was craftsmanship translated into a digital process. Every micron mattered. You could literally see how a good algorithm meant a better fit in someone’s mouth. Gaussian Splatting isn’t about surface reconstruction, it’s about appearance reconstruction. It doesn’t care about explicit topology, it captures how light interacts with the scene. In a sense, it’s the opposite philosophy of the dental world: instead of modeling what the object is, it models how the object looks. 3D Gaussian Splatting enables applications like training self driving cars, teaching robots to understand their environment, creating virtual worlds, or monitoring real sites. It represents scenes as millions of small Gaussians rendered in real time without the need for meshes or textures. Coming from a world where precision geometry was everything, this shift felt natural. It’s still about reconstruction, but with a different goal: not manufacturing a perfect object, but reproducing how the world actually looks. Two weeks ago I got my first dental crown, made with the same software, reconstruction algorithms, and Swiss precision I once helped develop. I haven’t worked there in two years, but sitting in that chair and seeing the process from the other side was a proud moment. It reminded me why I love this field.
0:18

Two weeks ago I fixed one of my teeth with algorithms I wrote a couple of years ago! I got hooked by 3D scanning when I started to work for a software shop in Zurich that was programming 3D computational geometry algorithms for denture scanning to produce crowns (and more). Back then, a typical reconstruction pipeline was like: scan the patient’s teeth using an intraoral scanner, reconstruct the surface mesh, design the restoration digitally, and finally mill the crown out of ceramic. We were working mostly with point clouds and meshes, but it wasn’t just math, it was craftsmanship translated into a digital process. Every micron mattered. You could literally see how a good algorithm meant a better fit in someone’s mouth. Gaussian Splatting isn’t about surface reconstruction, it’s about appearance reconstruction. It doesn’t care about explicit topology, it captures how light interacts with the scene. In a sense, it’s the opposite philosophy of the dental world: instead of modeling what the object is, it models how the object looks. 3D Gaussian Splatting enables applications like training self driving cars, teaching robots to understand their environment, creating virtual worlds, or monitoring real sites. It represents scenes as millions of small Gaussians rendered in real time without the need for meshes or textures. Coming from a world where precision geometry was everything, this shift felt natural. It’s still about reconstruction, but with a different goal: not manufacturing a perfect object, but reproducing how the world actually looks. Two weeks ago I got my first dental crown, made with the same software, reconstruction algorithms, and Swiss precision I once helped develop. I haven’t worked there in two years, but sitting in that chair and seeing the process from the other side was a proud moment. It reminded me why I love this field.

MrNeRF

290,319 Aufrufe • vor 1 Jahr

Single video → a reframeable 4D Gaussian Splatting scene. Not a sequence of separately built 3D frames played back like a video. This is one continuous space-time scene, reconstructed from a single clip shot on an iPhone 16. We combine feed-forward Gaussian generation, 3D tracking, and 4D Gaussian Splatting, aiming to deliver it as a compact 4D video file that runs on your phone. Still early R&D. The goal: make 4DGS something anyone can create and experience, not just researchers with a camera rig.
0:16

Single video → a reframeable 4D Gaussian Splatting scene. Not a sequence of separately built 3D frames played back like a video. This is one continuous space-time scene, reconstructed from a single clip shot on an iPhone 16. We combine feed-forward Gaussian generation, 3D tracking, and 4D Gaussian Splatting, aiming to deliver it as a compact 4D video file that runs on your phone. Still early R&D. The goal: make 4DGS something anyone can create and experience, not just researchers with a camera rig.

KIRI Engine - 3D Scanner App

31,871 Aufrufe • vor 3 Monaten

Universal Beta Splatting Contributions: • Universal Beta Splatting: A unified N-dimensional representation with per-dimension shape control, enabling simultaneous modeling of spatial, angular, and temporal properties through anisotropic Beta kernels with spatial-orthogonal Cholesky parameterization. • Efficient CUDA implementation: Fully accelerated differentiable rendering with custom kernels, achieving real-time performance. • Interpretable scene decomposition: Emergent separation of geometric, appearance, and motion components through learned Beta parameters without supervision. • Backward compatibility: Reduction to approximate existing methods as special cases, ensuring plug-in usability with performance lower bounds while enabling substantial improvements.
0:19

Universal Beta Splatting Contributions: • Universal Beta Splatting: A unified N-dimensional representation with per-dimension shape control, enabling simultaneous modeling of spatial, angular, and temporal properties through anisotropic Beta kernels with spatial-orthogonal Cholesky parameterization. • Efficient CUDA implementation: Fully accelerated differentiable rendering with custom kernels, achieving real-time performance. • Interpretable scene decomposition: Emergent separation of geometric, appearance, and motion components through learned Beta parameters without supervision. • Backward compatibility: Reduction to approximate existing methods as special cases, ensuring plug-in usability with performance lower bounds while enabling substantial improvements.

MrNeRF

10,281 Aufrufe • vor 1 Jahr

In a healthy adult male, the bladder doesn’t have a fixed size, it’s a dynamic organ. Most people start feeling the first urge to urinate around 150–250 mL, a stronger urge around 300–400 mL, and the usual functional capacity is roughly 400–600 mL. It can stretch beyond that, sometimes up to 700–800 mL, but that’s uncomfortable and not considered normal everyday function. Now, once a urinary catheter is in place, the physiology changes. The bladder is no longer “storing” urine the way it normally does. Instead, urine drains continuously through the catheter into the collection bag, so the bladder usually stays relatively empty. That’s why in catheterized patients, what you see accumulating is not bladder capacity, it’s just the total urine output over time. As for the urine bags, standard ones typically hold around 1500–2000 mL, which is accurate and designed to safely collect urine over several hours. So overall, the concept is right, just important to remember that with a catheter, you’re no longer measuring how much the bladder can hold, but how much the kidneys are producing.
0:10

In a healthy adult male, the bladder doesn’t have a fixed size, it’s a dynamic organ. Most people start feeling the first urge to urinate around 150–250 mL, a stronger urge around 300–400 mL, and the usual functional capacity is roughly 400–600 mL. It can stretch beyond that, sometimes up to 700–800 mL, but that’s uncomfortable and not considered normal everyday function. Now, once a urinary catheter is in place, the physiology changes. The bladder is no longer “storing” urine the way it normally does. Instead, urine drains continuously through the catheter into the collection bag, so the bladder usually stays relatively empty. That’s why in catheterized patients, what you see accumulating is not bladder capacity, it’s just the total urine output over time. As for the urine bags, standard ones typically hold around 1500–2000 mL, which is accurate and designed to safely collect urine over several hours. So overall, the concept is right, just important to remember that with a catheter, you’re no longer measuring how much the bladder can hold, but how much the kidneys are producing.

Op. Dr. Mehmet Bekir Şen

41,089 Aufrufe • vor 6 Monaten

🚨 Anthropic committed up to 1M TPU chips for Claude. Openai is leasing TPUs for chatgpt inference. Here's How kernels work on TPUs (deep dive 2/6 by emilio andere) pallas is Google's answer to kernel writing. a python kernel SDK built on JAX. still very experimental (jax.experimental.pallas). on TPU it compiles through mosaic; on GPU it lowers to triton. if you know CUDA, the syntax will feel familiar but the execution model is completely different. in CUDA, grid=(4,4) launches 16 blocks running simultaneously across SMs. in pallas, those 16 iterations run one after another in lexicographic order. no threads. no warps. no blocks. no occupancy tuning. a TPU is a sequential machine with a very wide vector register — more like a CPU than a GPU. performance comes from width: a 128x128 systolic array doing matmul and an 8x128 SIMD vector unit doing everything else. maximum parallelism on chip: 2, one per TensorCore in megacore mode. three concepts replace CUDA's thread/block/grid hierarchy. Refs are mutable memory references. because execution is sequential, each iteration safely accumulates without atomics. in CUDA you'd need atomics or a separate reduction pass. the memory model is also very different from NVIDIA's. zero hardware caches. VMEM is 32-128 MiB of software-managed scratchpad — 500-1000x larger than GPU shared memory per SM. all data must be explicitly DMA'd from HBM to VMEM before any computation touches it. four levels: HBM → VMEM → VREGs → MXU/VPU, plus SMEM for scalar control data. every byte of data movement is your responsibility. this is like CUDA shared memory except it's 500x bigger and there's no cache fallback. pipelining is mandatory. without double-buffering HBM→VMEM transfers, the MXU just stalls waiting for data. this is the single most important optimization on TPU. and because grid execution is sequential and deterministic, consecutive iterations that need the same input block skip the redundant HBM transfer automatically, impossible on GPU where block execution order is undefined. the compilation pipeline is unlike anything in this series: python → jaxpr → stableHLO → XLA HLO (71+ optimization passes) → LLO (78+ passes) → 322-bit VLIW bundles. the compiler packs instructions for scalar, vector, matrix, and DMA units into a single 322-bit word. everything in that bundle executes in parallel, with no runtime scheduling.
0:18

🚨 Anthropic committed up to 1M TPU chips for Claude. Openai is leasing TPUs for chatgpt inference. Here's How kernels work on TPUs (deep dive 2/6 by emilio andere) pallas is Google's answer to kernel writing. a python kernel SDK built on JAX. still very experimental (jax.experimental.pallas). on TPU it compiles through mosaic; on GPU it lowers to triton. if you know CUDA, the syntax will feel familiar but the execution model is completely different. in CUDA, grid=(4,4) launches 16 blocks running simultaneously across SMs. in pallas, those 16 iterations run one after another in lexicographic order. no threads. no warps. no blocks. no occupancy tuning. a TPU is a sequential machine with a very wide vector register — more like a CPU than a GPU. performance comes from width: a 128x128 systolic array doing matmul and an 8x128 SIMD vector unit doing everything else. maximum parallelism on chip: 2, one per TensorCore in megacore mode. three concepts replace CUDA's thread/block/grid hierarchy. Refs are mutable memory references. because execution is sequential, each iteration safely accumulates without atomics. in CUDA you'd need atomics or a separate reduction pass. the memory model is also very different from NVIDIA's. zero hardware caches. VMEM is 32-128 MiB of software-managed scratchpad — 500-1000x larger than GPU shared memory per SM. all data must be explicitly DMA'd from HBM to VMEM before any computation touches it. four levels: HBM → VMEM → VREGs → MXU/VPU, plus SMEM for scalar control data. every byte of data movement is your responsibility. this is like CUDA shared memory except it's 500x bigger and there's no cache fallback. pipelining is mandatory. without double-buffering HBM→VMEM transfers, the MXU just stalls waiting for data. this is the single most important optimization on TPU. and because grid execution is sequential and deterministic, consecutive iterations that need the same input block skip the redundant HBM transfer automatically, impossible on GPU where block execution order is undefined. the compilation pipeline is unlike anything in this series: python → jaxpr → stableHLO → XLA HLO (71+ optimization passes) → LLO (78+ passes) → 322-bit VLIW bundles. the compiler packs instructions for scalar, vector, matrix, and DMA units into a single 322-bit word. everything in that bundle executes in parallel, with no runtime scheduling.

wafer

33,655 Aufrufe • vor 4 Monaten

In a healthy adult male, the bladder doesn’t have a fixed size, it’s a dynamic organ. Most people start feeling the first urge to urinate around 150–250 mL, a stronger urge around 300–400 mL, and the usual functional capacity is roughly 400–600 mL. It can stretch beyond that, sometimes up to 700–800 mL, but that’s uncomfortable and not considered normal everyday function. Now, once a urinary catheter is in place, the physiology changes. The bladder is no longer “storing” urine the way it normally does. Instead, urine drains continuously through the catheter into the collection bag, so the bladder usually stays relatively empty. That’s why in catheterized patients, what you see accumulating is not bladder capacity, it’s just the total urine output over time. As for the urine bags, standard ones typically hold around 1500–2000 mL, which is accurate and designed to safely collect urine over several hours. So overall, the concept is right, just important to remember that with a catheter, you’re no longer measuring how much the bladder can hold, but how much the kidneys are producing.
0:10

In a healthy adult male, the bladder doesn’t have a fixed size, it’s a dynamic organ. Most people start feeling the first urge to urinate around 150–250 mL, a stronger urge around 300–400 mL, and the usual functional capacity is roughly 400–600 mL. It can stretch beyond that, sometimes up to 700–800 mL, but that’s uncomfortable and not considered normal everyday function. Now, once a urinary catheter is in place, the physiology changes. The bladder is no longer “storing” urine the way it normally does. Instead, urine drains continuously through the catheter into the collection bag, so the bladder usually stays relatively empty. That’s why in catheterized patients, what you see accumulating is not bladder capacity, it’s just the total urine output over time. As for the urine bags, standard ones typically hold around 1500–2000 mL, which is accurate and designed to safely collect urine over several hours. So overall, the concept is right, just important to remember that with a catheter, you’re no longer measuring how much the bladder can hold, but how much the kidneys are producing.

Op. Dr. Mehmet Bekir Şen

42,580 Aufrufe • vor 5 Monaten

More coming soon ! I manage to integrate for the first time volumetric 3D video made with Kartel.ai inside the AI world generated in gaussian splatting by World Labs !! I coded all in three.js! And it's possible to integrate elements on the fly ;) And so you can imagine soon what we will produce with that en AI : consistency of character and environment, relighting etc...
0:14

More coming soon ! I manage to integrate for the first time volumetric 3D video made with Kartel.ai inside the AI world generated in gaussian splatting by World Labs !! I coded all in three.js! And it's possible to integrate elements on the fly ;) And so you can imagine soon what we will produce with that en AI : consistency of character and environment, relighting etc...

Lovis Odin

33,984 Aufrufe • vor 1 Jahr

Getting a bunch of questions about this! It's a full gpu rendering pipeline with built in shader editor for Rive, which can interleave 2D + 3D. Think WebGPU (minus compute, for now) that runs anywhere, not just the browser. You can implement anything at all. Cell shading. Background blurs. Deformation effects. And yes, even Rive vector components as textures. Shaders are precompiled and ship with your .riv. Minimal runtime size impact. Still WIP.
0:13

Getting a bunch of questions about this! It's a full gpu rendering pipeline with built in shader editor for Rive, which can interleave 2D + 3D. Think WebGPU (minus compute, for now) that runs anywhere, not just the browser. You can implement anything at all. Cell shading. Background blurs. Deformation effects. And yes, even Rive vector components as textures. Shaders are precompiled and ship with your .riv. Minimal runtime size impact. Still WIP.

Guido Rosso

22,346 Aufrufe • vor 7 Monaten

New from PocketJS: quickjs-oxide. A JavaScript interpreter modeled after QuickJS, implemented from scratch by AI in Rust with zero unsafe code, and the first of its kind to achieve performance competitive with the original QuickJS. We spent one month achieving a high Test262 pass rate, then another month cleaning up the code, getting rid of the AI slop, and reaching competitive performance with just a stack VM and reference counting. Over the last weekend at RustChinaConf, we rewrote the core into a register VM and achieved this level of performance in just two days. Now quickjs-oxide outperforms most non-JIT JavaScript engines on benchmarks like V8 v7 and the Octane suite, using only around 30% more memory than QuickJS, while maintaining zero lines of unsafe implementation code and no external dependencies except for BigInt support. Huge credit to Ericoolen(薄肌版) for making this happen, and massive thanks to our sponsor Zephyr Cloud for supporting the journey!
0:18

New from PocketJS: quickjs-oxide. A JavaScript interpreter modeled after QuickJS, implemented from scratch by AI in Rust with zero unsafe code, and the first of its kind to achieve performance competitive with the original QuickJS. We spent one month achieving a high Test262 pass rate, then another month cleaning up the code, getting rid of the AI slop, and reaching competitive performance with just a stack VM and reference counting. Over the last weekend at RustChinaConf, we rewrote the core into a register VM and achieved this level of performance in just two days. Now quickjs-oxide outperforms most non-JIT JavaScript engines on benchmarks like V8 v7 and the Octane suite, using only around 30% more memory than QuickJS, while maintaining zero lines of unsafe implementation code and no external dependencies except for BigInt support. Huge credit to Ericoolen(薄肌版) for making this happen, and massive thanks to our sponsor Zephyr Cloud for supporting the journey!

PocketJS

23,822 Aufrufe • vor 21 Tagen

Despite the Framer hype, we still get tons of requests for custom Next.js + Tailwind sites. Because most founders and their teams don't want to learn another tool. They want to own their code, not rent it. Just shipped a landing page in Next.js for a SaaS. Their reasoning: - Full control over performance optimization - No vendor lock-in - Developers can maintain it without learning Framer - Custom functionality without workarounds - Better SEO control Sure, Framer is faster to build. But he no-code movement sold everyone on "anyone can build websites now" and forgot that someone still needs to maintain them. What's your take: Custom code for control or no-code for speed?
0:13

Despite the Framer hype, we still get tons of requests for custom Next.js + Tailwind sites. Because most founders and their teams don't want to learn another tool. They want to own their code, not rent it. Just shipped a landing page in Next.js for a SaaS. Their reasoning: - Full control over performance optimization - No vendor lock-in - Developers can maintain it without learning Framer - Custom functionality without workarounds - Better SEO control Sure, Framer is faster to build. But he no-code movement sold everyone on "anyone can build websites now" and forgot that someone still needs to maintain them. What's your take: Custom code for control or no-code for speed?

Namya @ Supafast

50,594 Aufrufe • vor 1 Jahr

Gaussian splats don't simulate light. They capture it. I took one photograph of this room and generated it as a fully walkable 3D world in Mint. The volumetric rays, the shadow play on the rug, all preserved. This is what real-time 3D is becoming
0:13

Gaussian splats don't simulate light. They capture it. I took one photograph of this room and generated it as a fully walkable 3D world in Mint. The volumetric rays, the shadow play on the rug, all preserved. This is what real-time 3D is becoming

mint

15,446 Aufrufe • vor 5 Monaten

3D Gaussian Splatting for Real-Time Radiance Field Rendering paper page: Radiance Field methods have recently revolutionized novel-view synthesis of scenes captured with multiple photos or videos. However, achieving high visual quality still requires neural networks that are costly to train and render, while recent faster methods inevitably trade off speed for quality. For unbounded and complete scenes (rather than isolated objects) and 1080p resolution rendering, no current method can achieve real-time display rates. We introduce three key elements that allow us to achieve state-of-the-art visual quality while maintaining competitive training times and importantly allow high-quality real-time (>= 30 fps) novel-view synthesis at 1080p resolution. First, starting from sparse points produced during camera calibration, we represent the scene with 3D Gaussians that preserve desirable properties of continuous volumetric radiance fields for scene optimization while avoiding unnecessary computation in empty space; Second, we perform interleaved optimization/density control of the 3D Gaussians, notably optimizing anisotropic covariance to achieve an accurate representation of the scene; Third, we develop a fast visibility-aware rendering algorithm that supports anisotropic splatting and both accelerates training and allows realtime rendering. We demonstrate state-of-the-art visual quality and real-time rendering on several established datasets.
0:18

3D Gaussian Splatting for Real-Time Radiance Field Rendering paper page: Radiance Field methods have recently revolutionized novel-view synthesis of scenes captured with multiple photos or videos. However, achieving high visual quality still requires neural networks that are costly to train and render, while recent faster methods inevitably trade off speed for quality. For unbounded and complete scenes (rather than isolated objects) and 1080p resolution rendering, no current method can achieve real-time display rates. We introduce three key elements that allow us to achieve state-of-the-art visual quality while maintaining competitive training times and importantly allow high-quality real-time (>= 30 fps) novel-view synthesis at 1080p resolution. First, starting from sparse points produced during camera calibration, we represent the scene with 3D Gaussians that preserve desirable properties of continuous volumetric radiance fields for scene optimization while avoiding unnecessary computation in empty space; Second, we perform interleaved optimization/density control of the 3D Gaussians, notably optimizing anisotropic covariance to achieve an accurate representation of the scene; Third, we develop a fast visibility-aware rendering algorithm that supports anisotropic splatting and both accelerates training and allows realtime rendering. We demonstrate state-of-the-art visual quality and real-time rendering on several established datasets.

AK

634,954 Aufrufe • vor 3 Jahren