Загрузка видео...

Не удалось загрузить видео

На главную

How can we reconstruct 3D objects under *extreme lighting variations*? 🌤️🌥️🌆🌃🌉 How about ... 🤔 appearance embedding? BUT it cannot capture view-dependent appearances. 🤔 inverse rendering? BUT it suffers from ambiguities. 💡 Our idea: Relighting comes to the rescue!

33,151 просмотров • 1 год назад •via X (Twitter)

Комментарии: 0

Нет доступных комментариев

Здесь появятся комментарии из оригинального поста

Похожие видео

Introducing Kaleido💮 from AI at Meta — a universal generative neural rendering engine for photorealistic, unified object and scene view synthesis. Kaleido is built on a simple but powerful design philosophy: 3D perception is a form of visual common sense. Following this idea, we formulate rendering purely as a sequence-to-sequence generation problem, successfully unifying neural rendering with the architecture principles behind modern language and video models. Unlike traditional neural rendering methods, Kaleido learns 3D purely in a data-driven way, without explicit 3D representations or structures. It acquires spatial understanding directly through large-scale video pretraining, then multi-view 3D data finetuning, inspired by how LLMs acquire textual common sense from large corpora before specialising in domains like coding. Through extensive ablations, we progressively modernised the architecture design and training strategies and tackled key scaling challenges in sequence-to-sequence generative rendering, arriving at a design that’s simple, versatile, and scalable. Kaleido significantly outperforms prior generative models in few-view settings, and remarkably is the first zero-shot generative method matches InstantNGP-level rendering quality in multi-view settings. We view Kaleido also as an alternative step towards world modeling that flexibly spans a spectrum of “realities": with many views, it faithfully reconstructs grounded reality; with fewer views, it imagines plausible unseen details. 🔗 Explore more results and paper:

Shikun Liu

22,430 просмотров • 10 месяцев назад

Most chiral molecules arise from carbons being bonded to 4 different atoms, which are called sterocenters. The makes the molecule have a different mirror image that cannot arise from simple rotation. But, you can have chiral molecules not from stereocenters. You can have chirality that doesn't come from a single point in the molecule. It comes from some global property. The classic example is helicene, which doesn't have any stereocenters, but has chirality because of hits helical structure. This means you cannot capture this molecule with a graph, and thus SMILES or a string representation cannot capture this. Of course natural language comes to the rescue (just say in words if it's left-handed or right-handed helix), but it's an interesting failure mode for viewing molecules as just a graph. Another example of a molecule with helical chirality is DNA. DNA is actually chiral in two ways, which is kind of confusing. It has both helical structure and stereocenters. You won't find the stereocenters ever flipped, but left-handed helical DNA can exist (called Z-DNA). Interestingly, making the flipped stereocenter of DNA could be part of an entire mirror organism (mirror RNA, DNA, AAs, sugars) that would then be potentially invisible to our immune systems. This has been recently proposed as a "mechanism" for how a runaway AI system could cause harm to Earth. I find it to be a pretty tedious and difficult way to cause harm, but it is intellectually cool. Anyway - this came up in a PhD defense and I have a lot of arcane knowledge about this I wanted to dump.

Andrew White 🐦‍⬛

15,039 просмотров • 4 месяцев назад

Two weeks ago I fixed one of my teeth with algorithms I wrote a couple of years ago! I got hooked by 3D scanning when I started to work for a software shop in Zurich that was programming 3D computational geometry algorithms for denture scanning to produce crowns (and more). Back then, a typical reconstruction pipeline was like: scan the patient’s teeth using an intraoral scanner, reconstruct the surface mesh, design the restoration digitally, and finally mill the crown out of ceramic. We were working mostly with point clouds and meshes, but it wasn’t just math, it was craftsmanship translated into a digital process. Every micron mattered. You could literally see how a good algorithm meant a better fit in someone’s mouth. Gaussian Splatting isn’t about surface reconstruction, it’s about appearance reconstruction. It doesn’t care about explicit topology, it captures how light interacts with the scene. In a sense, it’s the opposite philosophy of the dental world: instead of modeling what the object is, it models how the object looks. 3D Gaussian Splatting enables applications like training self driving cars, teaching robots to understand their environment, creating virtual worlds, or monitoring real sites. It represents scenes as millions of small Gaussians rendered in real time without the need for meshes or textures. Coming from a world where precision geometry was everything, this shift felt natural. It’s still about reconstruction, but with a different goal: not manufacturing a perfect object, but reproducing how the world actually looks. Two weeks ago I got my first dental crown, made with the same software, reconstruction algorithms, and Swiss precision I once helped develop. I haven’t worked there in two years, but sitting in that chair and seeing the process from the other side was a proud moment. It reminded me why I love this field.

MrNeRF

290,202 просмотров • 9 месяцев назад