正在加载视频...

视频加载失败

加载此视频时出现问题。这可能是由于临时网络问题，或视频可能不可用。

3DShape2VecSet: A 3D Shape Representation for Neural Fields and Generative Diffusion Models. SIGGRAPH 2023. Zhang (Biao Zhang ) Tang (Jiapeng Tang) Niessner (Matthias Niessner) Wonka.

Peter Wonka

1,006 subscribers

11,765 次观看 • 3 年前 •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

1 条评论

Andrea Tagliasacchi 的头像

Andrea Tagliasacchi3 年前

@MattNiessner @_BiaoZhang @jiapeng_tang would have loved to see the what, before the how…

相关视频

Check out 𝐌𝐨𝐭𝐢𝐨𝐧𝟐𝐕𝐞𝐜𝐒𝐞𝐭𝐬, a 4D diffusion model for dynamic surface reconstruction from imperfect observations of sparse, noisy, or partial point clouds. Main idea: we represent time-varying shapes via 4D neural representation with latent vector sets, and then explicitly learns the shape and motion distribution of non-rigid objects through an iterative denoising process of latent vector sets. Great work by Wei Cao るちゃん Biao Zhang Jiapeng Tang

Check out 𝐌𝐨𝐭𝐢𝐨𝐧𝟐𝐕𝐞𝐜𝐒𝐞𝐭𝐬, a 4D diffusion model for dynamic surface reconstruction from imperfect observations of sparse, noisy, or partial point clouds. Main idea: we represent time-varying shapes via 4D neural representation with latent vector sets, and then explicitly learns the shape and motion distribution of non-rigid objects through an iterative denoising process of latent vector sets. Great work by Wei Cao るちゃん Biao Zhang Jiapeng Tang

Matthias Niessner

29,149 次观看 • 2 年前

How can we generate high-fidelity, complex 3D scenes? Quan Meng's LT3SD decomposes 3D scenes into latent tree representations, with diffusion on the latent trees enabling seamless infinite 3D scene synthesis! w/ Lei Li, Matthias Niessner

How can we generate high-fidelity, complex 3D scenes? Quan Meng's LT3SD decomposes 3D scenes into latent tree representations, with diffusion on the latent trees enabling seamless infinite 3D scene synthesis! w/ Lei Li, Matthias Niessner

Angela Dai

29,469 次观看 • 2 年前

Gaussian Shell Maps are a new neural scene representation that connects fields and 3D Gaussians. This representation unlocks the full potential of 3D Gaussian splatting for generative AI applications, such as 3D avatar generation. 1/2

Gordon Wetzstein

52,555 次观看 • 2 年前

zhang boheng & tang chia-hung medal ceremony cut

zhang boheng & tang chia-hung medal ceremony cut

noé 🩵

117,138 次观看 • 2 年前

Looking for a challenging dataset for novel view synthesis and 3D semantics? Check out ScanNet++ at #ICCV23 (Oral)! 460+ scenes w/ 1mm laser scans, semantics, DSLR images, iPhone RGBD video Chandan Yeshwanth Yueh-Cheng Liu Matthias Niessner

Looking for a challenging dataset for novel view synthesis and 3D semantics? Check out ScanNet++ at #ICCV23 (Oral)! 460+ scenes w/ 1mm laser scans, semantics, DSLR images, iPhone RGBD video Chandan Yeshwanth Yueh-Cheng Liu Matthias Niessner

Angela Dai

10,289 次观看 • 3 年前

Synthesia (Synthesia 🎥) has raised $200 million in its latest funding round, bringing the company's valuation to $4 billion. Synthesia (Synthesia 🎥), headquartered in London, England, United Kingdom, was founded in 2017 by Victor Riparbelli (Victor Riparbelli), Lourdes Agapito (Lourdes Agapito), Matthias Niessner (Matthias Niessner), and Steffen Tjerrild (Steffen Tjerrild).

Synthesia (Synthesia 🎥) has raised $200 million in its latest funding round, bringing the company's valuation to $4 billion. Synthesia (Synthesia 🎥), headquartered in London, England, United Kingdom, was founded in 2017 by Victor Riparbelli (Victor Riparbelli), Lourdes Agapito (Lourdes Agapito), Matthias Niessner (Matthias Niessner), and Steffen Tjerrild (Steffen Tjerrild).

Silicon Valley Investclub

15,606 次观看 • 11 个月前

We present Neural Riemannian Motion Fields (NRMF) as a promising paradigm for generative modeling of articulated motion. Unlike various diffusion based models, NRMF is useful both in generation and in deployment as a prior across tasks: #CVPR #AI #CVML👇

We present Neural Riemannian Motion Fields (NRMF) as a promising paradigm for generative modeling of articulated motion. Unlike various diffusion based models, NRMF is useful both in generation and in deployment as a prior across tasks: #CVPR #AI #CVML👇

Tolga Birdal

16,795 次观看 • 1 年前

Excited to announce ScanNet++ v2!🎉 Chandan Yeshwanth and Yueh-Cheng Liu have been working tirelessly to bring: 🔹1006 high-fidelity 3D scans 🔹+ DSLR & iPhone captures 🔹+ rich semantics Elevating 3D scene understanding to the next level!🚀 w/ Matthias Niessner

Excited to announce ScanNet++ v2!🎉 Chandan Yeshwanth and Yueh-Cheng Liu have been working tirelessly to bring: 🔹1006 high-fidelity 3D scans 🔹+ DSLR & iPhone captures 🔹+ rich semantics Elevating 3D scene understanding to the next level!🚀 w/ Matthias Niessner

Angela Dai

47,629 次观看 • 1 年前

New AAA game from China, "Blood Message." A little background to this true historic event. The story is based in the year 848AD, after the Anshi rebellion, Tang dynasty was weakened and lost control of the Western Region (modern day Gansu and Xinjiang). With a shift in global temperature, the Tibetan empire rose up and occupied the region. After years of struggle, with one Tang city holding out a siege for 11 years. The Tibetan empire practiced a very oppressive form of serfdom and slavery, very much opposed to the ideals of Tang Dynasty of openness (many Tang officials, even a couple of prime ministers were foreigners). the guy giving the speech at 0:50 is Zhang Yichao, who led a local rebellion against the Tibetan empire. To seek help from the Tang Dynasty, Zhang Yichao sent 10 teams of messengers to Tang capital Chang'An modern day Xi'An. To tell the people of Tang, the rebellion of the Western Region and seek reinforcement. Of the 10 messenger teams, only 1 team arrived in Chang'An, the player is in that team. Historically, Tang Emperor gave Zhang Yichao a bunch of titles and all the land in the Western region, but the dynasty was unable to help, but through years of war, Zhang Yichao's rebel army was able to reconnect the Western region with Tang territory.

New AAA game from China, "Blood Message." A little background to this true historic event. The story is based in the year 848AD, after the Anshi rebellion, Tang dynasty was weakened and lost control of the Western Region (modern day Gansu and Xinjiang). With a shift in global temperature, the Tibetan empire rose up and occupied the region. After years of struggle, with one Tang city holding out a siege for 11 years. The Tibetan empire practiced a very oppressive form of serfdom and slavery, very much opposed to the ideals of Tang Dynasty of openness (many Tang officials, even a couple of prime ministers were foreigners). the guy giving the speech at 0:50 is Zhang Yichao, who led a local rebellion against the Tibetan empire. To seek help from the Tang Dynasty, Zhang Yichao sent 10 teams of messengers to Tang capital Chang'An modern day Xi'An. To tell the people of Tang, the rebellion of the Western Region and seek reinforcement. Of the 10 messenger teams, only 1 team arrived in Chang'An, the player is in that team. Historically, Tang Emperor gave Zhang Yichao a bunch of titles and all the land in the Western region, but the dynasty was unable to help, but through years of war, Zhang Yichao's rebel army was able to reconnect the Western region with Tang territory.

Zhao DaShuai 东北进修🇨🇳

72,049 次观看 • 1 年前

Generative Novel View Synthesis with 3D-Aware Diffusion Models abs: project page:

Generative Novel View Synthesis with 3D-Aware Diffusion Models abs: project page:

AK

304,708 次观看 • 3 年前

We also present another paper at ACM SIGGRAPH 2023 on neural implicit 3D Morphable Models that can be used to create a dynamic 3D avatar from a single in-the-wild image. (Lead author Connor Lin).

Koki Nagano

12,786 次观看 • 3 年前

Check out the Neural Graph Mapping for Dense SLAM with Efficient Loop Closure paper by Leonard Bruns, Jun Zhang, and Patric Jensfelt, visualized with Rerun. “Existing neural field-based SLAM methods typically employ a single monolithic field as their scene representation. This prevents efficient incorporation of loop closure constraints and limits scalability. To address these shortcomings, we propose a neural mapping framework which anchors lightweight neural fields to the pose graph of a sparse visual SLAM system.”

Check out the Neural Graph Mapping for Dense SLAM with Efficient Loop Closure paper by Leonard Bruns, Jun Zhang, and Patric Jensfelt, visualized with Rerun. “Existing neural field-based SLAM methods typically employ a single monolithic field as their scene representation. This prevents efficient incorporation of loop closure constraints and limits scalability. To address these shortcomings, we propose a neural mapping framework which anchors lightweight neural fields to the pose graph of a sparse visual SLAM system.”

Rerun

12,865 次观看 • 2 年前

(1/2) Mesh2NeRF: Direct Mesh Supervision for Neural Radiance Field Representation and Generation #ECCV2024! We show a theoretical derivation to create radiance fields directly from meshes. Thus, we can obtain GT training data for generative NeRF methods.

(1/2) Mesh2NeRF: Direct Mesh Supervision for Neural Radiance Field Representation and Generation #ECCV2024! We show a theoretical derivation to create radiance fields directly from meshes. Thus, we can obtain GT training data for generative NeRF methods.

Matthias Niessner

16,835 次观看 • 2 年前

📢ProcGen3D: Learning Neural Procedural Graphs for Image-to-3D Reconstruction Xinyi Zhang learns neural procedural graphs to generate high-fidelity 3D - MCTS-guided sampling maintains consistency with the input image, even from real images! Check it out:

📢ProcGen3D: Learning Neural Procedural Graphs for Image-to-3D Reconstruction Xinyi Zhang learns neural procedural graphs to generate high-fidelity 3D - MCTS-guided sampling maintains consistency with the input image, even from real images! Check it out:

Angela Dai

14,702 次观看 • 11 个月前

Can we apply diffusion directly on MLP weights? Yes!!! We show generalization to new 3D shapes and 4D mesh animations: "HyperDiffusion: Generating Implicit Neural Fields with Weight-Space Diffusion" #ICCV2023! Project: Video:

Can we apply diffusion directly on MLP weights? Yes!!! We show generalization to new 3D shapes and 4D mesh animations: "HyperDiffusion: Generating Implicit Neural Fields with Weight-Space Diffusion" #ICCV2023! Project: Video:

Matthias Niessner

86,084 次观看 • 3 年前

Dream3D: Zero-Shot Text-to-3D Synthesis Using 3D Shape Prior and Text-to-Image Diffusion Models abs: project page:

Dream3D: Zero-Shot Text-to-3D Synthesis Using 3D Shape Prior and Text-to-Image Diffusion Models abs: project page:

AK

121,020 次观看 • 3 年前

📢 Lyra: Generative 3D Scene Reconstruction via Video Diffusion Model Self-Distillation Got only one or a few images and wondering if recovering the 3D environment is a reconstruction or generation problem? Why not do it with a generative reconstruction model! We show that a camera-conditioned video diffusion model can be transformed into a generative reconstruction model that directly outputs a high-quality 3D Gaussian Splatting representation through self-distillation, without requiring real-world training data. Check out our results in the video (wait for dynamic scenes in the second half!) : Project Page: Code and Models: Paper:

📢 Lyra: Generative 3D Scene Reconstruction via Video Diffusion Model Self-Distillation Got only one or a few images and wondering if recovering the 3D environment is a reconstruction or generation problem? Why not do it with a generative reconstruction model! We show that a camera-conditioned video diffusion model can be transformed into a generative reconstruction model that directly outputs a high-quality 3D Gaussian Splatting representation through self-distillation, without requiring real-world training data. Check out our results in the video (wait for dynamic scenes in the second half!) : Project Page: Code and Models: Paper:

Sherwin Bahmani

66,918 次观看 • 1 年前

Today we are sharing three new research papers, each exploring a new way to generate 3D content by leveraging large-scale generative models and 2D priors. These projects were led by our incredible interns Hao Zhang @BDuisterhof @DrTunnels [1/4]

Today we are sharing three new research papers, each exploring a new way to generate 3D content by leveraging large-scale generative models and 2D priors. These projects were led by our incredible interns Hao Zhang @BDuisterhof @DrTunnels [1/4]

World Labs

123,355 次观看 • 4 个月前

Rodin: A Generative Model for Sculpting 3D Digital Avatars Using Diffusion abs: project page:

Rodin: A Generative Model for Sculpting 3D Digital Avatars Using Diffusion abs: project page:

AK

263,035 次观看 • 3 年前

An Object is Worth 64x64 Pixels: Generating 3D Object via Image Diffusion discuss: We introduce a new approach for generating realistic 3D models with UV maps through a representation termed "Object Images." This approach encapsulates surface geometry, appearance, and patch structures within a 64x64 pixel image, effectively converting complex 3D shapes into a more manageable 2D format. By doing so, we address the challenges of both geometric and semantic irregularity inherent in polygonal meshes. This method allows us to use image generation models, such as Diffusion Transformers, directly for 3D shape generation. Evaluated on the ABO dataset, our generated shapes with patch structures achieve point cloud FID comparable to recent 3D generative models, while naturally supporting PBR material generation.

An Object is Worth 64x64 Pixels: Generating 3D Object via Image Diffusion discuss: We introduce a new approach for generating realistic 3D models with UV maps through a representation termed "Object Images." This approach encapsulates surface geometry, appearance, and patch structures within a 64x64 pixel image, effectively converting complex 3D shapes into a more manageable 2D format. By doing so, we address the challenges of both geometric and semantic irregularity inherent in polygonal meshes. This method allows us to use image generation models, such as Diffusion Transformers, directly for 3D shape generation. Evaluated on the ABO dataset, our generated shapes with patch structures achieve point cloud FID comparable to recent 3D generative models, while naturally supporting PBR material generation.

AK

66,769 次观看 • 2 年前