正在加载视频...

视频加载失败

加载此视频时出现问题。这可能是由于临时网络问题，或视频可能不可用。

Excited to share our work, Know3D, which connects LLMs' reasoning ability and knowledge to 3D generative models. This increases the controllability and plausibility of unseen parts in the generated 3D shapes. Paper: Project page:

Yuan Liu

2,106 subscribers

14,382 次观看 • 6 个月前 •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

6 条评论

Yuan Liu 的头像

Yuan Liu6 个月前

We can control what would be hallucinated in the back view using text prompts and make it more structurally plausible. HF: Code is coming soon:

Jabed Bhuyan 的头像

Jabed Bhuyan5 个月前

This is such a clever and practical idea! 🔥 Integrating LLM knowledge to guide unseen regions in 3D diffusion models is a game-changer for making generated shapes way more plausible and controllable. The back-view examples look super impressive love how a simple text prompt can fix structural issues without any extra 3D data. Congrats on the work, Can't wait for the code release. This feels like a big step toward more intelligent 3D generation.

PaperGlide.net 的头像

PaperGlide.net6 个月前

Skim a digestible version of the paper here:

Rachmadio Noval L. 的头像

Rachmadio Noval L.6 个月前

I wonder if the fine-tuned Qwen also have the editing capability of the front-view image in a way that maintains the original pose? Would be interesting to enable the 3D edit. Very nice work!

Yuan Liu 的头像

Yuan Liu5 个月前

Yes, I think this could be a promising way to achieve 3D editing without using 3D editing training data, by leveraging strong image editing. In our current version, we only finetune it to generate the backview, which cannot do this. But I think this is promising.

PaperGlide.net 的头像

PaperGlide.net6 个月前

Digestible breakdown of this paper:

相关视频

Excited to share our recent work, UniSH, which unifies dynamic 3D scene reconstruction and SMPL estimation within a single framework. (Left-top is input video). Code has been released! Project page: Paper:

Excited to share our recent work, UniSH, which unifies dynamic 3D scene reconstruction and SMPL estimation within a single framework. (Left-top is input video). Code has been released! Project page: Paper:

Yuan Liu

19,258 次观看 • 8 个月前

I'm excited to share our new work, VistaDream, which generates a 3D Gaussian field from a single-view image. The codes have already been released. Project page: (with interactive demos) Code: Paper:

I'm excited to share our new work, VistaDream, which generates a 3D Gaussian field from a single-view image. The codes have already been released. Project page: (with interactive demos) Code: Paper:

Yuan Liu

79,610 次观看 • 1 年前

3D-LLM: Injecting the 3D World into Large Language Models paper page: Large language models (LLMs) and Vision-Language Models (VLMs) have been proven to excel at multiple tasks, such as commonsense reasoning. Powerful as these models can be, they are not grounded in the 3D physical world, which involves richer concepts such as spatial relationships, affordances, physics, layout, and so on. In this work, we propose to inject the 3D world into large language models and introduce a whole new family of 3D-LLMs. Specifically, 3D-LLMs can take 3D point clouds and their features as input and perform a diverse set of 3D-related tasks, including captioning, dense captioning, 3D question answering, task decomposition, 3D grounding, 3D-assisted dialog, navigation, and so on. Using three types of prompting mechanisms that we design, we are able to collect over 300k 3D-language data covering these tasks. To efficiently train 3D-LLMs, we first utilize a 3D feature extractor that obtains 3D features from rendered multi- view images. Then, we use 2D VLMs as our backbones to train our 3D-LLMs. By introducing a 3D localization mechanism, 3D-LLMs can better capture 3D spatial information. Experiments on ScanQA show that our model outperforms state-of-the-art baselines by a large margin (e.g., the BLEU-1 score surpasses state-of-the-art score by 9%). Furthermore, experiments on our held-in datasets for 3D captioning, task composition, and 3D-assisted dialogue show that our model outperforms 2D VLMs. Qualitative examples also show that our model could perform more tasks beyond the scope of existing LLMs and VLMs.

3D-LLM: Injecting the 3D World into Large Language Models paper page: Large language models (LLMs) and Vision-Language Models (VLMs) have been proven to excel at multiple tasks, such as commonsense reasoning. Powerful as these models can be, they are not grounded in the 3D physical world, which involves richer concepts such as spatial relationships, affordances, physics, layout, and so on. In this work, we propose to inject the 3D world into large language models and introduce a whole new family of 3D-LLMs. Specifically, 3D-LLMs can take 3D point clouds and their features as input and perform a diverse set of 3D-related tasks, including captioning, dense captioning, 3D question answering, task decomposition, 3D grounding, 3D-assisted dialog, navigation, and so on. Using three types of prompting mechanisms that we design, we are able to collect over 300k 3D-language data covering these tasks. To efficiently train 3D-LLMs, we first utilize a 3D feature extractor that obtains 3D features from rendered multi- view images. Then, we use 2D VLMs as our backbones to train our 3D-LLMs. By introducing a 3D localization mechanism, 3D-LLMs can better capture 3D spatial information. Experiments on ScanQA show that our model outperforms state-of-the-art baselines by a large margin (e.g., the BLEU-1 score surpasses state-of-the-art score by 9%). Furthermore, experiments on our held-in datasets for 3D captioning, task composition, and 3D-assisted dialogue show that our model outperforms 2D VLMs. Qualitative examples also show that our model could perform more tasks beyond the scope of existing LLMs and VLMs.

AK

249,798 次观看 • 3 年前

Thrilled to share our new work on Reconviagen✨! A key challenge in 3D creation is the alignment of generative 3D with observational input. Our method solves this by grounding the generative process in 3D reconstruction. Try it at: #Reconstruction #AIGC

Thrilled to share our new work on Reconviagen✨! A key challenge in 3D creation is the alignment of generative 3D with observational input. Our method solves this by grounding the generative process in 3D reconstruction. Try it at: #Reconstruction #AIGC

Chongjie Ye

11,624 次观看 • 1 年前

Excited to share our new #SIGGRAPH2025 paper! In this work, we show how to combine Gaussian splatting and computer-generated holography using Gaussian Wave Splatting. This enables photorealistic 3D holograms for emerging holographic VR/AR displays. 1/4

Excited to share our new #SIGGRAPH2025 paper! In this work, we show how to combine Gaussian splatting and computer-generated holography using Gaussian Wave Splatting. This enables photorealistic 3D holograms for emerging holographic VR/AR displays. 1/4

Gordon Wetzstein

21,471 次观看 • 1 年前

HoloPart is here, and it’s open-source! Our generative model splits 3D shapes into complete parts, even the hidden bits. Unlock seamless editing, animation, and creation. 🎨

HoloPart is here, and it’s open-source! Our generative model splits 3D shapes into complete parts, even the hidden bits. Unlock seamless editing, animation, and creation. 🎨

Yanpei Cao

74,403 次观看 • 1 年前

HoloPart is now OPEN-SOURCED! Our generative model splits 3D shapes into complete parts, even the hidden bits. Unlock seamless editing, animation, and creation. Try now:

HoloPart is now OPEN-SOURCED! Our generative model splits 3D shapes into complete parts, even the hidden bits. Unlock seamless editing, animation, and creation. Try now:

Tripo

61,113 次观看 • 1 年前

I'm excited to share our new work Align3R that estimates camera poses and consistent depth maps from a monocular video of a dynamic scene. Project page: Code: Paper:

I'm excited to share our new work Align3R that estimates camera poses and consistent depth maps from a monocular video of a dynamic scene. Project page: Code: Paper:

Yuan Liu

56,547 次观看 • 1 年前

(1/4) Excited to share our #ICCV2023 paper Text2Room! We generate scene-scale textured 3D meshes from a given text prompt leveraging 2D text-to-image models such as StableDiffusion. Project: Code: Video:

(1/4) Excited to share our #ICCV2023 paper Text2Room! We generate scene-scale textured 3D meshes from a given text prompt leveraging 2D text-to-image models such as StableDiffusion. Project: Code: Video:

Matthias Niessner

74,907 次观看 • 3 年前

Excited to share Track4World, feedforward 3D tracking of all pixels in the world-centric coordinate system. Code has been released, and welcome to try it! Homepage: Code: Paper:

Excited to share Track4World, feedforward 3D tracking of all pixels in the world-centric coordinate system. Code has been released, and welcome to try it! Homepage: Code: Paper:

Yuan Liu

20,337 次观看 • 6 个月前

We are excited to share EMDB, a novel dataset of 3D human poses for in-the-wild monocular videos, including global trajectories. Data and toolkit code is now available. More details in the thread below. Project Page:

We are excited to share EMDB, a novel dataset of 3D human poses for in-the-wild monocular videos, including global trajectories. Data and toolkit code is now available. More details in the thread below. Project Page:

AIT Lab

14,625 次观看 • 3 年前

An Object is Worth 64x64 Pixels: Generating 3D Object via Image Diffusion discuss: We introduce a new approach for generating realistic 3D models with UV maps through a representation termed "Object Images." This approach encapsulates surface geometry, appearance, and patch structures within a 64x64 pixel image, effectively converting complex 3D shapes into a more manageable 2D format. By doing so, we address the challenges of both geometric and semantic irregularity inherent in polygonal meshes. This method allows us to use image generation models, such as Diffusion Transformers, directly for 3D shape generation. Evaluated on the ABO dataset, our generated shapes with patch structures achieve point cloud FID comparable to recent 3D generative models, while naturally supporting PBR material generation.

An Object is Worth 64x64 Pixels: Generating 3D Object via Image Diffusion discuss: We introduce a new approach for generating realistic 3D models with UV maps through a representation termed "Object Images." This approach encapsulates surface geometry, appearance, and patch structures within a 64x64 pixel image, effectively converting complex 3D shapes into a more manageable 2D format. By doing so, we address the challenges of both geometric and semantic irregularity inherent in polygonal meshes. This method allows us to use image generation models, such as Diffusion Transformers, directly for 3D shape generation. Evaluated on the ABO dataset, our generated shapes with patch structures achieve point cloud FID comparable to recent 3D generative models, while naturally supporting PBR material generation.

AK

66,717 次观看 • 2 年前

#CVPR2026 Can frontier LLMs write PhD-level 3D vision code? We introduce GeoCodeBench, a benchmark that asks models to read real 3D geometric vision papers and implement core functions. Best result so far: GPT-5 reaches only 36.6%. This suggests that scientific coding in 3D vision remains far from solved. Paper: Project:

#CVPR2026 Can frontier LLMs write PhD-level 3D vision code? We introduce GeoCodeBench, a benchmark that asks models to read real 3D geometric vision papers and implement core functions. Best result so far: GPT-5 reaches only 36.6%. This suggests that scientific coding in 3D vision remains far from solved. Paper: Project:

Hao Zhao

18,324 次观看 • 4 个月前

Generative image to scene is getting incredibly good. This is pure 3D without v2v generated from the image in the reply ...

Generative image to scene is getting incredibly good. This is pure 3D without v2v generated from the image in the reply ...

martin_casado

57,176 次观看 • 1 年前

Today we presented our work on Action Recognition with 3D pose and tracking at #CVPR. We also released our code, models, and dataset of 1.5 million human trajectories. Code: Project Page:

Today we presented our work on Action Recognition with 3D pose and tracking at #CVPR. We also released our code, models, and dataset of 1.5 million human trajectories. Code: Project Page:

Jathushan Rajasegaran

132,016 次观看 • 3 年前

3D Object Tracking without Training Data? In our nature Machine Intelligence paper ( we recast 3D tracking as an inverse neural rendering task where we fit a scene graph to an image that best explains this image. The method generalizes to completely unseen datasets and is explainable. Project and Code: Fun collaboration between Princeton Computer Science and Torc Robotics, with Julian Ost and Tanushree Banerjee leading this project.

3D Object Tracking without Training Data? In our nature Machine Intelligence paper ( we recast 3D tracking as an inverse neural rendering task where we fit a scene graph to an image that best explains this image. The method generalizes to completely unseen datasets and is explainable. Project and Code: Fun collaboration between Princeton Computer Science and Torc Robotics, with Julian Ost and Tanushree Banerjee leading this project.

Felix Heide

27,858 次观看 • 1 年前

We are introducing Hunyuan3D-Part: an open-source part-level 3D shape generation model that outperforms all existing open and close-source models. Highlights: 🔹P3-SAM: The industry's first native 3D part segmentation model. 🔹X-Part: A part generation model that achieves state-of-the-art results in controllability and shape quality. Key-features: 1️⃣Eliminates the use of 2D SAM during training, relying solely on a large-scale dataset with 3.7 million shapes and clean part annotations. 2️⃣Introduces a new automated segmentation pipeline in 3D without user intervention. 3️⃣Implements a diffusion-based part decomposition pipeline utilizing both geometry and semantic clues. Code: Weights: Tech reports: 🔸P3-SAM： → Paper: → Project page: 🔸X-Part： → Paper: → Project page: Try it now： → (Light version) Hugging Face demo: → (Full version) Hunyuan3D Studio:

We are introducing Hunyuan3D-Part: an open-source part-level 3D shape generation model that outperforms all existing open and close-source models. Highlights: 🔹P3-SAM: The industry's first native 3D part segmentation model. 🔹X-Part: A part generation model that achieves state-of-the-art results in controllability and shape quality. Key-features: 1️⃣Eliminates the use of 2D SAM during training, relying solely on a large-scale dataset with 3.7 million shapes and clean part annotations. 2️⃣Introduces a new automated segmentation pipeline in 3D without user intervention. 3️⃣Implements a diffusion-based part decomposition pipeline utilizing both geometry and semantic clues. Code: Weights: Tech reports: 🔸P3-SAM： → Paper: → Project page: 🔸X-Part： → Paper: → Project page: Try it now： → (Light version) Hugging Face demo: → (Full version) Hunyuan3D Studio:

Tencent Hy

72,640 次观看 • 1 年前

📢 Lyra: Generative 3D Scene Reconstruction via Video Diffusion Model Self-Distillation Got only one or a few images and wondering if recovering the 3D environment is a reconstruction or generation problem? Why not do it with a generative reconstruction model! We show that a camera-conditioned video diffusion model can be transformed into a generative reconstruction model that directly outputs a high-quality 3D Gaussian Splatting representation through self-distillation, without requiring real-world training data. Check out our results in the video (wait for dynamic scenes in the second half!) : Project Page: Code and Models: Paper:

📢 Lyra: Generative 3D Scene Reconstruction via Video Diffusion Model Self-Distillation Got only one or a few images and wondering if recovering the 3D environment is a reconstruction or generation problem? Why not do it with a generative reconstruction model! We show that a camera-conditioned video diffusion model can be transformed into a generative reconstruction model that directly outputs a high-quality 3D Gaussian Splatting representation through self-distillation, without requiring real-world training data. Check out our results in the video (wait for dynamic scenes in the second half!) : Project Page: Code and Models: Paper:

Sherwin Bahmani

66,869 次观看 • 1 年前