Video yükleniyor...

Video Yüklenemedi

Bu video yüklenirken bir sorun oluştu. Bu geçici bir ağ sorunundan kaynaklanıyor olabilir veya video kullanılamıyor olabilir.

Ana Sayfaya Dön

Google announces InseRF Text-Driven Generative Object Insertion in Neural 3D Scenes paper page: InseRF generates an object in a 3D scene via a text prompt and one 2D bounding box

AK

506,881 subscribers

205,987 görüntüleme • 2 yıl önce •via X (Twitter)

Haberler & Politika Bilim & Teknoloji Eğitim

Anya Rossi• Live Now

Private livecam show

10 Yorum

Georgia Gkioxari profil fotoğrafı

Georgia Gkioxari2 yıl önce

The adding of the panettone got me. Panetonnes all year round please in NeRFs and the real world!

Mohamad Shahbazi profil fotoğrafı

Mohamad Shahbazi2 yıl önce

Thanks a lot for featuring our work @_akhaliq! Here is the project page for more details and results:

DAS profil fotoğrafı

DAS2 yıl önce

I bet this works as good as most google AI products we’ve seen demo(ed)

The AI Edge profil fotoğrafı

The AI Edge2 yıl önce

Existing methods for 3D scene editing are mostly effective for style and appearance changes or removing objects. But generating new objects is a challenge for them. InseRF addresses this by combining advances in NeRFs with advances in generative AI and also shows potential for future improvements in generative 2D and 3D models.

Vaibhav Tulsyan profil fotoğrafı

Vaibhav Tulsyan2 yıl önce

@adyaman

Cédric Limousin profil fotoğrafı

Cédric Limousin2 yıl önce

If real, that's the future of video. Starting from a blank scene, adding assets, then actors and animating them while being able to move the camera where you want.

Supreme profil fotoğrafı

Supreme2 yıl önce

what the

Poe Allen profil fotoğrafı

Poe Allen2 yıl önce

What is the big deal my bro

Nick Moran profil fotoğrafı

Nick Moran2 yıl önce

"Add a panettone on the tray" is an odd prompt if you're already drawing a bounding box over the tray. The figure in the paper makes it look like the actual prompt would just be "a panettone".

𝑫𝒂𝒏𝒊𝒆𝒍 𝑺𝒄𝒐𝒕𝒕 𝑴𝒂𝒕𝒕𝒉𝒆𝒘𝒔 🇦🇺 profil fotoğrafı

𝑫𝒂𝒏𝒊𝒆𝒍 𝑺𝒄𝒐𝒕𝒕 𝑴𝒂𝒕𝒕𝒉𝒆𝒘𝒔 🇦🇺2 yıl önce

Nice, but perhaps they need to use the existing scene to do a little bit of global illumination and environment mapping from the scene to the inserted object?

Benzer Videolar

Object Cutter Create high-quality HD background removal for ANY object in your image with a text prompt or bounding boxes!

Object Cutter Create high-quality HD background removal for ANY object in your image with a text prompt or bounding boxes!

Gradio

136,153 görüntüleme • 2 yıl önce

Tencent presents TIP-Editor An Accurate 3D Editor Following Both Text-Prompts And Image-Prompts paper page: Text-driven 3D scene editing has gained significant attention owing to its convenience and user-friendliness. However, existing methods still lack accurate control of the specified appearance and location of the editing result due to the inherent limitations of the text description. To this end, we propose a 3D scene editing framework, TIPEditor, that accepts both text and image prompts and a 3D bounding box to specify the editing region. With the image prompt, users can conveniently specify the detailed appearance/style of the target content in complement to the text description, enabling accurate control of the appearance. Specifically, TIP-Editor employs a stepwise 2D personalization strategy to better learn the representation of the existing scene and the reference image, in which a localization loss is proposed to encourage correct object placement as specified by the bounding box. Additionally, TIPEditor utilizes explicit and flexible 3D Gaussian splatting as the 3D representation to facilitate local editing while keeping the background unchanged. Extensive experiments have demonstrated that TIP-Editor conducts accurate editing following the text and image prompts in the specified bounding box region, consistently outperforming the baselines in editing quality, and the alignment to the prompts, qualitatively and quantitatively.

Tencent presents TIP-Editor An Accurate 3D Editor Following Both Text-Prompts And Image-Prompts paper page: Text-driven 3D scene editing has gained significant attention owing to its convenience and user-friendliness. However, existing methods still lack accurate control of the specified appearance and location of the editing result due to the inherent limitations of the text description. To this end, we propose a 3D scene editing framework, TIPEditor, that accepts both text and image prompts and a 3D bounding box to specify the editing region. With the image prompt, users can conveniently specify the detailed appearance/style of the target content in complement to the text description, enabling accurate control of the appearance. Specifically, TIP-Editor employs a stepwise 2D personalization strategy to better learn the representation of the existing scene and the reference image, in which a localization loss is proposed to encourage correct object placement as specified by the bounding box. Additionally, TIPEditor utilizes explicit and flexible 3D Gaussian splatting as the 3D representation to facilitate local editing while keeping the background unchanged. Extensive experiments have demonstrated that TIP-Editor conducts accurate editing following the text and image prompts in the specified bounding box region, consistently outperforming the baselines in editing quality, and the alignment to the prompts, qualitatively and quantitatively.

AK

56,428 görüntüleme • 2 yıl önce

3D Text Around Object

3D Text Around Object

Photoshop Tricks!

20,888 görüntüleme • 2 ay önce

Blended-NeRF: Zero-Shot Object Generation and Blending in Existing Neural Radiance Fields paper page: Editing a local region or a specific object in a 3D scene represented by a NeRF is challenging, mainly due to the implicit nature of the scene representation. Consistently blending a new realistic object into the scene adds an additional level of difficulty. We present Blended-NeRF, a robust and flexible framework for editing a specific region of interest in an existing NeRF scene, based on text prompts or image patches, along with a 3D ROI box. Our method leverages a pretrained language-image model to steer the synthesis towards a user-provided text prompt or image patch, along with a 3D MLP model initialized on an existing NeRF scene to generate the object and blend it into a specified region in the original scene. We allow local editing by localizing a 3D ROI box in the input scene, and seamlessly blend the content synthesized inside the ROI with the existing scene using a novel volumetric blending technique. To obtain natural looking and view-consistent results, we leverage existing and new geometric priors and 3D augmentations for improving the visual fidelity of the final result. We test our framework both qualitatively and quantitatively on a variety of real 3D scenes and text prompts, demonstrating realistic multi-view consistent results with much flexibility and diversity compared to the baselines. Finally, we show the applicability of our framework for several 3D editing applications, including adding new objects to a scene, removing/replacing/altering existing objects, and texture conversion.

Blended-NeRF: Zero-Shot Object Generation and Blending in Existing Neural Radiance Fields paper page: Editing a local region or a specific object in a 3D scene represented by a NeRF is challenging, mainly due to the implicit nature of the scene representation. Consistently blending a new realistic object into the scene adds an additional level of difficulty. We present Blended-NeRF, a robust and flexible framework for editing a specific region of interest in an existing NeRF scene, based on text prompts or image patches, along with a 3D ROI box. Our method leverages a pretrained language-image model to steer the synthesis towards a user-provided text prompt or image patch, along with a 3D MLP model initialized on an existing NeRF scene to generate the object and blend it into a specified region in the original scene. We allow local editing by localizing a 3D ROI box in the input scene, and seamlessly blend the content synthesized inside the ROI with the existing scene using a novel volumetric blending technique. To obtain natural looking and view-consistent results, we leverage existing and new geometric priors and 3D augmentations for improving the visual fidelity of the final result. We test our framework both qualitatively and quantitatively on a variety of real 3D scenes and text prompts, demonstrating realistic multi-view consistent results with much flexibility and diversity compared to the baselines. Finally, we show the applicability of our framework for several 3D editing applications, including adding new objects to a scene, removing/replacing/altering existing objects, and texture conversion.

AK

62,768 görüntüleme • 3 yıl önce

DreamBooth3D: Subject-Driven Text-to-3D Generation Personalized 3D models from just a few casual photos, with text-driven modifications abs: project page:

DreamBooth3D: Subject-Driven Text-to-3D Generation Personalized 3D models from just a few casual photos, with text-driven modifications abs: project page:

AK

107,503 görüntüleme • 3 yıl önce

(1/4) Excited to share our #ICCV2023 paper Text2Room! We generate scene-scale textured 3D meshes from a given text prompt leveraging 2D text-to-image models such as StableDiffusion. Project: Code: Video:

(1/4) Excited to share our #ICCV2023 paper Text2Room! We generate scene-scale textured 3D meshes from a given text prompt leveraging 2D text-to-image models such as StableDiffusion. Project: Code: Video:

Matthias Niessner

74,907 görüntüleme • 3 yıl önce

Current 3D generative models are slow and low quality. We present GRM, a large-scale model that reconstructs 3D Gaussians in 0.1s and generates high-quality 3D assets from text or single images in a few seconds. Demo: 1/4

Current 3D generative models are slow and low quality. We present GRM, a large-scale model that reconstructs 3D Gaussians in 0.1s and generates high-quality 3D assets from text or single images in a few seconds. Demo: 1/4

Gordon Wetzstein

19,241 görüntüleme • 2 yıl önce

NeuralLift-360: Lifting An In-the-wild 2D Photo to A 3D Object with 360° Views github:

NeuralLift-360: Lifting An In-the-wild 2D Photo to A 3D Object with 360° Views github:

AK

142,744 görüntüleme • 3 yıl önce

Our Puffin-World technical paper is now available. From one image or text prompt, it generates camera-controlled views with appearance, geometry and physical grounding, then reconstructs them into an explorable 3D world—all in one unified model. Paper: Blog: Dataset: Code:

Our Puffin-World technical paper is now available. From one image or text prompt, it generates camera-controlled views with appearance, geometry and physical grounding, then reconstructs them into an explorable 3D world—all in one unified model. Paper: Blog: Dataset: Code:

Chen Change Loy

10,439 görüntüleme • 1 ay önce

SceneScape: Text-Driven Consistent Scene Generation abs: project page: text-driven perpetual view generation -- synthesizing long videos of arbitrary scenes solely from an input text describing the scene and camera poses

SceneScape: Text-Driven Consistent Scene Generation abs: project page: text-driven perpetual view generation -- synthesizing long videos of arbitrary scenes solely from an input text describing the scene and camera poses

AK

73,258 görüntüleme • 3 yıl önce

Today we're releasing WildDet3D—an open model for monocular 3D object detection in the wild. It works with text, clicks, or 2D boxes, and on zero-shot evals it nearly doubles the best prior scores. 🧵

Today we're releasing WildDet3D—an open model for monocular 3D object detection in the wild. It works with text, clicks, or 2D boxes, and on zero-shot evals it nearly doubles the best prior scores. 🧵

Ai2

88,758 görüntüleme • 6 ay önce

Excited to share HOI-PAGE, to appear at #ICML2026! 🚀 Lei Li generates 4D human-object interactions zero-shot from text A part-affordance graph grounds interactions via LLM+video priors, enabling complex multi-person, multi-object interactions 👉

Excited to share HOI-PAGE, to appear at #ICML2026! 🚀 Lei Li generates 4D human-object interactions zero-shot from text A part-affordance graph grounds interactions via LLM+video priors, enabling complex multi-person, multi-object interactions 👉

Angela Dai

12,487 görüntüleme • 6 ay önce

Talk2Move Reinforcement Learning for Text-Instructed Object-Level Geometric Transformation in Scenes

Talk2Move Reinforcement Learning for Text-Instructed Object-Level Geometric Transformation in Scenes

AK

10,801 görüntüleme • 9 ay önce

🔥 DreamEngine revolutionizes image generation with its text-guided object fusion capabilities! The demo and code for Text Guided Object Fustion are released! Let's unlock the Imaginations! Run it locally now in: Paper:

🔥 DreamEngine revolutionizes image generation with its text-guided object fusion capabilities! The demo and code for Text Guided Object Fustion are released! Let's unlock the Imaginations! Run it locally now in: Paper:

Liang Chen

11,376 görüntüleme • 1 yıl önce

One-2-3-45: Any Single Image to 3D Mesh in 45 Seconds without Per-Shape Optimization paper page: Single image 3D reconstruction is an important but challenging task that requires extensive knowledge of our natural world. Many existing methods solve this problem by optimizing a neural radiance field under the guidance of 2D diffusion models but suffer from lengthy optimization time, 3D inconsistency results, and poor geometry. In this work, we propose a novel method that takes a single image of any object as input and generates a full 360-degree 3D textured mesh in a single feed-forward pass. Given a single image, we first use a view-conditioned 2D diffusion model, Zero123, to generate multi-view images for the input view, and then aim to lift them up to 3D space. Since traditional reconstruction methods struggle with inconsistent multi-view predictions, we build our 3D reconstruction module upon an SDF-based generalizable neural surface reconstruction method and propose several critical training strategies to enable the reconstruction of 360-degree meshes. Without costly optimizations, our method reconstructs 3D shapes in significantly less time than existing methods. Moreover, our method favors better geometry, generates more 3D consistent results, and adheres more closely to the input image. We evaluate our approach on both synthetic data and in-the-wild images and demonstrate its superiority in terms of both mesh quality and runtime. In addition, our approach can seamlessly support the text-to-3D task by integrating with off-the-shelf text-to-image diffusion models.

One-2-3-45: Any Single Image to 3D Mesh in 45 Seconds without Per-Shape Optimization paper page: Single image 3D reconstruction is an important but challenging task that requires extensive knowledge of our natural world. Many existing methods solve this problem by optimizing a neural radiance field under the guidance of 2D diffusion models but suffer from lengthy optimization time, 3D inconsistency results, and poor geometry. In this work, we propose a novel method that takes a single image of any object as input and generates a full 360-degree 3D textured mesh in a single feed-forward pass. Given a single image, we first use a view-conditioned 2D diffusion model, Zero123, to generate multi-view images for the input view, and then aim to lift them up to 3D space. Since traditional reconstruction methods struggle with inconsistent multi-view predictions, we build our 3D reconstruction module upon an SDF-based generalizable neural surface reconstruction method and propose several critical training strategies to enable the reconstruction of 360-degree meshes. Without costly optimizations, our method reconstructs 3D shapes in significantly less time than existing methods. Moreover, our method favors better geometry, generates more 3D consistent results, and adheres more closely to the input image. We evaluate our approach on both synthetic data and in-the-wild images and demonstrate its superiority in terms of both mesh quality and runtime. In addition, our approach can seamlessly support the text-to-3D task by integrating with off-the-shelf text-to-image diffusion models.

AK

179,061 görüntüleme • 3 yıl önce

Just tried Lux3D: an AI 3D generator built by Manycore Tech You drop in an image or text prompt… and it gives you a full, editable 3D asset with solid geometry + materials. No 3D modeling skills needed. This one actually feels different.

Just tried Lux3D: an AI 3D generator built by Manycore Tech You drop in an image or text prompt… and it gives you a full, editable 3D asset with solid geometry + materials. No 3D modeling skills needed. This one actually feels different.

Sharon Riley

21,697 görüntüleme • 2 ay önce

1/ Spent the past few days testing Lux3D, a 3D generative model. You give it an image or a text prompt and it hands you back a full, editable 3D asset. No modeling skills required. Tried all 3 core features. A thread:

1/ Spent the past few days testing Lux3D, a 3D generative model. You give it an image or a text prompt and it hands you back a full, editable 3D asset. No modeling skills required. Tried all 3 core features. A thread:

Chubby♨️

21,355 görüntüleme • 2 ay önce

Check out Christian Diller's CG-HOI :) We generate realistic 3D human-object interactions, from object geometry and text description. A key ingredient is explicit modeling of contact, during training and as guidance during inference.

Check out Christian Diller's CG-HOI :) We generate realistic 3D human-object interactions, from object geometry and text description. A key ingredient is explicit modeling of contact, during training and as guidance during inference.

Angela Dai

20,520 görüntüleme • 2 yıl önce

The relationship between vertices, sides, and edges in a 3D object

The relationship between vertices, sides, and edges in a 3D object

Interesting STEM

25,571 görüntüleme • 8 ay önce

Text2Room: Extracting Textured 3D Meshes from 2D Text-to-Image Models abs: project page:

Text2Room: Extracting Textured 3D Meshes from 2D Text-to-Image Models abs: project page:

AK

277,316 görüntüleme • 3 yıl önce