Video yรผkleniyor...

Video Yรผklenemedi

Introducing ๐†๐š๐ฎ๐ฌ๐ฌ๐ข๐š๐ง ๐€๐ง๐ฒ๐ญ๐ก๐ข๐ง๐ , a new 3D generative model with two key properties: - A structured point-cloud latent space enabling flexible editing! - Support multi-modal conditions, e.g., point cloud, text, single/multi-view images arXiv:

55,847 gรถrรผntรผleme โ€ข 1 yฤฑl รถnce โ€ขvia X (Twitter)

8 Yorum

Xingang Pan profil fotoฤŸrafฤฑ
Xingang Pan1 yฤฑl รถnce

Project page:

Zoe Wang profil fotoฤŸrafฤฑ
Zoe Wang1 yฤฑl รถnce

The paper introduces a comprehensive 3D generation framework with a point cloud-structured 3D latent space, addressing the limitations of existing methods.

HELLCAT profil fotoฤŸrafฤฑ
HELLCAT1 yฤฑl รถnce

gaussian anything? more like reality collapse. can't wait to break this one too.

Angel profil fotoฤŸrafฤฑ
Angel1 yฤฑl รถnce

The ability to edit in a structured point-cloud space is a huge leap forward.

AnonymousDestiny profil fotoฤŸrafฤฑ
AnonymousDestiny1 yฤฑl รถnce

I'm super glad there's finally a 3D generative model that allows multiple images to improve the geometry of the output!

Ostap Hembara profil fotoฤŸrafฤฑ
Ostap Hembara1 yฤฑl รถnce

This animation has bought me right away, I`ll have to read this now.

Artur Formella profil fotoฤŸrafฤฑ
Artur Formella1 yฤฑl รถnce

What if I have 8 images of the object?

Synthical profil fotoฤŸrafฤฑ
Synthical1 yฤฑl รถnce

Dark mode for this paper ๐ŸŒ™

Benzer Videolar

Want to create an avatar from a single image? FlexAvatar is a transformer model that creates full 360ยฐ, high-quality, and expressive 3D head avatar from just a single portrait image in minutes. Real-time Demo: FlexAvatar's lightweight architecture allows both animation and rendering in real-time, enabling interactive user experiences. To create a new 3D head avatar, only one image is required, e.g., from a webcam. The final avatar is ready after 2 minutes. Architecture: Under the hood, FlexAvatar adopts a transformer-based encoder-decoder design. The encoder maps the input image onto a latent avatar space, while the decoder produces 3D Gaussian attribute maps by incorporating the animation signal via cross-attention. The model learns all facial animations directly from the data without relying on pre-built 3D face models. This equips the avatars with realistic facial expressions. The internal avatar latent space can be conveniently used to integrate additional observations of a person via fitting. This enables use-cases where more than one image of a person is available, e.g., from a phone scan of the person. We train jointly on 2D monocular videos and multi-view data. However, in monocular videos, the animation signal leaks the target viewpoint, causing the model to produce incomplete 3D heads. We call this phenomenon entanglement of driving signal and target viewpoint. To prevent entanglement, we introduce bias sinks. These are learnable tokens that indicate whether a training sample stems from a monocular or a multi-view dataset. During training, the model learns to produce incomplete 3D heads only when the monocular token is present. During inference, FlexAvatar then always uses the multi-view token for which the model has learned to produce complete 3D heads. This simple design allows to combine the generalizability from monocular data with the quality of multi-view data. FlexAvatar summary: - Input: Single-image, phone scan, or monocular video - Output: Full 360ยฐ head avatar - Expressive animations - Real-time rendering and animation - Generalization to any portrait - Create a new avatar in 2 minutes - Use bias sinks to combine 2D and 3D data ๐Ÿ  ๐ŸŒ ๐ŸŽฅ Great work by Tobias Kirschstein and Simon Giebenhain!

Matthias Niessner

95,991 gรถrรผntรผleme โ€ข 7 ay รถnce

Blended-NeRF: Zero-Shot Object Generation and Blending in Existing Neural Radiance Fields paper page: Editing a local region or a specific object in a 3D scene represented by a NeRF is challenging, mainly due to the implicit nature of the scene representation. Consistently blending a new realistic object into the scene adds an additional level of difficulty. We present Blended-NeRF, a robust and flexible framework for editing a specific region of interest in an existing NeRF scene, based on text prompts or image patches, along with a 3D ROI box. Our method leverages a pretrained language-image model to steer the synthesis towards a user-provided text prompt or image patch, along with a 3D MLP model initialized on an existing NeRF scene to generate the object and blend it into a specified region in the original scene. We allow local editing by localizing a 3D ROI box in the input scene, and seamlessly blend the content synthesized inside the ROI with the existing scene using a novel volumetric blending technique. To obtain natural looking and view-consistent results, we leverage existing and new geometric priors and 3D augmentations for improving the visual fidelity of the final result. We test our framework both qualitatively and quantitatively on a variety of real 3D scenes and text prompts, demonstrating realistic multi-view consistent results with much flexibility and diversity compared to the baselines. Finally, we show the applicability of our framework for several 3D editing applications, including adding new objects to a scene, removing/replacing/altering existing objects, and texture conversion.

AK

62,768 gรถrรผntรผleme โ€ข 3 yฤฑl รถnce

๐Ÿ“ข๐Ÿ“ข ๐๐ž๐ซ๐œ๐‡๐ž๐š๐: ๐๐ž๐ซ๐œ๐ž๐ฉ๐ญ๐ฎ๐š๐ฅ ๐‡๐ž๐š๐ ๐Œ๐จ๐๐ž๐ฅ ๐Ÿ๐จ๐ซ ๐’๐ข๐ง๐ ๐ฅ๐ž-๐ˆ๐ฆ๐š๐ ๐ž ๐Ÿ‘๐ƒ ๐‡๐ž๐š๐ ๐‘๐ž๐œ๐จ๐ง๐ฌ๐ญ๐ซ๐ฎ๐œ๐ญ๐ข๐จ๐ง & ๐„๐๐ข๐ญ๐ข๐ง๐ ๐Ÿ“ข๐Ÿ“ข PercHead reconstructs realistic 3D heads from a single image and enables disentangled 3D editing via geometric controls and style inputs from images or text. At its core is a generalized 3D head decoder trained with perceptual supervision from DINOv2 and SAM 2.1. We find that our new perceptual loss formulation improves reconstruction fidelity compared to commonly-used methods such as LPIPS. Our trained reconstruction model is able to generate 3D-consistent heads from a single input image. Even with challenging side-view inputs, the model robustly infers missing regions for a coherent, high-fidelity output. In addition, our architecture seamlessly adapts to downstream tasks: by swapping the encoder, we can transform the model into a disentangled 3D editing pipeline. In this scenario, we can control geometry through - potentially hand-drawn - segmentation maps, and condition style via image or text prompt. We also provide an interactive GUI to enable the exploration of our editing pipeline. ๐ŸŒ ๐Ÿ“ฝ๏ธ Great work by Antonio Oroz and Tobias Kirschstein

Matthias Niessner

18,855 gรถrรผntรผleme โ€ข 8 ay รถnce