正在加载视频...

视频加载失败

Introducing ShapeR, a method for robust conditional 3D shape generation from casually captured sequences. ShapeR leverages a rectified flow transformer conditioned on per-object multimodal data to turn casual image sequences into full metric scene reconstructions. Project Page: Paper: Links to code and huggingface below ⬇️

71,078 次观看 • 9 个月前 •via X (Twitter)

15 条评论

Yawar Siddiqui 的头像
Yawar Siddiqui9 个月前

Code: Model: Evaluation Dataset: Video:

Jim 的头像
Jim9 个月前

This is going to be amazing for so many applications especially robotics training data

Zwecharki of Immense AGI fundamentalism 的头像
Zwecharki of Immense AGI fundamentalism9 个月前

Its incredible. Can it be used with unstructured lidar data so we can combine it with iphone lidar + camera mount setups? Or does the lidar have to be specifically aligned with the cameras like on the Aria glasses?

David Ferrera 的头像
David Ferrera9 个月前

What do I use to capture the data: Meta Quest, iPhone, lidar?

Not Indexed 的头像
Not Indexed9 个月前

@droneforge

X-Files 的头像
X-Files9 个月前

@intuition_trust You need a drone automated like prometheus

Zwecharki of Immense AGI fundamentalism 的头像
Zwecharki of Immense AGI fundamentalism9 个月前

Can we use this to clear out floaters, low data areas that fuse together or white wall glitches for gaussian splat scans?

Joshua Poddoku 的头像
Joshua Poddoku9 个月前

Impressive direction, curious how it holds up in cluttered indoor scenes

Amit 的头像
Amit9 个月前

Rectified flow transformers for 3D generation are a step beyond vanilla diffusion. Conditioning on multimodal signals unlocks compositionality, but metric accuracy from casual sequences still hinges on how you handle scale drift and occlusions.

Reza Sayar 的头像
Reza Sayar9 个月前

wow!! awesome work!! and thanks for releasing everything! 🤗 @pablovelagomez1 lookie 🔥

mrkelly 的头像
mrkelly9 个月前

Casual sequences work great. Until your real data doesn't match the training distribution.

Micah Alpern 的头像
Micah Alpern9 个月前

What do you want to use it for? Making a robot that moves?

Nick Harris 的头像
Nick Harris9 个月前

Is this what Houzz Pro uses to build their mockups with the iphone scans

ObotX - Open Source Robotics 的头像
ObotX - Open Source Robotics9 个月前

nice!

indoor positioning 的头像
indoor positioning9 个月前

@grok 这个的应用场景

相关视频

Blended-NeRF: Zero-Shot Object Generation and Blending in Existing Neural Radiance Fields paper page: Editing a local region or a specific object in a 3D scene represented by a NeRF is challenging, mainly due to the implicit nature of the scene representation. Consistently blending a new realistic object into the scene adds an additional level of difficulty. We present Blended-NeRF, a robust and flexible framework for editing a specific region of interest in an existing NeRF scene, based on text prompts or image patches, along with a 3D ROI box. Our method leverages a pretrained language-image model to steer the synthesis towards a user-provided text prompt or image patch, along with a 3D MLP model initialized on an existing NeRF scene to generate the object and blend it into a specified region in the original scene. We allow local editing by localizing a 3D ROI box in the input scene, and seamlessly blend the content synthesized inside the ROI with the existing scene using a novel volumetric blending technique. To obtain natural looking and view-consistent results, we leverage existing and new geometric priors and 3D augmentations for improving the visual fidelity of the final result. We test our framework both qualitatively and quantitatively on a variety of real 3D scenes and text prompts, demonstrating realistic multi-view consistent results with much flexibility and diversity compared to the baselines. Finally, we show the applicability of our framework for several 3D editing applications, including adding new objects to a scene, removing/replacing/altering existing objects, and texture conversion.

AK

62,768 次观看 • 3 年前