Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

Introducing 𝐒𝐀𝐑𝟑𝐃, which tokenizes 3D objects into multiscale tokens and generates 3D objects by autoregressive next-scale prediction. 𝐒𝐀𝐑𝟑𝐃 enables fast 3D generation and comprehensive 3D understanding. arXiv: Project:

Xingang Pan

3,057 subscribers

28,248 views • 1 year ago •via X (Twitter)

Education Arts Science & Technology

Anya Rossi• Live Now

Private livecam show

3 Comments

Xingang Pan1 year ago

This work is mainly done by @yongweichen001

Toad1 year ago

how many Mac minis do I need to run it?

E Schneider1 year ago

Impressive work on SAR3D! I'm excited to see how this novel framework will advance 3D object generation and understanding.

Related Videos

Introducing 3D Projections & Noise Effects - Wrap layers into 3D objects - Perspective and lighting - Noise Glass + Distortion Available on Spline Hana now.

Introducing 3D Projections & Noise Effects - Wrap layers into 3D objects - Perspective and lighting - Noise Glass + Distortion Available on Spline Hana now.

Spline

63,222 views • 10 months ago

Introducing SAM 3D, the newest addition to the SAM collection, bringing common sense 3D understanding of everyday images. SAM 3D includes two models: 🛋️ SAM 3D Objects for object and scene reconstruction 🧑‍🤝‍🧑 SAM 3D Body for human pose and shape estimation Both models achieve state-of-the-art performance transforming static 2D images into vivid, accurate reconstructions. 🔗 Learn more:

Introducing SAM 3D, the newest addition to the SAM collection, bringing common sense 3D understanding of everyday images. SAM 3D includes two models: 🛋️ SAM 3D Objects for object and scene reconstruction 🧑‍🤝‍🧑 SAM 3D Body for human pose and shape estimation Both models achieve state-of-the-art performance transforming static 2D images into vivid, accurate reconstructions. 🔗 Learn more:

AI at Meta

857,258 views • 6 months ago

SAM 3D enables accurate 3D reconstruction from a single image, supporting real-world applications in editing, robotics, and interactive scene generation. Matt, a SAM 3D researcher, explains how the two-model design makes this possible for both people and complex environments. 🔗 Read the SAM 3D Objects research paper: 🔗 Read the SAM 3D Body research paper:

SAM 3D enables accurate 3D reconstruction from a single image, supporting real-world applications in editing, robotics, and interactive scene generation. Matt, a SAM 3D researcher, explains how the two-model design makes this possible for both people and complex environments. 🔗 Read the SAM 3D Objects research paper: 🔗 Read the SAM 3D Body research paper:

AI at Meta

17,858 views • 6 months ago

3D Gaussian Splats are cool, but they're static (Part 25). DG-Mesh can reconstruct high-quality and time-consistent 3D meshes from a single video and track mesh vertices over time, which enables texture editing on dynamic objects.

3D Gaussian Splats are cool, but they're static (Part 25). DG-Mesh can reconstruct high-quality and time-consistent 3D meshes from a single video and track mesh vertices over time, which enables texture editing on dynamic objects.

Dreaming Tulpa 🥓👑

79,033 views • 2 years ago

We are thrilled to announce a major upgrade to our open-source 3D generation model, introducing two groundbreaking new versions: 3D 2.0 MV (Multi-View Generation) and 3D 2.0 Mini! 3D 2.0 MV : 3D 2.0Mini:

We are thrilled to announce a major upgrade to our open-source 3D generation model, introducing two groundbreaking new versions: 3D 2.0 MV (Multi-View Generation) and 3D 2.0 Mini! 3D 2.0 MV : 3D 2.0Mini:

Tencent HY

134,410 views • 1 year ago

3D arrived to Krea. this new feature lets you turn images into 3D objects and use them in our Real-time tool. free for everyone.

3D arrived to Krea. this new feature lets you turn images into 3D objects and use them in our Real-time tool. free for everyone.

Krea

665,421 views • 1 year ago

LLaVA-3D A Simple yet Effective Pathway to Empowering LMMs with 3D-awareness Recent advancements in Large Multimodal Models (LMMs) have greatly enhanced their proficiency in 2D visual understanding tasks, enabling them to effectively process and understand images and videos. However, the development of LMMs with 3D-awareness for 3D scene understanding has been hindered by the lack of large-scale 3D vision-language datasets and powerful 3D encoders. In this paper, we introduce a simple yet effective framework called LLaVA-3D. Leveraging the strong 2D understanding priors from LLaVA, our LLaVA-3D efficiently adapts LLaVA for 3D scene understanding without compromising 2D understanding capabilities. To achieve this, we employ a simple yet effective representation, 3D Patch, which connects 2D CLIP patch features with their corresponding positions in 3D space. By integrating the 3D Patches into 2D LMMs and employing joint 2D and 3D vision-language instruction tuning, we establish a unified architecture for both 2D image understanding and 3D scene understanding. Experimental results show that LLaVA-3D converges 3.5x faster than existing 3D LMMs when trained on 3D vision-language datasets. Moreover, LLaVA-3D not only achieves state-of-the-art performance across various 3D tasks but also maintains comparable 2D image understanding and vision-language conversation capabilities with LLaVA.

LLaVA-3D A Simple yet Effective Pathway to Empowering LMMs with 3D-awareness Recent advancements in Large Multimodal Models (LMMs) have greatly enhanced their proficiency in 2D visual understanding tasks, enabling them to effectively process and understand images and videos. However, the development of LMMs with 3D-awareness for 3D scene understanding has been hindered by the lack of large-scale 3D vision-language datasets and powerful 3D encoders. In this paper, we introduce a simple yet effective framework called LLaVA-3D. Leveraging the strong 2D understanding priors from LLaVA, our LLaVA-3D efficiently adapts LLaVA for 3D scene understanding without compromising 2D understanding capabilities. To achieve this, we employ a simple yet effective representation, 3D Patch, which connects 2D CLIP patch features with their corresponding positions in 3D space. By integrating the 3D Patches into 2D LMMs and employing joint 2D and 3D vision-language instruction tuning, we establish a unified architecture for both 2D image understanding and 3D scene understanding. Experimental results show that LLaVA-3D converges 3.5x faster than existing 3D LMMs when trained on 3D vision-language datasets. Moreover, LLaVA-3D not only achieves state-of-the-art performance across various 3D tasks but also maintains comparable 2D image understanding and vision-language conversation capabilities with LLaVA.

AK

41,713 views • 1 year ago

Holographic fans to visualise 3D objects

Holographic fans to visualise 3D objects

Science girl

671,081 views • 7 months ago

Apple Vision Pro’s AirDraw app converts 3D drawings into physical objects

Apple Vision Pro’s AirDraw app converts 3D drawings into physical objects

Apple Design

1,061,826 views • 4 months ago

Just made Pascal's item catalog infinite. Pulling 3D objects straight out of images, live in the browser. SAM 3D Objects + Three.js + WebGPU. 👇

Just made Pascal's item catalog infinite. Pulling 3D objects straight out of images, live in the browser. SAM 3D Objects + Three.js + WebGPU. 👇

Wawa Sensei

46,257 views • 1 month ago

Introducing Objaverse-XL, an open dataset of over 10 million 3D objects! With it, we train Zero123-XL, a foundation model for 3D, observing incredible 3D generalization abilities: 🧵👇 📝 Paper:

Introducing Objaverse-XL, an open dataset of over 10 million 3D objects! With it, we train Zero123-XL, a foundation model for 3D, observing incredible 3D generalization abilities: 🧵👇 📝 Paper:

Matt Deitke

368,681 views • 2 years ago

Step into a new era of creation. With Cube 3D, turn simple text prompts into 3D objects in seconds. What will you build?

Step into a new era of creation. With Cube 3D, turn simple text prompts into 3D objects in seconds. What will you build?

Roblox

1,058,293 views • 1 year ago

🚀 Introducing #DreamCraft3D, our breakthrough hierarchical technique for 3D content generation. Leading the way in AIGC, we're redefining 3D generation: Project page: Paper: Code:

🚀 Introducing #DreamCraft3D, our breakthrough hierarchical technique for 3D content generation. Leading the way in AIGC, we're redefining 3D generation: Project page: Paper: Code:

DeepSeek

16,400 views • 2 years ago

Today we’re excited to unveil a new generation of Segment Anything Models: 1️⃣ SAM 3 enables detecting, segmenting and tracking of objects across images and videos, now with short text phrases and exemplar prompts. 🔗 Learn more about SAM 3: 2️⃣ SAM 3D brings the model collection into the 3rd dimension to enable precise reconstruction of 3D objects and people from a single 2D image. 🔗 Learn more about SAM 3D: These models offer innovative capabilities and unique tools for developers and researchers to create, experiment and uplevel media workflows.

Today we’re excited to unveil a new generation of Segment Anything Models: 1️⃣ SAM 3 enables detecting, segmenting and tracking of objects across images and videos, now with short text phrases and exemplar prompts. 🔗 Learn more about SAM 3: 2️⃣ SAM 3D brings the model collection into the 3rd dimension to enable precise reconstruction of 3D objects and people from a single 2D image. 🔗 Learn more about SAM 3D: These models offer innovative capabilities and unique tools for developers and researchers to create, experiment and uplevel media workflows.

AI at Meta

1,087,064 views • 6 months ago

🚨 Hunyuan 3D 3.1 Pro and Rapid is here on fal! 🎯 Pro: High-fidelity Image-to-3D and Text-to-3D generation ⚡ Rapid: Speed-optimized 3D generation ✨ Smart Topology and Part generation for advanced 3D workflows

🚨 Hunyuan 3D 3.1 Pro and Rapid is here on fal! 🎯 Pro: High-fidelity Image-to-3D and Text-to-3D generation ⚡ Rapid: Speed-optimized 3D generation ✨ Smart Topology and Part generation for advanced 3D workflows

fal

26,048 views • 4 months ago

Your next 3D photo shoot will be done with AI 3D Scenes generates full environments from any image → Place your objects in the scene → Move the camera like a real shoot → Consistent lightning and detail across every angle Available now on Freepik 👇

Your next 3D photo shoot will be done with AI 3D Scenes generates full environments from any image → Place your objects in the scene → Move the camera like a real shoot → Consistent lightning and detail across every angle Available now on Freepik 👇

Magnific

324,509 views • 2 months ago

BREAKING 🚨: Microsoft is working on a Copilot 3D experiment where users will be able to turn images into 3D objects. * Early development stage

BREAKING 🚨: Microsoft is working on a Copilot 3D experiment where users will be able to turn images into 3D objects. * Early development stage

🚨 AI News | TestingCatalog

18,163 views • 1 year ago

A UV printer adds perfect logos to 3D objects

A UV printer adds perfect logos to 3D objects

Science girl

43,612 views • 7 months ago

Introducing AI 3D Generation ✨ 🔥 Text to 3D 🖼️ Image to 3D 🌳 Remix 📐 Editor integration & Exports Available now!

Introducing AI 3D Generation ✨ 🔥 Text to 3D 🖼️ Image to 3D 🌳 Remix 📐 Editor integration & Exports Available now!

Spline

705,628 views • 2 years ago