Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

Introducing Objaverse, a massive open dataset of text-paired 3D objects! Nearly 1 million annotated 3D objects to pave the way to build incredible large-scale 3D generative models: 🧵👇 🤗 Hugging Face: 📝ArXiv: #CVPR2023

mattdeitke's profile picture

Matt Deitke

13,878 subscribers

1,601,100 Aufrufe • vor 3 Jahren •via X (Twitter)

BildungWissenschaft & Technologie#CVPR2023
LIVE

Anya Rossi• Live Now

Private livecam show

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

Introducing 📦𝗔𝗿𝘁𝗶𝗟𝗮𝘁𝗲𝗻𝘁🔧 (SIGGRAPH Asia 2025) — a high-quality 3D diffusion model that explicitly models object articulation, paving the way for richer, more realistic assets in embodied AI and simulation: – Generates fully articulated 3D objects – Physically plausible joints & motion – High-fidelity 3D Gaussian appearance – Supports generation from a single real image arXiv: Project: Code (coming soon):
0:16

Introducing 📦𝗔𝗿𝘁𝗶𝗟𝗮𝘁𝗲𝗻𝘁🔧 (SIGGRAPH Asia 2025) — a high-quality 3D diffusion model that explicitly models object articulation, paving the way for richer, more realistic assets in embodied AI and simulation: – Generates fully articulated 3D objects – Physically plausible joints & motion – High-fidelity 3D Gaussian appearance – Supports generation from a single real image arXiv: Project: Code (coming soon):

Xingang Pan

11,520 Aufrufe • vor 9 Monaten

(1/3) Can we turn text-to-image models into photorealistic 3D generators? ViewDiff (#CVPR2024) produces realistic, multi-view consistent images of real-world 3D objects in authentic surroundings. Website Video How does it work?
0:18

(1/3) Can we turn text-to-image models into photorealistic 3D generators? ViewDiff (#CVPR2024) produces realistic, multi-view consistent images of real-world 3D objects in authentic surroundings. Website Video How does it work?

Matthias Niessner

34,761 Aufrufe • vor 2 Jahren

Diffusion for virtual worlds: we've trained a new model to create 3D objects from text. And it's 50x faster than any alternative. We made each of these cute characters in just one minute with Genmo's new text-to-3D model. Build your own world at
0:16

Diffusion for virtual worlds: we've trained a new model to create 3D objects from text. And it's 50x faster than any alternative. We made each of these cute characters in just one minute with Genmo's new text-to-3D model. Build your own world at

Genmo

18,532 Aufrufe • vor 3 Jahren

✨ Any static 3D assets ➡️ 4D dynamic worlds. Introducing CHORD, a universal framework for generating scene-level 4D dynamic motion from any static 3D inputs. It generalizes surprisingly well across a wide range of objects 🤯 and can even be used to learn robotics manipulation policy 🤖! Project page: Dive deeper in a 🧵: 1/n
0:12

✨ Any static 3D assets ➡️ 4D dynamic worlds. Introducing CHORD, a universal framework for generating scene-level 4D dynamic motion from any static 3D inputs. It generalizes surprisingly well across a wide range of objects 🤯 and can even be used to learn robotics manipulation policy 🤖! Project page: Dive deeper in a 🧵: 1/n

Chen Geng

43,382 Aufrufe • vor 7 Monaten

(1/2) #NVIDIA and Shutterstock have announced a landmark deal to build new generative #AIart tools that will let users create custom 3D models simply by typing in text descriptions #3Dmodeling #gamedev #motiongraphics #VFX #archviz
0:17

(1/2) #NVIDIA and Shutterstock have announced a landmark deal to build new generative #AIart tools that will let users create custom 3D models simply by typing in text descriptions #3Dmodeling #gamedev #motiongraphics #VFX #archviz

CG Channel

240,483 Aufrufe • vor 3 Jahren

As announced in partnership with NVIDIA at CES, we’re excited to introduce Stable Point Aware 3D (SPAR3D), setting a new standard in 3D generation. Ideal for running on NVIDIA RTX AI PCs, SPAR3D enables real-time editing and complete structure generation of 3D objects from a single image in under a second. You can download the weights on Hugging Face and code on GitHub, or access the model through the Stability AI API. Learn more here: (1/3)
0:13

As announced in partnership with NVIDIA at CES, we’re excited to introduce Stable Point Aware 3D (SPAR3D), setting a new standard in 3D generation. Ideal for running on NVIDIA RTX AI PCs, SPAR3D enables real-time editing and complete structure generation of 3D objects from a single image in under a second. You can download the weights on Hugging Face and code on GitHub, or access the model through the Stability AI API. Learn more here: (1/3)

Stability AI

181,554 Aufrufe • vor 1 Jahr

Space objects in 3D NASA created 3D models of stars and supernova remnants by combining data from the Chandra X-ray Observatory with computer calculations. These models can not only be viewed on the screen, but also printed on a 3D printer. Here they are, from left to right — the supernova remnant Cassiopeia A, the young star BP Tauri, the planetary nebula Cygnus Loop, and the supernova remnant G292.0+1.8.
0:15

Space objects in 3D NASA created 3D models of stars and supernova remnants by combining data from the Chandra X-ray Observatory with computer calculations. These models can not only be viewed on the screen, but also printed on a 3D printer. Here they are, from left to right — the supernova remnant Cassiopeia A, the young star BP Tauri, the planetary nebula Cygnus Loop, and the supernova remnant G292.0+1.8.

Black Hole

13,099 Aufrufe • vor 1 Jahr

Combining the explicit control of 3D software with the creativity of generative AI models is a promising yet underrated workflow. Build your 3D scenes procedurally by describing them in natural language, then take them all the way with your image & video models of choice. Tools like intangible are built around such a workflow so you don't need to duct-tape apps together. Pretty cool!
0:15

Combining the explicit control of 3D software with the creativity of generative AI models is a promising yet underrated workflow. Build your 3D scenes procedurally by describing them in natural language, then take them all the way with your image & video models of choice. Tools like intangible are built around such a workflow so you don't need to duct-tape apps together. Pretty cool!

Bilawal Sidhu

37,629 Aufrufe • vor 1 Jahr

Announcing our latest open medical AI models for developers: MedGemma 1.5, which is small enough to run offline & improves performance on 3D imaging (CT & MRI), & MedASR, a speech-to-text model for medical dictation. Both available on Hugging Face + Vertex AI. #MedGemma #HealthAI #GenerativeAI
0:18

Announcing our latest open medical AI models for developers: MedGemma 1.5, which is small enough to run offline & improves performance on 3D imaging (CT & MRI), & MedASR, a speech-to-text model for medical dictation. Both available on Hugging Face + Vertex AI. #MedGemma #HealthAI #GenerativeAI

Google Research

395,456 Aufrufe • vor 7 Monaten

Today, we released Lyra 2.0, a framework for generating persistent, explorable 3D worlds at scale, from NVIDIA Research. Generating large-scale, complex environments is difficult for AI models. Current models often “forget” what spaces look like and lose track of movement over time, causing objects to shift, blur, or appear inconsistent. This prevents them from creating the reliable 3D environments required for downstream simulations. Lyra 2.0 solves these issues by: ✅ Maintaining per-frame 3D geometry to retrieve past frames and establish spatial correspondences ✅ Using self-augmented training to correct its own temporal drifting. Lyra 2.0 turns an image into a 3D world you can walk through, look back, and drop a robot into for real-time rendering, simulation, and immersive applications. ➡️ Learn more: 📄 Read the paper:
0:15

Today, we released Lyra 2.0, a framework for generating persistent, explorable 3D worlds at scale, from NVIDIA Research. Generating large-scale, complex environments is difficult for AI models. Current models often “forget” what spaces look like and lose track of movement over time, causing objects to shift, blur, or appear inconsistent. This prevents them from creating the reliable 3D environments required for downstream simulations. Lyra 2.0 solves these issues by: ✅ Maintaining per-frame 3D geometry to retrieve past frames and establish spatial correspondences ✅ Using self-augmented training to correct its own temporal drifting. Lyra 2.0 turns an image into a 3D world you can walk through, look back, and drop a robot into for real-time rendering, simulation, and immersive applications. ➡️ Learn more: 📄 Read the paper:

NVIDIA AI Developer

437,660 Aufrufe • vor 4 Monaten

Create a 3D model from a single image, set of images or a text prompt in < 1 minute 😮‍💨 This new AI paper called CAT3D shows us that it’ll keep getting easier to produce 3D models from 2D images — whether it’s a sparser real world 3D scan (a few photos instead of hundreds) or your favorite 2D image generator like Midjourney (just an image). How does this magic work? “This architecture is similar to video diffusion models, but with camera pose embeddings for each image instead of time embeddings. The generated views are passed into a robust 3D reconstruction pipeline to create the 3D representation (Zip-NeRF or 3DGS)”
0:18

Create a 3D model from a single image, set of images or a text prompt in < 1 minute 😮‍💨 This new AI paper called CAT3D shows us that it’ll keep getting easier to produce 3D models from 2D images — whether it’s a sparser real world 3D scan (a few photos instead of hundreds) or your favorite 2D image generator like Midjourney (just an image). How does this magic work? “This architecture is similar to video diffusion models, but with camera pose embeddings for each image instead of time embeddings. The generated views are passed into a robust 3D reconstruction pipeline to create the 3D representation (Zip-NeRF or 3DGS)”

Bilawal Sidhu

92,867 Aufrufe • vor 2 Jahren

We’re excited to share the OVER roadmap for 2026. The roadmap is focused on one core objective: scaling a machine-readable, world-scale 3D map of the physical world. Everything we’re building in 2026 ties back to this 🧵👇
0:14

We’re excited to share the OVER roadmap for 2026. The roadmap is focused on one core objective: scaling a machine-readable, world-scale 3D map of the physical world. Everything we’re building in 2026 ties back to this 🧵👇

Over the Reality 🌐

176,393 Aufrufe • vor 7 Monaten

Today we’re releasing SAIR, the Structurally Augmented IC50 Repository. SAIR is the Largest Open-Sourced Binding Affinity Dataset with Cofolded 3D Structures. It includes more than 5 million protein-ligand structures, generated using our Large Quantitative Models and labeled with binding affinity data. By providing this unprecedented scale of structure-activity data, we aim to enable researchers to train and evaluate new AI models for drug discovery, bridging the historical gap between molecular structures and drug potency prediction. The SAIR dataset was created using the NVIDIA DGX Cloud and is now publicly available on the Google Cloud Platform. Access and build with SAIR today! 📰Read the Press Release: 📥Learn More and Download the Dataset at #DrugDiscovery #LQMs #SAIR #AIforScience #SandboxAQ
0:16

Today we’re releasing SAIR, the Structurally Augmented IC50 Repository. SAIR is the Largest Open-Sourced Binding Affinity Dataset with Cofolded 3D Structures. It includes more than 5 million protein-ligand structures, generated using our Large Quantitative Models and labeled with binding affinity data. By providing this unprecedented scale of structure-activity data, we aim to enable researchers to train and evaluate new AI models for drug discovery, bridging the historical gap between molecular structures and drug potency prediction. The SAIR dataset was created using the NVIDIA DGX Cloud and is now publicly available on the Google Cloud Platform. Access and build with SAIR today! 📰Read the Press Release: 📥Learn More and Download the Dataset at #DrugDiscovery #LQMs #SAIR #AIforScience #SandboxAQ

SandboxAQ

129,416 Aufrufe • vor 1 Jahr

Wonderland: Navigating 3D Scenes from a Single Image Contributions: • First, we introduce a representation for controllable 3D generation by leveraging the generative priors from camera-guided video diffusion models. Unlike image models, video diffusion models are trained on extensive video datasets. This enables them to capture comprehensive spatial relationships within scenes across multiple views and embed a form of "3D awareness" in their latent space, which allows us to maintain 3D consistency in novel view synthesis. • Second, to achieve controllable novel view generation, we empower video models with precise control over specified camera motions. We introduce a novel dual-branch conditioning mechanism that effectively incorporates desired diverse camera trajectories into the video diffusion model. This enables expansion of a single image into a multi-view consistent capture of a 3D scene with precise pose control. • Third, to achieve efficient 3D reconstruction, we directly transform video latents into 3DGS. We propose a novel latent-based large reconstruction model (LaLRM) that lifts video latents to 3D in a feed-forward manner. With this design, during inference, our model directly predicts 3DGS from a single input image, effectively aligning the generation and reconstruction tasks—and bridging image space and 3D space—through the video latent space. Compared with reconstructing scenes from images, the video latent space offers a 256× spatial-temporal reduction while retaining essential and consistent 3D structural details. Such a high degree of compression is crucial, as it allows the LaLRM to handle a wider range of 3D scenes within the reconstruction framework, with the same memory constraints.
0:12

Wonderland: Navigating 3D Scenes from a Single Image Contributions: • First, we introduce a representation for controllable 3D generation by leveraging the generative priors from camera-guided video diffusion models. Unlike image models, video diffusion models are trained on extensive video datasets. This enables them to capture comprehensive spatial relationships within scenes across multiple views and embed a form of "3D awareness" in their latent space, which allows us to maintain 3D consistency in novel view synthesis. • Second, to achieve controllable novel view generation, we empower video models with precise control over specified camera motions. We introduce a novel dual-branch conditioning mechanism that effectively incorporates desired diverse camera trajectories into the video diffusion model. This enables expansion of a single image into a multi-view consistent capture of a 3D scene with precise pose control. • Third, to achieve efficient 3D reconstruction, we directly transform video latents into 3DGS. We propose a novel latent-based large reconstruction model (LaLRM) that lifts video latents to 3D in a feed-forward manner. With this design, during inference, our model directly predicts 3DGS from a single input image, effectively aligning the generation and reconstruction tasks—and bridging image space and 3D space—through the video latent space. Compared with reconstructing scenes from images, the video latent space offers a 256× spatial-temporal reduction while retaining essential and consistent 3D structural details. Such a high degree of compression is crucial, as it allows the LaLRM to handle a wider range of 3D scenes within the reconstruction framework, with the same memory constraints.

MrNeRF

52,849 Aufrufe • vor 1 Jahr

Another big drop from Hunyuan3D by Tencent. Hunyuan3D-Buffalo 1.0 is not just another text-to-3D model. The more interesting part is its approach to 3D understanding and editing. It can understand separate parts of a complete asset, extract them, remove or replace them with prompts, and edit only the selected region while keeping the rest of the geometry unchanged. That feels like a much more useful direction for 3D AI in general: not just generating a mesh once, but actually understanding its structure and letting you continue working with it. The model combines Qwen-VL, TRELLIS and Hunyuan3D, and was trained on an impressive dataset of 87 million 3D samples. Source:
0:18

Another big drop from Hunyuan3D by Tencent. Hunyuan3D-Buffalo 1.0 is not just another text-to-3D model. The more interesting part is its approach to 3D understanding and editing. It can understand separate parts of a complete asset, extract them, remove or replace them with prompts, and edit only the selected region while keeping the rest of the geometry unchanged. That feels like a much more useful direction for 3D AI in general: not just generating a mesh once, but actually understanding its structure and letting you continue working with it. The model combines Qwen-VL, TRELLIS and Hunyuan3D, and was trained on an impressive dataset of 87 million 3D samples. Source:

Stefan 3D AI

36,034 Aufrufe • vor 22 Tagen

You can't 3D reconstruct glass from images... ...WRONG! Thanks for video diffusion, now just about anything is possible! Introducing...Diffusion Knows Transparency (DKT) Transparent and reflective objects usually break robot vision and photogrammetry pipelines because they don't follow the "solid object" rules standard cameras expect. DKT is a new AI model that repurposes the "internal physics engine" found in video generation models to solve this problem. Researchers took a massive video diffusion model (WAN) and fine-tuned it using a custom-built synthetic dataset to turn it into a high-precision depth sensor. To train the AI, they built the first massive synthetic video library of transparent objects, 1.32 million frames of perfectly labeled glass and metal objects in motion. Without ever seeing a "real" labeled video of glass during training, the model (DKT) outperformed all previous specialized systems on real-world benchmarks (ClearPose, DREDS). They created a "lightweight" 1.3B parameter version that runs fast enough (0.17s per frame) to be used on actual robot hardware. Two reasons I find this project important: 1. It further proves that synthetic data will be essential for training the next generation vision models. 2. In real-world robotic tests, using DKT's depth maps nearly doubled the success rate of robot arms trying to pick up objects on tricky reflective or translucent surfaces. At home robots will need to interact with these types of objects on a daily basis. Check out the project page here: Code is LIVE! #Computervision #Robotics #AI
0:14

You can't 3D reconstruct glass from images... ...WRONG! Thanks for video diffusion, now just about anything is possible! Introducing...Diffusion Knows Transparency (DKT) Transparent and reflective objects usually break robot vision and photogrammetry pipelines because they don't follow the "solid object" rules standard cameras expect. DKT is a new AI model that repurposes the "internal physics engine" found in video generation models to solve this problem. Researchers took a massive video diffusion model (WAN) and fine-tuned it using a custom-built synthetic dataset to turn it into a high-precision depth sensor. To train the AI, they built the first massive synthetic video library of transparent objects, 1.32 million frames of perfectly labeled glass and metal objects in motion. Without ever seeing a "real" labeled video of glass during training, the model (DKT) outperformed all previous specialized systems on real-world benchmarks (ClearPose, DREDS). They created a "lightweight" 1.3B parameter version that runs fast enough (0.17s per frame) to be used on actual robot hardware. Two reasons I find this project important: 1. It further proves that synthetic data will be essential for training the next generation vision models. 2. In real-world robotic tests, using DKT's depth maps nearly doubled the success rate of robot arms trying to pick up objects on tricky reflective or translucent surfaces. At home robots will need to interact with these types of objects on a daily basis. Check out the project page here: Code is LIVE! #Computervision #Robotics #AI

Jonathan Stephens

17,712 Aufrufe • vor 7 Monaten

There are way too many AI 3D models claiming to be the best. So we’re testing them head-to-head. Tripo P1 vs Meshy V6 vs Hunyuan V3.1 Pro vs Rodin 2.5 Same image. Same prompt. No cherry-picking, only 1 generation. This week’s test: a shiny metal helmet. reflections, materials, geometry, and texture accuracy. Next up: characters, objects, environments, and more. By the end, we want one answer: Which AI 3D model should you actually use for your next game, website, or project?
0:12

There are way too many AI 3D models claiming to be the best. So we’re testing them head-to-head. Tripo P1 vs Meshy V6 vs Hunyuan V3.1 Pro vs Rodin 2.5 Same image. Same prompt. No cherry-picking, only 1 generation. This week’s test: a shiny metal helmet. reflections, materials, geometry, and texture accuracy. Next up: characters, objects, environments, and more. By the end, we want one answer: Which AI 3D model should you actually use for your next game, website, or project?

mint

20,027 Aufrufe • vor 14 Tagen

Introducing Kaleido💮 from AI at Meta — a universal generative neural rendering engine for photorealistic, unified object and scene view synthesis. Kaleido is built on a simple but powerful design philosophy: 3D perception is a form of visual common sense. Following this idea, we formulate rendering purely as a sequence-to-sequence generation problem, successfully unifying neural rendering with the architecture principles behind modern language and video models. Unlike traditional neural rendering methods, Kaleido learns 3D purely in a data-driven way, without explicit 3D representations or structures. It acquires spatial understanding directly through large-scale video pretraining, then multi-view 3D data finetuning, inspired by how LLMs acquire textual common sense from large corpora before specialising in domains like coding. Through extensive ablations, we progressively modernised the architecture design and training strategies and tackled key scaling challenges in sequence-to-sequence generative rendering, arriving at a design that’s simple, versatile, and scalable. Kaleido significantly outperforms prior generative models in few-view settings, and remarkably is the first zero-shot generative method matches InstantNGP-level rendering quality in multi-view settings. We view Kaleido also as an alternative step towards world modeling that flexibly spans a spectrum of “realities": with many views, it faithfully reconstructs grounded reality; with fewer views, it imagines plausible unseen details. 🔗 Explore more results and paper:
0:12

Introducing Kaleido💮 from AI at Meta — a universal generative neural rendering engine for photorealistic, unified object and scene view synthesis. Kaleido is built on a simple but powerful design philosophy: 3D perception is a form of visual common sense. Following this idea, we formulate rendering purely as a sequence-to-sequence generation problem, successfully unifying neural rendering with the architecture principles behind modern language and video models. Unlike traditional neural rendering methods, Kaleido learns 3D purely in a data-driven way, without explicit 3D representations or structures. It acquires spatial understanding directly through large-scale video pretraining, then multi-view 3D data finetuning, inspired by how LLMs acquire textual common sense from large corpora before specialising in domains like coding. Through extensive ablations, we progressively modernised the architecture design and training strategies and tackled key scaling challenges in sequence-to-sequence generative rendering, arriving at a design that’s simple, versatile, and scalable. Kaleido significantly outperforms prior generative models in few-view settings, and remarkably is the first zero-shot generative method matches InstantNGP-level rendering quality in multi-view settings. We view Kaleido also as an alternative step towards world modeling that flexibly spans a spectrum of “realities": with many views, it faithfully reconstructs grounded reality; with fewer views, it imagines plausible unseen details. 🔗 Explore more results and paper:

Shikun Liu

22,442 Aufrufe • vor 10 Monaten

We’ve seen humanoid robots walk around for a while, but when will they actually help with useful tasks in daily life? The challenge here is the diversity and complexity of real-world scenes. Our new work tackles this problem via 3D visuomotor policy learning. Using data from only 1 scene, our Improved 3D Diffusion Policy (iDP3) enables a full-sized humanoid robot to autonomously pick&place objects, pour water, and wipe tables, in the wild open world. (and all these skills are useful, right?) Web: Fully open-sourced code:
0:18

We’ve seen humanoid robots walk around for a while, but when will they actually help with useful tasks in daily life? The challenge here is the diversity and complexity of real-world scenes. Our new work tackles this problem via 3D visuomotor policy learning. Using data from only 1 scene, our Improved 3D Diffusion Policy (iDP3) enables a full-sized humanoid robot to autonomously pick&place objects, pour water, and wipe tables, in the wild open world. (and all these skills are useful, right?) Web: Fully open-sourced code:

Yanjie Ze

75,312 Aufrufe • vor 1 Jahr