Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

HY-World 2.0 A Multi-Modal World Model for Reconstructing, Generating, and Simulating 3D Worlds paper:

AK

508,735 subscribers

13,879 views • 2 months ago •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

New open-source 3D world-generation model. I'm rendering a couple of worlds in the video, so check it out. You'll find the GitHub and the Hugging Face links to the model below. This is a multi-modal world model that you can use for a bunch of things: • To generate new worlds • To reconstruct worlds • To simulate 3D interactive worlds from a prompt, images, or a video You can edit the 3D outputs in Unity and Unreal Engine (they export as meshes, 3DGS files, and point clouds). You can also generate 3D characters in the world and walk around. Pretty fun stuff!

New open-source 3D world-generation model. I'm rendering a couple of worlds in the video, so check it out. You'll find the GitHub and the Hugging Face links to the model below. This is a multi-modal world model that you can use for a bunch of things: • To generate new worlds • To reconstruct worlds • To simulate 3D interactive worlds from a prompt, images, or a video You can edit the 3D outputs in Unity and Unreal Engine (they export as meshes, 3DGS files, and point clouds). You can also generate 3D characters in the world and walk around. Pretty fun stuff!

Santiago

65,446 views • 2 months ago

Google's Genie 3 was just announced. This is a world model capable of generating fully interactive 3D worlds. This has major implications for video games and giving agents infinite self-play environments.

Google's Genie 3 was just announced. This is a world model capable of generating fully interactive 3D worlds. This has major implications for video games and giving agents infinite self-play environments.

Matthew Berman

76,291 views • 11 months ago

Excited to share our #ICML2023 paper ✨LIV✨! Extending VIP, LIV is at once a pre-training, fine-tuning, and (zero-shot!) multi-modal reward method for (real-world!) language-conditioned robotic control. Project: Code & Model: 🧵:

Excited to share our #ICML2023 paper ✨LIV✨! Extending VIP, LIV is at once a pre-training, fine-tuning, and (zero-shot!) multi-modal reward method for (real-world!) language-conditioned robotic control. Project: Code & Model: 🧵:

Jason Ma

55,045 views • 3 years ago

We are thrilled to announce a major upgrade to our open-source 3D generation model, introducing two groundbreaking new versions: 3D 2.0 MV (Multi-View Generation) and 3D 2.0 Mini! 3D 2.0 MV : 3D 2.0Mini:

We are thrilled to announce a major upgrade to our open-source 3D generation model, introducing two groundbreaking new versions: 3D 2.0 MV (Multi-View Generation) and 3D 2.0 Mini! 3D 2.0 MV : 3D 2.0Mini:

Tencent HY

134,410 views • 1 year ago

Today, we are open-sourcing Hunyuan World 1.1 (WorldMirror), a universal feed-forward 3D reconstruction model. 🚀🚀🚀 While our previously released Hunyuan World 1.0 (open-sourced, lite version deployable on consumer GPUs) focused on generating 3D worlds from text or single-view images, Hunyuan World 1.1 significantly expands the input scope by unlocking video-to-3D and multi-view-to-3D world creation. Highlights: 🔹Any Input, Maximized Flexibility and Fidelity: Flexibly integrates diverse geometric priors (camera poses, intrinsics, depth maps) to resolve structural ambiguities and ensure geometrically consistent 3D outputs. 🔹Any Output, SOTA Results：This elegant architecture simultaneously generates multiple 3D representations: dense point clouds, multi-view depth maps, camera parameters, surface normals, and 3D Gaussian Splattings. 🔹Single-GPU & Fast Inference: As an all-in-one, feed-forward model, Hunyuan World 1.1 runs on a single GPU and delivers all 3D attributes in a single forward pass, within seconds. 🌐Project Page: 🔗Github： 🤗Hugging Face： ✨Demo: 📄Technical Report:

Today, we are open-sourcing Hunyuan World 1.1 (WorldMirror), a universal feed-forward 3D reconstruction model. 🚀🚀🚀 While our previously released Hunyuan World 1.0 (open-sourced, lite version deployable on consumer GPUs) focused on generating 3D worlds from text or single-view images, Hunyuan World 1.1 significantly expands the input scope by unlocking video-to-3D and multi-view-to-3D world creation. Highlights: 🔹Any Input, Maximized Flexibility and Fidelity: Flexibly integrates diverse geometric priors (camera poses, intrinsics, depth maps) to resolve structural ambiguities and ensure geometrically consistent 3D outputs. 🔹Any Output, SOTA Results：This elegant architecture simultaneously generates multiple 3D representations: dense point clouds, multi-view depth maps, camera parameters, surface normals, and 3D Gaussian Splattings. 🔹Single-GPU & Fast Inference: As an all-in-one, feed-forward model, Hunyuan World 1.1 runs on a single GPU and delivers all 3D attributes in a single forward pass, within seconds. 🌐Project Page: 🔗Github： 🤗Hugging Face： ✨Demo: 📄Technical Report:

Tencent Hy

168,451 views • 8 months ago

Map2World. turns text descriptions and sketch maps into full 3D worlds. -multi-diffusion in SLAT latent space for global consistency; - beaats SynCity - supports arbitrary world sizes.

Map2World. turns text descriptions and sketch maps into full 3D worlds. -multi-diffusion in SLAT latent space for global consistency; - beaats SynCity - supports arbitrary world sizes.

Wildminder

15,679 views • 2 months ago

Gaussian Garments: Reconstructing Simulation-Ready Clothing with Photorealistic Appearance from Multi-View Video Contribution quote from the paper: In summary, our main contributions are • a comprehensive pipeline for reconstructing the shape, appearance, and behavior of real-world garments using Gaussian splatting, • an algorithm for registering garment meshes to multi- view videos with an optimization procedure based on Gaussian splatting, and • a Gaussian Garment representation that combines triangle meshes with Gaussian textures to capture photorealistic appearance and can be used as a fully controllable 3D asset.

Gaussian Garments: Reconstructing Simulation-Ready Clothing with Photorealistic Appearance from Multi-View Video Contribution quote from the paper: In summary, our main contributions are • a comprehensive pipeline for reconstructing the shape, appearance, and behavior of real-world garments using Gaussian splatting, • an algorithm for registering garment meshes to multi- view videos with an optimization procedure based on Gaussian splatting, and • a Gaussian Garment representation that combines triangle meshes with Gaussian textures to capture photorealistic appearance and can be used as a fully controllable 3D asset.

MrNeRF

27,277 views • 1 year ago

ByteDance has released Seed3D 2.0, a model capable of generating 3D objects from a single image or a text prompt.

ByteDance has released Seed3D 2.0, a model capable of generating 3D objects from a single image or a text prompt.

Kashberg

164,747 views • 2 months ago

SAM 3D enables accurate 3D reconstruction from a single image, supporting real-world applications in editing, robotics, and interactive scene generation. Matt, a SAM 3D researcher, explains how the two-model design makes this possible for both people and complex environments. 🔗 Read the SAM 3D Objects research paper: 🔗 Read the SAM 3D Body research paper:

SAM 3D enables accurate 3D reconstruction from a single image, supporting real-world applications in editing, robotics, and interactive scene generation. Matt, a SAM 3D researcher, explains how the two-model design makes this possible for both people and complex environments. 🔗 Read the SAM 3D Objects research paper: 🔗 Read the SAM 3D Body research paper:

AI at Meta

17,858 views • 7 months ago

Introducing Cube 2.0, a significant step towards controllable 3D game-world and video generation from any input. Powered by: 🌟 Image-to-3D in seconds w/ a new foundation model 🐎 Text-to-animation 🌎 AI rendering to create beautiful worlds (mesh+splats) Web app → Discord → Blog →

Introducing Cube 2.0, a significant step towards controllable 3D game-world and video generation from any input. Powered by: 🌟 Image-to-3D in seconds w/ a new foundation model 🐎 Text-to-animation 🌎 AI rendering to create beautiful worlds (mesh+splats) Web app → Discord → Blog →

Common Sense Machines

411,381 views • 2 years ago

Marble is the first product from World Labs and is powered by our multimodal world model, which lets anyone create high-fidelity, persistent 3D worlds from just a single image, video, text prompt, or 3D layout. Read more at

Marble is the first product from World Labs and is powered by our multimodal world model, which lets anyone create high-fidelity, persistent 3D worlds from just a single image, video, text prompt, or 3D layout. Read more at

World Labs

117,146 views • 8 months ago

Introducing Agora-1, a multi-agent world model. Multiple participants—human or AI—can now interact inside the same world simulation, all in real-time. Try our playable research preview today, with Agora-1 simulating a multiplayer GoldenEye deathmatch!

Introducing Agora-1, a multi-agent world model. Multiple participants—human or AI—can now interact inside the same world simulation, all in real-time. Try our playable research preview today, with Agora-1 simulating a multiplayer GoldenEye deathmatch!

Odyssey

911,043 views • 1 month ago

Building TikTok for 3D worlds We just reached our 1000th world generated!

Building TikTok for 3D worlds We just reached our 1000th world generated!

tamrat

26,406 views • 5 months ago

Robbyant released LingBot-World 2.0 (Infinity), a new open-source World model capable of generating hour-long experiences. > No quality decay, stable 720p / 60 fps real-time output. > Users can interact with the world via Actions. > The world is dynamic, with built-in agents generating new events in real time. > Multiplayer is supported within the same world. Testing time 👀

Robbyant released LingBot-World 2.0 (Infinity), a new open-source World model capable of generating hour-long experiences. > No quality decay, stable 720p / 60 fps real-time output. > Users can interact with the world via Actions. > The world is dynamic, with built-in agents generating new events in real time. > Multiplayer is supported within the same world. Testing time 👀

🚨 AI News | TestingCatalog

15,272 views • 5 days ago

Did you try Tencent HY World model on fal?

Did you try Tencent HY World model on fal?

Aykut

14,373 views • 11 months ago

Multi-modal #LLMs understand a lot about humans. But do they understand our 3D pose? We train #PoseGPT to estimate, generate, and reason about 3D human pose (#SMPL) in images and text. This is the first true foundation model for understanding 3D humans.

Multi-modal #LLMs understand a lot about humans. But do they understand our 3D pose? We train #PoseGPT to estimate, generate, and reason about 3D human pose (#SMPL) in images and text. This is the first true foundation model for understanding 3D humans.

Michael Black

81,406 views • 2 years ago

Introducing Spell, a model to generate 3D worlds. A new era for 3D. ✨ (Thread - Link Below)

Introducing Spell, a model to generate 3D worlds. A new era for 3D. ✨ (Thread - Link Below)

Spline

381,852 views • 1 year ago

Glad to have the opportunity to collaborate with the World Labs team on their new tool Marble! It’s currently the best tool for generating 3D worlds with Gaussian Splatting. Here are some of the results I’ve been getting and putting together in a video.

Glad to have the opportunity to collaborate with the World Labs team on their new tool Marble! It’s currently the best tool for generating 3D worlds with Gaussian Splatting. Here are some of the results I’ve been getting and putting together in a video.

Cristian Peñas ░░░░░░░░

42,022 views • 10 months ago

NEWS: Waymo has introduced the Waymo World Model, "a frontier generative model built on Google DeepMind’s Genie 3 that sets a new bar for large-scale, hyper-realistic autonomous driving simulation." "By simulating the “impossible”, we proactively prepare the Waymo Driver for some of the most rare and complex scenarios—from tornadoes to planes landing on freeways—long before it encounters them in the real world. The model’s architecture offers high controllability, allowing our engineers to modify simulations with simple language prompts, driving inputs, and scene layouts. Notably, the Waymo World Model generates high-fidelity, multi-sensor outputs that include both camera and lidar data. This combination of broad world knowledge, fine-grained controllability, and multi-modal realism enhances Waymo’s ability to safely scale our service across more places and new driving environments. In the following sections we showcase the Waymo World Model in action, featuring simulations of the Waymo Driver navigating diverse rare edge-case scenarios." More info below.

NEWS: Waymo has introduced the Waymo World Model, "a frontier generative model built on Google DeepMind’s Genie 3 that sets a new bar for large-scale, hyper-realistic autonomous driving simulation." "By simulating the “impossible”, we proactively prepare the Waymo Driver for some of the most rare and complex scenarios—from tornadoes to planes landing on freeways—long before it encounters them in the real world. The model’s architecture offers high controllability, allowing our engineers to modify simulations with simple language prompts, driving inputs, and scene layouts. Notably, the Waymo World Model generates high-fidelity, multi-sensor outputs that include both camera and lidar data. This combination of broad world knowledge, fine-grained controllability, and multi-modal realism enhances Waymo’s ability to safely scale our service across more places and new driving environments. In the following sections we showcase the Waymo World Model in action, featuring simulations of the Waymo Driver navigating diverse rare edge-case scenarios." More info below.

Sawyer Merritt

133,168 views • 5 months ago

📢 A Recipe for Generating 3D Worlds From a Single Image 📢 Our recipe explains how existing generative models can be adapted with minimal training effort to generate 3D worlds from a single input image.

📢 A Recipe for Generating 3D Worlds From a Single Image 📢 Our recipe explains how existing generative models can be adapted with minimal training effort to generate 3D worlds from a single input image.

Katja Schwarz

13,970 views • 1 year ago