Video wird geladen...

Video konnte nicht geladen werden

Beim Laden dieses Videos ist ein Problem aufgetreten. Dies könnte an einem vorübergehenden Netzwerkproblem liegen oder das Video ist möglicherweise nicht verfügbar.

World generation is a bottleneck for robotics. We’re exploring how generative 3D worlds can reduce manual simulation setup and enable broader, more realistic evaluation 🧵

World Labs

115,490 subscribers

150,616 Aufrufe • vor 9 Monaten •via X (Twitter)

Bildung Wissenschaft & Technologie

Anya Rossi• Live Now

Private livecam show

21 Kommentare

Profilbild von World Labs

World Labsvor 9 Monaten

One example: integrating generative scenes directly into NVIDIA Isaac Sim. From a single image or prompt → simulation-ready environment with visuals + physics, without manual scene construction. Tutorial + workflow below:

Profilbild von World Labs

World Labsvor 9 Monaten

This isn’t limited to a single simulator or task. Researchers have used Marble-generated worlds across MuJoCo, RoboSuite, and Isaac Sim for manipulation, locomotion, and industrial scenarios. Some examples in the case study here:

Profilbild von World Labs

World Labsvor 9 Monaten

World generation doesn’t have to be a manual step. Using the World API, users are generating full 3D worlds on demand and loading them directly into simulators like MuJoCo, including collision geometry, all in the browser.

Profilbild von World Labs

World Labsvor 9 Monaten

When world generation scales, what we can evaluate and imagine scales with it 🌎

Profilbild von Rahul Chhabra

Rahul Chhabravor 9 Monaten

world labs is so goated

Profilbild von Mario Bollini

Mario Bollinivor 9 Monaten

Hi Spot! Looking good without the crash panels

Profilbild von Erwin

Erwinvor 9 Monaten

Looks awesome. Visually it appealing but how interactive and physically realistic can it be for interactions?Without that sim2real transfer is going to be really hard

Profilbild von Anish

Anishvor 9 Monaten

robocop, please raid the shire

Profilbild von #INTERNETofAGENTS

#INTERNETofAGENTSvor 9 Monaten

Outstanding! Infinite potential >_

Profilbild von Brian Chew

Brian Chewvor 9 Monaten

so cool!

Profilbild von YogenshaSilver

YogenshaSilvervor 9 Monaten

“tHiS is jUSt Ai slOP”

Profilbild von 岡田泰彦｜光邦｜AI×印刷エヴァンジェリスト

岡田泰彦｜光邦｜AI×印刷エヴァンジェリストvor 9 Monaten

Exciting direction. If world generation becomes cheap and diverse, iteration speed and generalization could improve overnight. とてもワクワクする方向性です。環境生成が低コストかつ多様になれば、反復速度も汎化性能も一気に変わりそうですね。

Profilbild von BoldHyun

BoldHyunvor 9 Monaten

Generative 3D worlds have wild potential! Not just for robotics, but imagine the impact on digital runways, bold product prototyping, and virtual styling. Tech-driven creativity keeps raising the bar.

Profilbild von @grok

@grokvor 9 Monaten

bruh.....

Profilbild von @profondob1ue

@profondob1uevor 9 Monaten

In case you want to run this but don’t have a GPU for cloud simulation, you can give shot.

Profilbild von Nick Landolfi

Nick Landolfivor 9 Monaten

🤖🌎

Profilbild von Amaar

Amaarvor 9 Monaten

Amazing!

Profilbild von Vinay Sharma

Vinay Sharmavor 9 Monaten

Excited to see it in home

Profilbild von just10101

just10101vor 9 Monaten

Shameless @danfei_xu Stop stealing from students/ interviewees Shame on @ICatGT

Profilbild von Tashi Network

Tashi Networkvor 9 Monaten

These robots gotta learn somehow 🤖

Profilbild von Tool Side Room

Tool Side Roomvor 9 Monaten

Teaching robots to think outside the box We’re still stuck trying to make the box look good.

Ähnliche Videos

Researchers are using Marble to generate simulation-ready robotics environments (scenes + collider meshes) then bring them into NVIDIA Robotics Isaac Sim for training + evaluation without any manual environment setup. Case Study:

Researchers are using Marble to generate simulation-ready robotics environments (scenes + collider meshes) then bring them into NVIDIA Robotics Isaac Sim for training + evaluation without any manual environment setup. Case Study:

World Labs

29,857 Aufrufe • vor 10 Monaten

We’re open-sourcing HY-World 2.0, a multimodal world model that generates, reconstructs, and simulates interactive *3D worlds* from text, images, and videos. Outputs can be integrated into game engines and embodied simulation pipelines. Key highlights: 🔹 One-click world generation Turn text or image into interactive 3D worlds automatically. 🔹 Pipeline-ready 3D outputs Editable 3D worlds for Unity and Unreal Engine, with standard 3D exports including mesh, 3DGS, and point clouds. 🔹 Unified world model system One model family for world generation and reconstruction across synthetic and real-world scenes. 🔹 Interactive character mode Explore generated 3D worlds in real time with physics-aware movement and collision support. ✨ Apply for access: 🔗 GitHub: 🤗 Hugging Face: 📄 Technical Report:

We’re open-sourcing HY-World 2.0, a multimodal world model that generates, reconstructs, and simulates interactive 3D worlds from text, images, and videos. Outputs can be integrated into game engines and embodied simulation pipelines. Key highlights: 🔹 One-click world generation Turn text or image into interactive 3D worlds automatically. 🔹 Pipeline-ready 3D outputs Editable 3D worlds for Unity and Unreal Engine, with standard 3D exports including mesh, 3DGS, and point clouds. 🔹 Unified world model system One model family for world generation and reconstruction across synthetic and real-world scenes. 🔹 Interactive character mode Explore generated 3D worlds in real time with physics-aware movement and collision support. ✨ Apply for access: 🔗 GitHub: 🤗 Hugging Face: 📄 Technical Report:

Tencent Hy

373,065 Aufrufe • vor 6 Monaten

From a single image to a full robotics simulation. 360 image → Marble → 3D environment → Robotics sim This is how we scale world creation for embodied AI with Lightwheel, turning lightweight capture into simulation-ready environments in minutes, not weeks.

From a single image to a full robotics simulation. 360 image → Marble → 3D environment → Robotics sim This is how we scale world creation for embodied AI with Lightwheel, turning lightweight capture into simulation-ready environments in minutes, not weeks.

World Labs

46,356 Aufrufe • vor 9 Monaten

Meet SceneSmith: An agentic system that generates entire simulation-ready environments from a single text prompt. VLM agents collaborate to build scenes with dozens of objects per room, articulated furniture, and full physics properties. We believe environment generation is no longer the bottleneck for scalable robot training and evaluation in simulation. Website: 👇🧵(1/8)

Meet SceneSmith: An agentic system that generates entire simulation-ready environments from a single text prompt. VLM agents collaborate to build scenes with dozens of objects per room, articulated furniture, and full physics properties. We believe environment generation is no longer the bottleneck for scalable robot training and evaluation in simulation. Website: 👇🧵(1/8)

Nicholas Pfaff

92,004 Aufrufe • vor 8 Monaten

Generative 3D worlds are amazing! Here is a generative AI video showing fine grained camera control. Camera movements are all hand places within a generated 3D world.

Generative 3D worlds are amazing! Here is a generative AI video showing fine grained camera control. Camera movements are all hand places within a generated 3D world.

Brittani Natali

21,061 Aufrufe • vor 1 Jahr

The biggest bottleneck in AI for biology is still the wet lab. AI can generate more ideas than our labs can test. Closing that gap is how we turn better models into faster discovery. Today, we’re introducing X-Automate at Xaira: bringing agentic AI and physical robotics into the lab. 🧵

The biggest bottleneck in AI for biology is still the wet lab. AI can generate more ideas than our labs can test. Closing that gap is how we turn better models into faster discovery. Today, we’re introducing X-Automate at Xaira: bringing agentic AI and physical robotics into the lab. 🧵

Bo Wang

32,915 Aufrufe • vor 8 Tagen

#ICCV2025 🤩3D world generation is cool, but it is cooler to play with the worlds using 3D actions 👆💨, and see what happens! — Introducing *WonderPlay*: Now you can create dynamic 3D scenes that respond to your 3D actions from a single image! Web: 🧵1/7

#ICCV2025 🤩3D world generation is cool, but it is cooler to play with the worlds using 3D actions 👆💨, and see what happens! — Introducing WonderPlay: Now you can create dynamic 3D scenes that respond to your 3D actions from a single image! Web: 🧵1/7

Hong-Xing (Koven) Yu

58,065 Aufrufe • vor 1 Jahr

We’re releasing the RoboArena today!🤖🦾 Fair & scalable evaluation is a major bottleneck for research on generalist policies. We’re hoping that RoboArena can help! We provide data, model code & sim evals for debugging! Submit your policies today and join the leaderboard! :) 🧵

We’re releasing the RoboArena today!🤖🦾 Fair & scalable evaluation is a major bottleneck for research on generalist policies. We’re hoping that RoboArena can help! We provide data, model code & sim evals for debugging! Submit your policies today and join the leaderboard! :) 🧵

Karl Pertsch

111,801 Aufrufe • vor 1 Jahr

🧵What if you could build, simulate, and collaborate on any 3D world—digitally and physically—to power the future of industry and robotics? Aaron Luk, Director of Product Management for Simulation Tech at NVIDIA, explains how #OpenUSD is making this possible. 🌟

🧵What if you could build, simulate, and collaborate on any 3D world—digitally and physically—to power the future of industry and robotics? Aaron Luk, Director of Product Management for Simulation Tech at NVIDIA, explains how #OpenUSD is making this possible. 🌟

NVIDIA AI

16,570 Aufrufe • vor 1 Jahr

📢 A Recipe for Generating 3D Worlds From a Single Image 📢 Our recipe explains how existing generative models can be adapted with minimal training effort to generate 3D worlds from a single input image.

📢 A Recipe for Generating 3D Worlds From a Single Image 📢 Our recipe explains how existing generative models can be adapted with minimal training effort to generate 3D worlds from a single input image.

Katja Schwarz

14,176 Aufrufe • vor 1 Jahr

Everything you love about generative models — now powered by real physics! Announcing the Genesis project — after a 24-month large-scale research collaboration involving over 20 research labs — a generative physics engine able to generate 4D dynamical worlds powered by a physics simulation platform designed for general-purpose robotics and physical AI applications. Genesis's physics engine is developed in pure Python, while being 10-80x faster than existing GPU-accelerated stacks like Isaac Gym and MJX. It delivers a simulation speed ~430,000 faster than in real-time, and takes only 26 seconds to train a robotic locomotion policy transferrable to the real world on a single RTX4090 (see tutorial: The Genesis physics engine and simulation platform is fully open source at We'll gradually roll out access to our generative framework in the near future. Genesis implements a unified simulation framework all from scratch, integrating a wide spectrum of state-of-the-art physics solvers, allowing simulation of the whole physical world in a virtual realm with the highest realism. We aim to build a universal data engine that leverages an upper-level generative framework to autonomously create physical worlds, together with various modes of data, including environments, camera motions, robotic task proposals, reward functions, robot policies, character motions, fully interactive 3D scenes, open-world articulated assets, and more, aiming towards fully automated data generation for robotics, physical AI and other applications. Open Source Code: Project webpage: Documentation: 1/n

Everything you love about generative models — now powered by real physics! Announcing the Genesis project — after a 24-month large-scale research collaboration involving over 20 research labs — a generative physics engine able to generate 4D dynamical worlds powered by a physics simulation platform designed for general-purpose robotics and physical AI applications. Genesis's physics engine is developed in pure Python, while being 10-80x faster than existing GPU-accelerated stacks like Isaac Gym and MJX. It delivers a simulation speed ~430,000 faster than in real-time, and takes only 26 seconds to train a robotic locomotion policy transferrable to the real world on a single RTX4090 (see tutorial: The Genesis physics engine and simulation platform is fully open source at We'll gradually roll out access to our generative framework in the near future. Genesis implements a unified simulation framework all from scratch, integrating a wide spectrum of state-of-the-art physics solvers, allowing simulation of the whole physical world in a virtual realm with the highest realism. We aim to build a universal data engine that leverages an upper-level generative framework to autonomously create physical worlds, together with various modes of data, including environments, camera motions, robotic task proposals, reward functions, robot policies, character motions, fully interactive 3D scenes, open-world articulated assets, and more, aiming towards fully automated data generation for robotics, physical AI and other applications. Open Source Code: Project webpage: Documentation: 1/n

Zhou Xian

3,824,721 Aufrufe • vor 1 Jahr

New open-source 3D world-generation model. I'm rendering a couple of worlds in the video, so check it out. You'll find the GitHub and the Hugging Face links to the model below. This is a multi-modal world model that you can use for a bunch of things: • To generate new worlds • To reconstruct worlds • To simulate 3D interactive worlds from a prompt, images, or a video You can edit the 3D outputs in Unity and Unreal Engine (they export as meshes, 3DGS files, and point clouds). You can also generate 3D characters in the world and walk around. Pretty fun stuff!

New open-source 3D world-generation model. I'm rendering a couple of worlds in the video, so check it out. You'll find the GitHub and the Hugging Face links to the model below. This is a multi-modal world model that you can use for a bunch of things: • To generate new worlds • To reconstruct worlds • To simulate 3D interactive worlds from a prompt, images, or a video You can edit the 3D outputs in Unity and Unreal Engine (they export as meshes, 3DGS files, and point clouds). You can also generate 3D characters in the world and walk around. Pretty fun stuff!

Santiago

65,446 Aufrufe • vor 6 Monaten

We've been exploring how clinical trial participants can control a powered wheelchair — with their minds. Here's how we’re doing it 🧵

We've been exploring how clinical trial participants can control a powered wheelchair — with their minds. Here's how we’re doing it 🧵

Neuralink

7,164,686 Aufrufe • vor 3 Monaten

#AGIBOTAIWeek Day4: Introducing GE-Sim 2.0 (Genie Envisioner World Simulator 2.0) , a comprehensive embodied world simulator for robotic manipulation. By integrating future video generation, proprioceptive state estimation, and reward-based policy assessment, GE-Sim 2.0 moves beyond visual simulation toward an embodied world simulator with built-in evaluation capability, laying the foundation for a world-model-centric platform for closed-loop and interactive policy learning. It provides a practical framework for scalable evaluation and training of robotic manipulation policies. Project page: #AGIBOT #AGIBOTAIWeek #GenieEnvisioner #Robotics #visualsimulation

#AGIBOTAIWeek Day4: Introducing GE-Sim 2.0 (Genie Envisioner World Simulator 2.0) , a comprehensive embodied world simulator for robotic manipulation. By integrating future video generation, proprioceptive state estimation, and reward-based policy assessment, GE-Sim 2.0 moves beyond visual simulation toward an embodied world simulator with built-in evaluation capability, laying the foundation for a world-model-centric platform for closed-loop and interactive policy learning. It provides a practical framework for scalable evaluation and training of robotic manipulation policies. Project page: #AGIBOT #AGIBOTAIWeek #GenieEnvisioner #Robotics #visualsimulation

AGIBOT

28,219 Aufrufe • vor 6 Monaten

One month after launching SR Platform. We started with simple simulation assets and layouts. Today, thanks to continuous improvements, SR Platform generates environments with richer geometry, higher-fidelity surface materials, and significantly more realistic assets, ready for modern robotics simulation. Most importantly, SR Platform is now fully compatible with NVIDIA Robotics Isaac Sim. Built with: • Agentic workflow & tool calling • Dynamic context retrieval • Self-evaluation & iterative refinement • Latest GPT-5.6 model variants

One month after launching SR Platform. We started with simple simulation assets and layouts. Today, thanks to continuous improvements, SR Platform generates environments with richer geometry, higher-fidelity surface materials, and significantly more realistic assets, ready for modern robotics simulation. Most importantly, SR Platform is now fully compatible with NVIDIA Robotics Isaac Sim. Built with: • Agentic workflow & tool calling • Dynamic context retrieval • Self-evaluation & iterative refinement • Latest GPT-5.6 model variants

Strike Robot

15,277 Aufrufe • vor 3 Monaten

Before AI can understand the physical world, it needs data about the physical world. Not just images. Spatial environments. Real streets, buildings, parks, stores, museums, and public spaces captured in 3D. OVER has already reconstructed 290,000+ real-world 3D scenes, creating a growing dataset for the next generation of spatial and embodied AI. And this is exactly the kind of real-world spatial data AI companies are looking for. We're already in conversations with multiple AI and Robotics companies exploring how OVER's dataset can support the development and training of models that need to understand the physical world. The next AI race will be about real-world data.

Before AI can understand the physical world, it needs data about the physical world. Not just images. Spatial environments. Real streets, buildings, parks, stores, museums, and public spaces captured in 3D. OVER has already reconstructed 290,000+ real-world 3D scenes, creating a growing dataset for the next generation of spatial and embodied AI. And this is exactly the kind of real-world spatial data AI companies are looking for. We're already in conversations with multiple AI and Robotics companies exploring how OVER's dataset can support the development and training of models that need to understand the physical world. The next AI race will be about real-world data.

Over the Reality 🌐

24,343 Aufrufe • vor 1 Monat

Generate persistent 3D worlds from a single image, bigger and better than ever! We’re excited to share our latest results and invite you to try out our world generation model in a limited beta preview.

Generate persistent 3D worlds from a single image, bigger and better than ever! We’re excited to share our latest results and invite you to try out our world generation model in a limited beta preview.

World Labs

836,159 Aufrufe • vor 1 Jahr

🧵 Evaluating robot policies in the real world is slow, expensive, and hard to scale. During my internship at SceniX this summer, we had many discussions around the two key questions: how accurate must a simulator be for evaluation to be meaningful, and how do we get there? Our new framework, Real2Sim-Eval, takes a step toward that answer. By combining Gaussian Splatting for photorealistic rendering and soft-body digital twins for realistic dynamics, we make simulation predictive of real-world performance. 👉

🧵 Evaluating robot policies in the real world is slow, expensive, and hard to scale. During my internship at SceniX this summer, we had many discussions around the two key questions: how accurate must a simulator be for evaluation to be meaningful, and how do we get there? Our new framework, Real2Sim-Eval, takes a step toward that answer. By combining Gaussian Splatting for photorealistic rendering and soft-body digital twins for realistic dynamics, we make simulation predictive of real-world performance. 👉

Kaifeng Zhang

67,703 Aufrufe • vor 11 Monaten

Robot trained entirely in simulation! 🤯 Sudo AI just released #sudo R1. This could be the moment the data bottleneck in robotics finally breaks, a major step forward for getting simulation-trained robots to actually work in the real world. #sudo R1 is a manipulation foundation model that reliably picks objects it has never seen: glass, fabric, metal, irregular shapes. All with zero real-world training data. No tele-op. No demos. No manual labeling. Pure simulation. And picking isn't just any skill, it's the foundation. Almost everything we want robots to do starts with being able to reliably pick something up. Data collection is the bottleneck holding the whole field back. Every new task means hours of tele-op and armies of labelers. If simulation alone can actually deliver production-grade picking, that bottleneck starts to disappear. Really exciting to see! If the results hold, the path to general-purpose robots just got a whole lot shorter. 👇 Read more here: ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →

Robot trained entirely in simulation! 🤯 Sudo AI just released #sudo R1. This could be the moment the data bottleneck in robotics finally breaks, a major step forward for getting simulation-trained robots to actually work in the real world. #sudo R1 is a manipulation foundation model that reliably picks objects it has never seen: glass, fabric, metal, irregular shapes. All with zero real-world training data. No tele-op. No demos. No manual labeling. Pure simulation. And picking isn't just any skill, it's the foundation. Almost everything we want robots to do starts with being able to reliably pick something up. Data collection is the bottleneck holding the whole field back. Every new task means hours of tele-op and armies of labelers. If simulation alone can actually deliver production-grade picking, that bottleneck starts to disappear. Really exciting to see! If the results hold, the path to general-purpose robots just got a whole lot shorter. 👇 Read more here: ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →

Lukas Ziegler

25,901 Aufrufe • vor 6 Monaten