Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

Wow. That's cool. Researchers just released World, an open-source Unreal Engine 5 simulator for training and testing LLM and VLM agents in realistic 3D environments. The platform supports RGB, depth, and segmentation sensors, along with navigation, vehicles, pedestrians, robots and procedural city generation. It's built with a Gym-like Python... show more

SciTech Era

22,520 subscribers

47,751 views • 4 months ago •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

16 Comments

SciTech Era4 months ago

GitHub repo: source of video:

SimWorld4 months ago

@Lianhuiq Thanks for sharing our work!🔥 Also check out the latest development on SimWorld Studio

SciTech Era4 months ago

SimWorld is a simulation platform for developing and evaluating LLM/VLM AI agents in complex physical and social environments!

AI Explained4 months ago

Feels like video games are slowly becoming training grounds for real world AI now

NFAdancer7774 months ago

@grok

unemployable4 months ago

Can you run over prostitutes with a vehicle here?

Chuck Petras4 months ago

@BrianRoemmele

Pedro 'XeonCat' Miranda ✵4 months ago

🫠🫠🫠 i expected this to only happen in, like, 3 years

Kekko D’Amato4 months ago

The Gym-like Python interface is the key detail here. Grounding LLM agent training in physics-based 3D environments closes a gap that's hard to fix with text-only benchmarks — spatial reasoning, occlusion, timing. The open-source + procedural city gen combo makes this actually usable for research.

Robotics Alpha4 months ago

open source means anyone can add custom robot joint models for better manipulation training.

NightGazer 🌌4 months ago

Wait a minute. Is that us?

Trevor Humongous4 months ago

Procedural city generation is useful. 'Social reasoning' though? Still just pattern matching with better props. LLMs don't reason. They just have nicer environments to not reason in now.

Quadruple Games(クアドラプルゲームス)4 months ago

最適化はされてますか？また、モジュラーはAIにより生成されたのでしょうか？

swev994 months ago

such a bad idea to give ai this info

BossFightGames4 months ago

Terminator is near

Hans Blitz4 months ago

I had full intention of hooking up an API to Unreal but I was waiting for the next version to drop, I wouldn't get it to do stuff that has already been done or your wasting cash, I'd like to test how A.I handles socketing items to a Daz3d body I have added to a mannequin using a bridge, I want to test how well the A.I can handle the shaping morphs for the body sets with gear and how it manages to animate a full control rig to interact with vehicles and the environment by using animations I already have in library so it's not creating them all from scratch, just altering animation timelines to suit needed criteria. I need it to build an inventory system to those items socketed to control rigs and just set up an easy method to add new characters in for games. Either this or I create my own web-based editor and hook my entire library of content into it.

Related Videos

🚨🚨Can agents earn money, run a business, or even self-organize a society in the physical social world? 🤖🤖 Can agents learn continually to survive and thrive in embodied environments, like how human babies grow? 👶 Super excited to introduce SimWorld, an open-ended simulator of LLM agents in infinite, realistic embodied worlds. SimWorld features 3 key designs: 1⃣Open-ended realistic world simulation - built on Unreal Engine 5, with accurate physical social dynamics - 100+ built-in environments (city, island, wilderness ...) - language-controllable procedural generation - text-to-3D asset generation 2⃣Native interface for LLM/VLM agents - Gym-like agent-environment interaction APIs - plug in any LLMs/VLMs (GPTs, Gemini, Qwen ...) - rich multi-modal perception - open-vocabulary natural-language action outputs 3⃣Diverse physical and social reasoning scenarios - long-horizon embodied reasoning - multi-agent collaboration / competition - easily customizable for any reasoning tasks SimWorld is fully open-sourced, with a hope to become a foundational infrastructure for real-world agent research across disciplines: robotics, economy, public health, education, etc. Project website + more details in the thread👇 ...1/

🚨🚨Can agents earn money, run a business, or even self-organize a society in the physical social world? 🤖🤖 Can agents learn continually to survive and thrive in embodied environments, like how human babies grow? 👶 Super excited to introduce SimWorld, an open-ended simulator of LLM agents in infinite, realistic embodied worlds. SimWorld features 3 key designs: 1⃣Open-ended realistic world simulation - built on Unreal Engine 5, with accurate physical social dynamics - 100+ built-in environments (city, island, wilderness ...) - language-controllable procedural generation - text-to-3D asset generation 2⃣Native interface for LLM/VLM agents - Gym-like agent-environment interaction APIs - plug in any LLMs/VLMs (GPTs, Gemini, Qwen ...) - rich multi-modal perception - open-vocabulary natural-language action outputs 3⃣Diverse physical and social reasoning scenarios - long-horizon embodied reasoning - multi-agent collaboration / competition - easily customizable for any reasoning tasks SimWorld is fully open-sourced, with a hope to become a foundational infrastructure for real-world agent research across disciplines: robotics, economy, public health, education, etc. Project website + more details in the thread👇 ...1/

Lianhui Qin@COLM2026

65,310 views • 10 months ago

#NVIDIACosmos Reason, an open, customizable, 7-billion-parameter reasoning VLM for #PhysicalAI, enables robots, autonomous vehicles and visual AI agents to: 👀 See, reason, and act in the physical world. 🛠️ Solve multistep tasks and handle ambiguous or new experiences. Get started ➡️ #SIGGRAPH2025

#NVIDIACosmos Reason, an open, customizable, 7-billion-parameter reasoning VLM for #PhysicalAI, enables robots, autonomous vehicles and visual AI agents to: 👀 See, reason, and act in the physical world. 🛠️ Solve multistep tasks and handle ambiguous or new experiences. Get started ➡️ #SIGGRAPH2025

NVIDIA AI Developer

32,772 views • 1 year ago

for physical AI to work in the real world, it needs high-fidelity 3D models of the real world. not just for visualization — for localization, spatial reasoning, and real-to-sim workflows that close the gap between training and deployment. at Niantic Spatial 🌎, we're turning Spexi drone imagery into exactly that — city-scale Gaussian splats grounded in the environments that matter most.

for physical AI to work in the real world, it needs high-fidelity 3D models of the real world. not just for visualization — for localization, spatial reasoning, and real-to-sim workflows that close the gap between training and deployment. at Niantic Spatial 🌎, we're turning Spexi drone imagery into exactly that — city-scale Gaussian splats grounded in the environments that matter most.

asim ᯅ

49,566 views • 4 months ago

Before AI can understand the physical world, it needs data about the physical world. Not just images. Spatial environments. Real streets, buildings, parks, stores, museums, and public spaces captured in 3D. OVER has already reconstructed 290,000+ real-world 3D scenes, creating a growing dataset for the next generation of spatial and embodied AI. And this is exactly the kind of real-world spatial data AI companies are looking for. We're already in conversations with multiple AI and Robotics companies exploring how OVER's dataset can support the development and training of models that need to understand the physical world. The next AI race will be about real-world data.

Before AI can understand the physical world, it needs data about the physical world. Not just images. Spatial environments. Real streets, buildings, parks, stores, museums, and public spaces captured in 3D. OVER has already reconstructed 290,000+ real-world 3D scenes, creating a growing dataset for the next generation of spatial and embodied AI. And this is exactly the kind of real-world spatial data AI companies are looking for. We're already in conversations with multiple AI and Robotics companies exploring how OVER's dataset can support the development and training of models that need to understand the physical world. The next AI race will be about real-world data.

Over the Reality 🌐

24,019 views • 12 days ago

Learning to play even 1️⃣ video game is an achievement for AI - but understanding instructions and applying them to unseen games is even harder. 🎮 Join host Hannah Fry and researcher Frederic Besse as they explore the complex world of AI agents – programs that can learn, adapt, and act in virtual environments. ↓

Learning to play even 1️⃣ video game is an achievement for AI - but understanding instructions and applying them to unseen games is even harder. 🎮 Join host Hannah Fry and researcher Frederic Besse as they explore the complex world of AI agents – programs that can learn, adapt, and act in virtual environments. ↓

Google DeepMind

31,734 views • 2 years ago

The coming merger of AI agents with the physical world. @iddris, founder/CEO of Open Matter and I have a wide-ranging conversation about how the next platform shift may not be just AI chat, robotics, smart glasses, or autonomous cars separately. It may be the connective tissue between all of them. This video teaches viewers that the next wave of AI agents will not just operate inside apps, browsers, calendars, and spreadsheets. The real frontier is agents that can understand, coordinate, and act in the physical world. Iddris explains a vision for “agentic infrastructure for atoms”: systems that let AI agents manage physical objects, deliveries, storage, location, identity, logistics, robots, vehicles, and real-world context.

The coming merger of AI agents with the physical world. @iddris, founder/CEO of Open Matter and I have a wide-ranging conversation about how the next platform shift may not be just AI chat, robotics, smart glasses, or autonomous cars separately. It may be the connective tissue between all of them. This video teaches viewers that the next wave of AI agents will not just operate inside apps, browsers, calendars, and spreadsheets. The real frontier is agents that can understand, coordinate, and act in the physical world. Iddris explains a vision for “agentic infrastructure for atoms”: systems that let AI agents manage physical objects, deliveries, storage, location, identity, logistics, robots, vehicles, and real-world context.

Robert Scoble

17,484 views • 4 months ago

World Simulator, reimagined — now alive with humans, robots, and their vibrant society unfolding in 3D real-world geospatial scenes across the globe! 🚀 One day soon, humans and robots will co-exist in the same world. To prepare, we must address: 1️⃣ How can robots cooperate or compete intelligently? 2️⃣ How do humans build social bonds and communities? 3️⃣ How can both co-exist in an open, dynamic world? Announcing Virtual Community Project — a social-physical world simulator, where human characters and robotic agents can interact, grow, and co-evolve within open-world societies, stretching from London to New York, and beyond! Key features include: ✅ Unified multi-agent physics simulations for rich social + physical interactions of humans and robots ✅ Massive auto-generated 3D scenes grounded with the rea-world geospatial data ✅ Agent communities populated by robots and LLM-driven human characters with rich appearances, personalities, and social ties. 🌍 Enter our Virtual Community, an open world to study embodied AI at scale— one social-physical world model at a time! 🔗 Project: 💻 Code: Paper: 1/n

World Simulator, reimagined — now alive with humans, robots, and their vibrant society unfolding in 3D real-world geospatial scenes across the globe! 🚀 One day soon, humans and robots will co-exist in the same world. To prepare, we must address: 1️⃣ How can robots cooperate or compete intelligently? 2️⃣ How do humans build social bonds and communities? 3️⃣ How can both co-exist in an open, dynamic world? Announcing Virtual Community Project — a social-physical world simulator, where human characters and robotic agents can interact, grow, and co-evolve within open-world societies, stretching from London to New York, and beyond! Key features include: ✅ Unified multi-agent physics simulations for rich social + physical interactions of humans and robots ✅ Massive auto-generated 3D scenes grounded with the rea-world geospatial data ✅ Agent communities populated by robots and LLM-driven human characters with rich appearances, personalities, and social ties. 🌍 Enter our Virtual Community, an open world to study embodied AI at scale— one social-physical world model at a time! 🔗 Project: 💻 Code: Paper: 1/n

Chuang Gan

90,723 views • 1 year ago

Mansa AI is an enterprise-grade AI + Web3 platform designed to move artificial intelligence from experimentation into real-world execution. Built for creators, developers, and businesses, it focuses on deploying AI that actually works across modern digital systems, not just in isolated demos. 🚀 Production-ready AI infrastructure Mansa AI enables teams to deploy AI systems designed for live environments, handling real workflows, real data, and real operational demands without constant manual oversight. 🧠 Autonomous AI agents At its core, Mansa AI allows users to build autonomous agents that automate decision-making, coordinate tasks, monitor live signals, and execute complex workflows across dynamic environments. ⚙️ Fully customizable logic Agents can be configured with custom behaviors, triggers, and responses. From content generation and analytics to operational automation and intelligent orchestration, logic adapts to specific business strategies. 🔗 Web3 and off-chain integration Mansa AI bridges blockchain ecosystems with traditional systems, enabling cross-chain coordination, smart contract interactions, and seamless integration with existing enterprise infrastructure. 📊 Real-world use cases The platform supports automation for operations, customer engagement, analytics, data pipelines, content workflows, and AI-driven optimization across products and teams. 📈 Built for scale Whether launching as a startup or deploying across enterprise systems, Mansa AI is designed to scale AI operations without adding complexity or fragmentation. Mansa AI transforms artificial intelligence into deployable infrastructure. By combining autonomy, customization, interoperability, and scalability, it enables teams to own, operate, and grow intelligent systems that deliver real value in production environments.

Mansa AI is an enterprise-grade AI + Web3 platform designed to move artificial intelligence from experimentation into real-world execution. Built for creators, developers, and businesses, it focuses on deploying AI that actually works across modern digital systems, not just in isolated demos. 🚀 Production-ready AI infrastructure Mansa AI enables teams to deploy AI systems designed for live environments, handling real workflows, real data, and real operational demands without constant manual oversight. 🧠 Autonomous AI agents At its core, Mansa AI allows users to build autonomous agents that automate decision-making, coordinate tasks, monitor live signals, and execute complex workflows across dynamic environments. ⚙️ Fully customizable logic Agents can be configured with custom behaviors, triggers, and responses. From content generation and analytics to operational automation and intelligent orchestration, logic adapts to specific business strategies. 🔗 Web3 and off-chain integration Mansa AI bridges blockchain ecosystems with traditional systems, enabling cross-chain coordination, smart contract interactions, and seamless integration with existing enterprise infrastructure. 📊 Real-world use cases The platform supports automation for operations, customer engagement, analytics, data pipelines, content workflows, and AI-driven optimization across products and teams. 📈 Built for scale Whether launching as a startup or deploying across enterprise systems, Mansa AI is designed to scale AI operations without adding complexity or fragmentation. Mansa AI transforms artificial intelligence into deployable infrastructure. By combining autonomy, customization, interoperability, and scalability, it enables teams to own, operate, and grow intelligent systems that deliver real value in production environments.

King

155,637 views • 9 months ago

Introducing Genie 2: our AI model that can create an endless variety of playable 3D worlds - all from a single image. 🖼️ These types of large-scale foundation world models could enable future agents to be trained and evaluated in an endless number of virtual environments. →

Introducing Genie 2: our AI model that can create an endless variety of playable 3D worlds - all from a single image. 🖼️ These types of large-scale foundation world models could enable future agents to be trained and evaluated in an endless number of virtual environments. →

Google DeepMind

1,443,049 views • 1 year ago

This first-person shooter was built from a single prompt, using 404-generated 3D assets. On the latest Novelty Search, 404 explains how SN17 is moving from Titan, its open image-to-3D model built with miners, toward 3D assets generated as code that AI agents can edit and use directly in games. That work is part of a broader direction for 404: building world models where AI agents can learn and act. Hosted by const Full episode in the first comment

This first-person shooter was built from a single prompt, using 404-generated 3D assets. On the latest Novelty Search, 404 explains how SN17 is moving from Titan, its open image-to-3D model built with miners, toward 3D assets generated as code that AI agents can edit and use directly in games. That work is part of a broader direction for 404: building world models where AI agents can learn and act. Hosted by const Full episode in the first comment

Openτensor Foundaτion

19,046 views • 1 month ago

While Anthropic and OpenAI race on AI Agents/LLMs, Mistral continues expanding its footprint in physical AI. They just released Robostral Navigate (8B VLA model). It shows strong potential in delivery, logistics, manufacturing, and hospitality--for example, enabling humanoid robots to autonomously navigate multiple rooms to hand a glass of water to an employee,all via natural language instructions in complex environments. Powered by just a single RGB camera (no LiDAR or depth sensors), it achieves 76.6% success on R2R-CE (Room-to-Room in Continuous Environments) unseen benchmarks, outperforming multi-sensor systems. Trained purely in simulation, it generalizes across wheeled, legged, and flying robots.

While Anthropic and OpenAI race on AI Agents/LLMs, Mistral continues expanding its footprint in physical AI. They just released Robostral Navigate (8B VLA model). It shows strong potential in delivery, logistics, manufacturing, and hospitality--for example, enabling humanoid robots to autonomously navigate multiple rooms to hand a glass of water to an employee,all via natural language instructions in complex environments. Powered by just a single RGB camera (no LiDAR or depth sensors), it achieves 76.6% success on R2R-CE (Room-to-Room in Continuous Environments) unseen benchmarks, outperforming multi-sensor systems. Trained purely in simulation, it generalizes across wheeled, legged, and flying robots.

CyberRobo

10,990 views • 2 months ago

Announcing Axol: a dual-arm robot designed for teams working with physical AI. Made in America. Axol is for builders who believe robots should work, in the real world, not just staged environments, and that the future of physical AI should be open, not closed. #robotics #automation #ai #hardware

Announcing Axol: a dual-arm robot designed for teams working with physical AI. Made in America. Axol is for builders who believe robots should work, in the real world, not just staged environments, and that the future of physical AI should be open, not closed. #robotics #automation #ai #hardware

Almond Robotics

63,842 views • 3 months ago

Welcome to Youmio Agents, the launchpad where you can create groundbreaking 3D agents and shape the world. 🌱 Create powerful onchain agents designed to live, learn and adapt in dynamic worlds. Customize their personality, agentic capabilities and equip them with traits that make them truly one of a kind. Agents can be tokenized and monetized as influencers, entertainers, investors, players and workers—bridging digital worlds and the real world. With real-time interaction, emergent storytelling and limitless gameplay opportunities, Youmio Agents is the new frontier of creativity, play and entertainment. The future of AI Agents will be built by you…in 3D. 🚀

Welcome to Youmio Agents, the launchpad where you can create groundbreaking 3D agents and shape the world. 🌱 Create powerful onchain agents designed to live, learn and adapt in dynamic worlds. Customize their personality, agentic capabilities and equip them with traits that make them truly one of a kind. Agents can be tokenized and monetized as influencers, entertainers, investors, players and workers—bridging digital worlds and the real world. With real-time interaction, emergent storytelling and limitless gameplay opportunities, Youmio Agents is the new frontier of creativity, play and entertainment. The future of AI Agents will be built by you…in 3D. 🚀

Youmio

233,076 views • 1 year ago

🎤 Overheard at #CES2026: Physical AI is moving intelligence from the cloud into the real world, and AMD is powering that shift with a full portfolio that enables robots to sense, decide, and act in real time across industries like healthcare, agriculture, and manufacturing.

🎤 Overheard at #CES2026: Physical AI is moving intelligence from the cloud into the real world, and AMD is powering that shift with a full portfolio that enables robots to sense, decide, and act in real time across industries like healthcare, agriculture, and manufacturing.

AMD

19,488 views • 8 months ago

🔥 JUST IN: Open-source robotics dataset from 100% real-world scenarios! 🤯 Chinese robotics company AGIBOT just released AGIBOT WORLD 2026, an open-source dataset systematically covering key embodied AI research directions. Built entirely from real-world environments: commercial spaces, and homes. Collected using AGIBOT G2 robots in free-form collection mode, providing structured, accurately annotated, high-quality data. Digital twin technology creates 1:1 scale replicas in simulation matching the real environments. Both real-world and simulation data are open-sourced. The AGIBOT G2 platform collects multiple data types simultaneously: RGB(D) cameras, tactile sensors, force sensors, LiDAR, IMU, and full-body joint states. Whole-body control coordinates arms, waist, and hands for complex tasks. First-person teleoperation lets operators control the robot from its perspective. The tasks covered are fine-grained manipulation, ultra-long-horizon tasks, spatial navigation, dual-arm coordination, and multi-agent/human-robot collaboration. The dataset includes error-recovery trajectories with annotations. Most datasets only show successful demonstrations. AGIBOT includes failures and how the robot recovers, teaching models how to handle mistakes. After collection, data is tested through policy training and real-robot deployment to ensure quality. Then processed through industrial quality control with multiple screening and cleaning rounds. Making it open-source accelerates embodied AI research by giving researchers access to high-quality real-world robot data at scale. 🇨🇳 Learn more here: ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →

🔥 JUST IN: Open-source robotics dataset from 100% real-world scenarios! 🤯 Chinese robotics company AGIBOT just released AGIBOT WORLD 2026, an open-source dataset systematically covering key embodied AI research directions. Built entirely from real-world environments: commercial spaces, and homes. Collected using AGIBOT G2 robots in free-form collection mode, providing structured, accurately annotated, high-quality data. Digital twin technology creates 1:1 scale replicas in simulation matching the real environments. Both real-world and simulation data are open-sourced. The AGIBOT G2 platform collects multiple data types simultaneously: RGB(D) cameras, tactile sensors, force sensors, LiDAR, IMU, and full-body joint states. Whole-body control coordinates arms, waist, and hands for complex tasks. First-person teleoperation lets operators control the robot from its perspective. The tasks covered are fine-grained manipulation, ultra-long-horizon tasks, spatial navigation, dual-arm coordination, and multi-agent/human-robot collaboration. The dataset includes error-recovery trajectories with annotations. Most datasets only show successful demonstrations. AGIBOT includes failures and how the robot recovers, teaching models how to handle mistakes. After collection, data is tested through policy training and real-robot deployment to ensure quality. Then processed through industrial quality control with multiple screening and cleaning rounds. Making it open-source accelerates embodied AI research by giving researchers access to high-quality real-world robot data at scale. 🇨🇳 Learn more here: ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →

Lukas Ziegler

40,583 views • 6 months ago

Halluminate is building reinforcement learning environments and benchmarks to help AI models do knowledge work beyond coding, starting with finance. With a team of fewer than 10 people, the company works with four of the top five closed-source US AI labs and recently raised a $30M Series A. In this fireside, co-founders Jerry Wu and wyatt marshall sit down with YC's Jon Xu to share how testing browser agents led them to building the environments models now use to practice tasks like financial modeling. They explain why better verification, not just more training tasks, is critical to making models smarter, and how that work could eventually lead to simulated companies where teams of AI agents learn to work together. 00:00 — Halluminate’s Pivot From Evals to Training Environments 03:48 — Teaching AI to Do Financial Work 07:13 — Why Verification Matters More Than Volume 10:13 — From Finance to Simulated Companies 13:30 — AI Safety and Building the Team

Halluminate is building reinforcement learning environments and benchmarks to help AI models do knowledge work beyond coding, starting with finance. With a team of fewer than 10 people, the company works with four of the top five closed-source US AI labs and recently raised a $30M Series A. In this fireside, co-founders Jerry Wu and wyatt marshall sit down with YC's Jon Xu to share how testing browser agents led them to building the environments models now use to practice tasks like financial modeling. They explain why better verification, not just more training tasks, is critical to making models smarter, and how that work could eventually lead to simulated companies where teams of AI agents learn to work together. 00:00 — Halluminate’s Pivot From Evals to Training Environments 03:48 — Teaching AI to Do Financial Work 07:13 — Why Verification Matters More Than Volume 10:13 — From Finance to Simulated Companies 13:30 — AI Safety and Building the Team

Y Combinator

258,252 views • 6 days ago

Introducing Cosmos 3 Edge: our open frontier world model built to run on-device. Cosmos 3 Edge helps robots learn and act, autonomous vehicles understand road scenes and predict intent, and vision AI agents reason across live video for smart infrastructure. With 4B parameters and a 2B Nemotron-based reasoner, you can run it on DGX Spark, NVIDIA Jetson, and more.

Introducing Cosmos 3 Edge: our open frontier world model built to run on-device. Cosmos 3 Edge helps robots learn and act, autonomous vehicles understand road scenes and predict intent, and vision AI agents reason across live video for smart infrastructure. With 4B parameters and a 2B Nemotron-based reasoner, you can run it on DGX Spark, NVIDIA Jetson, and more.

NVIDIA AI

212,153 views • 2 months ago

"The “ChatGPT moment” for physical AI is nearly here. 👏 Cosmos open world foundation models understand reality to create and reason about rich synthetic worlds—so self-driving cars and robots can learn from endlessly varied, physics-aware scenarios. From a single image, 3D scene, or simulator trace, Cosmos generates realistic video, brings edge cases to life, and helps autonomous systems train for the long tail. Watch the #CES2026 demo now 🎥

"The “ChatGPT moment” for physical AI is nearly here. 👏 Cosmos open world foundation models understand reality to create and reason about rich synthetic worlds—so self-driving cars and robots can learn from endlessly varied, physics-aware scenarios. From a single image, 3D scene, or simulator trace, Cosmos generates realistic video, brings edge cases to life, and helps autonomous systems train for the long tail. Watch the #CES2026 demo now 🎥

NVIDIA AI

49,809 views • 9 months ago