Wow. That's cool. Researchers just released World, an open-source Unreal Engine 5 simulator for training and testing LLM and VLM agents in realistic 3D environments. The platform supports RGB, depth, and segmentation sensors, along with navigation, vehicles, pedestrians, robots and procedural city generation. It's built with a Gym-like Python... interface, it allows AI agents to learn physical and social reasoning in complex virtual worlds before real world deployment. We're moving from AI that only understands information to AI that can perceive, reason and act inside realistic simulated environments.show more

GitHub repo: source of video:

@Lianhuiq Thanks for sharing our work!🔥 Also check out the latest development on SimWorld Studio

SimWorld is a simulation platform for developing and evaluating LLM/VLM AI agents in complex physical and social environments!

Feels like video games are slowly becoming training grounds for real world AI now

@grok

Can you run over prostitutes with a vehicle here?

@BrianRoemmele

🫠🫠🫠 i expected this to only happen in, like, 3 years

The Gym-like Python interface is the key detail here. Grounding LLM agent training in physics-based 3D environments closes a gap that's hard to fix with text-only benchmarks — spatial reasoning, occlusion, timing. The open-source + procedural city gen combo makes this actually usable for research.

open source means anyone can add custom robot joint models for better manipulation training.

Wait a minute. Is that us?

Procedural city generation is useful. 'Social reasoning' though? Still just pattern matching with better props. LLMs don't reason. They just have nicer environments to not reason in now.

最適化はされてますか？ また、モジュラーはAIにより生成されたのでしょうか？

such a bad idea to give ai this info

Terminator is near

I had full intention of hooking up an API to Unreal but I was waiting for the next version to drop, I wouldn't get it to do stuff that has already been done or your wasting cash, I'd like to test how A.I handles socketing items to a Daz3d body I have added to a mannequin using a bridge, I want to test how well the A.I can handle the shaping morphs for the body sets with gear and how it manages to animate a full control rig to interact with vehicles and the environment by using animations I already have in library so it's not creating them all from scratch, just altering animation timelines to suit needed criteria. I need it to build an inventory system to those items socketed to control rigs and just set up an easy method to add new characters in for games. Either this or I create my own web-based editor and hook my entire library of content into it.