Chuang Gan
@gan_chuang • 10,292 subscribers
Faculty Member at UMass Amherst; Principal researcher at MIT-IBM Watson AI Lab; Homepage: https://t.co/Pc8WeREfTz
Shorts
Videos
World Simulator, reimagined — now alive with humans, robots, and their vibrant society unfolding in 3D real-world geospatial scenes across the globe! 🚀 One day soon, humans and robots will co-exist in the same world. To prepare, we must address: 1️⃣ How can robots cooperate or compete intelligently? 2️⃣ How do humans build social bonds and communities? 3️⃣ How can both co-exist in an open, dynamic world? Announcing Virtual Community Project — a social-physical world simulator, where human characters and robotic agents can interact, grow, and co-evolve within open-world societies, stretching from London to New York, and beyond! Key features include: ✅ Unified multi-agent physics simulations for rich social + physical interactions of humans and robots ✅ Massive auto-generated 3D scenes grounded with the rea-world geospatial data ✅ Agent communities populated by robots and LLM-driven human characters with rich appearances, personalities, and social ties. 🌍 Enter our Virtual Community, an open world to study embodied AI at scale— one social-physical world model at a time! 🔗 Project: 💻 Code: Paper: 1/n
Chuang Gan89,505 görüntüleme • 11 ay önce
Robot Learning needs 4D world models! Robot Learning needs 4D world models! Robot Learning needs 4D world models! We introduce TesserAct, a 4D embodied world model that can simulate how agents interact with the 3D world over time! We achieve this by simply extending a pre-trained 2D video generation model to jointly predict RGB, depth, and surface normals. It enables: 1️⃣ Much better policy learning in the wild 2️⃣ Temporal + spatial coherence in 4D dynamic prediction 3️⃣ Novel view synthesis for embodied scenes Code: Paper Link: Project page:
Chuang Gan43,265 görüntüleme • 1 yıl önce
🚀 Introducing Articulate Anymesh – now open-sourced! An automated framework behind our Genesis simulator, capable of transforming any rigid 3D mesh into its articulated counterpart using an open-vocabulary manner! Given a 3D mesh, our framework uses VLMs + visual prompting to extract rich semantics — enabling part segmentation and functional joint construction automatically! 🔗 Code: 📄 Paper: 🌐 Project: #EmbodiedAI #3D #OpenSource #VLM #MeshProcessing
Chuang Gan35,320 görüntüleme • 1 yıl önce
Building intelligent embodied agents should address the diverse needs of all people, including those with physical constraints! Yet, this crucial aspect has often been overlooked within embodied AI communities. Our NeurIPS project, Constrained Human-AI Cooperation (CHAIC), seeks to fill this gap by introducing an inclusive embodied social intelligence challenge. Project page: Code:
Chuang Gan18,742 görüntüleme • 1 yıl önce
Daha fazla içerik yok.