Loading video...

Video Failed to Load

Go Home

Genesis's generative framework supports generating 3D and fully interactive scenes for training robotic skills 5/n

63,536 views • 1 year ago •via X (Twitter)

14 Comments

Zhou Xian's profile picture
Zhou Xian1 year ago

Everything you love about generative models — now powered by real physics! Announcing the Genesis project — after a 24-month large-scale research collaboration involving over 20 research labs — a generative physics engine able to generate 4D dynamical worlds powered by a physics simulation platform designed for general-purpose robotics and physical AI applications. Genesis's physics engine is developed in pure Python, while being 10-80x faster than existing GPU-accelerated stacks like Isaac Gym and MJX. It delivers a simulation speed ~430,000 faster than in real-time, and takes only 26 seconds to train a robotic locomotion policy transferrable to the real world on a single RTX4090 (see tutorial: The Genesis physics engine and simulation platform is fully open source at We'll gradually roll out access to our generative framework in the near future. Genesis implements a unified simulation framework all from scratch, integrating a wide spectrum of state-of-the-art physics solvers, allowing simulation of the whole physical world in a virtual realm with the highest realism. We aim to build a universal data engine that leverages an upper-level generative framework to autonomously create physical worlds, together with various modes of data, including environments, camera motions, robotic task proposals, reward functions, robot policies, character motions, fully interactive 3D scenes, open-world articulated assets, and more, aiming towards fully automated data generation for robotics, physical AI and other applications. Open Source Code: Project webpage: Documentation: 1/n

Zhou Xian's profile picture
Zhou Xian1 year ago

Nvidia brought GPU acceleration to robotic simulation, speeding up simulation speed by more than one order of magnitude compared to CPU-based simulation. This brought numerous amazing robotic skills to life by leveraging large-scale GPU-parallelized simulation. Genesis pushes up this speed by another order of magnitude. Note that the speed improvement is achieved with no compromise in simulation accuracy. 2/n

Zhou Xian's profile picture
Zhou Xian1 year ago

Genesis supports simulating various types of physical phenomena. We developed from scratch a unified physics engine that integrates various SOTA physics solvers (MPM, SPH, FEM, Rigid Body, PBD, etc.), supporting simulation of a wide range of materials: rigid body, articulated body, Cloth, Liquid, Smoke, Deformables, Thin-shell materials, Elastic/Plastic Body, Robot Muscles, etc. 3/n

Zhou Xian's profile picture
Zhou Xian1 year ago

Genesis is the first-ever platform providing comprehensive support for soft muscles and soft robot and their interaction with rigid robots. Genesis also ships with a URDF-like soft-robot configuration system. 4/n

Zhou Xian's profile picture
Zhou Xian1 year ago

Our generative agent autonomously proposes robotic tasks, design environments, write reward functions, and ultimately leading to automated generation of robotic policies. 6/n

Zhou Xian's profile picture
Zhou Xian1 year ago

Genesis's generative framework supports data generation beyond robotics, such as character motion: 7/n

Zhou Xian's profile picture
Zhou Xian1 year ago

Genesis's GPU parallellized IK solver is able to solve IK for 10,000 Franka arms simultaneously, under 2ms: 8/n

Zhou Xian's profile picture
Zhou Xian1 year ago

We support native non-convex collision handling: 9/n

Zhou Xian's profile picture
Zhou Xian1 year ago

Genesis supports a physically accurate tactile sensing simulation module: (Will be integrated into the main branch in a future release soon) 10/n

Zhou Xian's profile picture
Zhou Xian1 year ago

Finally, a cute interactive physical Tetris game made with Genesis :) Thanks to all the amazing collaborators who together made everything possible over the last two years! There's no space here to @ every single one, but a huge kudos to the whole Genesis team! We welcome everyone from the open-source community to come join us and build Genesis with us together! 11/11

Digital Currency's profile picture
Digital Currency2 years ago

From 3D modeling to VR/AR development, our MSc in Metaverse program equips you with the technical skills to excel in the rapidly evolving digital world. Don't miss out—enroll today! #UNIC #MScMetaverse

Squaremusher's profile picture
Squaremusher1 year ago

I’m confused, so does it create a 3d scene somewhere that we can access or does it just generate the end resulting 2d video? This is mind blowing either way.

Bill Harris's profile picture
Bill Harris1 year ago

AttributeError: module 'genesis' has no attribute 'generate'

slimemax's profile picture
slimemax1 year ago

Huh???

Related Videos

Everything you love about generative models — now powered by real physics! Announcing the Genesis project — after a 24-month large-scale research collaboration involving over 20 research labs — a generative physics engine able to generate 4D dynamical worlds powered by a physics simulation platform designed for general-purpose robotics and physical AI applications. Genesis's physics engine is developed in pure Python, while being 10-80x faster than existing GPU-accelerated stacks like Isaac Gym and MJX. It delivers a simulation speed ~430,000 faster than in real-time, and takes only 26 seconds to train a robotic locomotion policy transferrable to the real world on a single RTX4090 (see tutorial: The Genesis physics engine and simulation platform is fully open source at We'll gradually roll out access to our generative framework in the near future. Genesis implements a unified simulation framework all from scratch, integrating a wide spectrum of state-of-the-art physics solvers, allowing simulation of the whole physical world in a virtual realm with the highest realism. We aim to build a universal data engine that leverages an upper-level generative framework to autonomously create physical worlds, together with various modes of data, including environments, camera motions, robotic task proposals, reward functions, robot policies, character motions, fully interactive 3D scenes, open-world articulated assets, and more, aiming towards fully automated data generation for robotics, physical AI and other applications. Open Source Code: Project webpage: Documentation: 1/n

Zhou Xian

3,814,717 views • 1 year ago