正在加载视频...

视频加载失败

Besides reading cool papers, my Twitter account is mostly used for catching up Formula 1 news. Very excited about Lando Norris 's great performance recently. Now we combine Formula Racing to AI research: The following video shows we train a Reinforcement Learning policy to drive a Dallara F317 in...

32,361 次观看 • 2 年前 •via X (Twitter)

9 条评论

Xiaolong Wang 的头像
Xiaolong Wang2 年前

@LandoNorris Simulation plays a critical role in training drivers, one of the reasons why Red Bull has not been dominating recently:

Xiaolong Wang 的头像
Xiaolong Wang2 年前

And drivers play games as well:

Xiaolong Wang 的头像
Xiaolong Wang2 年前

Now go back to our paper. We open source and evaluate RL and imitation learning on 4 different circuits with 3 different cars. We have humans drive and collect 927 laps across these environments. Participants included a professional e-sports driver, four experts, five casual drivers, and five beginners.

Xiaolong Wang 的头像
Xiaolong Wang2 年前

The following table shows lap time (in seconds). Reinforcement Learning agent is able to beat the professional driver in 3 different cases, but still loses to the guy in most cases.

Xiaolong Wang 的头像
Xiaolong Wang2 年前

We collect human data from a fancy simulator in Austria.

Xiaolong Wang 的头像
Xiaolong Wang2 年前

And also not so fancy one in my lab at @UCSDJacobs

Xiaolong Wang 的头像
Xiaolong Wang2 年前

Check out our paper here: Everything is open sourced. A very fun collaboration with Adrian Remonda, @ncklashansen , Ayoub Raji, Nicola Musiu, Marko Bertogna, Eduardo Veas

Cern Basher 的头像
Cern Basher2 年前

@F1 @LandoNorris One day, it will be fun to see Lando race an autonomous F1 car!

varad 的头像
varad2 年前

@F1 @LandoNorris 10 sec penalty for Ocon NEEDS to be implemented as a bit of fun

相关视频

In my past research experience, finding or developing an appropriate simulation environment, dataset, and benchmark has always been a challenge. Missing features, limited support, or unexpected bugs often occupied my days and nights. Moreover, current simulation platforms are relatively fragmented—making it challenging to replicate the success of the RT-X dataset in unifying community efforts. Introducing RoboVerse, we provide a unified platform, dataset, and benchmark for scalable and generalizable robot learning. We hope to build a shared foundation to combine the community efforts. RoboVerse includes: MetaSim: We carefully designed a configuration system and a universal interface to align current robotic simulators. With MetaSim, you can use any simulator with the same code—bringing together the community’s diverse efforts under one framework! RoboVerse Dataset and Benchmark: We unify popular simulation environments and benchmarks into a single cohesive system and introduce the RoboVerse dataset—a large-scale, high-quality synthetic dataset. Additionally, we propose a standardized benchmark across both imitation learning and reinforcement learning. A cool feature enabled by our unified framework: Hybrid Simulation! You can now integrate physics engines and renderers from different simulators—e.g., using MuJoCo precise physics with Isaac photorealistic rendering. This not only elevates simulation fidelity but also significantly enhances real-world transfer performance across complex robotic applications. Hopefully, our team’s efforts could serve the robotic community to thrive vibrantly in the years to come. RoboVerse is open-sourced🥳!!! Project Page: Documentation: Github Repo: Paper:

Haoran Geng

84,318 次观看 • 1 年前

Everything you love about generative models — now powered by real physics! Announcing the Genesis project — after a 24-month large-scale research collaboration involving over 20 research labs — a generative physics engine able to generate 4D dynamical worlds powered by a physics simulation platform designed for general-purpose robotics and physical AI applications. Genesis's physics engine is developed in pure Python, while being 10-80x faster than existing GPU-accelerated stacks like Isaac Gym and MJX. It delivers a simulation speed ~430,000 faster than in real-time, and takes only 26 seconds to train a robotic locomotion policy transferrable to the real world on a single RTX4090 (see tutorial: The Genesis physics engine and simulation platform is fully open source at We'll gradually roll out access to our generative framework in the near future. Genesis implements a unified simulation framework all from scratch, integrating a wide spectrum of state-of-the-art physics solvers, allowing simulation of the whole physical world in a virtual realm with the highest realism. We aim to build a universal data engine that leverages an upper-level generative framework to autonomously create physical worlds, together with various modes of data, including environments, camera motions, robotic task proposals, reward functions, robot policies, character motions, fully interactive 3D scenes, open-world articulated assets, and more, aiming towards fully automated data generation for robotics, physical AI and other applications. Open Source Code: Project webpage: Documentation: 1/n

Zhou Xian

3,818,940 次观看 • 1 年前

Physics-based Motion Retargeting from Sparse Inputs paper page: Avatars are important to create interactive and immersive experiences in virtual worlds. One challenge in animating these characters to mimic a user's motion is that commercial AR/VR products consist only of a headset and controllers, providing very limited sensor data of the user's pose. Another challenge is that an avatar might have a different skeleton structure than a human and the mapping between them is unclear. In this work we address both of these challenges. We introduce a method to retarget motions in real-time from sparse human sensor data to characters of various morphologies. Our method uses reinforcement learning to train a policy to control characters in a physics simulator. We only require human motion capture data for training, without relying on artist-generated animations for each avatar. This allows us to use large motion capture datasets to train general policies that can track unseen users from real and sparse data in real-time. We demonstrate the feasibility of our approach on three characters with different skeleton structure: a dinosaur, a mouse-like creature and a human. We show that the avatar poses often match the user surprisingly well, despite having no sensor information of the lower body available. We discuss and ablate the important components in our framework, specifically the kinematic retargeting step, the imitation, contact and action reward as well as our asymmetric actor-critic observations. We further explore the robustness of our method in a variety of settings including unbalancing, dancing and sports motions.

AK

106,527 次观看 • 3 年前