Video yükleniyor...

Video Yüklenemedi

Ana Sayfaya Dön

For roboticists, one challenge towers above the others: there isn’t enough data. To accelerate the deployment of intelligent robots in the real world, MIT CSAIL’s "LucidSim" uses genAI & physics engines to create diverse & realistic virtual training grounds for robots. W/o any real-world data, their robot achieved expert-level...

82,915 görüntüleme • 1 yıl önce •via X (Twitter)

9 Yorum

MIT CSAIL profil fotoğrafı
MIT CSAIL1 yıl önce

ChatGPT comes up w/diverse descriptions of the environment, which are then transformed into realistic pictures by a ControlNet. Guidance from the physics engine ensures that images reflect correct collision geometry and real-world physics.

MIT CSAIL profil fotoğrafı
MIT CSAIL1 yıl önce

To make short, 140 millisecond videos that serve as visual "experiences" for the robot, the scientists hacked together a trick called "Dreams In Motion (DIM)" using a mix of image magic. This trick made LucidSim 7x times faster by moving pixels of a single generated image according to the head movement of the robot, & piece together a short, multi-frame video.

MIT CSAIL profil fotoğrafı
MIT CSAIL1 yıl önce

LucidSim used AI-generated images to train a robot dog to do parkour — w/o real-world data.

MIT CSAIL profil fotoğrafı
MIT CSAIL1 yıl önce

Having the dog learn from its own actions is essential. The team put LucidSim against the alternative, where an expert teacher provides substantially more training data. Robots learn from expert data only succeeded 15% of the time — and even quadrupling the amount of expert training data barely moved the needle. When robots collected their own training data through LucidSim, the story changed dramatically: Just doubling the dataset size catapulted success rates to 88%.

MIT CSAIL profil fotoğrafı
MIT CSAIL1 yıl önce

LucidSim outperforms domain randomization, a go-to method from 2017 that produces diverse data, but lacks realism. LucidSim addresses both diversity & realism problems, helping a robot recognize and navigate obstacles in real environments.

MIT CSAIL profil fotoğrafı
MIT CSAIL1 yıl önce

LucidSim is paving the way toward a new generation of intelligent machines that elevate & amplify human effort. The researchers envision these machines could safely and autonomously learn how to navigate our complex world w/o ever setting foot in it. The team’s immediate plan is to scale up to a human-sized robot and teach it to manipulate objects on the move, using purely synthesized images.

MIT CSAIL profil fotoğrafı
MIT CSAIL1 yıl önce

Authors: Alan Yu (@alany1_), Ge Yang (@EpisodeYang), Ran Choi, Yajvan Ravan, John Leonard (@jleonardmit), & Phillip Isola (@phillip_isola) Paper: Full video: Website:

Freedom_Aint_Free profil fotoğrafı
Freedom_Aint_Free1 yıl önce

There's no enough data ? You mean current algorithms are dumb as a brick, children don't need to look at 1 million pictures of cats to learn what a cat is.

wes george profil fotoğrafı
wes george1 yıl önce

The idea of walking your beta robot...seems like there is a song in that....

Benzer Videolar

Elon just dropped a MAJOR nugget on how Tesla is going to be training Optimus to do real world tasks. They are building an Optimus Academy, which is a large scale, dedicated real-world training facility to accelerate the development of Optimus. The Academy will deploy thousands of Optimus units, potentially 10,000 to 30,000 robots, in a controlled realistic environment where they perform self-play, experiment with tasks, iterate on behaviors, and continuously generate training data through trial and error. The Tesla bots will also run millions of simulations in Tesla’s high-fidelity physics-accurate engine, allowing Optimus to close the “sim-to-real gap” by using these real-world observations to refine and validate the simulations! “You’re actually highlighting an important limitation and difference from cars. We’ll soon have 10 million cars on the road. It’s hard to duplicate that massive training flywheel. For the robot, what we’re going to need to do is build a lot of robots and put them in kind of an Optimus Academy so they can do self-play in reality. We’re actually building that out. We can have at least 10,000 Optimus robots, maybe 20-30,000, that are doing self-play and testing different tasks. Tesla has quite a good reality generator, a physics-accurate reality generator, that we made for the cars. We’ll do the same thing for the robots. We actually have done that for the robots. So you have a few tens of thousands of humanoid robots doing different tasks. You can do millions of simulated robots in the simulated world. You use the tens of thousands of robots in the real world to close the simulation to reality gap. Close the sim-to-real gap.”

Teslaconomics

42,563 görüntüleme • 6 ay önce