Google's DeepMind trained robots how to play soccer. Here's what I found:

DeepMind used miniature humanoids with 20 controllable joints. They applied Deep Reinforcement Learning (Deep RL) to develop motor skills for: • Rapid fall recovery • Walking, turning, kicking • Basic strategic understanding

This enabled a 1v1 soccer game. The humanoids learned target joint angles using only the onboard computer. They also used a real-time motion capture system to observe the location of the robot and the ball.

The agents were rewarded for scoring goals and penalised for approaching closer than 1M to the other robot. The agents were then trained entirely in simulation using a deep reinforcement learning algorithm. They learned: • How to get up from the floor • How to score goals

The trained soccer agents showed agile and reactive behaviour during gameplay. • Getting up and turning • Kicking a moving ball • Blocking defensively I find it fascinating how the agents can quickly transition between various emergent skills and combine them effectively.

The performance of the agents was compared with scripted baseline skills in simulation and real environments. The agents: • Walked 156% faster • Took 63% less time to get up • Kicked 24% faster It's clear Deep RL training can produce significant improvements in performance.

DeepMind is now exploring the possibility of agents learning directly from vision. It's clear deep reinforcement learning produces high-quality individual skills. I'm genuinely excited about the potential not only in soccer but in other real-world applications and industries.

Follow me @thealexbanks for more on AI. If you liked this thread, you'll love the newsletter. Cut through the noise in AI each week, subscribe here:

Help everyone learn and retweet this thread:

You can read the full paper here:

Barely touched it. Already taking flops