One of the hardest challenges in developing AI for autonomous vehicles is evaluating the performance of our driving models. Why? (A short 🧵on our latest research on multi-agent RL).

On-road testing is important and the final arbiter of quality. But it is time-consuming, expensive, and unrepeatable. This slows the development iteration cycle and thus the rate of improvement of the AI driver.

The typical approach in AI development, a static held-out validation set, also doesn’t cut it, since the ‘closed loop’ actions that the vehicle takes affects the world around it. So what can you do?

We recently introduced Wayve Infinity Simulator, built for realism, diversity, controllability, and scale. But for Infinity to reflect the dynamic complexity of real-world urban driving, we need to populate it with diverse reactive agents. How can we achieve this?

Our new research blog describes how multi-agent reinforcement learning is allowing hundreds of independently acting virtual cars, bicycles, and other vehicles to populate our simulated environments.

This is an exciting challenge. Unlike typical uses of RL, the task we’re learning to solve is ‘non-stationary’ - it changes and gets harder as the agents that make the task themselves change.

The emergent result is a virtual world populated with enormous complexity of dynamic and reactive driving agents.

Like real world drivers, some of these agents drive well, others poorly. This provides a strong test-bed for evaluating our driving models that have to react safely to whatever the real-world throws at them.

You can read the full blog on our combined multi-agent reinforcement learning technique here and our blog on Wayve Infinity Simulator here

This is only the beginning, and we’re working to improve the realism of behaviour and the diversity of types of agents. If you'd be interested in working on this with us, check out our open roles here

@JamieDJS Will this simulator be opensourced?