Emergent Tool Use From Multi-Agent Autocurricula

ABSTRACT

We demonstrate that simple multi-agent competition can drive the emergence of highly complex, intelligent behaviors without explicit human design. We train agents using reinforcement learning to play a physics-based game of hide-and-seek in a simulated 3D environment. Through millions of episodes of competitive self-play, the agents naturally develop an 'autocurriculum' of increasingly sophisticated strategies. The hiding agents learn to use physical tools, such as moving boxes to build barricades and locking ramps in place, while the seeking agents learn to overcome these defenses by using ramps to jump over walls or 'surfing' on boxes. This work provides compelling evidence that multi-agent co-adaptation is a scalable path to general intelligence.