OpenAI Five | OpenAI

OpenAI Five | OpenAI Switch to ChatGPT (opens in a new window) 
 Sora (opens in a new window) 
 API Platform (opens in a new window) 
 OpenAI June 25, 2018

 Milestone OpenAI Five

 Our team of five neural networks, OpenAI Five, has started to defeat amateur human teams at Dota 2.

 Loading… Share Our team of five neural networks, OpenAI Five, has started to  defeat ⁠  amateur human teams at  Dota 2 ⁠ (opens in a new window) . While today we play with  restrictions ⁠ , we aim to beat a team of top professionals at  The International ⁠ (opens in a new window)  in August subject only to a limited set of heroes. We may not succeed: Dota 2 is one of the most popular and  complex ⁠ (opens in a new window)  esports games in the world, with creative and motivated professionals who  train ⁠ (opens in a new window)  year-round to earn part of Dota’s annual $40M  prize pool ⁠ (opens in a new window)  (the largest of any esports game). 

 OpenAI Five plays 180 years worth of games against itself every day, learning via self-play. It trains using a scaled-up version of  Proximal Policy Optimization ⁠  running on 256 GPUs and 128,000 CPU cores—a larger-scale version of the system we built to play the much-simpler  solo variant ⁠  of the game last year. Using a separate  LSTM ⁠ (opens in a new window)  for each hero and no human data, it learns recognizable strategies. This indicates that  reinforcement learning ⁠ (opens in a new window)  can yield long-term planning with large but achievable scale—without fundamental advances, contrary to our own expectations upon starting the project. 

 To benchmark our progress, we’ll host a match versus top players on August 5th.  Follow ⁠ (opens in a new window)  us on Twitch to view the live broadcast, or  request ⁠ (opens in a new window)  an invite to attend in person! 

 The problem

 One AI milestone is to exceed human capabilities in a complex video game like  StarCraft ⁠ (opens in a new window)  or Dota. Relative to previous AI milestones like  Chess ⁠ (opens in a new window)  or  Go ⁠ (opens in a new window) , complex video games start to capture the messiness and continuous nature of the real world. The hope is that systems which solve complex video games will be highly general, with applications outside of games. 

 Dota 2 is a real-time strategy game played between two teams of five players, with each player controlling a character called a “hero”. A Dota-playing AI must master the following: 

 Long time horizons.  Dota games run at 30 frames per second for an average of 45 minutes, resulting in 80,000 ticks per game. Most actions (like ordering a hero to  move ⁠ (opens in a new window)  to a location) have minor impact individually, but some individual actions like  town portal ⁠ (opens in a new window)  usage can affect the game strategically; some  strategies ⁠ (opens in a new window)  can play out over an entire game. OpenAI Five observes every fourth frame, yielding 20,000 moves.  Chess ⁠ (opens in a new

... (truncated, 21 KB total)