A team of computer-controlled bots took in a Dota 2 show-match a team of top players on the weekend, and the Computer not only beat people – it smashed them them.
We were first introduced Dota 2 Bot by OpenAI at the International, the multimillion dollar valve-hosted tournament, which is the pinnacle of the competitive season. Supported by Elon Musk, OpenAI's goal is to ensure that when artificial intelligence is created, it is good for humans: think Lt. Cmdr. Data rather than Skynet or The Matrix . The OpenAI team considers Dota 2 because the game is enormously more complex than games like chess or go. Unlike these games Dota 2 is played with imperfect knowledge (you can not see the whole card at once so that enemy moves can be hidden), it consists of thousands of moves over tens of minutes, and His goals are relatively long term, making it difficult to judge at the moment which action is best or which side has the advantage.
The Bots learned to play Dota 2 through hundreds of games against themselves, earlier versions of themselves, and pre-programmed script bots. Training for last year's bot was done on the Microsoft Azure platform with about 60,000 processor cores. This time, OpenAI uses 1
Last year's bot played a very limited form of play: a single hero (Shadow Fiend) in a first-to-two kills a one-on-one game. Although it is a simplistic form of the game, it is still used regularly in show matches to demonstrate the abilities of individual players. At that time, the OpenAI bot beat even top players successfully. Throughout many games, people have managed to find strategies the bot could not handle, and to show one of the things that remains (for the time being) a unique human trait: the ability to improvise and undermine the spirit of the game even if you follow the rules. Human players can operate at this meta-level by not only reacting to the state of the game, but also showing that they understand the goals and limitations of the game and then undermine it to its own end. The bot could not do it and let it exploit.
Over the past year, the OpenAI team has created a better bot: OpenAI Five. Dota 2 is a five-on-five team game with 115 different playable heroes. OpenAI is not yet able to handle this full game, but it is far more capable than it was: The bots can play 18 different heroes in five-on-five games. Almost all of the game's items and mechanics are now supported, and the bot even takes on the writing of a team: it selects heroes in response to the decisions made by the human team to maximize their chances of success.
The human team consists of a number of former Pros and Casters, with a skill level of an estimated 99.95 percent of all Dota 2 players. This is not quite the level of a trained, coordinated professional team, but it's not far. They played OpenAI Five in a best-of-three match.
The result? Well, it was not great for the humans. The bot won the first two games with ease, with a game style that was highly focused on goals – pushing the opposing team's towers, controlling the card – and maintaining the health and mana of its heroes during long battles. This deprived the humans of the time they needed to collect the items and steps they needed to repel the thrust. The result was two quick wins, with both games ending in under half an hour compared to more typical 30-40 minutes for real matches.
The bots also did things that human teams do not do. In typical Dota 2 teams, there is a prioritization of who should get the most gold; Heroes (the main damage dealers while the game continues) are prioritized before the support heroes (those with numbness and slowdown, who are used to control the pace of the battles and set kills for their transmissions). In general, OpenAI Five was much more generous to its support heroes than humans. This does not mean that there is no prioritization – its transmissions were richer than its support – but it is much less distorted than human teams prefer.
The bots are also much more willing to shoot human players with damaging spells. A particularly impressive example was Sniper's Assassinate. This is a very long-range, high-damage, single-target spell: Sniper takes a moment to aim his weapon and then fires a high-powered shot. Human players usually use this to land deadly blows, especially to enemies who want to flee from a bad engagement. The OpenAI Five sniper, on the other hand, would use Assassinate early in battles to keep the support hero's health very low and early in team fights.
On the other hand, the bots show some surprising weaknesses: their last blow (the fatal blow to computer-controlled "creeps" with gold, the main source of revenue within the game) was mediocre. In principle, the bots should have been hit almost perfectly lately: they can immediately determine the exact state of health of all creeps and know which creep they are attacking and when to maximize their income. In fact, one of the hallmarks of programmed bots (rather than these machine-learning bots) is that they've almost met almost perfectly. Not so OpenAI Five, or at least not yet.
Most important of all, the bots were Trash Talkers experts. The OpenAI Five-Bot calculates its probability of winning during the game and from time to time he would chatter that probability. It turned out that one does not have to be racist or sexist in order to effectively demoralize an enemy. The mere statement "We estimate the profit probability at over 95 percent" is enough to smash the mind.
After betting victorious 2-0, the bots were shaken for the third game. Instead of allowing OpenAI to design his team, Twitch chat and the live audience were recruited to select a terrific lineup: heroes who lacked many good slow and stuns to control heroes, all of whom had a lot of items and levels effectively needed and heroes who were weak in the early stages of the game. The result? "We estimate the probability of winning at 2.9 percent," the bots announced. Despite this unfavorable lineup, the bots showed a remarkable result, at least to begin with, and it took people more than half an hour to assert themselves.
This game also pointed to further weaknesses in OpenAI Five. At various times in the game, the computer-controlled Slark behaves (a carry-hero who is notoriously weak in early play, but late play can duck in fights with powerful regeneration powers and damage that gets bigger the more he attacks enemies) inhumane ways to its detriment. Weak and outnumbered, what the Slark should have done was hiding in the trees out of sight of humans and possibly teleporting away without attracting their attention or waiting for the rest of their team. Even low-level human players would have done well and ensured that they would not die and continue to accumulate gold to obtain their essential items. But the bot? It did not know how to stay silent. It would be safe to hide in the trees, but then it would consistently cover and be killed.
The three matches (and a fourth match against a low-level team) can be viewed here. The next venue for OpenAI Five will be the International in Vancouver later this month when the computer becomes a real professional team for its ultimate test.