Computers are becoming more and more complex when it comes to understanding and playing complicated games. DeepMind, one of the leading manufacturers of artificial intelligence, has this again today with its latest A.I. Agent named AlphaStar. During a livestream, this program recorded two StarCraft II pros in a series of five games each, and AlphaStar swept all 10 games.
StarCraft II pros Dario "TLO" Wünsch and Greegorz "MaNa" Komincz are two of the first players in the world. But even with the neural network AlphaStar no problem is possible. Blizzard opened StarCraft II for A.I. Researchers in the last year, and that has led to huge leaps in computer performance.
DeepMind already knows the game of chess and goes with AlphaZero or AlphaGo. And these games are so complicated that no computer on earth could brutally calculate every possible game in these games. However, a real-time strategy game like StarCraft II is exponentially more complicated in terms of what's possible at any given moment. And this reveals the power of deep learning. Something like AlphaStar does not have to learn every possible game in StarCraft to understand it. Instead, she focuses on winning strategies.
How AlphaStar Learns
The reason why AlphaStar is such a big deal lies in the way it learns. It uses several techniques, and DeepMind has worked through its operation.
"We take many repetitions of professionals and players and try to understand AlphaStar by looking at a situation where a human player is." DeepMind research co-lead Oriol Vinyals said. "And then we try to mimic those moves."
DeepMind does not just use professional games, either. The company also considers public games by players with a high matchmaking rating.
However, Imitation Training only creates the most basic iteration of AlphaStar. DeepMind says that this version 0.1
To prepare AlphaStar for a pro fight, DeepMind needed to use his Neural Network training.
The AlphaStar League
How do you get that? a little better? Study and practice. AlphaStart has nailed the session with imitation learning. For practice, however, DeepMind has set up the so-called AlphaStar League. This is a neural network training program where different versions of AlphaStar play together for a week.
This is the heart of modern machine learning. DeepMind sets success parameters for the A.I. Programs like "win the game". And then every A.I. Agent, as they are called, makes decisions to achieve this goal. Then the A.I. wins will continue in the AlphaStar League.
But the training goes deeper. DeepMind also increases the possibility for mutations from one generation of AlphaStar to the next by setting certain agents to try to win while, for example, favoring a particular type of unit.
DeepMind uses its AlphaStar agents to both randomly and mutate properties of the agents that gain the most. This process works so well because the A.I. is able to play many games in quick succession. After one or two training weeks, AlphaStar has played 200 years of StarCraft II.
But is not the computer cheating?
DeepMind knew that some StarCraft players are skeptical of a computer-controlled opponent. It got StarCraft experts to talk about the games and ask the questions that the community wants answers for. These experts focused on how AlphaStar actually plays and perceives the game. Can see it, for example, through the fog of war that acts like a veil to human players. Or is it just a thousand times faster spamming of keystrokes than human hands could physically move?
But DeepMind said it was trying to keep things level. This limits the Alpha-Star Actions (APM) per minute to ensure that the computer does not win out of pure speed.
"Oveall, AlphaStar uses far fewer APMs than a human professional," said David Silver, co-lead of DeepMind. "That's a sign that it's not amazing clicking, but something cleverer."
AlphaStar also has no superhuman response time.
"We measured how fast it reacts to things," said Silver. "When you measure the time between which AlphaStar perceives the game. From the moment it observes what happens, it must be processed and then re-tell the game what it chooses. This time is actually closer to 350ms. That's on the slow side of human players. "
Finally, DeepMind explained how AlphaStar visualizes the game world. It does not look at the code, but does not move the camera like a human player. Instead, it looks at the map that was zoomed out completely, but it can not see through the fog of war or anything like that. It can only see parts of the map that contain units. However, DeepMind says that AlphaStar still shares its attention in the same way as a human player.
AlphaStar lost a match
The livestream focused mainly on the five games, the AlphaStar against TLO and played MaNa a few weeks ago. But DeepMind has allowed MaNa to see a rematch live in front of the viewers on YouTube and Twitch. And then MaNa won revenge with a victory against the machine.
But the live game of MaNa against AlphaStar had some variations compared to their last game. DeepMind has used a new prototype version of AlphaStar that uses exactly the same camera view as the players. This means that AlphaStar can not only sit in a zoomed out perspective, but must approach the action to see the details of the fight.
This version of AlphaStar did not have much time to train. Instead of going through 200 years of AlphaStar League, it played a little closer than 20 years. But despite this "limited" experience, she still showed strategies that shocked all viewers.
"The way AlphaStar played the matchup was not something I had experience with," MaNa said. "It was a different kind of StarCraft. It was a great opportunity to learn something new from an A.I. "
And that's one of the things DeepMind is most proud of. That a professional player against a computer can take new strategy ideas, what nobody would have thought possible before.
"At the end of the day against A.I. is great, "said Vinyals. "However, due to the way we train AlphaStar, some of the movements – such as oversaturated probes – could possibly challenge some of the wisdom that has spread among the best players."