Developers from OpenAI and Uber have presented their AI system called Go-Explore in the specialist journal “Nature”, which masters Atari classic games such as Pitfall or Montezuma’s Revenge better than humans. Previous AI systems had difficulties with the corresponding type of game, as they lacked a clear reference point for evaluating a randomly executed move in the “reinforcement learning” method they used. Go-Explore does not explore the way through the game levels according to the pure random principle, but checks whether the procedure has really brought you closer to the goal.
With the Atari classic Montezuma’s Revenge, the AI managed to beat the existing world record, with Pitfall its performance at least exceeded that of an average player. The researchers speak of a breakthrough and assume that the inclusion of prior human knowledge (domain knowledge), for example in the field of robotics, could lead to progress. The writers have Go-Explore too made available via Github.
70 year old dream
Jan Peters, Professor of Intelligent Autonomous Systems at TU Darmstadt, sees it differently: “If we have good simulators and can clearly define both the problem and the situation, a problem in robotics can usually be solved easily.” he explains to the Science Media Center.
Nevertheless, there is still a big surprise for him in the details of the Nature article: “If Domain Knowledge is included, Go-Explore can beat the human world record. If this statement turns out to be generalizable, then this can be a change of time in AI, where the two AI families of statistical-neural processes and domain knowledge engineering finally unite. For many AI researchers this would be an almost 70 year old dream! ”