AI learns curiosity during OpenAI experiment, gets hooked on TV
A team of researchers has discovered that when an artificial intelligence (AI) algorithm is given a basic definition of curiosity, it can successfully explore and even conquer over 50 video games without human input. However, this curiosity has its drawbacks, as the AI agent can sometimes become sidetracked by new experiences, such as watching TV or encountering the Game Over screen. To note, the research team had members from OpenAI, UC Berkeley, and the University of Edinburgh.
Defining artificial curiosity
OpenAI's researchers described artificial curiosity as an algorithm that attempts to predict what its surroundings will look like one frame into the future. When the next frame appears, the algorithm is rewarded based on how inaccurate its prediction was. This method led to AI agents excelling in games like Super Mario, which focus on exploration and progressing to the next level.
Noisy TV problem
OpenAI researcher Harri Edwards shared that the inspiration for allowing the AI agent to change channels came from a thought experiment called the noisy TV problem. The random static on a TV screen makes it nearly impossible for a curious AI agent to predict what will happen next, causing it to become engrossed in watching TV indefinitely. In real-life situations, this can be compared to entirely random events, such as light reflecting off a waterfall.
How was the theory tested?
The research team tested its theory by putting a digital TV in a 3D environment. The AI agent was then permitted to press a button to change channels. When the agent began flipping through channels, it could not resist the stream of new pictures on the TV. Edwards claimed there were situations when the AI could take its eyes off the TV only when the "AI's surroundings somehow seemed more interesting than the next thing on TV."
Beyond video games
The goal of this research extends beyond simply defeating video games with AI. It also aims to comprehend how algorithms can better understand the world surrounding them. Since these algorithms demonstrated effectiveness in exploring all aspects of video games, researchers believe they could be adapted for other tasks, like debugging code or playing through a game to ensure no glitches.
