Move 37

Game 2, alphago vs lee-sedol (March 2016): a fifth-line shoulder hit that commentators first called a mistake. AlphaGo's own policy network rated it at a 1 in 10,000 chance of a human playing it, so the system knew it was un-human and played it anyway because search and value said it was good. Lee left the room, came back and thought for more than two minutes; Fan Hui: "all the stones played before work together... like a network."

It's the canonical example of a learning system going "beyond its human guide": imitation gives the starting prior, and self-play reinforcement learning finds what humans never tried. andrej-karpathy uses it as the template for reasoning that emerges from RL in verifiable domains (rlhf-is-not-rl). The film's coda: "move 37 begat move 78 begat a new attitude": the machine's creativity pushed human creativity forward too.

Source: report