{
  "id": "alphago-zero",
  "title": "AlphaGo Zero",
  "lang": "en",
  "source_lang": "en",
  "summary": "AlphaGo Zero is a Go-playing system developed by DeepMind that achieved superhuman performance through reinforcement learning from self-play, without training on human games.",
  "category": "computer-science",
  "subcategory": "artificial-intelligence",
  "type": "product",
  "aliases": [],
  "generated_at": "2026-11-03",
  "model": "pro/gpt-6.1-sol",
  "url": "https://aiwiki.page/en/alphago-zero/",
  "translations": {
    "zh-Hans": {
      "url": "https://aiwiki.page/zh/alphago-zero/",
      "json": "https://aiwiki.page/zh/alphago-zero.json"
    }
  },
  "sections": [
    {
      "heading": "",
      "text": "AlphaGo Zero is a artificial intelligence system developed by DeepMind to play the board game Go. Introduced in October 2017 in the Nature paper “Mastering the game of Go without human knowledge,” it learned through reinforcement learning from games played against itself, rather than from examples of human play. It combined a neural network with tree search and surpassed earlier versions of AlphaGo. The name “Zero” refers to the absence of human game records and handcrafted strategic knowledge in its training, not to an absence of human-designed rules, architecture, or algorithms."
    },
    {
      "heading": "Background and development",
      "text": "Earlier AlphaGo systems combined several learning and search techniques. The version that defeated Lee Sedol in 2016 initially learned to predict expert moves using supervised learning on human games. Reinforcement learning subsequently improved its play, while separate policy and value networks supported search. Its search also used fast simulations, known as rollouts, to help assess positions.\n\nAlphaGo Zero removed the human-game training stage, combined policy and value prediction in one network, and eliminated rollouts. Starting with randomly initialized network parameters, it generated its own training data through self-play.\n\nThe 2017 paper’s authors included David Silver, Julian Schrittwieser, Karen Simonyan, and Demis Hassabis. Its central result was that learning without human demonstrations could exceed the performance of systems whose training began with expert examples."
    },
    {
      "heading": "Learning architecture",
      "text": "Board representation and neural network\n\nAlphaGo Zero uses a deep residual neural network, a form of convolutional neural network with connections that help train many layers.\n\nThe input describes the board position, including recent board history and the player to move. Unlike earlier AlphaGo versions, it does not include handcrafted strategic features such as liberties or specialized tactical patterns. It nevertheless uses a representation designed specifically for Go.\n\nFor a position \\(s\\), the network produces two outputs:\n\n\\[(p,v)=f_\\theta(s).\\]\n\nHere:\n\n- \\(\\theta\\) represents the network’s learned parameters.\n- \\(p\\) is a probability distribution over moves, called the policy output.\n- \\(v\\) estimates the eventual game outcome from the current player’s perspective.\n\nThe policy guides search toward promising moves, while the value estimate assesses positions without requiring every explored line to be played to completion.\n\nSearch-guided self-play\n\nTo choose moves, AlphaGo Zero runs Monte Carlo tree search using its current network. The search builds a partial game tree, repeatedly selecting moves, expanding positions, evaluating them with the network, and propagating the resulting estimates back through the tree.\n\nSelection balances two objectives: exploring moves that the network considers promising and exploiting moves that have produced favorable evaluations during search. Despite the method’s name, AlphaGo Zero does not use the fast random rollouts employed by earlier AlphaGo systems.\n\nSearch produces a move distribution \\(\\pi\\), derived from how often different moves were visited. This distribution generally provides a stronger training target than the network’s unaided policy prediction.\n\nDuring self-play, exploration is encouraged by adding noise to the root’s move probabilities. Move selection also varies with the stage of the game, allowing diversity in training games while making later play more decisive.\n\nTraining objective\n\nEach completed self-play game supplies training examples consisting of:\n\n- a board position \\(s\\);\n- the search-derived move distribution \\(\\pi\\);\n- the final game result \\(z\\), expressed from the perspective of the player to move at that position.\n\nThe network is trained with a combined loss function:\n\n\\[\\ell(\\theta) = (z-v)^2 - \\pi^{\\mathsf T}\\log p + c\\lVert\\theta\\rVert^2.\\]\n\nThe first term trains outcome prediction. The second trains the policy to match the search distribution. The final term applies regularization to the network weights, with \\(c\\) controlling its strength.\n\nParameters are updated using gradient-based optimization and backpropagation. The policy-learning component uses targets generated by search, rather than human move labels; the overall learning process is reinforcement learning because the system improves through interaction with the game and its eventual outcomes."
    },
    {
      "heading": "The improvement cycle",
      "text": "AlphaGo Zero alternates between generating experience and learning from it:\n\n- The current best network guides self-play searches.\n- Completed games provide positions, search distributions, and outcomes.\n- Training produces candidate networks.\n- Candidates are evaluated against the current best network.\n- A sufficiently stronger candidate becomes the network used to generate further games.\n\nIn the published procedure, a candidate replaced the incumbent after winning more than 55% of their evaluation games.\n\nThis creates a feedback loop: better networks guide better searches, better searches produce stronger policy targets, and training on those targets improves the network. Terminal game outcomes simultaneously train the value output.\n\nThe process is closely related to policy iteration: search acts as a policy-improvement operator, while network training approximates the improved policy and its value function."
    },
    {
      "heading": "Reported performance",
      "text": "The original publication described multiple training runs rather than a single fixed model.\n\nTraining run | Reported result\nThree-day run | Generated approximately 4.9 million self-play games and defeated AlphaGo Lee, the version associated with the Lee Sedol match, by 100 games to 0\nForty-day run | Generated approximately 29 million self-play games and defeated AlphaGo Master by 89 games to 11\n\nAlphaGo Master was a stronger predecessor than AlphaGo Lee. The two results therefore concern different opponents and different stages of AlphaGo Zero’s development.\n\nThe three-day and forty-day experiments also used different network sizes. Their results should not be interpreted as points on one otherwise unchanged training run.\n\nThe publication reported match play using a single machine with four tensor processing units. That configuration does not represent the total resources required to generate self-play data and train the networks. Playing hardware, training hardware, and cumulative computation are distinct measures."
    },
    {
      "heading": "What “without human knowledge” means",
      "text": "AlphaGo Zero did not learn from human game records, human opening books, or manually specified Go strategy. Nevertheless, it depended on substantial human-designed structure:\n\n- the rules and legal moves of Go;\n- a board representation and neural-network architecture;\n- a search procedure;\n- a training objective and optimization process;\n- exploration settings and model-evaluation criteria;\n- board symmetries used in processing training examples.\n\nThus, the system demonstrated learning without human examples of expert play, not learning without prior assumptions. Its architecture and procedures supplied inductive biases that made the learning problem tractable.\n\nIts network was randomly initialized, but its environment and learning machinery were not discovered from scratch."
    },
    {
      "heading": "Relationship to AlphaZero",
      "text": "AlphaZero, introduced later in 2017, extended the central approach to chess and shogi as well as Go. AlphaGo Zero was specifically designed and evaluated for Go; AlphaZero adapted the framework to several games with different rules and action spaces.\n\nThe systems also differed in training details. For example, AlphaZero used a continuously updated network for self-play rather than retaining AlphaGo Zero’s evaluation gate for replacing the best network. The names are therefore not interchangeable."
    },
    {
      "heading": "Significance and limitations",
      "text": "AlphaGo Zero established a prominent example of expert-level behavior emerging from self-generated experience rather than imitation of human demonstrations. It also showed how a learned policy, a learned value function, and explicit search could reinforce one another within a single system.\n\nIts success, however, depended on favorable properties of the task. Go has fully observable states, precisely specified legal actions, deterministic transitions, and an unambiguous terminal result. Self-play can generate large numbers of valid interactions without the costs or risks associated with physical experimentation.\n\nThose conditions do not hold in many other applications. Real-world environments may have incomplete observations, uncertain dynamics, expensive actions, or objectives that cannot be reduced to winning a game. Applying the same approach elsewhere requires addressing those differences.\n\nAlphaGo Zero’s results also do not establish that human-generated data is generally unnecessary. They demonstrate that, in a domain with a reliable simulator and a clear objective, sufficiently resourced self-play can outperform a particular approach built partly on human demonstrations. Nor do its match victories constitute a mathematical solution of Go: the system learned an exceptionally strong playing strategy, not a proof of optimal play from every position."
    }
  ],
  "links": [
    "artificial-intelligence",
    "deepmind",
    "reinforcement-learning",
    "alphago",
    "supervised-learning",
    "training-data",
    "residual-neural-network",
    "convolutional-neural-network",
    "probability-distribution",
    "monte-carlo-tree-search",
    "game-tree",
    "loss-function",
    "regularization",
    "backpropagation",
    "tensor-processing-unit",
    "inductive-bias",
    "alphazero"
  ],
  "sources": [],
  "license": "https://creativecommons.org/licenses/by/4.0/"
}