{
  "id": "alphazero",
  "title": "AlphaZero",
  "lang": "en",
  "source_lang": "en",
  "summary": "AlphaZero is a reinforcement-learning system that learned superhuman chess, shogi, and Go through self-play, combining neural networks with tree search.",
  "category": "computer-science",
  "subcategory": "artificial-intelligence",
  "type": "product",
  "aliases": [
    "Alpha Zero"
  ],
  "generated_at": "2026-11-03",
  "model": "pro/gpt-6.1-sol",
  "url": "https://aiwiki.page/en/alphazero/",
  "translations": {
    "zh-Hans": {
      "url": "https://aiwiki.page/zh/alphazero/",
      "json": "https://aiwiki.page/zh/alphazero.json"
    }
  },
  "sections": [
    {
      "heading": "",
      "text": "AlphaZero is a game-playing system developed by DeepMind that combines reinforcement learning, an artificial neural network, and search to learn strong play from self-generated experience. Introduced in December 2017 and described in an expanded paper in Science in December 2018, it achieved superhuman performance in chess, shogi, and Go without training on recorded human games. A separate model was trained for each game, using essentially the same learning and search framework.\n\nIts central innovation was not any single component, but a tightly integrated improvement cycle: a neural network guides search; search produces stronger move choices than the network alone; and those choices, together with game outcomes, become the network’s next training examples. AlphaZero demonstrated that this cycle could replace much of the handcrafted evaluation machinery traditionally used in strong board-game programs. Its significance lies both in its competitive results and in the evidence it provided for a reusable method of learning to plan in environments with precisely specified rules."
    },
    {
      "heading": "Origins and relationship to AlphaGo",
      "text": "AlphaZero emerged from a sequence of DeepMind systems developed for Go. The original AlphaGo, whose match against Lee Sedol in March 2016 attracted international attention, combined neural networks, search, human game records, and reinforcement learning. Its initial move-prediction network learned from expert games through supervised learning, while further training and self-play improved its decisions. Different networks supported move selection and position evaluation, and its search also used rapid simulated continuations.\n\nAlphaGo Zero, described in Nature in October 2017, removed the requirement for human game records. Starting with randomly initialized network parameters, it learned entirely through self-play. It also simplified the architecture by using a single network with two outputs: a policy over moves and a value estimate for the position. Search transformed these outputs into improved decisions, and the resulting games supplied the next round of training data. This established the main learning mechanism subsequently generalized in AlphaZero.\n\nThe December 2017 AlphaZero preprint, led by David Silver and colleagues, extended the approach beyond Go to chess and shogi. This mattered because the games differ substantially. Chess and shogi contain multiple piece types with asymmetric movement rules; shogi also permits captured pieces to return to the board. Go instead uses uniform stones on an intersection grid. A method that worked across all three therefore offered stronger evidence of algorithmic generality than another improvement confined to Go.\n\nThe expanded 2018 Science paper, titled “A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play,” reported additional evaluations and analysis. The preliminary and final publications should be distinguished: their match sizes, testing conditions, and reported scores were not identical. In particular, the widely circulated 100-game chess result belonged to the initial report, whereas the later publication included a 1,000-game main evaluation against Stockfish.\n\nAlphaZero was not a single network that switched freely among three games. Each game required its own training run, input representation, action encoding, and learned parameters. What transferred was the general algorithm and its principal design, not the contents of one trained model. Nor did AlphaZero discover the games’ rules. Legal moves, state transitions, and terminal outcomes were supplied by the environment. Its achievement was learning how to play within those rules without human examples of good play or a handcrafted positional scoring system."
    },
    {
      "heading": "Games, state representations, and prior knowledge",
      "text": "AlphaZero’s three principal environments are deterministic, alternating-turn, two-player games with perfect information. Both players can observe the relevant game state, and moves do not depend on hidden cards or random dice. These properties make it possible to construct a game tree in which branches represent legal actions and paths represent possible continuations. Nevertheless, exhaustive analysis is impractical because the trees are enormous.\n\nThe games impose different representational demands. Chess is played on an \\(8 \\times 8\\) board and includes castling, promotion, en passant, and drawing rules. Shogi uses a \\(9 \\times 9\\) board, promotion zones, and the ability to drop captured pieces back into play. Standard Go uses a \\(19 \\times 19\\) grid, with rules governing connected groups, liberties, captures, and repetition. An effective state description must preserve information relevant to legality and outcomes, not merely the visible arrangement of pieces or stones.\n\nAlphaZero represented positions as stacks of spatial feature planes. These encoded piece or stone locations, a history of recent positions, and game-dependent state information. In chess, for example, the representation included information relevant to castling and move-count rules. Shogi required additional features for pieces held in hand. Such inputs were supplied to the network as structured arrays rather than images photographed from a physical board.\n\nActions were also encoded in a fixed structured form. For chess, the published representation used \\(8 \\times 8 \\times 73\\) potential move entries, describing a starting square and a move type. Most entries are illegal in any particular position and are excluded when forming the move distribution used by search. Go actions correspond more directly to board locations, together with passing. These encodings allow a network with fixed-size outputs to represent a position-dependent set of legal choices.\n\nThe phrase “learning from scratch” therefore has a specific meaning. AlphaZero did not receive opening books, expert game collections, handcrafted piece values, or conventional positional evaluation terms. It did receive a rule implementation, a spatially organized representation, a network architecture, and an optimization procedure. These are substantial forms of prior structure, although they do not tell it which legal move is best.\n\nThis distinction separates learned strategic knowledge from engineered problem formulation. Knowing that a bishop moves diagonally belongs to the rules; assigning a numerical preference to a long diagonal belongs to evaluation. AlphaZero was given the former and learned the latter indirectly from competitive outcomes. Its results show the strength of that separation in these games, not the absence of all human design."
    },
    {
      "heading": "Neural network architecture and learned evaluation",
      "text": "AlphaZero used a deep convolutional neural network built around a residual architecture. Convolution applies learned filters across spatial locations, making it suitable for board positions in which similar local relationships recur in different places. Residual connections allow information and gradients to pass through many layers, facilitating the training of a substantially deeper model than a simple stack of isolated transformations.\n\nThe main network in the published system used a shared trunk of 19 residual blocks, with 256 filters in its principal convolutional layers. Two output branches, conventionally called heads, interpreted the resulting internal representation. The policy head predicted which moves were promising. The value head estimated the expected result from the perspective of the player to move.\n\nWriting the network as \\(f_\\theta\\), with trainable parameters \\(\\theta\\), its outputs can be expressed as\n\n\\[(\\mathbf{p},v)=f_\\theta(s),\\]\n\nwhere \\(s\\) is a game state, \\(\\mathbf{p}\\) is a distribution over encoded actions, and \\(v\\) is a scalar value. The value lies on a scale in which a win, draw, and loss correspond to \\(+1\\), \\(0\\), and \\(-1\\). It is not a material count or a prediction of the number of moves remaining.\n\nThe two outputs serve different computational purposes. The policy focuses search on a relatively small set of plausible moves rather than distributing effort uniformly across all legal actions. The value estimates the desirability of a newly encountered position without requiring a full game to be played from that position. Their shared internal representation allows features useful for move selection and outcome prediction to influence one another during training.\n\nThis differs from a traditional heuristic evaluation function assembled from explicitly named components such as material, pawn structure, mobility, and king safety. AlphaZero could represent relationships among these factors without exposing them as separately adjustable terms. A sacrificed pawn might receive a favorable evaluation because the resulting configuration predicts successful outcomes, even though no programmer had assigned a particular bonus to that sacrifice.\n\nThe value estimate remains conditional on what the system has learned and on the distribution of positions it encounters. It is not a proof that a position is objectively won or drawn under perfect play. Likewise, a high policy probability indicates a learned preference, not a guarantee of tactical correctness. Search is needed partly because direct neural-network inference can overlook critical continuations.\n\nThe architecture also imposed useful biases. Spatial convolutions favor the reuse of local patterns, while repeated layers combine those patterns into broader board relationships. These choices helped make learning feasible, but they also show why a common learning algorithm need not imply a completely task-independent representation."
    },
    {
      "heading": "Monte Carlo tree search and move selection",
      "text": "AlphaZero selected moves using Monte Carlo tree search, or MCTS, guided by its neural network. The search incrementally builds a tree rooted at the current position. Rather than examining every continuation to a uniform depth, it repeatedly chooses promising paths, evaluates newly reached positions, and updates statistics along the paths it has explored.\n\nEach explored state–action edge stores quantities including a visit count \\(N(s,a)\\), a mean action value \\(Q(s,a)\\), and a prior probability \\(P(s,a)\\) supplied by the network. A typical selection rule chooses the action maximizing\n\n\\[Q(s,a)+U(s,a),\\]\n\nwith an exploration term of the general form\n\n\\[U(s,a)\\propto P(s,a)\\frac{\\sqrt{\\sum_b N(s,b)}}{1+N(s,a)}.\\]\n\nThis balances moves that have produced favorable evaluations against moves whose prior probabilities warrant further investigation. The precise exploration schedule and constants are implementation details, but the central mechanism is the interaction of learned priors with accumulated search evidence.\n\nWhen a simulation reaches an unexpanded nonterminal position, the network evaluates it. Its policy initializes the new node’s move priors, and its value is propagated backward through the visited path. Because the players alternate, the sign of an evaluation must be interpreted consistently from each player’s perspective. If the search reaches an actual terminal state, the known game result replaces the network’s estimate.\n\nUnlike earlier forms of Monte Carlo game search, AlphaZero did not rely on completing every simulation through random moves until the end of the game. A neural value estimate could stop the simulation at a newly expanded position. This was especially important in chess and shogi, where largely random continuations are poor indicators of strategic merit and games may continue for many moves.\n\nAfter a fixed search budget, the root visit counts define an improved move distribution:\n\n\\[\\pi(a\\mid s)= \\frac{N(s,a)^{1/\\tau}} {\\sum_b N(s,b)^{1/\\tau}},\\]\n\nwhere \\(\\tau\\) controls how sharply the distribution concentrates on heavily visited actions. During training, sampling and added root noise encourage exploration. For competitive play, selection favors the move most strongly supported by the search.\n\nThis procedure differs from conventional minimax search with alpha–beta pruning, which dominated strong chess engines before neural evaluation became widespread. Alpha–beta methods eliminate branches that cannot affect the minimax decision under the current bounds. AlphaZero instead allocates simulations through learned priors and search statistics. Both use selective reasoning about possible futures, but their evaluation machinery and mechanisms for directing computational effort differ."
    },
    {
      "heading": "Self-play and the training objective",
      "text": "The learning process begins with a randomly initialized network, whose judgments are initially uninformative. Even at this stage, the rules ensure that generated actions are legal. Search, exploration, and repeated self-play gradually create examples from which the network can learn. No external teacher supplies an expert move for each position.\n\nFor every position encountered in a self-play game, the system records a training example containing the state \\(s\\), the search-derived move distribution \\(\\boldsymbol{\\pi}\\), and the eventual game result \\(z\\). The outcome is expressed from the perspective of the player associated with that state. A victory by White, for example, supplies positive labels for White-to-move positions and negative labels for Black-to-move positions from the same game.\n\nThe network is trained with a combined loss function of the form\n\n\\[\\ell(\\theta)= (z-v)^2 -\\boldsymbol{\\pi}^{\\mathsf T}\\log\\mathbf{p} +c\\|\\theta\\|_2^2.\\]\n\nThe first term trains the value output to predict the result. The second trains the policy to match the improved move distribution produced by search. The last is a regularization term that penalizes large parameter values. Gradients calculated through backpropagation update the shared network and both output heads.\n\nThe policy target is particularly important. AlphaZero does not merely reinforce whichever move happened to be selected. It learns from the distribution of search effort across candidate moves. If search substantially revises the network’s initial preferences, the network is trained toward that revised judgment. In this sense, search acts as a repeatedly generated policy-improvement operator.\n\nAt the same time, the value target comes from actual completed games rather than directly copying the previous value estimate. This anchors learning to outcomes. The network nevertheless learns under a changing distribution: as its policy improves, it generates different openings, middlegames, and endings. Its training examples are therefore endogenous products of its own development rather than samples from a fixed human dataset.\n\nThe resulting feedback loop can be summarized as network-guided search, self-play, outcome collection, and parameter updates. It resembles generalized policy iteration: evaluation and improvement alternate, but neither is carried out exactly. Both depend on finite computation and function approximation.\n\nSelf-play also creates difficulties. Two instances of the current system may share weaknesses, and exploratory choices influence which positions are encountered. Randomness at the root and diversity across many generated games help broaden experience, but they do not establish that every strategically important possibility has been explored. The practical success of the method depends on the interaction among search quality, network capacity, optimization, and the scale of self-generated data."
    },
    {
      "heading": "Computational scale and experimental results",
      "text": "AlphaZero’s results required substantial computation. The published training method used large-scale self-play distributed across thousands of tensor processing units, or TPUs, alongside separate accelerators for neural-network optimization. The reported setup used 5,000 first-generation TPUs for self-play and 64 second-generation TPUs for training. These figures describe the training infrastructure, not the resources required by every individual move during evaluation.\n\nThe reported training schedule comprised 700,000 update steps with minibatches of 4,096 positions. Producing these examples involved many simultaneously generated games. The wall-clock cost depended on the game, network evaluation throughput, search settings, and hardware arrangement. Consequently, a statement that a system learned in a particular number of hours is incomplete unless it also specifies the computational resources used.\n\nThe preliminary December 2017 chess experiment attracted attention for a 100-game match against Stockfish 8. AlphaZero won 28 games, drew 72, and lost none. That result belonged to a particular testing setup, including its time control and hardware allocation; it was not a statement that AlphaZero would never lose to Stockfish under other conditions.\n\nThe expanded 2018 publication reported a 1,000-game main chess match against Stockfish 8, with AlphaZero scoring 155 wins, 839 draws, and 6 losses. The paper also examined alternative conditions, including opening positions and resource variations. In shogi, AlphaZero was evaluated against the strong engine Elmo. In Go, its principal comparison was with an AlphaGo Zero reference system. It achieved winning results across all three game families.\n\nThe chess comparison illustrated a striking difference in search organization. The published account reported approximately 80,000 positions searched per second by AlphaZero versus roughly 70 million by Stockfish under the stated conditions. These are not directly interchangeable units of work: AlphaZero’s neural evaluation of a position is far more computationally substantial than a conventional chess engine’s evaluation of a typical search node. The comparison shows that examining more positions is not the only route to stronger play, rather than establishing a hardware-independent efficiency ratio.\n\nTraining cost and playing cost must also be separated. A trained model can be reused for many games, whereas generating it requires a large initial investment. Competitive performance therefore answers a different question from whether the method is economical to reproduce on ordinary hardware.\n\nThe experiments demonstrated strength against specified contemporary opponents, not a permanently fixed ranking among all future engines. Engine software, neural architectures, hardware, time controls, and tournament procedures continued to change after publication. AlphaZero’s lasting experimental contribution was the successful application of one broad learning framework across several demanding games."
    },
    {
      "heading": "Learned strategy and the interpretation of play",
      "text": "AlphaZero’s published chess games became notable for sacrifices, long-term pressure, and a willingness to accept material deficits in exchange for activity or positional constraints. Commentators highlighted attacks built from restricted enemy pieces, vulnerable kings, and advantages that developed over many moves. Such games were striking because the system had not been given a human-written hierarchy of strategic principles.\n\nIts behavior can be explained without attributing human motives or aesthetic preferences to it. A move that sacrifices material can be favored when the network and search judge the resulting position more likely to produce a good outcome. Because the evaluation function is learned, it need not begin with the conventional assumption that each piece has a fixed baseline value adjusted by a limited collection of positional bonuses.\n\nSearch remains essential to this interpretation. A network can recognize a promising attacking configuration, but concrete analysis must still assess whether a defensive resource refutes it. Conversely, search may discover a sequence whose final position looks favorable to the value network even though a shallow material-based evaluation would reject the initial move. The observed style therefore results from the combination of learned representation and selective calculation, not from the neural network alone.\n\nDescriptions such as “creative,” “intuitive,” or “human-like” are analogies for observable decisions. They do not establish that AlphaZero experienced intentions or understood a sacrifice in the same way as a player explaining it verbally. Nor does a compelling annotated game reveal exactly which internal features caused the decision. The distinction is relevant to explainable artificial intelligence: a plausible strategic explanation can be useful without being a faithful account of the model’s internal computation.\n\nLater research analyzed how chess concepts developed during AlphaZero training. A 2022 study by Thomas McGrath and colleagues investigated the acquisition of human-associated concepts and changes in opening preferences. Such work used analysis of internal representations and behavior to connect learning stages with recognizable chess ideas. It provided evidence that structured strategic information can emerge from outcome-driven learning, while leaving open how completely particular probes capture the network’s representations.\n\nAlphaZero was also used in research on chess variants, including changes to rules governing castling, pawn movement, and other aspects of play. Training strong agents under modified rules enabled researchers to examine how those changes affected patterns of play without waiting for large human tournament records. These experiments extended its role beyond defeating existing engines: the system could act as an analytical instrument for exploring how a game’s rules shape its strategic possibilities."
    },
    {
      "heading": "Limitations, generality, and scientific interpretation",
      "text": "AlphaZero’s three-game success was a substantial demonstration of generality within a restricted class of problems. All three environments have explicit rules, observable states, discrete actions, and an unambiguous competitive objective. These shared properties provide unusually favorable conditions for simulation and learning. The method’s success does not imply that the same implementation can be applied unchanged to every decision problem.\n\nAn exact simulator is especially important. AlphaZero can explore a hypothetical move because the environment supplies the resulting state and identifies whether subsequent actions are legal. In physical control, economic decision-making, or scientific experimentation, transitions may be uncertain, expensive, or only partially understood. Simulated experience may also differ from real experience, creating problems of distribution shift that do not arise in the same form on a precisely implemented chessboard.\n\nThe reward structure is another advantage. Winning, drawing, and losing are clearly defined. Many real-world tasks involve several competing objectives, delayed consequences, or outcomes that cannot be reduced to a single agreed score. AlphaZero demonstrates effective optimization of a specified objective; it does not solve the separate problem of deciding what objective should be optimized.\n\nIts learned models were also game-specific. Training a chess network did not automatically provide a shogi network, even though the learning machinery was shared. Thus AlphaZero should be distinguished from transfer learning across tasks and from evidence of artificial general intelligence. Reusing a successful algorithm is a weaker and more precise claim than possessing broadly transferable knowledge.\n\nCompetitive strength likewise differs from mathematical certainty. AlphaZero did not solve chess, shogi, or Go in the game-theoretic sense. It did not prove the result of the initial position under perfect play, and it did not provide exhaustive correctness guarantees for its moves. Search and learned evaluation are approximate, leaving room for tactical errors, misjudged positions, or weaknesses outside frequently encountered self-play distributions.\n\nReproducibility also requires distinguishing the published method from the exact original system. DeepMind described the architecture, training procedure, and experiments, but independent projects did not simply receive the complete original training infrastructure and all trained artifacts. Reproducing the approach involved engineering choices, computational budgets, and implementation differences.\n\nFinally, “tabula rasa” should not be interpreted literally as an absence of prior assumptions. Rules, encodings, convolutional structure, search, optimization, and exploration were all deliberately chosen. The scientifically important result is narrower and stronger than a claim of knowledge-free intelligence: extensive expert-designed evaluation knowledge could be replaced by a general learning-and-search procedure when paired with sufficient computation and a suitable environment."
    },
    {
      "heading": "Influence on later systems and game-playing research",
      "text": "AlphaZero helped establish neural-network-guided search as a central approach in modern board-game artificial intelligence. Its influence was methodological rather than confined to a single record-setting program: learned policies guide where to search, learned values evaluate frontier positions, and search supplies improved targets for further learning. This pattern connects representation learning with explicit planning.\n\nLeela Chess Zero, launched in 2018, became an important independent, open-source implementation inspired by AlphaZero’s approach. It used community-contributed computation to generate self-play games and train chess networks. The project showed that related techniques could be developed outside the original industrial infrastructure, although its software, networks, training history, and hardware arrangements were not identical to AlphaZero’s.\n\nKataGo developed the neural-search approach further for Go, adding training techniques and auxiliary objectives that improved practical learning efficiency and analysis capabilities. Such systems illustrate that AlphaZero was not an endpoint. Subsequent work changed architectures, targets, exploration methods, and computational organization while retaining the interaction between learned evaluation and search.\n\nThe broader chess-engine ecosystem also adopted neural evaluation through approaches that differ from AlphaZero. In particular, efficiently updatable neural networks, commonly called NNUE, became important in conventional engines including Stockfish. NNUE-based engines typically retain highly optimized alpha–beta search and use neural networks designed for economical incremental evaluation. Their success shows that the historical contrast is not simply “neural networks versus traditional search”: neural evaluation can be combined with several different search architectures.\n\nDeepMind’s MuZero, introduced in 2019, addressed another limitation by learning a model suitable for planning rather than requiring search to use the environment’s exact transition rules directly. MuZero learned representations and predictions of rewards, policies, and values, and was demonstrated on board games and Atari tasks. It extended the research direction toward learned dynamics, but it did not eliminate the need for environment interaction or make arbitrary real-world simulation straightforward.\n\nAlphaZero also contributed to research on the relationship between training-time learning and decision-time computation. A trained policy can make a rapid direct prediction, while additional search can improve that decision at extra computational cost. Studying how these resources interact offers a concrete setting for investigating when better representations, more training examples, or deeper planning provide the greatest benefit.\n\nThe enduring technical distinction is between learning a policy and learning a system that can improve its policy through planning. AlphaZero made search part of both action selection and the production of training targets. Later systems continue to investigate that interaction under different constraints: cheaper inference, imperfect models, hidden information, broader action spaces, and environments where generating experience is substantially more difficult than playing another simulated game."
    }
  ],
  "links": [
    "deepmind",
    "reinforcement-learning",
    "artificial-neural-network",
    "chess",
    "go-game",
    "alphago",
    "supervised-learning",
    "training-data",
    "algorithm",
    "game-tree",
    "convolutional-neural-network",
    "heuristic-evaluation-function",
    "neural-network-inference",
    "monte-carlo-tree-search",
    "minimax",
    "alpha-beta-pruning",
    "loss-function",
    "regularization",
    "backpropagation",
    "tensor-processing-unit",
    "explainable-artificial-intelligence",
    "distribution-shift",
    "transfer-learning",
    "artificial-general-intelligence",
    "artificial-intelligence",
    "leela-chess-zero",
    "open-source-software"
  ],
  "sources": [],
  "license": "https://creativecommons.org/licenses/by/4.0/"
}