{
  "id": "belief-propagation",
  "title": "Belief propagation",
  "lang": "en",
  "source_lang": "en",
  "summary": "Belief propagation computes marginal probabilities or optimal configurations in graphical models by exchanging local messages, exactly on trees and approximately on graphs with cycles.",
  "category": "computer-science",
  "subcategory": "artificial-intelligence",
  "type": "concept",
  "aliases": [
    "Sum–product algorithm",
    "Belief-propagation algorithm"
  ],
  "generated_at": "2026-11-05",
  "model": "pro/gpt-6.1-sol",
  "url": "https://aiwiki.page/en/belief-propagation/",
  "translations": {
    "zh-Hans": {
      "url": "https://aiwiki.page/zh/belief-propagation/",
      "json": "https://aiwiki.page/zh/belief-propagation.json"
    }
  },
  "sections": [
    {
      "heading": "",
      "text": "Belief propagation is a family of message-passing algorithms for inference in probabilistic graphical models. It computes information about a joint probability distribution by passing functions, called messages, between neighboring parts of a graph. Its standard sum–product form calculates marginal probabilities: the distribution of one variable, or a small group of variables, after the others have been summed out. Related max–product and min–sum forms seek the most probable or lowest-cost joint configuration.\n\nThe central idea is to exploit factorization. A distribution involving many variables may be represented by small functions, each involving only a few variables. Rather than enumerate every possible joint assignment, belief propagation combines these local functions and reuses intermediate results. On a tree-structured graph, it produces exact answers after a finite exchange of messages. On a graph containing cycles, the same local rules define loopy belief propagation, an iterative approximation that can work remarkably well but is not guaranteed to converge or to return exact probabilities.\n\nBelief propagation connects artificial intelligence, statistics, communication theory, and statistical physics. It provides a common language for algorithms used in error-correcting codes, sequence models, image analysis, and distributed estimation. Its mathematical foundations also clarify the distinction between exact inference, approximate local consistency, and global optimization."
    },
    {
      "heading": "Historical development and unification",
      "text": "Belief propagation emerged from several traditions that initially used different terminology. In communication theory, Robert Gallager’s 1962 paper on low-density parity-check codes described iterative probabilistic decoding on sparse parity constraints. His 1963 monograph developed the subject further. These decoding procedures exchanged local information between bits and checks, anticipating the factor-graph interpretation that later became standard.\n\nIn artificial intelligence, Judea Pearl developed local probability-propagation methods for singly connected belief networks during the 1980s. His 1982 work on distributed probabilistic reasoning and his 1988 book Probabilistic Reasoning in Intelligent Systems helped establish belief propagation as a general inference method. A major concern was how a network representing uncertain dependencies could update its beliefs when observations arrived without repeatedly recomputing an entire joint probability table.\n\nRelated algorithms were already established in other settings. The forward–backward algorithm for hidden Markov models calculates state probabilities from sequential observations. The Viterbi algorithm finds a most probable state sequence. The BCJR algorithm, introduced by L. R. Bahl, J. Cocke, F. Jelinek, and J. Raviv in 1974, computes posterior probabilities for decoding convolutional codes. These procedures differ in purpose and algebra, but all exploit a decomposition into local computations along a graph.\n\nR. Michael Tanner’s 1981 graphical formulation of codes supplied an influential bipartite representation linking symbols to constraints. The introduction of turbo codes by Claude Berrou, Alain Glavieux, and Punya Thitimajshima in 1993, followed by renewed interest in low-density parity-check codes, demonstrated the practical power of iterative information exchange on graphs with cycles. Successful decoding did not require the underlying graph to be a tree.\n\nFrank Kschischang, Brendan Frey, and Hans-Andrea Loeliger’s 2001 paper “Factor Graphs and the Sum-Product Algorithm” presented a particularly broad unification. It showed how many familiar inference and signal-processing algorithms could be viewed as instances of one local computational rule. Work by Jonathan Yedidia, William Freeman, and Yair Weiss around the same period connected loopy belief propagation to approximate free-energy optimization. That connection explained why converged messages can satisfy meaningful variational equations even when they do not give exact marginals.\n\nHistorically, belief propagation is therefore not just a single algorithm invented for one task. It is also a recognition that decoding, probabilistic reasoning, and certain forms of dynamic programming share a common computational structure."
    },
    {
      "heading": "Factorized distributions and graphical representations",
      "text": "Let \\(x=(x_1,\\ldots,x_n)\\) denote a collection of discrete random variables. A factorized distribution can be written as\n\n\\[p(x)=\\frac{1}{Z}\\prod_{a\\in F}\\psi_a(x_{\\partial a}), \\qquad Z=\\sum_x\\prod_{a\\in F}\\psi_a(x_{\\partial a}).\\]\n\nHere \\(F\\) indexes factors, \\(\\partial a\\) is the set of variables involved in factor \\(a\\), and each \\(\\psi_a\\) is nonnegative. The partition function \\(Z\\) normalizes the product. It is assumed to be finite and positive. A factor may express a probabilistic dependence, an observation likelihood, or a hard constraint that is zero for forbidden assignments.\n\nA factor graph represents this expression as a bipartite graph. One class of nodes represents variables; the other represents factors. An edge connects variable \\(i\\) to factor \\(a\\) exactly when \\(x_i\\) appears in \\(\\psi_a\\). The graph records the scopes of local functions rather than a direction of causal influence. The same distribution may admit several factorizations, and the choice can materially affect the cost and accuracy of message passing.\n\nFor example, a chain with unary evidence terms and pairwise interactions has the form\n\n\\[p(x_1,\\ldots,x_n) \\propto \\prod_{i=1}^{n}\\phi_i(x_i) \\prod_{i=1}^{n-1}\\psi_{i,i+1}(x_i,x_{i+1}).\\]\n\nThe unary factor \\(\\phi_i\\) can encode an observation at position \\(i\\), while the pairwise factor describes the compatibility of adjacent states. Such chains occur in sequence labeling and state estimation. A two-dimensional image model has a similar local structure but usually contains many cycles.\n\nA Bayesian network supplies a different representation: a directed acyclic graph with one conditional probability factor for each variable given its parents. It can be converted to a factor graph by treating those conditional distributions as factors. A Markov random field instead typically expresses an undirected model through clique potentials. Belief propagation operates on the resulting factorization, so its essential requirements are not tied to either directed or undirected notation.\n\nObservations are commonly incorporated by fixing variables or adding likelihood factors. An observed discrete value \\(x_i=e_i\\), for example, can be represented by an indicator factor that equals one at \\(e_i\\) and zero elsewhere. More uncertain measurements use nontrivial likelihood functions.\n\nThe principal inference targets must be distinguished. Marginal inference asks for distributions such as \\(p(x_i)\\). Most-probable-explanation inference asks for a joint maximizer of \\(p(x)\\). Computing \\(Z\\) is a further task. These quantities are related, but a method that estimates one well need not estimate the others well."
    },
    {
      "heading": "Sum–product messages and local beliefs",
      "text": "For each variable–factor edge, sum–product belief propagation maintains two messages. The variable-to-factor message combines information arriving from the variable’s other neighboring factors:\n\n\\[m_{i\\to a}(x_i) = \\prod_{b\\in\\partial i\\setminus\\{a\\}} m_{b\\to i}(x_i).\\]\n\nThe factor-to-variable message combines the factor with information from its other variables and sums those variables out:\n\n\\[m_{a\\to i}(x_i) = \\sum_{x_{\\partial a\\setminus\\{i\\}}} \\psi_a(x_{\\partial a}) \\prod_{j\\in\\partial a\\setminus\\{i\\}} m_{j\\to a}(x_j).\\]\n\nThe notation \\(\\partial i\\) denotes the factors adjacent to variable \\(i\\). Unary evidence can be represented as an ordinary factor, making these equations sufficient without a separate evidence term.\n\nThe exclusion of the recipient is essential. A message from \\(i\\) to \\(a\\) does not use the message from \\(a\\) to \\(i\\) in the same update. On a tree, this ensures that each message summarizes information from one side of an edge without immediately sending the recipient’s information back to it. In physical terminology, such an exclusion is related to a cavity construction: the interaction with the recipient is temporarily removed.\n\nA variable belief is obtained by combining all incoming factor messages:\n\n\\[b_i(x_i) = \\frac{1}{Z_i} \\prod_{a\\in\\partial i}m_{a\\to i}(x_i).\\]\n\nSimilarly, a factor belief is\n\n\\[b_a(x_{\\partial a}) = \\frac{1}{Z_a} \\psi_a(x_{\\partial a}) \\prod_{i\\in\\partial a}m_{i\\to a}(x_i).\\]\n\nThe constants \\(Z_i\\) and \\(Z_a\\) normalize these functions. On a tree, the resulting beliefs are exact marginals. On a loopy graph, the word belief deliberately allows for their approximate status.\n\nAn individual message is not generally a posterior probability. Its scale is arbitrary: multiplying a message by a positive constant independent of its argument does not alter the normalized beliefs obtained from the corresponding updates. Messages are therefore often normalized to sum to one for numerical convenience. Such normalization must be accounted for if the computation also seeks the partition function.\n\nA binary example illustrates the factor update. Suppose \\(x,y\\in\\{0,1\\}\\), and a factor has value \\(3\\) when \\(x=y\\) and \\(1\\) otherwise. If the incoming message for \\(y\\) is \\((0.8,0.2)\\), the outgoing message to \\(x\\) is\n\n\\[m_{a\\to x}(0)=3(0.8)+1(0.2)=2.6, \\qquad m_{a\\to x}(1)=1(0.8)+3(0.2)=1.4.\\]\n\nAfter normalization it is \\((0.65,0.35)\\). The equality preference thus transmits evidence favoring \\(y=0\\) into a weaker preference for \\(x=0\\). Additional neighboring factors contribute through multiplication, allowing multiple sources of evidence to be combined locally."
    },
    {
      "heading": "Exact inference on trees and its computational cost",
      "text": "The exactness of belief propagation on a tree follows from a separation property. Removing any edge splits the graph into two disconnected components. A message across that edge can be interpreted as the result of multiplying all factors on the sender’s side and summing over every variable there except the boundary variable. The tree structure guarantees that the information arriving from different branches involves disjoint subproblems conditional on that boundary.\n\nThis interpretation gives an inductive proof of correctness. At a leaf, the message is obtained directly from a local factor or an empty product. Once all inward messages to an internal node have been computed, its outgoing message correctly summarizes the union of their subtrees. Continuing toward a chosen root produces its exact marginal. A subsequent outward pass supplies the remaining messages and hence all variable and factor marginals.\n\nA complete inward–outward schedule sends one message in each direction along every edge. Its communication count is therefore linear in the number of graph edges. This does not mean that every tree-structured model is cheap: a factor involving \\(r\\) variables, each with \\(q\\) states, can require work proportional to \\(q^r\\) for a straightforward table-based calculation. Factor arity and internal algebra matter as much as graph size.\n\nFor a pairwise tree with \\(n\\) variables and at most \\(q\\) states per variable, dense message updates ordinarily require \\(O(nq^2)\\) arithmetic operations overall. Storing all directed messages requires \\(O(nq)\\) space, apart from storage of the factors themselves. A chain is the special case that yields forward–backward inference; replacing summation with maximization yields a Viterbi-style dynamic program.\n\nBelief propagation is closely related to variable elimination. Both repeatedly apply the distributive law to avoid enumerating the full joint distribution. Variable elimination emphasizes an order for summing out variables and the intermediate factors that result. Tree message passing emphasizes reusable summaries attached to graph edges. On a suitable tree representation, these are complementary descriptions of essentially the same calculation.\n\nFor graphs with cycles, exact inference can be recovered by clustering variables into larger nodes and using the junction tree algorithm. The price is that messages now concern assignments to groups of variables. The size of those groups is governed by treewidth, and the resulting tables can grow exponentially with that width.\n\nThis distinction explains the main computational complexity trade-off. A sparse graph may still have large treewidth, so few edges alone do not guarantee efficient exact inference. Conversely, a large chain or tree can be tractable because its separators remain small. Belief propagation is efficient when the chosen representation exposes small, manageable boundaries between subproblems."
    },
    {
      "heading": "Loopy propagation, convergence, and approximation error",
      "text": "Loopy belief propagation applies the same update equations to a graph containing cycles. Messages are initialized, often to positive uniform values, and repeatedly updated until a stopping condition is reached. Unlike the tree case, a message no longer corresponds to a fixed, nonoverlapping component of the original graph. Information can circulate around a cycle and eventually influence the location from which it originated.\n\nA useful interpretation unfolds the iterations into a computation tree. Each repeated encounter with an original variable or factor becomes a separate copy in this expanded tree. After a finite number of iterations, the message at an edge can be understood as exact inference on a corresponding finite computation tree with boundary conditions induced by initialization. The approximation arises because the copies represent the same original variables but are treated as distinct within the unfolding.\n\nThis explains both success and failure. In sparse graphs with long cycles, nearby neighborhoods can look tree-like for several rounds, allowing informative local computations before substantial recirculation occurs. Where correlations decay rapidly with distance, the artificial boundary can have limited influence. Strong interactions, short feedback cycles, or competing modes can instead make the unfolded model a poor proxy for the original distribution.\n\nConvergence and accuracy are separate questions. Messages can oscillate, grow unstable before normalization, converge to different fixed points from different initializations, or settle to beliefs that differ substantially from exact marginals. A stable fixed point does not certify correctness. Even locally consistent beliefs can fail to be marginals of any global joint distribution.\n\nScheduling affects these behaviors. Synchronous updates compute all new messages from the preceding iteration. Asynchronous schedules use newly available values immediately. Residual schedules prioritize messages whose proposed changes are largest. Damping replaces a full update by a mixture, such as\n\n\\[m^{(t+1)} = (1-\\alpha)m^{(t)}+\\alpha\\widehat m^{(t+1)}, \\qquad 0<\\alpha\\leq 1,\\]\n\nwhere \\(\\widehat m^{(t+1)}\\) is the ordinary update. With a consistent normalization convention, damping preserves fixed points, but it does not universally guarantee convergence.\n\nRigorous convergence results apply under particular assumptions. Weak enough interactions can make the update map contractive, while some Gaussian models admit conditions expressed through matrix properties. These are sufficient conditions for specified settings, not blanket guarantees for all graphical models.\n\nPractical stopping tests usually measure changes in normalized messages or beliefs, sometimes together with violations of local consistency. A small numerical residual only shows that the implementation is close to a fixed point of its update equations. It does not measure the distance from the true posterior, so comparisons with exact calculations on smaller instances remain important when evaluating an approximation."
    },
    {
      "heading": "Bethe free energy and local consistency",
      "text": "A major theoretical interpretation of belief propagation comes from variational inference. Instead of directly summing over all assignments, variational methods express inference as optimization over distributions or summaries of distributions. For belief propagation, the relevant approximation is the Bethe approximation, which replaces the exact entropy by a combination of local entropies.\n\nLet \\(b_i(x_i)\\) and \\(b_a(x_{\\partial a})\\) be normalized candidate variable and factor beliefs. They are required to be nonnegative and locally consistent:\n\n\\[\\sum_{x_{\\partial a\\setminus\\{i\\}}} b_a(x_{\\partial a})=b_i(x_i)\\]\n\nfor every edge \\((i,a)\\). These conditions ensure that neighboring beliefs agree on shared single-variable marginals. On a tree, this local agreement is sufficient to construct a global joint distribution. On a graph with cycles, it generally is not.\n\nIf \\(d_i\\) is the number of factors adjacent to variable \\(i\\), the Bethe entropy is\n\n\\[H_B(b)= \\sum_a H(b_a)+\\sum_i(1-d_i)H(b_i),\\]\n\nwhere \\(H\\) denotes Shannon entropy. The subtraction corrects the repeated counting of variable uncertainty across overlapping factors. This correction is exact for a tree but only approximate for a general loopy graph.\n\nThe corresponding dimensionless Bethe free energy is\n\n\\[F_B(b)= -\\sum_a\\sum_{x_{\\partial a}} b_a(x_{\\partial a})\\log\\psi_a(x_{\\partial a}) -H_B(b).\\]\n\nFor hard constraints, beliefs must respect the zero-probability support, with the expression interpreted accordingly. On a tree, minimizing this free energy over locally consistent beliefs recovers the exact marginals, and the minimum equals \\(-\\log Z\\).\n\nFor positive models and appropriate interior beliefs, belief-propagation fixed points correspond to stationary points of the constrained Bethe free energy. Lagrange multipliers enforcing local consistency can be re-expressed as messages, yielding the familiar update equations. This establishes a connection between local computation and mathematical optimization.\n\nThe connection does not make ordinary belief propagation a general descent algorithm. On loopy graphs the Bethe objective is typically nonconvex, and standard message updates need not decrease it at every iteration. Multiple stationary points can exist. Some may reflect competing modes or phases; others can be poor approximations to the exact distribution.\n\nThe Bethe perspective also explains why belief propagation differs from a fully factorized mean-field approximation. Mean field represents the approximate joint distribution as a product of independent variable distributions. Bethe methods retain explicit factor-level correlations and require agreement where those factors overlap. This can capture dependencies that a product approximation misses, although neither approach is uniformly superior.\n\nExtensions alter either the entropy approximation or the regions over which beliefs are maintained. Their purpose is to improve the representation of correlations or the optimization landscape, usually at the cost of larger messages and more expensive consistency constraints."
    },
    {
      "heading": "Max–product, Gaussian, and generalized forms",
      "text": "The local message-passing pattern extends beyond summation and multiplication. In max–product propagation, a factor-to-variable update replaces the sum over neighboring assignments with a maximum. On a tree, the resulting dynamic program can recover a globally most probable assignment. It normally stores maximizing choices and performs a traceback to reconstruct a mutually consistent solution.\n\nTraceback matters because independently selecting a maximizing state from every variable’s max-marginal can fail when ties occur. Separate selections may combine pieces of different optimal configurations into a configuration that is not itself optimal. On loopy graphs, even converged max–product messages do not generally guarantee a global optimum, although specialized models and additional conditions support stronger conclusions.\n\nTaking negative logarithms converts multiplication into addition. A model written as\n\n\\[p(x)\\propto \\exp[-E(x)], \\qquad E(x)=\\sum_a E_a(x_{\\partial a}),\\]\n\nleads to min–sum updates:\n\n\\[M_{a\\to i}(x_i)= \\min_{x_{\\partial a\\setminus\\{i\\}}} \\left[ E_a(x_{\\partial a}) + \\sum_{j\\in\\partial a\\setminus\\{i\\}} M_{j\\to a}(x_j) \\right].\\]\n\nThis connects inference to discrete energy minimization. The broader sum–product framework can be formulated using suitable algebraic structures, often called semirings, in which the two combination operations obey distributive rules. Ordinary marginalization, maximum-weight calculations, and some constraint-processing algorithms consequently share a common graph-based structure.\n\nFor continuous variables, sums become integrals, and messages are functions rather than finite arrays. Exact functional messages are usually difficult to represent. A major exception is Gaussian inference: products and marginalizations of Gaussian functions remain Gaussian, so messages can be represented by precision matrices and information vectors.\n\nGaussian belief propagation is exact on trees. For loopy Gaussian models with suitable positive-definiteness and convergence conditions, the converged mean estimates can be exact even when the variance estimates are not. Convergence is a separate requirement; a proper Gaussian distribution alone does not ensure that every ordinary message schedule converges. The method is also related to distributed solution of linear systems.\n\nNon-Gaussian continuous models require approximations such as discretized grids, mixtures, numerical integration, or particles. These introduce representation error in addition to any error caused by loops. A fine discretization can make each message expensive, especially when factors connect several continuous variables.\n\nGeneralized belief propagation exchanges beliefs over larger clusters or regions, often using a region graph to account for overlaps. Capturing a short cycle inside one region can improve the approximation, but region states grow exponentially with the number of discrete variables they contain. Tree-reweighted methods take another route, combining tree-based constructions and modified entropy weights. These are related message-passing approaches rather than interchangeable names for ordinary belief propagation."
    },
    {
      "heading": "Error-correcting codes and other applications",
      "text": "One of belief propagation’s most consequential applications is decoding error-correcting codes, especially low-density parity-check codes. Such a code is specified by a sparse binary parity-check matrix \\(H\\). A valid codeword \\(x\\) satisfies\n\n\\[Hx=0\\pmod 2.\\]\n\nIts factor graph contains one variable node per transmitted bit and one check node per parity equation. Channel observations provide likelihoods for the bits. A check factor is one for assignments satisfying its parity constraint and zero otherwise.\n\nFor binary variables, messages can be represented compactly by log-likelihood ratios. With a conventional sign choice,\n\n\\[L=\\log\\frac{\\Pr(x=0)}{\\Pr(x=1)}.\\]\n\nVariable nodes add incoming check messages to the channel log-likelihood ratio. For an even-parity check, the sum–product check update is\n\n\\[L_{a\\to i} = 2\\,\\operatorname{atanh} \\left[ \\prod_{j\\in\\partial a\\setminus\\{i\\}} \\tanh\\left(\\frac{L_{j\\to a}}{2}\\right) \\right].\\]\n\nSpecial structure makes this far cheaper than enumerating every assignment to a high-degree parity factor. Min-sum decoding approximates the check magnitude using the smallest incoming magnitude and combines signs; normalized and offset variants modify this approximation. Their reduced arithmetic cost is useful in hardware, though the approximations change decoding behavior.\n\nCode design and inference interact. Sparse graphs reduce per-iteration work, while short cycles can create correlated messages. Particular subgraphs, including trapping sets, contribute to decoding failures. Density evolution analyzes message distributions in large code ensembles under assumptions that make finite neighborhoods effectively tree-like. Practical performance still depends on block length, degree distribution, channel model, numerical precision, and iteration limit.\n\nIn computer vision, belief propagation has been used for stereo matching, image restoration, and labeling. A variable may represent the disparity or class at a pixel, with unary factors describing local evidence and pairwise factors encouraging compatible neighboring labels. Such grids contain many short cycles and can have large state spaces, making update acceleration and approximation quality central concerns.\n\nIn natural language processing, chain-structured conditional random fields permit exact forward–backward inference, whereas richer dependency structures can require loopy or clustered methods. In robotics and simultaneous localization and mapping, factor graphs express relationships among poses, landmarks, and sensor measurements. Many practical systems solve the resulting estimation problem by nonlinear least squares rather than sum–product propagation, so use of a factor graph does not by itself establish that belief propagation is being used.\n\nStatistical physics supplies another important setting. Variables can represent spins, and factors encode interactions. Cavity calculations and Bethe approximations help describe sparse disordered systems. Across these applications, the shared attraction is the ability to transform a large global calculation into repeated local operations whose cost is governed by small neighborhoods."
    },
    {
      "heading": "Numerical implementation and relationships to modern learning",
      "text": "A robust implementation must separate the mathematical update rule from its numerical representation. Directly multiplying many small probabilities causes underflow in floating-point arithmetic. Log-domain messages replace products with sums and evaluate marginalization using the log-sum-exp identity:\n\n\\[\\log\\sum_k e^{z_k} = c+\\log\\sum_k e^{z_k-c}, \\qquad c=\\max_k z_k.\\]\n\nMessages can also be rescaled after every update. Such rescaling preserves normalized beliefs, but implementations estimating a partition function must retain the relevant normalization information.\n\nHard constraints require special care. Zero probabilities become negative infinity in a log representation, and entirely impossible local configurations can produce undefined normalization. Division-based implementations that derive an outgoing message from a product of all incoming messages can encounter zero-over-zero problems. Prefix–suffix accumulation or explicit exclusion of the recipient avoids some of these difficulties. Detecting inconsistent evidence is distinct from merely avoiding a numerical exception.\n\nPerformance depends strongly on factor structure. A dense table update is exponential in factor arity, whereas parity checks, Gaussian factors, sparse constraints, and some convolutional relationships admit specialized computations. For pairwise factors, matrix–vector operations are natural primitives. If a variable has many neighbors, reusable partial products or sums can avoid recomputing the same aggregate separately for every outgoing edge.\n\nThe local structure supports parallel computing and distributed computing. Independent updates can be assigned to processors, and large regular graphs can be mapped to a graphics processing unit. Nevertheless, irregular connectivity, small message sizes, memory bandwidth, and synchronization overhead can limit acceleration. A synchronous schedule is straightforward to parallelize but may require more iterations than a carefully chosen sequential or residual schedule.\n\nBelief propagation also appears as a component inside machine learning systems. When model parameters determine factor values, a fixed number of message-passing iterations can be unrolled into a computational graph. Gradients can then be obtained through backpropagation. This trains parameters for the behavior of the finite inference procedure, which need not match training under exact probabilistic inference.\n\nAlternatively, derivatives may be calculated implicitly through a converged fixed point, subject to differentiability and nonsingularity conditions. Changing normalization, damping, stopping criteria, or factor parameterization can affect these derivatives. Differentiating an unstable or poorly converged inference procedure is therefore not equivalent to differentiating exact posterior probabilities.\n\nGraph neural networks share the broad pattern of aggregating messages from neighbors, but the resemblance has limits. Their messages are usually learned vector representations rather than functions with a prescribed probabilistic interpretation. Learned message-passing systems may imitate, initialize, accelerate, or modify belief propagation; once the update equations are replaced, tree exactness and Bethe fixed-point results do not automatically carry over. The computational architecture can remain similar even when the mathematical meaning of a message has changed."
    }
  ],
  "links": [
    "message-passing",
    "probabilistic-graphical-model",
    "probability-distribution",
    "artificial-intelligence",
    "statistics",
    "hidden-markov-model",
    "dynamic-programming",
    "random-variable",
    "partition-function",
    "factor-graph",
    "bayesian-network",
    "markov-random-field",
    "computational-complexity",
    "variational-inference",
    "information-entropy",
    "mathematical-optimization",
    "integral",
    "normal-distribution",
    "error-correcting-code",
    "low-density-parity-check-code",
    "computer-vision",
    "pixel",
    "natural-language-processing",
    "conditional-random-field",
    "simultaneous-localization-and-mapping",
    "floating-point-arithmetic",
    "parallel-computing",
    "distributed-computing",
    "graphics-processing-unit",
    "machine-learning",
    "computational-graph",
    "backpropagation"
  ],
  "sources": [],
  "license": "https://creativecommons.org/licenses/by/4.0/"
}