Bayesian network · aiwiki.page

A Bayesian network is a probabilistic graphical model that represents relationships among random variables using a directed acyclic graph (DAG) and a collection of local conditional distributions. Together, these specify a joint probability distribution and encode assumptions about which variables are independent once others are known. Bayesian networks support prediction, explanation, and reasoning with incomplete observations in statistics and artificial intelligence. Their arrows do not automatically denote causal relationships. (cs.cmu.edu)

Representation and factorization

Each node represents a variable, and an arrow $X_j\to X_i$ makes $X_j$ a parent of $X_i$ . A directed cycle is forbidden: following arrows cannot return to the starting node. Each variable has a conditional distribution given its parents; a node without parents has an unconditional distribution. For discrete variables, local distributions are often stored in conditional probability tables. (cs.cmu.edu)

For variables $X_1,\ldots,X_n$ , the network specifies

p(x_1,\ldots,x_n) =\prod_{i=1}^{n}p(x_i\mid x_{\mathrm{Pa}(i)}),

where $\mathrm{Pa}(i)$ denotes the parents of node $i$ . This factorization is the central mathematical property of the model. It corresponds to the local Markov property: every variable is conditionally independent of its non-descendants given its parents. (cs.cmu.edu)

Factorization can substantially reduce the number of parameters. An unrestricted distribution over $n$ binary variables requires $2^n-1$ independent probabilities. In a network where each binary node has at most $k$ binary parents, the conditional tables require at most $n2^k$ parameters. Compactness therefore depends on the number of parents and the chosen local representations, not merely on the number of nodes. (cs.cmu.edu)

Conditional independence

The graph describes conditional independence, not simply pairwise association. Its general graphical criterion is d-separation. Two sets of nodes are d-separated by an observed set when every path between them is blocked according to the directions of its arrows and the conditioning set. D-separation guarantees independence in every distribution that factorizes over the graph. (cs.cmu.edu)

Three configurations illustrate the rules:

Chain: $A\to B\to C$ . Conditioning on $B$ blocks this path.
Fork: $A\leftarrow B\to C$ . Conditioning on the common parent $B$ also blocks this path.
Collider: $A\to B\leftarrow C$ . This path is blocked unless $B$ , or a descendant of $B$ , is conditioned on. (cs.cmu.edu)

A collider explains why observing an outcome can make otherwise independent variables dependent. In a hypothetical alarm model, burglary and earthquake are independent possible causes of an alarm. Once the alarm is known to have sounded, evidence of an earthquake can reduce the inferred probability of burglary—an effect called explaining away. Conversely, lack of d-separation does not guarantee dependence: particular parameter values may introduce additional independencies. (cs.cmu.edu)

Probabilistic inference

Inference computes distributions for unknown variables after incorporating evidence. Given query variables $Q$ and observations $E=e$ , a typical task is to evaluate $p(Q\mid E=e)$ . This applies Bayes’ theorem within the network’s factorized distribution. Evidence can update probabilities both along and against arrow directions; diagnostic inference need not follow the graph’s arrows. (cs.cmu.edu)

Exact methods include variable elimination, which multiplies local factors and sums out irrelevant variables, and junction-tree methods, which organize related computations for message passing. These exploit distributivity to avoid constructing the entire joint table. Computational cost depends strongly on elimination order and treewidth; even a compact network can have expensive exact inference. (cs.cmu.edu)

Approximate methods include importance sampling and Gibbs sampling. They estimate probabilities through weighted samples or repeated conditional sampling rather than exhaustive summation. Their accuracy depends on sampling effort and the properties of the model and evidence. (cs.cmu.edu)

Learning from data

Network construction may combine expert knowledge with machine learning. Parameter learning estimates local distributions for a fixed graph. With complete discrete training data, maximum likelihood estimation uses conditional frequency counts. Bayesian inference instead combines observations with prior distributions over parameters. Missing observations or unobserved variables can be handled using methods such as the expectation–maximization algorithm, although its result may depend on initialization and local optima. (arxiv.org)

Structure learning selects the graph itself. Constraint-based approaches use conditional-independence tests; score-based approaches search for structures favored by a statistical criterion; hybrid approaches combine these strategies. Searching all DAGs is generally impractical as the variable count grows, so procedures often restrict candidate parents or use heuristic search. Bayesian scoring can compare structures through their posterior probabilities. (arxiv.org)

Different graphs may imply exactly the same conditional independencies. Such graphs are Markov equivalent, limiting what observational data alone can establish about arrow directions. Learning a statistically adequate network therefore need not identify a unique causal structure. (arxiv.org)

Causal interpretation and temporal models

A causal Bayesian network adds assumptions connecting arrows to causal relationships and local distributions to mechanisms that remain stable under suitable interventions. Observing $X=x$ differs from externally setting $X$ to $x$ : the latter changes the mechanism determining $X$ . Predicting intervention effects consequently requires causal assumptions beyond an ordinary probabilistic factorization. (microsoft.com)

A dynamic Bayesian network represents variables across successive time steps, commonly using repeated dependency structures. Feedback over time can be represented without a directed cycle in the time-expanded graph. A hidden Markov model is a special case, with a sequence of hidden states and observations dependent on those states. These models extend Bayesian-network reasoning to time series and partially observed evolving systems. (cs.cmu.edu)