Probability & Statistics

In probability theory, a property holds almost surely when the set of outcomes on which it holds has probability one, although exceptions may exist.

A statistical proposition specifying parameter values or distributions against which a null hypothesis is tested.

Analysis of variance partitions observed variation into components to test differences among group means and effects of explanatory factors.

Andrey Kolmogorov was a Soviet mathematician who established modern probability’s axiomatic foundations and made major contributions to analysis, dynamical systems, turbulence, and algorithmic information.

Andrey Markov was a Russian mathematician whose work on dependent random variables established the foundations of Markov chains.

Bayes’ theorem relates conditional probabilities and provides the mathematical basis for updating probabilities in light of evidence.

Bayesian inference updates probability distributions for unknown quantities by combining prior information with observed data through Bayes’ theorem.

A Bayesian network represents a joint probability distribution through a directed acyclic graph and local conditional distributions, enabling reasoning under uncertainty.

A discrete probability distribution describing a single binary outcome, with probability p of taking the value 1 and probability 1 − p of taking the value 0.

Bessel’s correction replaces n with n−1 in sample variance calculations to obtain an unbiased estimator of population variance when the mean is estimated from the same sample.

A flexible family of continuous probability distributions on the unit interval, widely used to model proportions and uncertainty about probabilities.

The bias of an estimator is the difference between its expected value under repeated sampling and the parameter it estimates.

The bias–variance tradeoff describes how systematic prediction error and sensitivity to training data jointly affect a model’s expected performance on unseen observations.

The binomial distribution describes the number of successes in a fixed number of independent trials with the same success probability.

A multiple-testing procedure that controls the probability of at least one false rejection by dividing the overall significance level among tests.

Bootstrap sampling estimates statistical uncertainty by repeatedly resampling observed data or simulating from a fitted model.

Causal inference uses data and explicit assumptions to determine how interventions change outcomes, rather than merely describing associations.

A family of probability theorems describing when suitably standardized sums of random variables converge in distribution to a normal distribution.

The Chapman–Kolmogorov equations express how Markov transition probabilities compose across successive time intervals by summing or integrating over intermediate states.

Chebyshev’s inequality bounds the probability that a random variable deviates from its mean using only its variance.

Cluster analysis groups observations according to similarity or statistical structure, revealing patterns without requiring predefined class labels.

The coefficient of determination, usually denoted R², measures a model’s reduction in squared prediction error relative to a specified baseline.

Conditional expectation is the mean of a random variable given specified information, defined rigorously by measurability and integral-preservation conditions.

Conditional independence means that, given specified information, learning one random variable provides no additional information about another.

Conditional probability measures the probability of an event when specified information or another event is taken as given.

A confidence interval estimates an unknown parameter using a procedure calibrated to cover its true value at a specified long-run frequency.

Confounding is the distortion of a causal comparison by differences between groups that also influence the outcome.

A conjugate prior is a prior distribution whose family is preserved when it is updated by a specified likelihood.

A consistent estimator converges in probability to its target parameter as the sample size tends to infinity.

A stochastic process on a finite or countable state space that evolves in continuous time and satisfies the Markov property.

Convergence in probability means that the probability of any fixed positive deviation from a limiting random variable tends to zero.

Correlation describes statistical association between variables, commonly quantified by coefficients measuring linear or monotonic relationships.

Covariance measures how two random variables vary together through the expected product of their deviations from their means.

A covariance matrix records the variances and pairwise covariances of a random vector, describing its second-order variability and linear dependence.

A credible interval is a range containing a specified posterior probability for an unknown quantity under a Bayesian statistical model.

Cross-validation estimates predictive performance by repeatedly fitting models on subsets of data and evaluating them on held-out observations.

A cumulative distribution function gives the probability that a real-valued random variable is less than or equal to a specified threshold, uniquely characterizing its distribution.

The curse of dimensionality describes geometric, statistical, and computational difficulties that arise as the number of dimensions in a problem increases.

A graphical criterion that identifies conditional independence relations implied by a directed acyclic graph.

Data science is an interdisciplinary field that combines statistical reasoning, computing, and domain knowledge to extract information from data and support investigation and decision-making.

Degrees of freedom describe independent variation remaining after constraints, with applications in mathematics, statistical inference, and mechanics.

Density estimation infers the probability density underlying observed data, using parametric models, nonparametric smoothing, or flexible learned distributions.

Detailed balance is the condition that every transition between two states is exactly balanced by its reverse under a specified stationary distribution.

Econometrics combines economic theory and statistical methods to estimate relationships, test hypotheses, forecast outcomes, and investigate causal effects using data.

An effect size quantifies the magnitude of a difference or association, complementing statistical tests with information about scale and practical importance.

Elastic net is a regularization method combining lasso and ridge penalties to shrink regression coefficients and select predictors.

Endogeneity arises when explanatory variables are related to a model’s unobserved disturbances, complicating estimation and causal interpretation.

An estimator is a rule that uses observed data to infer an unknown population parameter or other statistical quantity.

An event is a measurable set of outcomes in a probability model, representing a condition that may or may not occur.

The evidence lower bound is a variational objective that bounds log marginal likelihood from below and supports approximate inference and probabilistic model learning.

An iterative method for estimating statistical model parameters when observations are incomplete or depend on unobserved variables.

The probability-weighted average of a random variable, a basic tool in probability, statistics, and decision-making.

Experimental design is the statistical planning of experiments to estimate effects, control unwanted variation, and obtain interpretable evidence.

A family of evaluation measures combining precision and recall, most commonly through their harmonic mean, the F1 score.

A factorial design investigates multiple factors together by combining their levels, allowing estimation of individual effects and interactions.

The false discovery rate is the expected proportion of incorrect rejections among all hypotheses rejected by a multiple-testing procedure.

A filtration is an increasing family of sigma-algebras representing the information available over time in a probabilistic model.

Fisher information measures a statistical model’s local sensitivity to its parameters and determines bounds on estimation precision.

A Gaussian mixture model represents a probability distribution as a weighted combination of Gaussian components, supporting density estimation and probabilistic clustering.

A Gaussian process is a stochastic process whose every finite collection of values has a joint Gaussian distribution, specified by a mean function and a covariance kernel.