Probability & Statistics

A generalized linear model relates a response variable’s conditional mean to a linear predictor through a link function, accommodating several probability distributions.

A probabilistic model of sequential observations generated by unobserved states that evolve according to a Markov chain.

Instrumental variables identify and estimate causal effects using external variation in an explanatory variable, subject to assumptions about relevance and validity.

A joint probability distribution describes the simultaneous behavior of two or more random variables, including their individual distributions and dependence relationships.

Kullback–Leibler divergence measures the discrepancy between probability distributions through an expected logarithmic probability ratio.

Lasso regression is a regularized linear modeling method that shrinks coefficients and can set some exactly to zero, combining prediction with variable selection.

A family of probability theorems describing when averages of many random observations converge to their expected value.

A likelihood function evaluates how observed data support different parameter values within a specified statistical model.

A statistical method that models a response variable as a linear combination of predictors and estimates relationships or predicts numerical outcomes.

Logistic regression models categorical outcomes by relating their probabilities to linear combinations of explanatory variables through the logistic function.

The logit is the natural logarithm of the odds of an event, mapping probabilities between zero and one to the real line.

Marginal likelihood is the probability or density of observed data under a model, obtained by averaging its likelihood over a prior distribution.

A Markov chain is a stochastic process whose future evolution, conditional on its present state, does not depend on its past states.

A family of sampling methods that uses Markov chains to approximate probability distributions and expectations.

A Markov decision process models sequential choices under uncertainty, using states, actions, transition probabilities, and rewards to define and optimize decision-making policies.

The Markov property states that, conditional on a process’s present state, its future evolution does not depend on its past history.

A martingale is an integrable stochastic process whose conditional expected future value equals its present value, relative to a specified information flow.

Maximum a posteriori estimation selects a parameter value that maximizes its posterior probability or density, combining observed data with a prior distribution.

Maximum likelihood estimation selects model parameters that maximize the likelihood of observed data under a specified statistical model.

Mean squared error measures estimation or prediction error by averaging squared differences between estimated and reference values.

Missing data imputation replaces unavailable observations with plausible values while accounting for assumptions about missingness and, where required, statistical uncertainty.

A family of computational methods that use repeated random sampling to estimate numerical quantities and characterize uncertainty.

Multiple testing concerns inference on several hypotheses and methods for controlling false-positive errors across the resulting collection of decisions.

A probability distribution for random vectors whose every linear combination is normally distributed, characterized by a mean vector and covariance matrix.

A natural experiment uses externally generated differences in exposure to investigate causal effects without researcher-controlled assignment.

Norbert Wiener was an American mathematician who helped establish cybernetics and made foundational contributions to probability, mathematical analysis, and statistical prediction.

A continuous probability distribution defined by its mean and variance, with a symmetric bell-shaped density and a central role in statistical inference.

A null hypothesis is a statistical claim used as a reference model for evaluating whether observed data warrant its rejection.

An odds ratio compares the odds of an event between two groups and measures association between binary variables.

Ordinary least squares estimates linear regression coefficients by minimizing the sum of squared differences between observed and fitted responses.

A p-value measures how extreme an observed test statistic is under a specified null hypothesis and statistical model.

Pierre-Simon Laplace was a French mathematician and astronomer whose work transformed celestial mechanics, probability theory, and mathematical physics.

The Poisson distribution is a discrete probability distribution for event counts, characterized by a single parameter equal to both its mean and variance.

A posterior distribution describes uncertainty about unknown quantities after observed data are incorporated through a probabilistic model.

Posterior predictive checking assesses a Bayesian model by comparing observed data with replicated data generated from its posterior predictive distribution.

A posterior predictive distribution describes unobserved outcomes by averaging their conditional distributions over Bayesian posterior uncertainty about model parameters.

Principal component analysis transforms correlated variables into orthogonal components ordered by explained variance, enabling linear dimensionality reduction and data exploration.

A prior distribution represents uncertainty about an unknown quantity before the data used in a particular Bayesian analysis are incorporated.

A probabilistic graphical model represents a probability distribution through a graph and local functions, enabling structured reasoning about uncertainty.

Probability is the mathematical study of uncertainty, assigning numbers between 0 and 1 to events and governing how those numbers combine and update.

A probability density function describes an absolutely continuous probability distribution, with probabilities obtained by integration over sets of possible values.

A probability distribution specifies how probability is assigned to the possible values of a random variable or to measurable sets of outcomes.

A probability mass function assigns probabilities to the individual values of a discrete random variable.

A probability space is a mathematical structure consisting of possible outcomes, measurable events, and a measure assigning probabilities to those events.

A random variable is a measurable function that assigns a numerical value to each outcome in a probability space.

A random walk is a stochastic process whose position changes through successive randomly determined steps.

Randomization introduces chance into assignments, ordering, or computation to support statistical inference and reduce systematic bias.

Ridge regression estimates linear-model coefficients using a squared-magnitude penalty to reduce instability and balance bias against variance.

British statistician and geneticist who developed foundational methods of statistical inference, experimental design, and population genetics.

The sample mean is the arithmetic average of observed values, used to describe a dataset’s center and estimate a population mean.

A sample space is the set of all possible outcomes represented in a probability model, forming the underlying domain on which events and random variables are defined.

The probability distribution of a statistic across repeated samples drawn under a specified sampling process.

A sigma-algebra is a collection of subsets closed under complements and countable unions, providing the framework for measurable sets and probabilistic events.

A significance level is a prespecified bound on the probability of rejecting a true null hypothesis in statistical hypothesis testing.

Standard deviation measures dispersion around a mean as the square root of variance, expressed in the same units as the observations.

Standard error measures the sampling variability of a statistic, indicating how much its value varies across repeated samples.

A stationary distribution is a probability distribution preserved by the transition dynamics of a Markov process.

A stationary process is a stochastic process whose distributions, or specified statistical moments, are invariant under shifts of the time index.

Statistical hypothesis testing evaluates claims about populations or models using sample data and procedures with specified error properties.

Statistical identification concerns whether observable data and specified assumptions uniquely determine a parameter, distribution, or other quantity of interest.