Applied & Computational Math

Automatic differentiation computes derivatives of numerical programs by propagating local derivative rules through their operations.

A recursive equation relating a decision problem’s value to immediate rewards or costs and the value of subsequent states.

Channel capacity is the supremum of information rates achievable over a specified communication channel with decoding error probability tending to zero.

Cholesky decomposition factors a positive-definite matrix into a triangular matrix and its conjugate transpose, enabling efficient linear-system solutions and statistical computations.

Claude Shannon was an American mathematician and engineer who founded information theory and established key principles of digital circuit design.

A condition number measures how strongly small changes in a problem’s input can affect its solution.

Conditional entropy measures the average uncertainty remaining about a random variable when another variable is known.

Convex optimization studies the minimization of convex functions over convex feasible sets, combining global optimality guarantees with structured numerical methods.

Cross-entropy measures the expected logarithmic loss incurred when one probability distribution is used to describe outcomes generated by another.

Decision theory studies how choices can be evaluated using preferences, uncertainty, consequences, and formal criteria of rationality.

Differential entropy measures the spread of a continuous probability distribution relative to a specified coordinate scale and reference measure.

Dimensionality reduction represents data with fewer variables while seeking to preserve information relevant to analysis, visualization, or prediction.

Information entropy measures the average uncertainty of a probability distribution and establishes fundamental limits on lossless data compression.

Entropy rate measures the asymptotic average information generated per symbol by a stochastic process, accounting for dependence between successive observations.

A family of algorithms that efficiently compute discrete Fourier transforms and their inverses, typically using O(N log N) arithmetic operations.

A feasible set is the collection of all candidate solutions that satisfy an optimization problem’s constraints and domain restrictions.

A numerical method that approximates derivatives by weighted differences of function values, converting differential equations into discrete algebraic equations.

The finite-element method approximates differential equations by dividing a domain into elements and solving for a finite number of coefficients in locally defined functions.

Floating-point arithmetic represents and calculates with numbers using finite-precision significands and variable exponents, balancing numerical range, accuracy, and computational efficiency.

Game theory mathematically analyzes strategic interactions in which each participant’s outcomes depend on the choices of others.

Gradient descent is an iterative optimization method that reduces a differentiable objective by moving opposite to its gradient.

Hannah Fry is a British mathematician, author and broadcaster whose work combines research on human behaviour with public communication of mathematics and science.

A Householder transformation is a reflection across a hyperplane, used to eliminate vector components and construct stable matrix factorizations.

Information theory mathematically quantifies uncertainty and establishes fundamental limits on data compression and reliable communication.

Integer programming optimizes decisions subject to constraints requiring some or all variables to take integer values.

An inverse problem infers unknown parameters, structures, or states from observations using a model of how those observations are produced.

American mathematician (1928–2015) whose work founded modern non-cooperative game theory, advanced Riemannian geometry and PDEs, and earned the 1994 Nobel Prize in Economic Sciences and the 2015 Abel Prize.

John von Neumann was a Hungarian-born American mathematician whose work shaped quantum theory, game theory, and electronic computing.

First-order conditions characterizing constrained optima under suitable regularity assumptions, and certifying global optimality in convex problems.

Lagrangian duality associates a constrained optimization problem with a multiplier-based problem that provides bounds and, under suitable conditions, optimality certificates.

A numerical optimization procedure that selects a step size along a prescribed search direction.

Linear programming optimizes a linear objective subject to linear constraints on continuous decision variables.

A loss function assigns a numerical penalty to a decision or prediction, defining the errors that statistical and machine-learning procedures seek to minimize.

LU decomposition expresses a matrix as triangular factors, usually with permutations, enabling efficient solution of linear systems and related computations.

Mathematical optimization studies how to find the best feasible solution to a problem defined by an objective function and constraints.

A mixed strategy is a probability distribution over a player’s pure strategies, allowing randomized choice in a game.

Mutual information measures statistical dependence by quantifying how much observing one random variable reduces uncertainty about another.

A strategy profile in a game in which no player can improve their payoff by changing their strategy alone.

Newton’s method is an iterative numerical technique for solving nonlinear equations through successive local linear approximations.

The noisy-channel coding theorem identifies channel capacity as the limiting rate for communication with arbitrarily small decoding error.

Numerical analysis studies algorithms for approximating mathematical solutions and evaluates their accuracy, stability, convergence, and computational efficiency.

Numerical linear algebra studies accurate and efficient computational methods for solving linear systems, least-squares problems, and matrix eigenvalue problems.

Numerical stability describes how a computational method controls the propagation and amplification of errors, particularly those introduced by finite-precision arithmetic.

A mathematical function whose value is minimized or maximized to select the best feasible solution to an optimization problem.

Policy iteration solves sequential decision problems by repeatedly evaluating a policy and improving its action choices until an optimal policy is obtained.

The QR algorithm computes matrix eigenvalues and Schur decompositions through repeated orthogonal or unitary similarity transformations.

QR decomposition factors a matrix into orthogonal or unitary and triangular factors, supporting stable least-squares solutions and eigenvalue computations.

Regularization stabilizes mathematical estimation problems and controls model complexity through penalties, constraints, or changes to the learning process.

Richard Bellman was an American applied mathematician who developed dynamic programming and advanced the mathematical study of sequential decisions and control.

Self-information is the negative logarithm of an event’s probability, measuring how surprising its occurrence is under a specified probability model.

Sensitivity analysis examines how changes in inputs and assumptions affect model outputs, identifying influential factors and assessing the robustness of conclusions.

The source coding theorem identifies entropy as the fundamental limit on the rate of lossless compression of a discrete information source.

A matrix with sufficiently many zero entries that specialized storage and algorithms can reduce memory use and computational work.

Stochastic gradient descent is an iterative optimization method that updates parameters using gradients estimated from randomly sampled observations.

A value function assigns an optimal objective value or an expected cumulative reward to a parameter, state, or state–action pair.

Value iteration is a dynamic programming algorithm that computes optimal value functions and policies by repeatedly applying Bellman optimality updates.

A game in which players’ payoffs sum to zero at every outcome, so gains to some players are exactly balanced by losses to others.