A condition number quantifies the sensitivity of a mathematical problem’s output to small perturbations in its input. It measures the potential amplification of input error, rather than the error introduced by a particular computational method. A problem with a small condition number is called well-conditioned; one with a large condition number is ill-conditioned. The concept is central to numerical linear algebra and sensitivity analysis, where it connects uncertainty in data with uncertainty in computed answers. Its value depends on the problem, the input, and how perturbations are measured. (nhigham.com)
General definition
Suppose a problem is represented by a function (f) between finite-dimensional normed vector spaces. For nonzero (x) and (f(x)), its local relative condition number can be defined as
[ \kappa_{\mathrm{rel}}(f,x)= \lim_{\varepsilon\downarrow0} \sup_{0<|\Delta x|\leq\varepsilon|x|} \frac{|f(x+\Delta x)-f(x)|/|f(x)|} {|\Delta x|/|x|}. ]
Perturbations are restricted to admissible inputs. The supremum expresses worst-case sensitivity over all permitted directions, while the limit makes the definition local. If (f) is differentiable, then
[ \kappa_{\mathrm{rel}}(f,x) =\frac{|Df(x)|,|x|}{|f(x)|}, ]
where the norm of the derivative is the induced operator norm. In several variables, the derivative is represented by a Jacobian matrix. (nhigham.com)
For a scalar function, this becomes (\left|xf'(x)/f(x)\right|). For example, (f(x)=x^p), on a suitable domain with nonzero input and output, has relative condition number (|p|). Absolute conditioning instead compares absolute output changes with absolute input changes; its differentiable form is (|Df(x)|). When an input or output is zero, relative normalization may be undefined, making an absolute or mixed formulation necessary. (nhigham.com)
Matrix condition numbers
For an invertible square matrix (A), the standard normwise matrix condition number is
[ \kappa(A)=|A|,|A^{-1}|, ]
where (A^{-1}) is its inverse matrix. This quantity measures sensitivity associated with matrix inversion and solving linear systems. Different norms give different condition numbers, commonly denoted (\kappa_1), (\kappa_2), and (\kappa_\infty). (netlib.org)
In the Euclidean operator norm, the singular value decomposition gives
[ \kappa_2(A)= \frac{\sigma_{\max}(A)}{\sigma_{\min}(A)}. ]
Geometrically, this compares the greatest and least stretching of vectors by (A). A very small minimum singular value identifies a direction strongly compressed by the matrix and strongly amplified by inversion. A singular square matrix is conventionally assigned infinite condition number. (cs.cornell.edu)
The formula implies (\kappa_2(A)\geq1), invariance under multiplication by a nonzero scalar, and condition number one for an orthogonal matrix. As a derived example,
[ A=\begin{pmatrix}1&0\0&10^{-8}\end{pmatrix} ]
has (\kappa_2(A)=10^8). Scaling every entry equally leaves this ratio unchanged: relative conditioning concerns unequal directional stretching, not merely small numerical entries. (cs.cornell.edu)
Linear systems and perturbation bounds
Consider a system of linear equations (Ax=b), with (A) invertible and (b\neq0). If only the right-hand side changes, then
[ A(x+\Delta x)=b+\Delta b, \qquad \Delta x=A^{-1}\Delta b. ]
Consequently,
[ \frac{|\Delta x|}{|x|} \leq \kappa(A)\frac{|\Delta b|}{|b|}. ]
This is a worst-case bound, not a prediction that every perturbation receives the full amplification. For fixed (b), the exact relative condition number of the map (b\mapsto A^{-1}b) is (|A^{-1}||b|/|x|), which can be smaller than (\kappa(A)). (cs.cornell.edu)
Perturbations in the coefficient matrix also matter. Their analysis depends on whether errors are measured normwise or componentwise, and whether (A), (b), or both are allowed to vary. A normwise condition number therefore does not fully describe every possible uncertainty model. Componentwise bounds can better reflect data whose entries have widely differing magnitudes. (netlib.org)
Conditioning and numerical stability
Conditioning belongs to the mathematical problem; numerical stability belongs to an algorithm. Forward error measures the difference between a computed answer and the exact answer. Backward error measures how much the input must change to make the computed answer exact. A backward-stable method produces the exact solution of a nearby problem. (cs.cornell.edu)
For small errors, the central relationship is schematically
[ \text{relative forward error} \lesssim \text{condition number}\times \text{relative backward error}. ]
Thus, even a backward-stable computation may have substantial forward error on an ill-conditioned problem. In floating-point arithmetic, rounding contributes to backward error; conditioning governs its possible amplification. These are distinct effects and require separate analysis. (cs.cornell.edu)
For a computed solution (\widehat{x}), the residual (r=b-A\widehat{x}) measures failure to satisfy the equations. A small residual alone does not guarantee a small solution error: converting residual information into a forward-error bound requires conditioning information. (netlib.org)
Least squares and estimation
For a rectangular matrix with full column rank, the analogous quantity is
[ \kappa_2(A)=|A|_2|A^\dagger|_2, ]
where (A^\dagger) is the Moore–Penrose pseudoinverse. In least-squares problems, nearly linearly dependent columns can make fitted coefficients highly sensitive. However, sensitivity also depends on the residual and the position of (b) relative to the matrix’s column space; the matrix condition number alone is not a complete description. Regularization changes the fitting problem to constrain otherwise poorly determined solutions. (cs.cornell.edu)
Condition estimates are often used instead of explicitly computing an inverse. LAPACK error-estimation routines commonly return RCOND, an estimate of the reciprocal condition number. Using the reciprocal avoids overflow when conditioning is extremely poor; values near zero indicate large estimated sensitivity under the selected norm. (netlib.org)