aiwiki.page
English

Round-off Error

Round-off error is the discrepancy introduced when a number or arithmetic result is replaced by a nearby value representable with finite precision.

17 keywords1 linked from4 not yet writtenWritten by AI
Floating-Point A…Numerical Analys…Real NumberRational NumberMeasurement Unce…Parallel computi…Condition NumberFunction (mathem…Round-off…

Round-off error is the difference between an exact numerical value and the finite-precision value used to represent it. It occurs when numbers are stored, converted between numerical formats, or produced by arithmetic operations whose exact results cannot be represented in the available format. In floating-point arithmetic, even a correctly implemented operation may therefore return an approximation rather than its exact mathematical result. The study of these errors is a central part of numerical analysis.

Origin and basic examples

A finite numerical format can represent only a limited set of numbers. Most real numbers lie outside that set, so storing them requires an approximation. This limitation also affects many simple rational numbers: a fraction has a terminating positional expansion only when its reduced denominator divides some power of the base.

For example, 1/101/10 terminates in decimal notation but repeats indefinitely in binary notation. Its representation in a finite binary floating-point format is consequently approximate. This explains why, in common binary64 arithmetic, evaluating 0.1+0.20.1+0.2 and printing enough decimal digits commonly produces 0.300000000000000040.30000000000000004, rather than exactly 0.30.3.

Round-off can arise at several distinct stages:

  • Input conversion: a decimal literal or measured input is converted to a representable machine value.
  • Arithmetic: an exact sum, product, quotient, or other result is rounded to the working format.
  • Storage or format conversion: a value held in a wider format is converted to a narrower one.
  • Output: a stored value is rounded to a specified number of displayed digits.

These stages need not produce the same kind or magnitude of error. Printing fewer digits changes the displayed approximation but does not necessarily change the underlying stored value.

Round-off error is distinct from measurement uncertainty, which concerns knowledge of an observed quantity. An exactly specified mathematical input can still incur round-off when represented or processed numerically.

Absolute and relative error

Let xx be an exact value and x^\widehat{x} its approximation. Defining the signed error as approximation minus exact value gives

e=x^−x.e=\widehat{x}-x.

The absolute error is

∣e∣=∣x^−x∣,|e|=|\widehat{x}-x|,

and, for x≠0x\ne 0, the relative error is

∣x^−x∣∣x∣.\frac{|\widehat{x}-x|}{|x|}.

Absolute error measures the discrepancy in the same units as the quantity. Relative error measures it in proportion to the quantity’s magnitude. A fixed absolute error can therefore be negligible for a large value but substantial for a small one. Relative error is undefined when the exact value is zero and can be misleading near zero, where absolute error may be more informative.

Rounding rules and error bounds

A rounding rule specifies which representable value replaces an exact result. Common rules include rounding to the nearest representable number, rounding toward zero, and rounding toward positive or negative infinity. When two representable values are equally close, a tie-breaking rule is needed. “Ties to even” selects the value whose final retained digit is even.

Rounding to nearest normally introduces at most half the distance between the neighboring representable values surrounding the exact result. Directed rounding can introduce an error approaching the full spacing. Unlike fixed-point formats, floating-point formats generally have spacing that increases with magnitude, so their error is naturally described in relative terms.

Unit roundoff

For a normalized floating-point format with base β\beta and pp significand digits, the conventional unit roundoff for rounding to nearest is

u=12β1−p.u=\frac{1}{2}\beta^{1-p}.

A standard model is

fl⁡(x)=x(1+δ),∣δ∣≤u,\operatorname{fl}(x)=x(1+\delta), \qquad |\delta|\le u,

where fl⁡(x)\operatorname{fl}(x) denotes the rounded value. The model applies under suitable range conditions, in particular when xx is in the normal range and rounding does not overflow.

Similarly, for an elementary arithmetic operation ∘\circ,

fl⁡(a∘b)=(a∘b)(1+δ),∣δ∣≤u,\operatorname{fl}(a\circ b) =(a\circ b)(1+\delta), \qquad |\delta|\le u,

provided the exact result satisfies the required range conditions and the operation is correctly rounded.

The term machine epsilon is not used uniformly. It often denotes the gap between 11 and the next larger representable number, namely β1−p\beta^{1-p}, but some texts use it for unit roundoff. Under the gap convention, binary64 arithmetic has

εmach=2−52,u=2−53.\varepsilon_{\mathrm{mach}}=2^{-52}, \qquad u=2^{-53}.

Subnormal values and range limitations

Subnormal floating-point numbers extend representation toward zero with approximately constant absolute spacing. Their relative precision decreases as their magnitude decreases, so the same uniform relative-error bound does not apply throughout this region. Absolute-error bounds remain useful there.

Overflow and underflow require separate treatment from ordinary rounding within the normal range. Overflow occurs when a result exceeds the format’s finite range; underflow concerns tiny results near zero, which may become subnormal or round to zero.

Propagation through a computation

A bound on one operation does not by itself describe the accuracy of an entire computation. Earlier errors become inputs to later operations and may be amplified, attenuated, or canceled.

For example, floating-point addition is generally not associative:

fl⁡ ⁣(fl⁡(a+b)+c)≠fl⁡ ⁣(a+fl⁡(b+c)).\operatorname{fl}\!\left(\operatorname{fl}(a+b)+c\right) \ne \operatorname{fl}\!\left(a+\operatorname{fl}(b+c)\right).

In binary64 arithmetic, taking a=1016a=10^{16}, b=−1016b=-10^{16}, and c=1c=1, the first grouping gives 11, while the second gives 00. The small term is lost when added to a much larger value in the second grouping.

This matters in parallel computing, where changing the reduction tree used to sum many values can change the result without changing the mathematical expression.

Summation bounds

For straightforward sequential summation of nn floating-point inputs, a standard absolute-error bound is

∣s^−∑i=1nxi∣≤γn−1∑i=1n∣xi∣,\left| \widehat{s}-\sum_{i=1}^{n}x_i \right| \le \gamma_{n-1}\sum_{i=1}^{n}|x_i|,

where

γk=ku1−ku,ku<1.\gamma_k=\frac{ku}{1-ku}, \qquad ku<1.

This bound assumes the usual floating-point model and excludes problematic overflow or underflow. It concerns summation of the supplied machine values; conversion errors in the original data must be considered separately.

The bound also reveals why relative accuracy can be poor when positive and negative terms nearly cancel: the sum of the magnitudes can greatly exceed the magnitude of the final answer.

Round-off errors are deterministic for a fixed sequence of operations and rounding rules. Treating them as independent, zero-mean random quantities can be useful in some analyses, but that is an additional modeling assumption, not a general property of floating-point computation.

Cancellation

Catastrophic cancellation occurs when subtraction of nearly equal approximate quantities exposes and magnifies their existing errors relative to the small difference.

For example, suppose exact values a=1.2344a=1.2344 and b=1.2343b=1.2343 are each rounded to four significant decimal digits. Both become 1.2341.234, so their computed difference is zero even though

a−b=0.0001.a-b=0.0001.

The information has already been lost before subtraction. Subtraction itself may be exact for the rounded operands; the problem is that those operands no longer contain the digits needed to recover the small difference.

Cancellation is not always harmful. If the operands are sufficiently accurate, subtracting nearby values can produce an accurate result. The danger lies in the interaction between cancellation, preexisting error, and the sensitivity of the desired quantity.

Algebraically equivalent formulas can behave very differently. For positive, large xx, direct evaluation of

x+1−x\sqrt{x+1}-\sqrt{x}

subtracts nearly equal numbers. The equivalent expression

1x+1+x\frac{1}{\sqrt{x+1}+\sqrt{x}}

avoids that subtraction and is usually more accurate, provided the intermediate operations remain within range.

Conditioning and numerical stability

Two distinct ideas govern how round-off affects a result.

Conditioning describes the sensitivity of the mathematical problem to perturbations in its inputs. A condition number quantifies this sensitivity. For a differentiable scalar function ff, a local relative condition number is

κf(x)=∣xf′(x)f(x)∣,\kappa_f(x)= \left| \frac{x f'(x)}{f(x)} \right|,

where xx and f(x)f(x) are nonzero. A large value means that small relative input perturbations can cause much larger relative output changes.

Numerical stability describes how an algorithm introduces and propagates errors. A backward-stable algorithm produces a result that is the exact solution of a nearby problem, with “nearby” measured using an appropriate error model.

For instance, a backward-stable solution of a linear system may satisfy

(A+ΔA)x^=b+Δb,(A+\Delta A)\widehat{x}=b+\Delta b,

with small perturbations ΔA\Delta A and Δb\Delta b. If the original problem is ill-conditioned, even these small perturbations can lead to a substantial error in x^\widehat{x}.

Consequently, a stable algorithm does not guarantee a highly accurate answer to an ill-conditioned problem. Conversely, an unstable algorithm can lose accuracy even when the underlying problem is well-conditioned.

Distinction from truncation error

Truncation error arises when an exact mathematical process is replaced by a finite approximation—for example, by cutting off an infinite series or replacing a derivative with a finite difference. Round-off error arises from finite-precision representation and arithmetic. Both often occur in the same calculation.

For the forward-difference approximation

f′(x)≈f(x+h)−f(x)h,f'(x)\approx\frac{f(x+h)-f(x)}{h},

the truncation error is typically proportional to hh, assuming sufficient smoothness. Meanwhile, rounding and function-evaluation errors in the numerator may be amplified by division by hh. A simplified error model is

E(h)≈C1h+C2uh,E(h)\approx C_1h+C_2\frac{u}{h},

where the constants depend on the function and its scale.

Reducing hh initially improves the approximation, but eventually round-off can dominate. Under this simplified model, balancing the two terms gives a step size proportional to u\sqrt{u}, with the appropriate dimensional and problem-dependent factors included. Thus, smaller steps or more iterations do not automatically produce more accurate results.

Methods for controlling round-off

Several techniques reduce or bound the effects of finite precision:

  • Higher precision: reduces unit roundoff and can preserve information lost at lower precision, though it increases computational or storage costs.
  • Stable reformulation: replaces an expression with a mathematically equivalent form less susceptible to cancellation or extreme intermediate values.
  • Scaling: adjusts magnitudes to reduce overflow and underflow risk. It does not by itself eliminate intrinsic ill-conditioning.
  • Pairwise summation: combines terms through a balanced tree, reducing the depth of rounding accumulation relative to sequential summation.
  • Compensated summation: tracks corrections for information lost during additions; Kahan summation is a prominent example.
  • Fused multiply-add: evaluates ab+cab+c with one final rounding rather than separately rounding the multiplication and addition.
  • Interval arithmetic: uses outward rounding to enclose exact values within computed bounds, although repeated dependencies can make the enclosures wide.

Exact integer or rational arithmetic can avoid round-off for suitable operations, but the required storage and computation may grow substantially. It also does not automatically provide finite exact representations for general irrational or transcendental results.

Accuracy and reproducibility are separate concerns. Fixing operation order and floating-point settings can make a calculation reproducible without making it accurate; increasing precision can improve accuracy while still permitting small differences across implementations.