{
  "id": "quantization-signal-processing",
  "title": "Quantization (signal processing)",
  "lang": "en",
  "source_lang": "en",
  "summary": "Quantization maps signal values to a discrete set of levels, enabling digital representation while introducing an approximation error governed by resolution, signal statistics, and quantizer design.",
  "category": "technology",
  "subcategory": "engineering",
  "type": "concept",
  "aliases": [
    "Signal quantization",
    "Amplitude quantization"
  ],
  "generated_at": "2026-11-04",
  "model": "pro/gpt-6.1-sol",
  "url": "https://aiwiki.page/en/quantization-signal-processing/",
  "translations": {
    "zh-Hans": {
      "url": "https://aiwiki.page/zh/quantization-signal-processing/",
      "json": "https://aiwiki.page/zh/quantization-signal-processing.json"
    }
  },
  "sections": [
    {
      "heading": "",
      "text": "Quantization in signal processing is the mapping of values from a large or continuous set to a smaller, discrete set of representative values. A quantizer assigns each input value to a decision region and replaces it with that region’s reconstruction value. This enables finite-precision storage, transmission, and computation, but usually discards information. Quantization is fundamental to analog-to-digital conversion, digital audio and images, data compression, and finite-precision numerical processing.\n\nQuantization is distinct from sampling: sampling discretizes an independent variable, usually time or spatial position, whereas quantization discretizes the values of the signal. A sampled signal can still have continuously valued amplitudes until those amplitudes are quantized."
    },
    {
      "heading": "Mathematical description",
      "text": "A scalar quantizer operates on one number at a time. Let its decision regions be \\(R_1,\\ldots,R_M\\), with corresponding reconstruction values \\(y_1,\\ldots,y_M\\). Its mapping is\n\n\\[Q(x)=y_i \\quad \\text{if }x\\in R_i.\\]\n\nFor an ordinary ordered scalar quantizer, the regions are intervals separated by decision thresholds. An encoder produces an index identifying the selected region, and a decoder maps that index to a reconstruction value. The numerical value of the index need not equal the reconstructed signal amplitude.\n\nA fixed-length binary representation of \\(M\\) possible levels requires\n\n\\[b=\\left\\lceil\\log_2 M\\right\\rceil\\]\n\nbits per value. Conversely, a \\(b\\)-bit code can identify at most \\(2^b\\) distinct levels. Variable-length coding can reduce the average number of bits when some indices occur more frequently than others.\n\nThe quantization error is commonly defined as\n\n\\[e=Q(x)-x.\\]\n\nFor a random input \\(X\\), a frequently used distortion measure is the mean squared error,\n\n\\[D=\\mathbb{E}\\!\\left[(Q(X)-X)^2\\right],\\]\n\nwhere \\(\\mathbb{E}\\) denotes expected value. Other measures may place greater weight on large errors, relative errors, or perceptually important differences.\n\nBecause multiple inputs generally produce the same output, quantization is a many-to-one operation and is not exactly reversible without additional information."
    },
    {
      "heading": "Uniform quantization",
      "text": "A uniform quantizer has equally spaced reconstruction levels over its normal operating range. The spacing \\(\\Delta\\) is called the quantization step size. An ideal unbounded round-to-nearest quantizer can be written as\n\n\\[Q(x)=\\Delta\\,\\operatorname{round}\\!\\left(\\frac{x}{\\Delta}\\right).\\]\n\nAway from overload, its error satisfies\n\n\\[|e|\\leq\\frac{\\Delta}{2},\\]\n\napart from the choice of convention at exact decision thresholds.\n\nTwo common arrangements are:\n\n- Mid-tread: zero is a reconstruction level, with a decision interval centered on zero.\n- Mid-rise: zero is a decision threshold, and the nearest reconstruction levels are typically \\(-\\Delta/2\\) and \\(+\\Delta/2\\).\n\nActual converters have finitely many levels. If the nominal input span is \\(R\\) and it is divided into \\(2^b\\) equal-width bins, then\n\n\\[\\Delta=\\frac{R}{2^b}.\\]\n\nSpecifications must distinguish bin widths from distances between extreme reconstruction levels: if both endpoints are reconstruction values, the endpoint-to-endpoint span is instead \\((2^b-1)\\Delta\\).\n\nRounding, truncation, and overload\n\nRounding chooses a nearby representable value. Truncation discards lower-order information according to a particular arithmetic convention. Depending on whether truncation means rounding toward zero, toward negative infinity, or simply dropping bits in a specific representation, its error can have different signs and biases.\n\nA finite-range quantizer must also handle inputs beyond its designed range. A saturating quantizer assigns them to an extreme output level. The resulting overload distortion can greatly exceed the error within normal quantization cells. Arithmetic wraparound is another possible finite-word-length behavior, but it is distinct from saturation and can produce abrupt, large errors.\n\nIncreasing bit depth reduces the quantization step for a fixed input range; it does not prevent overload if that range is exceeded."
    },
    {
      "heading": "Quantization error and the noise model",
      "text": "Quantization is deterministic for a fixed quantizer and input. Its error is therefore not inherently random, independent, or spectrally white. Nevertheless, treating it as additive noise is often a useful approximation.\n\nFor a sufficiently fine uniform quantizer, when the input spans many cells and its probability distribution varies little across each cell, the error is often approximated as uniformly distributed over \\([-\\Delta/2,\\Delta/2]\\). Under this model,\n\n\\[\\mathbb{E}[e]\\approx0, \\qquad \\sigma_e^2\\approx\\frac{\\Delta^2}{12}.\\]\n\nFor input mean-square power \\(P_x\\), the signal-to-quantization-noise ratio is approximately\n\n\\[\\mathrm{SQNR} = 10\\log_{10}\\!\\left(\\frac{P_x}{\\sigma_e^2}\\right).\\]\n\nFor an ideal \\(b\\)-bit uniform quantizer driven by a full-scale sine wave, without overload and under the usual noise-model assumptions,\n\n\\[\\mathrm{SQNR}\\approx6.02b+1.76\\ \\mathrm{dB}.\\]\n\nThis is the origin of the rule that each additional bit improves ideal quantization-limited performance by about 6 dB. It is not a universal formula for signal-to-noise ratio: lower-amplitude signals, different waveforms, restricted measurement bandwidths, and correlated errors change the result.\n\nFor constant, slowly varying, low-amplitude, or coherently sampled periodic inputs, quantization error may be strongly signal-dependent. It can appear as bias, harmonic distortion, or discrete spectral tones rather than a smooth noise floor. Physical converters also introduce thermal noise, timing uncertainty, and nonlinearities, so nominal bit depth alone does not determine usable resolution or dynamic range."
    },
    {
      "heading": "Dither",
      "text": "Dither is a deliberately introduced signal, usually random noise, added before quantization to control the statistical behavior of quantization error. It can turn objectionable signal-dependent distortion into a more uniform noise background.\n\nDither can also make changes smaller than one quantization step observable statistically. A small input change may alter the probabilities of neighboring output codes even when it cannot be represented by a single output sample.\n\nTwo arrangements are important:\n\n- Nonsubtractive dither: the added signal remains in the reconstructed output and contributes noise.\n- Subtractive dither: a matching dither sequence is subtracted after reconstruction. With suitable dither statistics and no overload, this can make the quantization error uniform and independent of the input.\n\nNot every dither distribution provides the same guarantees. Dither generally exchanges deterministic artifacts for controlled noise; it does not make an individual finite-bit sample exact."
    },
    {
      "heading": "Nonuniform and adaptive quantization",
      "text": "A nonuniform quantizer uses unequal decision-region widths or unequal reconstruction-level spacing. It can allocate finer resolution where values occur more often, where errors are more consequential, or where a chosen perceptual model demands greater precision.\n\nCompanding\n\nCompanding applies a nonlinear compression function before uniform quantization and a corresponding expansion function after decoding. Smaller input amplitudes can thereby receive finer effective resolution than larger amplitudes.\n\nThe logarithmic-like μ-law and A-law systems are established examples in digital telecommunications. Their purpose is not to minimize absolute error uniformly, but to provide an appropriate distribution of resolution over signals with a wide amplitude range.\n\nOptimal scalar quantization\n\nFor a known input probability density \\(p(x)\\), a fixed number of levels, and mean squared error as the objective, an optimal scalar quantizer satisfies two familiar conditions:\n\n- Each decision threshold lies halfway between neighboring reconstruction values.\n- Each reconstruction value is the conditional mean of the input within its decision region.\n\nAlternating these updates gives the Lloyd–Max algorithm, an iterative optimization method. Its outcome can depend on initialization and need not be a global optimum.\n\nIn the high-resolution regime, the optimal density of reconstruction levels for squared-error scalar quantization is proportional to \\(p(x)^{1/3}\\). Thus, more levels are allocated to more probable regions, but generally not in direct proportion to probability density.\n\nAdaptive quantization\n\nAn adaptive quantizer changes its step size, range, or other parameters as signal statistics change. Adaptation can be based on recent samples, transmitted side information, or state maintained by both encoder and decoder.\n\nSuch methods can improve efficiency for nonstationary signals, but introduce complications involving synchronization, abrupt transitions, and error propagation."
    },
    {
      "heading": "Vector quantization and compression",
      "text": "Vector quantization maps a group of values to one representative vector from a finite codebook. Rather than separately quantizing every component, it partitions a multidimensional space into regions.\n\nThis can exploit dependencies between components and offer better rate–distortion performance than simple independent scalar quantization. Its costs include codebook storage, training, search complexity, and possible sensitivity to mismatches between training data and actual inputs.\n\nQuantization is also central to lossy compression. A typical system transforms or predicts a signal, quantizes the resulting coefficients or prediction errors, and then applies entropy coding to the indices.\n\nIn transform-based image and video coding, coarse quantization suppresses small coefficients and reduces the information needed to describe the signal. Depending on the transform and coding structure, visible consequences can include loss of texture, banding, ringing, or block boundaries.\n\nRate–distortion theory formalizes the trade-off between the information rate used to represent a source and the distortion tolerated in its reconstruction. Practical quantizers operate within additional constraints such as delay, computational cost, and perceptual quality."
    },
    {
      "heading": "Oversampling and noise shaping",
      "text": "Increasing the sampling rate does not, by itself, reduce the quantization step. However, under an approximately white-error model, oversampling distributes quantization-noise power over a wider frequency range. A subsequent low-pass filter and decimation stage can remove out-of-band noise.\n\nIf the signal bandwidth is \\(B\\) and the sampling rate is \\(f_s\\), the oversampling ratio is\n\n\\[\\mathrm{OSR}=\\frac{f_s}{2B}.\\]\n\nFor unshaped white quantization noise with fixed total power, the fraction remaining in the signal band is approximately \\(1/\\mathrm{OSR}\\). Doubling the oversampling ratio therefore improves in-band SQNR by about 3 dB under these assumptions.\n\nDelta–sigma modulation combines oversampling with feedback that shapes quantization error, usually suppressing it at low frequencies while increasing it at higher frequencies. Digital filtering then removes much of the out-of-band error.\n\nNoise shaping redistributes error rather than eliminating it. Practical limitations include loop stability, overload, idle tones, component imperfections, and the conditions under which a linearized noise model is valid."
    },
    {
      "heading": "Applications and finite-precision effects",
      "text": "Conversion and recording\n\nAnalog-to-digital converters quantize sampled voltages or other electrical quantities into codes. Digital-to-analog converters map codes back to physical output levels; reconstruction filtering then forms the desired continuous-time output.\n\nIn digital sound recording, sample rate determines the represented frequency range, while bit depth determines amplitude resolution. Reducing sample rate and reducing bit depth are therefore different operations with different artifacts.\n\nDigital computation and control\n\nQuantization also occurs when intermediate arithmetic results are rounded to a limited word length. Fixed-point arithmetic commonly has approximately constant absolute spacing over its range. Floating-point arithmetic has spacing that changes with magnitude, giving roughly constant relative precision within its normal range.\n\nIn digital filters, coefficient and state quantization can change frequency response, move poles, and produce persistent oscillations known as limit cycles. In control systems, finite sensor and actuator resolution can affect tracking, stability margins, and small-signal behavior.\n\nMachine learning\n\nIn machine learning, quantization represents model weights, activations, or intermediate values with reduced precision. Common schemes map a real-valued quantity \\(x\\) to an integer \\(q\\) using a scale \\(s>0\\) and an integer zero point \\(z\\):\n\n\\[q= \\operatorname{clip}\\!\\left( \\operatorname{round}(x/s)+z,\\, q_{\\min},q_{\\max} \\right), \\qquad \\hat{x}=s(q-z).\\]\n\nReduced precision can lower memory use and data movement and enable faster computation on suitable hardware. Accuracy depends on the numerical format, calibration data, treatment of outliers, and whether training accounts for quantization. A lower nominal bit count does not guarantee faster execution on every processor."
    },
    {
      "heading": "Historical development",
      "text": "Quantization became a central engineering problem with the development of pulse-code modulation in the twentieth century. Alec Reeves’s work in the late 1930s established an important basis for representing sampled communication signals by discrete numerical codes.\n\nDuring the mid-twentieth century, research connected quantizer design with statistical decision-making and information theory. Work by Stuart Lloyd and Joel Max established widely used conditions and iterative methods for minimum-mean-square-error scalar quantization.\n\nLater developments in digital telecommunications, consumer audio, image and video coding, oversampling converters, and low-precision computation extended quantization from a conversion-stage operation into a general method for allocating limited numerical precision."
    }
  ],
  "links": [
    "signal-processing",
    "expected-value",
    "probability-distribution",
    "signal-to-noise-ratio",
    "telecommunications",
    "mathematical-optimization",
    "lossy-data-compression",
    "signal-filter",
    "sound-recording",
    "control-theory",
    "machine-learning"
  ],
  "sources": [],
  "license": "https://creativecommons.org/licenses/by/4.0/"
}