{
  "id": "neural-network-quantization",
  "title": "Neural Network Quantization",
  "lang": "en",
  "source_lang": "en",
  "summary": "Neural network quantization represents model parameters and computations with reduced numerical precision to lower memory, bandwidth, and computational costs.",
  "category": "computer-science",
  "subcategory": "artificial-intelligence",
  "type": "concept",
  "aliases": [
    "Neural net quantization",
    "Neural-network quantization"
  ],
  "generated_at": "2026-11-03",
  "model": "pro/gpt-6.1-sol",
  "url": "https://aiwiki.page/en/neural-network-quantization/",
  "translations": {
    "zh-Hans": {
      "url": "https://aiwiki.page/zh/neural-network-quantization/",
      "json": "https://aiwiki.page/zh/neural-network-quantization.json"
    }
  },
  "sections": [
    {
      "heading": "",
      "text": "Neural network quantization is the representation of numerical quantities in an artificial neural network using a restricted set of values, typically to reduce storage requirements and execution costs. Instead of retaining every weight or intermediate activation in a conventional 32-bit floating-point format, a quantized model may use 8-bit integers, 4-bit codes, or low-precision floating-point numbers. Quantization can be applied after training, incorporated into training, or combined with limited adaptation of an existing model.\n\nIts importance comes from the cost of moving and processing neural-network data. Large models require substantial memory capacity and bandwidth, while small devices operate under strict power and latency constraints. Reduced precision can make inference practical on devices ranging from microcontrollers and smartphones to data-center accelerators. It also supports memory-efficient adaptation of large language models, although a model’s nominal bit width alone does not determine its speed or accuracy.\n\nQuantization is generally a form of lossy data compression: different original values can receive the same stored representation. Its central engineering problem is to choose numerical formats, scaling rules, and execution methods that save resources without introducing unacceptable changes in model behavior. The resulting trade-offs depend on network architecture, data distributions, hardware instructions, and the accuracy requirements of the application."
    },
    {
      "heading": "Numerical foundations and quantization error",
      "text": "A quantizer maps a numerical input to one member of a finite representable set. In uniform affine quantization, an input \\(x\\) is converted into an integer code \\(q\\) using a scale \\(s>0\\) and an integer zero point \\(z\\):\n\n\\[q=\\operatorname{clip}\\!\\left( \\operatorname{round}\\!\\left(\\frac{x}{s}\\right)+z,\\, q_{\\min},q_{\\max} \\right).\\]\n\nAn approximate reconstruction is then\n\n\\[\\hat{x}=s(q-z).\\]\n\nThe scale specifies the spacing between adjacent reconstructed values, while the zero point allows numerical zero to be represented exactly even when the code range is unsigned. For an unsigned 8-bit format, the available codes are typically 0 through 255. Signed formats commonly use codes from \\(-128\\) through 127, although some symmetric schemes deliberately exclude one endpoint.\n\nQuantization error has two main components. Rounding error arises when a value lies between representable levels. Clipping error arises when it lies outside the chosen range and is replaced by an endpoint. With round-to-nearest and no clipping, the absolute error of uniform quantization is at most \\(s/2\\). Under the additional idealized assumption that rounding errors are uniformly distributed across a quantization interval, their mean squared value is \\(s^2/12\\). Actual neural-network errors need not satisfy that assumption: they can be correlated with inputs, concentrated around particular values, and amplified by later computations.\n\nChoosing a range therefore creates a trade-off. A wide range preserves rare extreme values but increases the spacing between ordinary values. A narrower range improves resolution for the bulk of a distribution but clips its tails. At very low precision, this compromise becomes especially severe. Four bits provide only 16 codes, so a small number of outliers can consume a substantial fraction of the available numerical range. Symmetric quantization usually fixes the zero point at zero and represents values around the origin. Asymmetric quantization places the representable interval more flexibly, which can benefit nonnegative or skewed activations.\n\nNot all quantization uses uniformly spaced integer levels. Low-precision floating-point formats allocate bits to sign, exponent, and significand, providing finer absolute spacing near zero and coarser spacing at larger magnitudes. Nonuniform codebooks instead associate stored indices with selected reconstruction values. Binary and ternary networks restrict selected quantities to two or three values. These representations differ in numerical behavior and hardware cost; describing all of them simply as “low bit” conceals important distinctions."
    },
    {
      "heading": "Quantization targets, granularity, and mixed precision",
      "text": "A neural network contains several numerically distinct objects: learned weights, intermediate activations, biases, accumulators, gradients, and optimizer state. Quantizing one does not imply quantizing the others. Weight-only quantization compresses the stored parameters while leaving activations and much arithmetic at higher precision. Weight-and-activation quantization also reduces the precision of inputs to major operations. Quantized training extends the problem to backward computations and parameter updates, where small numerical changes can accumulate over many optimization steps.\n\nA common notation describes weight and activation precision separately. W8A8 usually denotes 8-bit weights and 8-bit activations; W4A16 usually denotes 4-bit weights with 16-bit activations. Such labels are incomplete specifications. They do not necessarily identify whether the numbers are integers or floating point, how values are grouped, what precision accumulators use, or whether some layers remain unquantized. A W4A16 implementation may unpack weights and perform multiplication in a higher-precision format rather than execute native 4-bit-by-16-bit arithmetic.\n\nGranularity describes which values share quantization parameters. Per-tensor quantization assigns one scale, and possibly one zero point, to an entire tensor. Per-channel quantization assigns separate parameters to channels, often the output channels of a convolution or linear transformation. This is valuable when channels have very different ranges, as can occur after folding batch normalization into a preceding layer. Group-wise quantization divides weights into smaller blocks, such as groups of 32, 64, or 128 elements, each with its own parameters. Finer granularity generally improves local numerical resolution but increases metadata and implementation complexity.\n\nFor \\(N\\) weights stored with \\(b\\) bits each and one \\(h\\)-bit scale per group of \\(G\\) weights, the approximate storage requirement, excluding other metadata, is\n\n\\[Nb+\\frac{N}{G}h \\quad \\text{bits}.\\]\n\nFour-bit weights with a 16-bit scale for every 32 weights therefore require about 4.5 bits per weight, not exactly four. Zero points, padding, exceptional values, and alignment can add further overhead. Quantizing the scales themselves can reduce this overhead.\n\nMixed-precision schemes allocate different formats to different parts of a model. Sensitive layers may retain 8-bit or 16-bit representations while others use four bits or fewer. The first and last layers have often received special treatment in aggressively quantized vision networks, but sensitivity is architecture-dependent. Within transformers, attention projections, feed-forward layers, normalization operations, and output projections can have different numerical requirements. Precision assignment is consequently a constrained optimization problem involving accuracy, memory, latency, and available hardware kernels rather than a uniform replacement of one data type with another."
    },
    {
      "heading": "Integer arithmetic and execution semantics",
      "text": "Integer quantization can replace expensive floating-point operations with integer arithmetic, but this requires more than storing weights as integers. Consider a dot product with quantized activations and weights:\n\n\\[a_i\\approx s_a(q_{a,i}-z_a),\\qquad w_i\\approx s_w(q_{w,i}-z_w).\\]\n\nIts reconstructed value is approximately\n\n\\[y\\approx s_as_w \\sum_i(q_{a,i}-z_a)(q_{w,i}-z_w).\\]\n\nThe inner sum can be evaluated with integer multiply–accumulate instructions. Common 8-bit inference implementations accumulate products in 32-bit integers, because summing many low-precision products requires substantially more range than representing an individual input.\n\nZero points affect implementation cost. Expanding the product gives\n\n\\[\\sum_i q_{a,i}q_{w,i} -z_w\\sum_i q_{a,i} -z_a\\sum_i q_{w,i} +nz_az_w.\\]\n\nWhen weights are symmetric and \\(z_w=0\\), the input-dependent correction involving \\(\\sum_i q_{a,i}\\) disappears. Weight-dependent sums can often be precomputed. This is one reason many integer inference systems combine symmetric weights with potentially asymmetric activations. Bias terms can be represented in the accumulator’s units, using a scale related to the product of activation and weight scales; per-channel weight scaling requires corresponding per-channel bias treatment.\n\nAn accumulated result must usually be converted to the next layer’s quantized representation. This operation, called requantization, applies a multiplier approximately equal to \\(s_as_w/s_y\\), followed by an output zero-point adjustment, rounding, and clipping. An integer-only implementation approximates the multiplier using an integer multiplier and a bit shift. The exact rounding convention matters: ties-to-even, ties-away-from-zero, and other rules can produce different outputs over many layers. Accumulator overflow behavior is equally important, particularly for unusually long dot products or narrow intermediate representations.\n\nWhole-network execution introduces additional constraints. Adding two residual branches requires compatible scales or explicit rescaling. Concatenation may require harmonizing the representations of several inputs. A rectified linear unit can often be implemented by clipping integer outputs below the code for zero, whereas normalization, exponentiation, and softmax require more elaborate approximations or higher-precision execution. Folding normalization into adjacent linear operations can simplify the graph, but changes the effective weight distributions that the quantizer sees.\n\nThese details make quantization partly a contract between model conversion and runtime execution. Two systems using identical nominal bit widths can disagree because their scales, clipping rules, operator implementations, or intermediate precision differ. Correct deployment therefore depends on preserving numerical semantics across the computational graph, not merely on converting parameter files."
    },
    {
      "heading": "Post-training quantization and calibration",
      "text": "Post-training quantization converts a trained model without repeating its original training process. The simplest form rounds weights to a chosen grid and determines activation ranges from representative examples. Because it avoids full retraining, it is attractive when the original training data are unavailable, computational resources are limited, or a pretrained model must be deployed across several devices. Depending on the method, it may require no sample data, a small calibration set, or a more substantial reconstruction procedure.\n\nCalibration estimates the numerical ranges encountered during inference. A min–max observer records the smallest and largest activation values, preserving the observed range but potentially sacrificing resolution to rare extremes. Percentile methods discard a selected fraction of tail values. Histogram-based methods search for thresholds that minimize a reconstruction criterion, such as mean squared error or divergence between approximate distributions. None of these criteria directly guarantees the smallest change in task accuracy: preserving a tensor numerically and preserving the network’s final decisions are related but distinct objectives.\n\nThe calibration set must exercise the paths and distributions relevant to deployment. Image resolution, preprocessing, sequence length, language, and task type can all influence activation statistics. A range fitted to short text inputs may poorly describe long-context inference; one fitted to common images may miss unusual lighting or sensor conditions. Distribution shift can therefore turn a well-behaved static quantizer into a source of frequent clipping. Dynamic quantization instead derives some parameters from runtime inputs, often per tensor or per token, trading additional computation for adaptation to changing ranges.\n\nMore advanced methods adjust the model before or during rounding. Cross-layer equalization rescales adjacent layers in ways that preserve their combined function under suitable architectural conditions, while making their parameter ranges easier to quantize. Bias correction compensates for systematic output shifts. AdaRound, introduced in 2020, optimizes weight-rounding choices rather than simply rounding every value to its nearest grid point. Layer-wise and block-wise reconstruction methods seek quantized parameters that reproduce intermediate outputs on calibration examples.\n\nFor a linear layer with weights \\(W\\) and calibration inputs \\(X\\), a reconstruction objective can be written as\n\n\\[\\min_{\\hat W}\\ \\lVert WX-\\hat W X\\rVert_F^2,\\]\n\nwhere \\(\\hat W\\) must obey the quantized representation. This objective accounts for which input directions are actually active, rather than treating all weight errors as equally important. Optimization remains constrained by calibration coverage, computational cost, and the mismatch between local reconstruction error and end-to-end model quality. Post-training quantization is therefore a family of methods, ranging from inexpensive rounding to substantial data-driven optimization."
    },
    {
      "heading": "Quantization-aware training and low-precision learning",
      "text": "Quantization-aware training exposes a network to quantization effects while its parameters are being optimized. A common implementation inserts fake-quantization operations into an otherwise floating-point graph. During the forward pass, these operations round and clip values as the eventual deployment quantizer would, then express the reconstructed values in floating point. This simulates numerical degradation without necessarily executing low-precision hardware instructions during training.\n\nRounding poses a difficulty for backpropagation because its derivative is zero almost everywhere and undefined at discontinuities. A widely used approximation is the straight-through estimator, which substitutes a useful surrogate derivative for the true derivative. Within a selected range, the backward pass may treat quantization as approximately the identity function, while clipped values may receive different or zero gradients. This estimator is not an exact derivative of the discrete operation; it is a practical optimization device whose effectiveness depends on the model and training setup.\n\nMany implementations retain full-precision master weights. These accumulate small updates that would vanish if each update were immediately rounded to a coarse grid. The forward pass uses quantized versions, while the optimizer updates the master parameters. This arrangement distinguishes quantization-aware training from fully low-precision training: simulating 4-bit inference does not imply that gradients, optimizer state, or all training arithmetic use four bits. Optimizers such as Adam maintain additional state whose memory footprint can remain substantial even when forward-pass weights are compressed.\n\nTraining may also optimize quantization scales or clipping thresholds. Learned Step Size Quantization, introduced in 2019, treats step sizes as trainable quantities with appropriately scaled gradient estimates. Other approaches learn activation ranges or use regularization to make parameter distributions easier to quantize. Knowledge distillation can provide an additional training signal, encouraging a quantized student to reproduce a higher-precision teacher’s outputs. These methods can recover accuracy at lower bit widths, but require training data, optimization effort, and careful matching between simulated and deployed operators.\n\nLow-precision training more broadly includes reduced-precision forward and backward arithmetic, quantized communication between workers, and compressed optimizer states. Stochastic rounding is useful in some settings because it can preserve small updates in expectation rather than always discarding them. However, these techniques solve different bottlenecks. Communication quantization in distributed training does not necessarily change the inference model, and quantized optimizer states do not imply quantized activations. Reporting the precision of each component is essential for understanding both numerical behavior and actual resource savings."
    },
    {
      "heading": "Large language models and transformer-specific methods",
      "text": "Large language models present distinctive quantization challenges because their parameter counts make weight storage expensive and their activation distributions can contain unusually large values. In some transformers, a small subset of hidden dimensions has much larger activation magnitudes than the others. A single scale shared across all dimensions then allocates much of its range to these outliers, leaving insufficient resolution for ordinary values. Layer-to-layer variation and token-dependent changes further complicate static range selection.\n\nSeveral influential methods address different parts of this problem. LLM.int8(), introduced in 2022, combines 8-bit matrix multiplication for most values with higher-precision treatment of selected outlier dimensions. SmoothQuant, released in 2022 and published in 2023, uses a function-preserving rescaling to transfer some activation quantization difficulty into the weights, making 8-bit weight-and-activation inference more practical. For a suitable diagonal scaling matrix \\(D\\),\n\n\\[XW=(XD^{-1})(DW).\\]\n\nThe unquantized product is unchanged, but the distributions presented to the two quantizers can become more favorable.\n\nWeight-only methods pursue a different trade-off. GPTQ, introduced in 2022, uses approximate second-order information derived from calibration inputs to guide sequential weight quantization and compensate for errors in remaining weights. AWQ, introduced in 2023, uses activation information to identify and protect important weight channels through scaling before low-bit quantization. These approaches are particularly relevant when reading weights from memory dominates execution time. Their names identify algorithmic approaches, not interchangeable storage formats: practical implementations differ in grouping, packing, kernels, and supported architectures.\n\nQLoRA, introduced in 2023, combines a frozen quantized base model with trainable low-rank adapters. Its 4-bit NormalFloat representation uses a nonuniform codebook designed for approximately normally distributed weights, while double quantization compresses quantization constants. This reduces the memory needed for fine-tuning without requiring every base-model parameter to be trained. It is not equivalent to fully 4-bit training, because adapter updates and important computations use higher precision.\n\nAutoregressive generation also stores a key–value cache for previously processed tokens. Its size increases with context length, batch size, layer count, and the number and dimension of stored attention heads. Quantizing weights does not reduce this cache automatically. Cache quantization introduces separate questions about key and value distributions, per-token versus per-channel scaling, and how errors affect later attention operations. Long-context workloads can therefore require a different precision strategy from short-prompt workloads, even when both use the same quantized weights."
    },
    {
      "heading": "Hardware, software, and realized performance",
      "text": "The main resource benefits of quantization are reduced storage, reduced data movement, and access to more efficient arithmetic. An ideal 4-bit representation uses one eighth the weight storage of 32-bit floating point before metadata and exceptions. Whether this translates into faster inference depends on the system bottleneck. If execution is limited by arithmetic throughput, low-precision instructions can help directly. If it is limited by memory bandwidth, compressed weights can help even when they are unpacked into a higher-precision representation before multiplication.\n\nWorkload shape strongly affects the outcome. Single-token decoding at small batch sizes often has relatively little arithmetic per weight read, making weight bandwidth especially important. Processing a long prompt or a large batch can reuse weights across many inputs, increasing arithmetic intensity. A method that accelerates memory-bound decoding may offer smaller gains during compute-bound prompt processing. At long contexts, reading the attention cache can also become a major cost. Latency, throughput, and maximum feasible batch size are therefore separate performance outcomes.\n\nHardware support is not uniform. CPUs may provide vector or matrix instructions for integer dot products; GPUs and specialized accelerators may provide matrix units for selected integer and floating-point formats. An application-specific integrated circuit can be designed around particular precisions and dataflows. Support for a nominal data type does not guarantee efficient support for every grouping scheme, matrix dimension, or operator. Packing order, alignment, padding, memory layout, and conversion instructions can determine whether the expected advantage is realized.\n\nSoftware must map the quantized graph onto available kernels. Frameworks such as PyTorch, interchange representations such as ONNX, and deployment systems such as TensorRT represent or transform quantization information in different ways. Explicit quantize and dequantize operations can communicate numerical intent, while a compiler or runtime may fuse them with surrounding operations. Unsupported layers can fall back to higher precision, introducing conversions and additional memory traffic. A model with many nominally quantized operators can still spend considerable time in higher-precision code.\n\nLow-bit arithmetic also has nontrivial overheads. Four-bit values must often be packed two per byte, scales must be loaded, and groups must be decoded. Small matrices may not contain enough work to amortize kernel-launch or setup costs. Conversely, fitting a model entirely in accelerator memory can eliminate transfers or offloading and produce a disproportionately large practical improvement. Meaningful performance claims therefore identify the hardware, software versions, batch size, sequence lengths, precision configuration, and whether timing includes preprocessing, cache management, and data transfer."
    },
    {
      "heading": "Accuracy measurement, robustness, and reproducibility",
      "text": "Quantization quality cannot be reduced to a single tensor error. Weight reconstruction error, intermediate activation error, and final task performance measure different effects. A large error in an insensitive direction may have little influence on predictions, whereas a small error near a classification boundary can change an output. Error propagation also depends on the architecture: residual connections, normalization, nonlinearities, and attention can suppress some disturbances while amplifying others. Numerical comparisons are useful diagnostics, but they do not replace end-to-end evaluation.\n\nFor vision systems, evaluation may include classification accuracy, detection metrics, or segmentation quality. For language models, perplexity measures predictive fit to a token sequence, while task benchmarks measure performance on particular forms of reasoning, knowledge, or generation. A small average perplexity change does not guarantee unchanged task results. Generated text can diverge after a single altered token, and sampling introduces additional variation. Comparisons therefore need clearly specified decoding settings and sufficient examples to distinguish quantization effects from measurement noise.\n\nCalibration data should be distinguished from the held-out test set. Repeatedly choosing scales, clipping thresholds, or bit allocations based on test outcomes can create evaluation leakage. A validation set can support model selection, while final evaluation remains separate. Reproducible reporting also identifies calibration sample counts and selection procedures, whether labels were used, and whether weights were reconstructed, fine-tuned, or otherwise updated. Two systems both described as post-training quantization may have used very different data and optimization budgets.\n\nRobustness requires attention to more than average performance on common inputs. Rare activation patterns can produce clipping that calibration never observed. Long sequences, unusual input magnitudes, different image preprocessing, and specialized language domains can reveal weaknesses hidden by an aggregate benchmark. Quantization does not inherently improve or worsen every form of robustness, and broad claims require evaluation under the relevant conditions. It also does not anonymize training data, remove memorized information, or establish security properties merely by reducing numerical precision.\n\nResource measurement needs similar care. Model-file size differs from resident memory, which may include unpacked weights, workspaces, caches, allocator overhead, and temporary buffers. Peak memory during loading can exceed steady-state inference memory. Energy per request differs from instantaneous power, and faster execution does not automatically imply the same proportional energy reduction. Comprehensive comparisons report accuracy together with latency distributions, throughput, peak memory, and workload details. This makes it possible to distinguish genuine improvements from changes caused by different batching, kernels, or evaluation procedures."
    },
    {
      "heading": "Historical development and continuing research",
      "text": "Neural-network quantization draws on older work in digital signal processing, fixed-point arithmetic, and numerical approximation. Limited precision was relevant to early neural-network hardware because memory and multiplier circuits were expensive. The rapid growth of deep learning in the 2010s renewed the problem at a much larger scale: networks achieved strong results with floating-point training but needed to run on mobile processors, embedded systems, and energy-constrained accelerators.\n\nSeveral mid-2010s developments established influential directions. Matthieu Courbariaux, Yoshua Bengio, and Jean-Pierre David’s BinaryConnect work in 2015 explored training with binary weights during forward and backward propagation while retaining higher-precision underlying parameters. Song Han, Huizi Mao, and William Dally’s Deep Compression, released in 2015, combined pruning, trained quantization through weight sharing, and Huffman coding. These techniques addressed complementary forms of redundancy rather than treating reduced precision as an isolated transformation. Mohammad Rastegari and colleagues’ XNOR-Net in 2016 investigated approximating convolutional computations using binary operations and scaling factors.\n\nPractical integer deployment became increasingly important as neural networks moved onto consumer devices. Benoit Jacob and colleagues’ 2018 work, “Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference,” described a widely influential framework linking affine quantization, training simulation, and integer execution. Subsequent research improved post-training calibration, optimized rounding, learned quantization parameters, and developed mixed-precision allocation. The focus expanded from whether low precision was possible to how closely training, conversion, and hardware execution could be coordinated.\n\nThe growth of transformers and large language models in the early 2020s shifted attention toward activation outliers, very large weight matrices, and memory-bound generation. Low-bit weight-only methods became especially significant because fitting a model into accelerator memory could be as important as reducing arithmetic cost. In parallel, low-precision floating-point formats, including different 8-bit exponent–significand allocations, became important for accelerated training and inference. Integer and floating-point approaches consequently developed alongside one another rather than forming a single progression toward ever smaller integers.\n\nContinuing research examines precision below four bits, vector and codebook quantization, learned transformations, and joint optimization of models and kernels. Extremely low bit widths create difficult questions about which information must be preserved and where additional computation should compensate for reduced storage. Training-aware architectures may be easier to quantize than architectures designed only for high-precision execution. Long-context cache compression, multimodal activation distributions, and irregular mixture-of-experts workloads introduce further challenges. Across these settings, numerical formats, statistical sensitivity, and physical execution remain closely coupled: improving one component in isolation does not necessarily improve the deployed system."
    }
  ],
  "links": [
    "artificial-neural-network",
    "neural-network-inference",
    "smartphone",
    "large-language-model",
    "data-compression",
    "probability-distribution",
    "floating-point-arithmetic",
    "batch-normalization",
    "transformer-architecture",
    "rectified-linear-unit",
    "computational-graph",
    "training-data",
    "distribution-shift",
    "backpropagation",
    "adam-optimizer",
    "knowledge-distillation",
    "distributed-computing",
    "low-rank-adaptation",
    "fine-tuning",
    "kv-cache",
    "attention-mechanism",
    "central-processing-unit",
    "graphics-processing-unit",
    "application-specific-integrated-circuit",
    "pytorch",
    "open-neural-network-exchange",
    "compiler",
    "perplexity",
    "test-set",
    "validation-set",
    "deep-learning",
    "huffman-coding"
  ],
  "sources": [],
  "license": "https://creativecommons.org/licenses/by/4.0/"
}