{
  "id": "predictive-coding",
  "title": "Predictive coding",
  "lang": "en",
  "source_lang": "en",
  "summary": "Predictive coding represents information through prediction errors, linking signal compression, theories of perception, and computational models of learning.",
  "category": "science",
  "subcategory": "biology",
  "type": "concept",
  "aliases": [
    "Predictive error coding"
  ],
  "generated_at": "2026-11-03",
  "model": "pro/gpt-6.1-sol",
  "url": "https://aiwiki.page/en/predictive-coding/",
  "translations": {
    "zh-Hans": {
      "url": "https://aiwiki.page/zh/predictive-coding/",
      "json": "https://aiwiki.page/zh/predictive-coding.json"
    }
  },
  "sections": [
    {
      "heading": "",
      "text": "Predictive coding is a computational strategy in which a system predicts an incoming signal and represents the difference between that prediction and the signal actually received. This difference, called a prediction error or residual, carries information that the prediction has not already captured. In engineering, the strategy supports efficient transmission and storage of correlated signals. In neuroscience, it describes a family of hypotheses about how the brain interprets sensory input, learns environmental regularities, and coordinates perception with action.\n\nThe central proposal in neuroscience is that neural systems maintain models of the causes of their sensory inputs. Predictions derived from these models are compared with incoming activity, and discrepancies update the models. Hierarchical versions assign different roles to descending predictions and ascending prediction errors. This offers a possible explanation for why perception depends on context and prior experience rather than being a passive reconstruction of stimulation.\n\nPredictive coding is not a single, universally specified theory. Engineering implementations can be precisely defined and directly evaluated, whereas biological proposals differ in their equations, neural circuitry, and experimental predictions. Its relationship to predictive processing, Bayesian inference, and active inference is therefore important: these ideas overlap, but they are not interchangeable."
    },
    {
      "heading": "Origins and conceptual scope",
      "text": "The engineering rationale for predictive coding arises from redundancy. Adjacent samples in speech, neighboring pixels in an image, and successive frames in a video are often correlated. If a receiver can construct a useful prediction of the next sample from information it already possesses, the sender need not transmit the complete sample independently. Transmitting the residual can require fewer bits, provided the residual has a more concentrated statistical distribution. The benefit depends on both the quality of the predictor and the method used to encode its errors.\n\nThis principle belongs to information theory and data compression, but its application to nervous systems introduces additional questions. A communication system normally has an identifiable transmitter, receiver, and coding objective. A brain must infer what exists in the world while simultaneously learning the statistical relationships that make its inputs predictable. Neural predictive coding consequently concerns not just economical communication but the construction and revision of internal representations.\n\nAn important precursor is Hermann von Helmholtz’s nineteenth-century account of perception as unconscious inference. Helmholtz argued that sensory stimulation underdetermines its external causes, so perception depends on learned assumptions. This was not yet a predictive-coding circuit model. Later work supplied more explicit computational mechanisms. In 1982, Mandyam Srinivasan, Simon Laughlin, and Andreas Dubs examined predictive coding in retinal function, relating neural responses to statistical redundancy in visual input.\n\nDavid Mumford’s 1992 work proposed a computational role for reciprocal connections between cortical areas, helping establish a hierarchical interpretation of cortical processing. In 1999, Rajesh Rao and Dana Ballard presented an influential model in which higher visual levels predicted lower-level activity and residual errors traveled upward. Their simulations connected the architecture to contextual effects in visual responses. Karl Friston’s subsequent work, including a 2005 treatment of cortical responses, developed connections between hierarchical inference, prediction errors, and probabilistic models.\n\nThese developments should be distinguished from the broader efficient coding hypothesis. Efficient coding asks how neural representations exploit environmental statistics under constraints such as limited bandwidth, noise, or metabolic cost. Predictive coding is one possible strategy for achieving such efficiency, but efficient representations can also arise through decorrelation, sparse coding, or other transformations. Likewise, a system can use prediction without representing errors in the specific message-passing arrangement proposed by hierarchical predictive-coding models. Historical continuity therefore does not imply that all these theories describe the same mechanism."
    },
    {
      "heading": "Mathematical formulation and hierarchical inference",
      "text": "A simple predictive coder starts with an observed signal \\(x_t\\), a prediction \\(\\hat{x}_t\\), and an error\n\n\\[e_t=x_t-\\hat{x}_t.\\]\n\nIn a temporal predictor, \\(\\hat{x}_t\\) might be a weighted sum of previous samples. In a hierarchical perceptual model, it may instead be generated from a higher-level representation. For example, an inferred object can predict the pattern of edges expected at particular image locations. The term “prediction” thus need not mean a forecast of the future: it can describe an expectation about currently available but not yet explained sensory information.\n\nA common probabilistic formulation uses a generative model, which specifies how hidden causes could produce observations. If \\(x\\) denotes sensory data and \\(z\\) denotes hidden causes, Bayesian inference relates the posterior distribution to the likelihood and prior:\n\n\\[p(z\\mid x)=\\frac{p(x\\mid z)p(z)}{p(x)}.\\]\n\nThe likelihood describes how well a proposed cause explains the data; the prior expresses expectations before those data are considered. Predictive-coding dynamics can approximate inference by iteratively adjusting representations to reduce a suitable objective.\n\nFor a hierarchy, let \\(z_0=x\\), and let a mapping \\(f_l\\) predict activity at level \\(l-1\\) from activity at level \\(l\\). A layer-specific error is\n\n\\[\\varepsilon_{l-1}=z_{l-1}-f_l(z_l).\\]\n\nUnder Gaussian assumptions with fixed covariance matrices, an objective can take the form\n\n\\[E=\\frac{1}{2}\\sum_{l=1}^{L} \\varepsilon_{l-1}^{\\mathsf T} \\Pi_{l-1}\\varepsilon_{l-1} -\\log p(z_L),\\]\n\nwhere \\(\\Pi_{l-1}\\) is an inverse covariance, or precision, matrix. Representations can then evolve through gradient descent,\n\n\\[\\tau_l\\frac{d z_l}{dt}=-\\frac{\\partial E}{\\partial z_l}.\\]\n\nFor an intermediate level, this update combines pressure to explain activity below with pressure to remain consistent with predictions from above.\n\nPrecision determines how strongly an error influences inference. A discrepancy associated with a reliable signal receives greater weight than an equally large discrepancy associated with noise. The objective therefore concerns weighted disagreement, not simply the total magnitude of raw residuals. If precisions are themselves inferred, normalization terms involving covariance determinants also matter; omitting them can produce an incomplete learning objective.\n\nThese equations illustrate one important model family rather than a universal definition. Non-Gaussian likelihoods, discrete hidden states, temporal dynamics, and different approximations produce different update rules. Some models estimate only the most probable hidden state, whereas others approximate uncertainty through variational inference. Predictive coding is consequently compatible with Bayesian reasoning under specified assumptions, but a network that minimizes squared errors is not automatically an exact Bayesian inference system."
    },
    {
      "heading": "Proposed neural circuits",
      "text": "Hierarchical predictive coding assigns computational roles to reciprocal connections in the cerebral cortex. Higher cortical areas generate predictions about activity in lower areas, while lower areas transmit discrepancies that higher areas have not explained. This arrangement draws motivation from the widespread presence of both ascending and descending anatomical connections. The existence of these connections is well established; their identification with particular mathematical messages remains a hypothesis requiring independent evidence.\n\nA widely discussed circuit proposal distinguishes representation units from error units. Representation units encode estimates of hidden causes. Error units compare incoming activity with a prediction and signal the residual. Descending pathways could recruit inhibitory circuitry to subtract predicted input, while ascending pathways could carry the remaining error. Individual neurons need not literally perform arithmetic subtraction in isolation: subtraction-like effects can emerge from combinations of excitation, inhibition, and changes in neuronal gain.\n\nSome canonical microcircuit accounts associate superficial pyramidal populations with ascending errors and deeper pyramidal populations with descending predictions. This proposal is motivated partly by the laminar origins and terminations of cortical projections. However, cortical organization varies across areas, and projection classes do not map neatly onto one computational label. Feedback can excite some cells, suppress others, or change their sensitivity to coincident input. Connections that primarily modulate responses may implement a different operation from connections that directly drive them.\n\nThe representation of signed errors is another difficulty. Mathematical residuals can be positive or negative, but firing rates cannot fall below zero. Proposed solutions include separate populations for positive and negative errors, deviations above and below a baseline, or population codes whose combined activity represents a signed quantity. Each solution generates different experimental expectations. For example, omission of an expected stimulus may preferentially recruit one population, while an unexpectedly strong stimulus may recruit another.\n\nTiming and rhythmic activity provide additional possible constraints. Some studies associate feedforward communication with relatively high-frequency activity and feedback with lower-frequency activity, but frequency bands do not uniquely identify predictions or errors. A signal’s meaning depends on its source, destination, behavioral context, and relation to competing variables. The same caution applies to measurements of synapses, dendritic activity, or action potentials: a response can correlate with surprise without being the error term required by a particular model.\n\nBiologically informative models must therefore explain more than the existence of feedback. They must specify which cells compare which quantities, how uncertainty alters their influence, and how local circuitry produces the predicted dynamics. Anatomical plausibility is a constraint on predictive coding, not by itself proof of the mechanism."
    },
    {
      "heading": "Sensory evidence: context, deviance, and omission",
      "text": "Visual processing provides several influential examples. A neuron’s receptive field is the region or stimulus configuration that influences its activity, but responses also depend on surrounding context. In the visual cortex, a line segment may evoke less activity when it forms part of a predictable extended contour than when it appears in a less expected arrangement. Rao and Ballard’s 1999 model demonstrated how contextual suppression and certain extra-classical receptive-field effects could emerge when higher levels explain lower-level input.\n\nSuch findings fit predictive coding but do not uniquely establish it. Surround suppression can also arise through normalization, lateral interactions, and other circuit mechanisms. Moreover, a familiar or expected stimulus can evoke reduced activity because of sensory adaptation, independently of an explicit prediction. Experiments therefore need to separate probability, repetition, attention, and physical stimulus differences rather than treating reduced responses as a direct readout of successful prediction.\n\nAuditory deviance detection has been especially important. In an oddball experiment, a frequent standard sound is interrupted by a rarer deviant. Human electroencephalography often reveals mismatch negativity, a negative-going event-related component commonly observed roughly 100–250 milliseconds after a change, with timing dependent on the stimulus and experimental conditions. Predictive accounts interpret part of this response as evidence that the auditory system has learned regularities and signals violations. Adaptation and stimulus-specific response properties also contribute, so the ordinary oddball contrast alone cannot settle the mechanism.\n\nControl designs improve discrimination among explanations. In an equiprobable sequence, the same sound can occur as often as it does when it serves as a deviant but without a dominant repeated standard. Other designs manipulate higher-order sequences while controlling immediate repetition. Responses to violations of a relational rule—for example, an unexpected change in the direction of a pitch pair—are especially relevant because the rule cannot be reduced to the frequency of one physical sound.\n\nOmission experiments test a complementary case: a predicted event fails to occur. Neural activity at the expected time can indicate that the system represented an event that was not delivered. In mice, Georg Keller and colleagues reported in 2012 that some visual cortical neurons responded when visual flow unexpectedly stopped during locomotion. These visuomotor mismatch responses provide evidence for comparisons between movement-related expectations and sensory input. Nevertheless, omission, arousal, movement, and behavioral relevance can interact. The strongest interpretation comes from a model that predicts the pattern of responses across controlled conditions, rather than from labeling every unusual response a prediction error."
    },
    {
      "heading": "Precision, attention, and perceptual interpretation",
      "text": "Prediction errors differ in reliability. A small discrepancy in a clear visual scene may deserve more influence than a large discrepancy in dense fog. In statistical terms, precision is the inverse of variance for a scalar Gaussian variable, and the inverse covariance matrix in a multivariate case. Precision weighting allows a predictive-coding system to distinguish informative disagreement from fluctuations that are likely to be noise.\n\nA simple cue-combination example makes this concrete. Suppose two independent Gaussian cues estimate the same location, with means \\(\\mu_1,\\mu_2\\) and precisions \\(\\pi_1,\\pi_2\\). With a sufficiently broad prior, the combined estimate is\n\n\\[\\hat{\\mu}= \\frac{\\pi_1\\mu_1+\\pi_2\\mu_2}{\\pi_1+\\pi_2}.\\]\n\nThe more reliable cue receives greater weight. Predictive-coding networks can approximate related computations through the gain applied to error signals. This is a computational account of reliability-sensitive inference; observing such behavior does not, on its own, demonstrate a particular neural implementation.\n\nOne influential proposal identifies attention with the selective enhancement of expected precision. Attending to a location, feature, or sensory modality would increase the influence of errors arising from that source. This helps explain why attention can increase neural responses even when expectation reduces them. A well-predicted stimulus might produce little unweighted error, yet its remaining discrepancy could exert substantial influence if the corresponding signal is assigned high precision.\n\nExpectation and attention are nevertheless distinct experimental variables. A stimulus can be probable but irrelevant to a task, or improbable but highly important. Behavioral goals, reward, and sensory quality can alter processing in different ways. Models that use precision to explain all such effects must specify how precision is estimated and which neural processes implement its modulation. Otherwise, precision risks becoming an adjustable label applied after the observed result.\n\nPrior expectations can also influence what is perceived when input is ambiguous or degraded. Familiar structure may support recognition of an obscured object, while an inappropriate expectation may bias interpretation. Predictive coding expresses this as competition between errors at different hierarchical levels: an interpretation must account both for sensory evidence and for constraints imposed by learned structure. Strong priors do not imply that perception can disregard arbitrarily reliable sensory data.\n\nThe distinction between inferred content and measured activity is crucial here. A stronger perceptual representation can coexist with reduced activity in an error-signaling population. Conversely, increased activity may indicate unresolved discrepancy rather than stronger perception. Experimental interpretation therefore requires an explicit account of what a recorded population represents and how its activity should change when reliability, expectation, and task demands are independently manipulated."
    },
    {
      "heading": "Action, sensorimotor prediction, and active inference",
      "text": "Movement changes sensory input in predictable ways. Turning the head alters the retinal image, moving a limb produces proprioceptive feedback, and speaking generates expected auditory consequences. Predictive computations can help distinguish changes caused by the organism’s own actions from changes originating elsewhere. Copies of motor-related signals, often described as efference copies or corollary discharge, provide information from which expected sensory consequences can be calculated.\n\nA forward model predicts the sensory outcome of a motor command. Comparing that prediction with subsequent feedback can support state estimation and error correction. Unexpected resistance during reaching, for example, creates a discrepancy between expected and observed movement. Such sensorimotor computations are closely related to predictive coding, but they do not require the claim that all ascending sensory activity consists of prediction errors. Motor systems can combine predictive and nonpredictive mechanisms.\n\nActive inference extends the inferential picture by allowing action to change observations so that they conform to predictions or preferred outcomes. Perceptual inference changes estimates of what caused the sensory signal; action changes the sensory signal itself. In some formulations, descending proprioceptive predictions are fulfilled through spinal and brainstem reflex mechanisms. This differs from treating descending signals solely as explicit motor commands, although the proposed relationship between computational variables and biological motor pathways remains under investigation.\n\nThe free-energy principle supplies a broader mathematical framework often associated with these models. A standard variational free-energy expression is\n\n\\[F[q]=\\mathbb{E}_{q(z)}[\\log q(z)-\\log p(x,z)].\\]\n\nIt can be rewritten as\n\n\\[F[q]=D_{\\mathrm{KL}}\\!\\left(q(z)\\,\\|\\,p(z\\mid x)\\right)-\\log p(x),\\]\n\nwhere \\(D_{\\mathrm{KL}}\\) measures divergence between the approximate and exact posterior. Minimizing \\(F\\) with respect to \\(q\\) improves the approximation to the posterior. This variational quantity is not simply the thermodynamic Gibbs free energy or Helmholtz free energy, despite the shared terminology.\n\nPredictive coding can implement a restricted form of such inference under appropriate assumptions, but the free-energy principle does not uniquely require one predictive-coding circuit. Likewise, active inference commonly evaluates possible actions using expected free energy, which is not identical to the current sensory-error objective. Some formulations balance preferred outcomes with the information actions are expected to provide.\n\nAn apparent difficulty is that reducing prediction error alone could favor uninformative, unchanging stimulation. Active-inference accounts address this through the organism’s model, prior preferences, temporal expectations, and information-seeking policies. These additions must be explicitly specified. Relations to model predictive control and optimal control are substantial, but shared use of prediction does not make their objectives or biological interpretations identical."
    },
    {
      "heading": "Learning, plasticity, and artificial neural networks",
      "text": "Predictive coding separates inference from learning conceptually. During inference, internal activity changes to explain a particular observation while model parameters remain approximately fixed. During learning, the parameters themselves change so that future observations can be predicted more accurately. Biological systems need not separate these processes into discrete stages, but the distinction clarifies why a transient surprise and a lasting change in expectation are different phenomena.\n\nFor a prediction \\(f(z;W)\\) with parameters \\(W\\), a squared-error objective produces a parameter update involving the residual and the sensitivity of the prediction to those parameters:\n\n\\[\\Delta W\\propto \\left(\\frac{\\partial f}{\\partial W}\\right)^{\\mathsf T} \\Pi\\bigl(x-f(z;W)\\bigr).\\]\n\nFor a linear mapping \\(f(z;W)=Wz\\), this reduces to\n\n\\[\\Delta W\\propto \\Pi\\varepsilon z^{\\mathsf T}.\\]\n\nThe update combines an error signal with presynaptic activity. Its apparent locality is one reason predictive coding attracts interest as a possible account of neuroplasticity. However, identifying each mathematical factor with a biologically available signal is a further task.\n\nLearning also changes uncertainty estimates. A repeatedly reliable sensory relationship may acquire greater precision, while an unstable relationship may receive less weight. Hierarchical models can distinguish ordinary observation noise from changes in the underlying environment. A large discrepancy can then lead either to a small perceptual correction, because the observation is considered noisy, or to substantial model revision, because a previously stable regularity appears to have changed. These outcomes depend on the model’s assumptions about variability and change.\n\nIn machine learning, predictive-coding networks offer alternatives to conventional training procedures for an artificial neural network. Rather than computing all error derivatives in one explicitly separated backward pass, some architectures iteratively adjust activity at multiple layers and then update weights using local quantities. The resemblance to backpropagation has been formalized in several models. Whittington and Bogacz showed in 2017 how, under specified conditions, predictive-coding networks could approximate error-backpropagation learning.\n\nThese relationships are conditional rather than universal. Their accuracy depends on matters such as convergence of activity, initialization, feedback weights, and the scale of perturbations. Some architectures require substantial iteration before each parameter update, which can increase computational cost. Others relax assumptions but change the learning rule or its theoretical guarantees. Local-looking updates do not automatically establish biological plausibility if they depend on precisely coordinated global phases or unavailable weight information.\n\nPredictive learning also differs from reinforcement learning. A sensory prediction error concerns a mismatch in observations or representations; a reward prediction error concerns the difference between obtained and expected reward-related quantities. They can interact in one agent, but their targets, learning signals, and computational roles should not be conflated."
    },
    {
      "heading": "Engineering implementations and related estimators",
      "text": "In signal processing, predictive coding is a concrete family of methods rather than a hypothesis about biological organization. A basic implementation uses a predictor available at both encoder and decoder. The encoder subtracts the prediction from the input and transmits a coded residual. The decoder adds the reconstructed residual to the same prediction. Compression is effective when the residual is easier to encode than the original signal, after accounting for predictor parameters and other overhead.\n\nDifferential pulse-code modulation illustrates the principle. To keep a lossy encoder and decoder synchronized, the encoder typically forms its prediction from previously reconstructed samples, not inaccessible original samples. If \\(Q\\) denotes residual quantization, then\n\n\\[\\tilde e_t=Q(x_t-\\hat x_t), \\qquad \\tilde x_t=\\hat x_t+\\tilde e_t.\\]\n\nBoth sides use the same reconstructed history to generate subsequent predictions. Otherwise, quantization differences can cause their predictors to diverge. The choice of quantizer and predictor affects distortion, bit rate, and error propagation.\n\nLinear predictive coding is particularly important in speech analysis and coding. A speech sample is approximated by a linear combination of earlier samples,\n\n\\[\\hat{x}_t=\\sum_{k=1}^{p}a_kx_{t-k}.\\]\n\nThe coefficients characterize an all-pole model associated with resonances of the vocal tract, while an excitation or residual accounts for what the filter prediction does not explain. Practical speech coders need additional components to represent excitation, voicing, and time variation; merely subtracting a linear prediction does not describe the entire system.\n\nImage and video compression use related ideas across space and time. An image block can be predicted from already reconstructed neighboring samples, while a video block can be predicted from regions in earlier or other reference frames. Motion compensation improves prediction when objects or the camera move. Standards such as H.264/AVC and HEVC combine prediction with transform coding, quantization, and entropy coding of residual information. Their performance depends on the full coding system, not on prediction alone.\n\nThe Kalman filter provides a useful comparison from state estimation. It predicts a hidden state and then corrects that prediction using an innovation, the discrepancy between an observation and its predicted value. The correction is weighted by uncertainty through the Kalman gain. For linear Gaussian state-space models, the filter gives exact recursive inference under its assumptions. This resembles precision-weighted predictive updating, but a shared innovation structure does not make every Kalman filter a hierarchical neural predictive-coding model.\n\nEngineering success demonstrates that residual-based representation can be efficient and effective. It does not establish that nervous systems use the same code, optimize the same objective, or separate prediction and error into the same physical channels."
    },
    {
      "heading": "Tests, competing explanations, and unresolved questions",
      "text": "A central challenge is distinguishing predictive coding from the broad observation that brains exploit regularities. Almost any adaptive perceptual system will respond differently to familiar and unfamiliar events. A strong test therefore requires a specified model that predicts which populations carry representations, which carry errors, how those signals depend on uncertainty, and when their interactions occur. Without these commitments, many different results can be accommodated by changing the interpretation of an unobserved variable.\n\nExpectation suppression illustrates the problem. Reduced responses to predictable stimuli are compatible with predictive coding, but also with repetition-dependent adaptation, changes in attention, sharpening of representations, or altered processing demands. Conversely, increased responses to expected stimuli need not refute every predictive-coding model if the measurement emphasizes representation units or precision-related gain. This flexibility becomes scientifically useful only when a model identifies the relevant populations and predicts the response direction before the experiment.\n\nPopulation recordings and causal interventions offer stronger constraints than simple average contrasts. An informative experiment might independently manipulate stimulus probability and reliability, record across cortical layers, and disrupt a candidate feedback pathway. A circuit model should predict not merely an overall change in activity but which error responses disappear, which representations remain, and how behavior changes. Controls must also establish whether an intervention altered sensory drive, movement, arousal, or general task engagement.\n\nMeasurement scale matters. Functional magnetic resonance imaging reflects aggregate hemodynamic consequences of neural activity and cannot directly distinguish prediction units from error units. Electrical and magnetic recordings provide finer timing but combine signals from multiple sources. Single-cell and cellular imaging methods offer greater specificity, yet the sampled population may not reveal the full computation. Conclusions are strongest when behavior, recordings, perturbations, and a quantitative model converge.\n\nAnother unresolved issue is how widely the architecture applies. Some sensory circuits may suppress predictable components efficiently, while others preserve detailed representations or emphasize behaviorally useful features. Brains contain recurrent pathways, parallel processing streams, neuromodulatory influences, and diverse learning mechanisms. A predictive-coding account must explain how its proposed variables coexist with this organization rather than assuming that every response serves one universal error-minimization function.\n\nModel comparison consequently requires explicit assumptions, matched complexity, and tests on data not used to choose the model. Relevant alternatives include adaptation models, normalization circuits, feedforward recognition networks, recurrent dynamical systems, and other probabilistic inference schemes. The decisive questions concern identifiable mechanisms: whether predicted content can be measured separately from residual error, whether reliability changes their coupling as specified, and whether disrupting the proposed comparison prevents the predicted computation."
    }
  ],
  "links": [
    "neuroscience",
    "brain",
    "information-theory",
    "data-compression",
    "bayesian-inference",
    "matrix",
    "variational-inference",
    "neuron",
    "synapse",
    "action-potential",
    "electroencephalography",
    "variance",
    "active-inference",
    "free-energy-principle",
    "gibbs-free-energy",
    "helmholtz-free-energy",
    "model-predictive-control",
    "neuroplasticity",
    "machine-learning",
    "artificial-neural-network",
    "backpropagation",
    "reinforcement-learning",
    "kalman-filter",
    "magnetic-resonance-imaging"
  ],
  "sources": [],
  "license": "https://creativecommons.org/licenses/by/4.0/"
}