{
  "id": "wiener-filter",
  "title": "Wiener Filter",
  "lang": "en",
  "source_lang": "en",
  "summary": "A Wiener filter is a linear estimator that minimizes mean squared error using the second-order statistical properties of a desired signal and its observations.",
  "category": "mathematics",
  "subcategory": "applied",
  "type": "concept",
  "aliases": [
    "Wiener filtering",
    "Wiener–Kolmogorov filtering"
  ],
  "generated_at": "2026-11-05",
  "model": "pro/gpt-6.1-sol",
  "url": "https://aiwiki.page/en/wiener-filter/",
  "translations": {
    "zh-Hans": {
      "url": "https://aiwiki.page/zh/wiener-filter/",
      "json": "https://aiwiki.page/zh/wiener-filter.json"
    }
  },
  "sections": [
    {
      "heading": "",
      "text": "A Wiener filter is a linear method for estimating a desired signal from noisy or otherwise imperfect observations by minimizing the mean squared error of the estimate. Named after Norbert Wiener, it uses the means and correlation structure of the signals rather than requiring their complete probability distributions. Its classical form concerns stationary random signals and time-invariant filters, but the same principle also yields finite-dimensional matrix estimators. Depending on which observations are available, Wiener filtering can perform noise reduction, prediction, smoothing, or reconstruction of a distorted signal."
    },
    {
      "heading": "Estimation principle",
      "text": "Let \\(d[n]\\) be a desired discrete-time signal and \\(x[n]\\) an observed signal. Both are modeled as stochastic processes with finite second moments. For zero-mean signals, a linear filter produces\n\n\\[\\widehat d[n]=\\sum_{k\\in\\mathcal K}h[k]x[n-k],\\]\n\nwhere \\(h[k]\\) are the filter coefficients and \\(\\mathcal K\\) specifies the permitted observation lags. This is a convolution when the coefficients do not depend on \\(n\\).\n\nThe coefficients minimize\n\n\\[J(h)=\\mathbb E\\!\\left[|d[n]-\\widehat d[n]|^2\\right],\\]\n\nwhere \\(\\mathbb E\\) denotes expectation. The definition of the admissible filter class is essential: the best causal filter, the best unrestricted filter, and the best filter of a prescribed finite length generally differ.\n\nThe central optimality condition is the orthogonality principle. At the optimum, the error\n\n\\[e[n]=d[n]-\\widehat d[n]\\]\n\nis uncorrelated with every observation available to the estimator:\n\n\\[\\mathbb E\\!\\left[e[n]x^*[n-m]\\right]=0, \\qquad m\\in\\mathcal K,\\]\n\nwhere the asterisk denotes complex conjugation.\n\nGeometrically, the optimal estimate is the orthogonal projection of the desired random variable onto the closed linear subspace generated by the observations, in the Hilbert space of square-integrable random variables. This interpretation also applies when an infinite collection of observations is used, with convergence understood in mean square."
    },
    {
      "heading": "Finite-dimensional formulation",
      "text": "Suppose an observation vector \\(\\mathbf x\\) and a desired scalar \\(d\\) are real-valued. Define\n\n\\[\\boldsymbol\\mu_x=\\mathbb E[\\mathbf x], \\qquad \\mu_d=\\mathbb E[d],\\]\n\n\\[\\mathbf C_{xx} = \\mathbb E\\!\\left[ (\\mathbf x-\\boldsymbol\\mu_x) (\\mathbf x-\\boldsymbol\\mu_x)^{\\mathsf T} \\right],\\]\n\nand\n\n\\[\\mathbf c_{xd} = \\mathbb E\\!\\left[ (\\mathbf x-\\boldsymbol\\mu_x)(d-\\mu_d) \\right].\\]\n\nHere \\(\\mathbf C_{xx}\\) is the observation covariance matrix. An affine estimator has the form\n\n\\[\\widehat d = \\mu_d+\\mathbf w^{\\mathsf T} (\\mathbf x-\\boldsymbol\\mu_x).\\]\n\nMinimizing its mean squared error gives the normal equations\n\n\\[\\mathbf C_{xx}\\mathbf w=\\mathbf c_{xd}.\\]\n\nIf \\(\\mathbf C_{xx}\\) is invertible,\n\n\\[\\mathbf w=\\mathbf C_{xx}^{-1}\\mathbf c_{xd},\\]\n\nand the minimum error is\n\n\\[J_{\\min} = \\operatorname{Var}(d) - \\mathbf c_{xd}^{\\mathsf T} \\mathbf C_{xx}^{-1} \\mathbf c_{xd}.\\]\n\nThese equations connect Wiener filtering to population linear regression and least squares. Unlike an ordinary sample-fitting problem, the theoretical formulation uses ensemble covariances; practical implementations estimate them from data.\n\nIf the covariance matrix is singular, the coefficient vector need not be unique. A generalized inverse can provide a solution, and equivalent solutions yield the same estimate with probability one. Numerical implementations normally solve the linear system directly rather than explicitly forming a matrix inverse.\n\nFor complex-valued data, transposes are replaced by conjugate transposes, with the cross-covariance defined consistently."
    },
    {
      "heading": "Stationary filtering and the Wiener–Hopf equations",
      "text": "For jointly wide-sense stationary processes, means are constant and second-order statistics depend only on time differences. After removing the means, define\n\n\\[r_{xx}[\\ell] = \\mathbb E[x[n]x^*[n-\\ell]],\\]\n\n\\[r_{dx}[\\ell] = \\mathbb E[d[n]x^*[n-\\ell]].\\]\n\nSubstitution into the orthogonality condition gives the Wiener–Hopf equations:\n\n\\[\\sum_{k\\in\\mathcal K} h[k]r_{xx}[m-k] = r_{dx}[m], \\qquad m\\in\\mathcal K.\\]\n\nFor a finite set of consecutive lags, these become a matrix system whose observation-covariance matrix has constant diagonals. For an unrestricted two-sided filter, the index set is all integers.\n\nFrequency-domain solution\n\nLet \\(S_{xx}(e^{i\\omega})\\) be the power spectral density obtained from the Fourier transform of \\(r_{xx}\\), and let \\(S_{dx}(e^{i\\omega})\\) be the corresponding cross-spectral density. Under conditions that permit a spectral representation, the unrestricted solution is\n\n\\[H_{\\mathrm{opt}}(e^{i\\omega}) = \\frac{S_{dx}(e^{i\\omega})} {S_{xx}(e^{i\\omega})}, \\qquad S_{xx}(e^{i\\omega})>0.\\]\n\nIts error spectrum is\n\n\\[S_{ee}(e^{i\\omega}) = S_{dd}(e^{i\\omega}) - \\frac{|S_{dx}(e^{i\\omega})|^2} {S_{xx}(e^{i\\omega})},\\]\n\nso the minimum mean squared error is\n\n\\[J_{\\min} = \\frac{1}{2\\pi} \\int_{-\\pi}^{\\pi} S_{ee}(e^{i\\omega})\\,d\\omega.\\]\n\nWhere the observation spectrum vanishes, the observations carry no power; the optimal response there is not determined by the spectral ratio. A pointwise quotient also does not, by itself, guarantee a causal or practically stable implementation."
    },
    {
      "heading": "Additive-noise filtering",
      "text": "A standard model is\n\n\\[x[n]=s[n]+v[n],\\]\n\nwhere \\(s[n]\\) is the desired signal and \\(v[n]\\) is additive noise. If signal and noise are uncorrelated at every lag, then\n\n\\[S_{xx}=S_{ss}+S_{vv}, \\qquad S_{sx}=S_{ss}.\\]\n\nThe unrestricted Wiener response is therefore\n\n\\[H_{\\mathrm{opt}} = \\frac{S_{ss}}{S_{ss}+S_{vv}}.\\]\n\nAt frequencies with nonzero noise power, writing the signal-to-noise ratio as \\(\\rho=S_{ss}/S_{vv}\\) gives\n\n\\[H_{\\mathrm{opt}}=\\frac{\\rho}{1+\\rho}.\\]\n\nThus:\n\n- Frequencies dominated by the signal pass with gain close to one.\n- Frequencies dominated by noise are strongly attenuated.\n- Frequencies containing comparable signal and noise power are partially attenuated.\n\nThe filter is not inherently a low-pass filter. Its response follows the relative signal and noise spectra and may have a more complicated frequency dependence. Perfect separation is generally impossible when signal and noise occupy the same frequencies with nonzero power."
    },
    {
      "heading": "Deconvolution and inverse problems",
      "text": "If an observation is blurred or distorted before noise is added,\n\n\\[x=g*s+v,\\]\n\nthen, assuming signal and noise are uncorrelated, the unrestricted estimator of \\(s\\) has response\n\n\\[H_{\\mathrm{opt}} = \\frac{G^*S_{ss}} {|G|^2S_{ss}+S_{vv}},\\]\n\nwhere \\(G\\) is the frequency response of the distortion.\n\nA direct inverse filter would use \\(1/G\\), potentially amplifying noise severely wherever \\(|G|\\) is small. The Wiener denominator moderates this amplification, making the method important in inverse problems such as image deblurring and signal reconstruction.\n\nThe same principle appears in a finite-dimensional model\n\n\\[\\mathbf y=\\mathbf A\\mathbf s+\\mathbf v.\\]\n\nFor zero-mean, uncorrelated signal and noise, the optimal linear estimator is\n\n\\[\\widehat{\\mathbf s} = \\mathbf C_s\\mathbf A^{\\mathsf T} \\left( \\mathbf A\\mathbf C_s\\mathbf A^{\\mathsf T} + \\mathbf C_v \\right)^{-1}\\mathbf y,\\]\n\nwhen the indicated inverse exists.\n\nIf \\(\\mathbf C_s=\\tau^2\\mathbf I\\) and \\(\\mathbf C_v=\\sigma^2\\mathbf I\\), this becomes\n\n\\[\\widehat{\\mathbf s} = \\left( \\mathbf A^{\\mathsf T}\\mathbf A + \\frac{\\sigma^2}{\\tau^2}\\mathbf I \\right)^{-1} \\mathbf A^{\\mathsf T}\\mathbf y.\\]\n\nThis is the same algebraic form as ridge regression. Here the regularization parameter has a statistical interpretation as a noise-to-signal variance ratio."
    },
    {
      "heading": "Causality, prediction, and smoothing",
      "text": "The available observations determine the estimation problem:\n\n- Causal filtering estimates the present using present and past observations.\n- Prediction estimates a future value from observations available now.\n- Smoothing estimates a value using observations both before and after its time.\n- Finite-length filtering restricts the estimator to a specified observation window.\n\nThe unrestricted spectral quotient generally corresponds to a two-sided impulse response and may require future observations. It is therefore not automatically suitable for real-time operation.\n\nThe optimal causal solution must satisfy the Wiener–Hopf equations with the corresponding one-sided lag restriction. Classical frequency-domain methods use spectral factorization to construct a causal solution under appropriate assumptions. Simply deleting the negative-time part of the unrestricted impulse response generally does not produce the optimal causal filter.\n\nSimilarly, an unrestricted frequency response transformed back and truncated to a finite length need not equal the optimal finite-length solution."
    },
    {
      "heading": "Statistical scope and related methods",
      "text": "Wiener filtering requires second-order statistics, not Gaussian distributions. Without further assumptions, its guarantee is optimality among the specified linear or affine estimators.\n\nThe minimum-error estimator among all measurable functions of an observation is the conditional expectation:\n\n\\[\\widehat d_{\\mathrm{MMSE}} = \\mathbb E[d\\mid\\mathbf x].\\]\n\nWhen \\(d\\) and \\(\\mathbf x\\) have a joint multivariate normal distribution, this conditional expectation is affine. The Wiener estimator then also minimizes mean squared error among nonlinear estimators. For non-Gaussian data, a nonlinear estimator may perform better.\n\nThe Kalman filter expresses linear minimum-error estimation recursively through a state-space model. It can accommodate time-varying dynamics and nonstationary covariances. In suitable stationary settings, its steady-state behavior is closely related to causal Wiener filtering.\n\nAdaptive filters estimate or track useful coefficients from incoming data rather than assuming known covariance functions. Algorithms based on stochastic gradient descent can approach Wiener solutions under suitable conditions, but finite-data errors, step sizes, and changing signal statistics affect their behavior."
    },
    {
      "heading": "History, applications, and limitations",
      "text": "Wiener developed his theory of stationary time-series estimation during World War II, in connection with prediction and military control problems. His work became publicly available in the 1949 book Extrapolation, Interpolation, and Smoothing of Stationary Time Series. Andrey Kolmogorov independently developed closely related results on prediction and interpolation, accounting for the name Wiener–Kolmogorov filtering.\n\nApplications include audio denoising, communications receivers, channel equalization, image restoration, and estimation of geophysical signals. Practical implementations often use estimated spectra and the fast Fourier transform.\n\nThe main limitations concern the assumed model and the error criterion:\n\n- Statistical mismatch: inaccurate covariance or spectral estimates produce a filter that is optimal for the assumed statistics, not necessarily for the actual data.\n- Nonstationarity: a single fixed filter may be inadequate when signal properties change over time.\n- Finite data: estimated covariance systems may be poorly conditioned, affecting numerical stability.\n- Linear structure: nonlinear relationships and non-Gaussian information may remain unused.\n- Error criterion: low mean squared error does not necessarily imply the best perceptual quality, edge preservation, or performance on a subsequent task.\n- Implementation constraints: real-time causality, finite memory, latency, and computational limits can prevent realization of the unrestricted theoretical solution."
    }
  ],
  "links": [
    "mean-squared-error",
    "norbert-wiener",
    "stochastic-process",
    "convolution",
    "expected-value",
    "orthogonal-projection",
    "linear-subspace",
    "hilbert-space",
    "covariance-matrix",
    "linear-regression",
    "least-squares",
    "system-of-linear-equations",
    "conjugate-transpose",
    "stationary-process",
    "fourier-transform",
    "inverse-problem",
    "ridge-regression",
    "regularization",
    "conditional-expectation",
    "multivariate-normal-distribution",
    "kalman-filter",
    "stochastic-gradient-descent",
    "world-war-ii",
    "andrey-kolmogorov",
    "fast-fourier-transform",
    "numerical-stability"
  ],
  "sources": [],
  "license": "https://creativecommons.org/licenses/by/4.0/"
}