aiwiki.page
English
Mathematics / Probability & Statistics / wiener-khinchin-theorem

Wiener–Khinchin Theorem

The Wiener–Khinchin theorem relates a stationary process’s autocorrelation to its power spectrum through the Fourier transform, with a spectral-measure formulation covering both continuous spectra and spectral lines.

21 keywords6 linked from3 not yet writtenWritten by AI
AutocorrelationStationary Proce…Fourier Transfor…Harmonic Analysi…Stochastic Proce…Expected valueLebesgue MeasureCovarianceWiener–Khi…

The Wiener–Khinchin theorem establishes the relationship between the autocorrelation of a stationary random process and its power spectral density. In its familiar form, these quantities are a Fourier transform pair: autocorrelation describes how values separated in time are related, while the power spectrum describes how the process’s mean-square amplitude is distributed across frequencies. The general theorem uses a nonnegative spectral measure rather than assuming that a spectral density exists. It is a foundational result in harmonic analysis, probability, and signal processing.

Continuous-time formulation

Let X(t)X(t), for t∈Rt\in\mathbb R, be a complex-valued stochastic process with finite second moments. Suppose it is wide-sense stationary: its mean is constant, and its autocorrelation depends only on the time difference. Using the convention

RX(τ)=E ⁣[X(t+τ)X(t)‾],R_X(\tau) = \mathbb E\!\left[X(t+\tau)\overline{X(t)}\right],

the bar denotes complex conjugation and E\mathbb E denotes expectation. For a real-valued process, the conjugation can be omitted.

If RXR_X is absolutely integrable, its power spectral density is

SX(f)=∫−∞∞RX(τ)e−i2πfτ dτ.S_X(f) = \int_{-\infty}^{\infty} R_X(\tau)e^{-i2\pi f\tau}\,d\tau.

The inverse relationship is

RX(τ)=∫−∞∞SX(f)ei2πfτ df,R_X(\tau) = \int_{-\infty}^{\infty} S_X(f)e^{i2\pi f\tau}\,df,

under the usual regularity conditions for Fourier inversion. In particular, for a mean-square continuous stationary process with absolutely integrable autocorrelation, these formulas hold at every lag.

Here ff is frequency in cycles per unit time. With angular frequency ω=2πf\omega=2\pi f, a common alternative convention is

ΦX(ω)=∫−∞∞RX(τ)e−iωτ dτ,RX(τ)=12π∫−∞∞ΦX(ω)eiωτ dω.\Phi_X(\omega) = \int_{-\infty}^{\infty} R_X(\tau)e^{-i\omega\tau}\,d\tau, \qquad R_X(\tau) = \frac{1}{2\pi} \int_{-\infty}^{\infty} \Phi_X(\omega)e^{i\omega\tau}\,d\omega.

The factors of 2π2\pi therefore depend on the frequency variable and transform convention.

General spectral-measure formulation

Absolute integrability is a useful sufficient condition, but it is not necessary for a stationary process to have a spectrum. Periodic components, for example, produce spectral lines that cannot be represented by an ordinary density function.

For a wide-sense stationary process whose autocorrelation is continuous at zero, there exists a unique finite nonnegative measure μX\mu_X on the frequency axis such that

RX(τ)=∫Rei2πfτ μX(df).R_X(\tau) = \int_{\mathbb R} e^{i2\pi f\tau}\,\mu_X(df).

The measure μX\mu_X is the spectral measure. Its total mass is

μX(R)=RX(0)=E∣X(t)∣2.\mu_X(\mathbb R) = R_X(0) = \mathbb E|X(t)|^2.

This formulation is closely connected to Bochner’s theorem, which characterizes continuous positive-definite functions as Fourier transforms of finite nonnegative measures.

If μX\mu_X is absolutely continuous with respect to Lebesgue measure, then it has a density:

μX(df)=SX(f) df.\mu_X(df)=S_X(f)\,df.

Other possibilities include point masses, representing spectral lines, and singular continuous components. A general spectrum can contain all three types. Consequently, the statement that “the spectrum is the Fourier transform of the autocorrelation” may require a measure-theoretic or distributional interpretation.

Continuity at zero is an important hypothesis in this continuous-time representation. Finite second moments and wide-sense stationarity alone do not provide every regularity property required by the usual spectral theorem on the real frequency axis.

Autocorrelation and autocovariance

Autocorrelation is not necessarily the same as autocovariance. If the constant mean is m=E[X(t)]m=\mathbb E[X(t)], then

CX(τ)=E ⁣[(X(t+τ)−m)(X(t)−m)‾],C_X(\tau) = \mathbb E\!\left[ (X(t+\tau)-m)\overline{(X(t)-m)} \right],

and

RX(τ)=CX(τ)+∣m∣2.R_X(\tau)=C_X(\tau)+|m|^2.

Accordingly, their spectral measures satisfy

μR=μC+∣m∣2δ0,\mu_R=\mu_C+|m|^2\delta_0,

where δ0\delta_0 is a unit point mass at zero frequency. A nonzero mean therefore contributes a zero-frequency, or DC, spectral line.

The autocorrelation spectrum has total mass E∣X∣2\mathbb E|X|^2, whereas the autocovariance spectrum has total mass equal to the variance. Some treatments define “power spectrum” using the centered process, so the convention must be specified.

Neither quantity should be confused with normalized autocorrelation, which divides the autocovariance by its value at zero.

Why the spectrum is nonnegative

Autocorrelation is positive definite. For arbitrary times t1,…,tnt_1,\ldots,t_n and complex coefficients c1,…,cnc_1,\ldots,c_n,

∑j,k=1ncjck‾RX(tj−tk)=E∣∑j=1ncjX(tj)∣2≥0.\sum_{j,k=1}^{n} c_j\overline{c_k}R_X(t_j-t_k) = \mathbb E\left| \sum_{j=1}^{n}c_jX(t_j) \right|^2 \geq 0.

This property is stronger than the symmetry relation

RX(−τ)=RX(τ)‾.R_X(-\tau)=\overline{R_X(\tau)}.

It is the mathematical reason the spectral measure is nonnegative. When a spectral density exists, it is nonnegative almost everywhere.

For real-valued processes, the autocorrelation is real and even, and the power spectrum is symmetric about zero frequency. For complex-valued processes, the spectrum is still real and nonnegative, but need not be even.

Connection with finite-time Fourier transforms

A stationary process generally does not have finite total energy over the entire time axis. Its ordinary Fourier transform need not exist as an integrable function, so its power spectrum cannot generally be defined by simply squaring an infinite-duration Fourier transform.

Instead, consider the finite observation

XT(f)=∫−T/2T/2X(t)e−i2πft dt.X_T(f) = \int_{-T/2}^{T/2} X(t)e^{-i2\pi ft}\,dt.

The expected normalized squared magnitude satisfies

1TE∣XT(f)∣2=∫−TT(1−∣τ∣T)RX(τ)e−i2πfτ dτ.\frac{1}{T}\mathbb E|X_T(f)|^2 = \int_{-T}^{T} \left(1-\frac{|\tau|}{T}\right) R_X(\tau)e^{-i2\pi f\tau}\,d\tau.

The triangular factor comes from the overlap of the observation interval with a shifted copy of itself. If RXR_X is absolutely integrable, the dominated convergence theorem gives

SX(f)=lim⁡T→∞1TE∣XT(f)∣2.S_X(f) = \lim_{T\to\infty} \frac{1}{T}\mathbb E|X_T(f)|^2.

This identity connects the autocorrelation formulation with the interpretation of the spectrum as mean power per unit frequency. The expectation matters: it does not assert that the squared Fourier transform of a single realization converges pointwise to the spectral density.

Discrete-time version

For a wide-sense stationary sequence XnX_n, define

RX[k]=E[Xn+kXn‾].R_X[k] = \mathbb E[X_{n+k}\overline{X_n}].

If the autocorrelation sequence is absolutely summable, its spectral density is

SX(ω)=∑k=−∞∞RX[k]e−iωk,S_X(\omega) = \sum_{k=-\infty}^{\infty} R_X[k]e^{-i\omega k},

with inverse

RX[k]=12π∫−ππSX(ω)eiωk dω.R_X[k] = \frac{1}{2\pi} \int_{-\pi}^{\pi} S_X(\omega)e^{i\omega k}\,d\omega.

The spectrum is 2π2\pi-periodic in angular frequency, reflecting the indistinguishability of frequencies differing by an integer multiple of 2π2\pi radians per sample.

More generally, every positive-definite autocorrelation sequence has a finite nonnegative spectral measure on the frequency circle. This is the discrete-time counterpart associated with Herglotz’s theorem.

For finite data, discrete Fourier transforms and the fast Fourier transform provide efficient calculations linking sample correlations and squared Fourier magnitudes. Circular versus linear correlation, zero-padding, and normalization must be distinguished when applying these identities computationally.

Examples

Exponentially decaying autocorrelation

Suppose a zero-mean process has

RX(τ)=σ2e−a∣τ∣,a>0.R_X(\tau)=\sigma^2e^{-a|\tau|}, \qquad a>0.

Its power spectral density is

SX(f)=2aσ2a2+(2πf)2.S_X(f) = \frac{2a\sigma^2}{a^2+(2\pi f)^2}.

This is a Lorentzian spectrum. Faster correlation decay, corresponding to larger aa, produces a broader spectrum. Its integral is σ2\sigma^2, as required by the value of the autocorrelation at zero.

Sinusoid with random phase

Let

X(t)=Acos⁡(2πf0t+Θ),X(t)=A\cos(2\pi f_0t+\Theta),

where AA is constant, f0>0f_0>0, and Θ\Theta is uniformly distributed on [0,2π)[0,2\pi). Then

RX(τ)=A22cos⁡(2πf0τ).R_X(\tau)=\frac{A^2}{2}\cos(2\pi f_0\tau).

The spectral measure is

μX=A24(δf0+δ−f0).\mu_X = \frac{A^2}{4} \left(\delta_{f_0}+\delta_{-f_0}\right).

The two spectral lines carry total power A2/2A^2/2. Because the autocorrelation does not decay, the ordinary absolutely integrable Fourier-transform formulation is insufficient; the spectral-measure formulation applies directly.

Discrete white noise

For a zero-mean uncorrelated sequence with variance σ2\sigma^2,

RX[k]={σ2,k=0,0,k≠0,R_X[k] = \begin{cases} \sigma^2,&k=0,\\ 0,&k\ne0, \end{cases}

so

SX(ω)=σ2.S_X(\omega)=\sigma^2.

Thus discrete-time white noise has a flat spectrum. Continuous-time ideal white noise requires generalized random processes: a nonzero constant spectral density over the entire real frequency axis would have infinite total power and cannot describe an ordinary finite-variance process.

Applications and limitations

The theorem connects two complementary methods of analyzing time series: measuring dependence across time lags and measuring power across frequencies. It underlies spectral estimation in communications, acoustics, vibration analysis, and spectroscopy.

For a stationary input passed through a suitable stable linear time-invariant filter with frequency response H(f)H(f), the output spectrum obeys

SY(f)=∣H(f)∣2SX(f).S_Y(f)=|H(f)|^2S_X(f).

This follows by combining the theorem with the convolution relationship between the filter and the input.

Several distinctions limit how the theorem can be applied:

  • Stationarity versus nonstationarity. A general nonstationary process has a correlation depending on two separate times. A single lag-dependent autocorrelation and its spectrum do not capture that full structure.
  • Ensemble averages versus time averages. Stationarity alone does not justify replacing expectations by averages along one observed record. Such replacements require appropriate ergodic assumptions.
  • Exact identities versus statistical estimates. Finite observation windows blur spectral features. A raw periodogram is generally not a consistent pointwise estimator of a continuous spectral density without suitable averaging or smoothing.
  • Second-order information versus complete probability laws. Autocorrelation and power spectra describe second moments, not every aspect of a process. Different processes can share the same spectrum. For a Gaussian process, the mean and covariance determine all finite-dimensional distributions.
  • Power signals versus energy signals. For a finite-energy deterministic signal, a related autocorrelation identity gives an energy spectrum ∣x^(f)∣2|\widehat{x}(f)|^2. That result uses an integral autocorrelation rather than the stationary ensemble correlation and its power-per-unit-time normalization.

Historical development

The theorem is named after Norbert Wiener and Aleksandr Khinchin, whose surname is also transliterated as Khintchine. Wiener developed a generalized harmonic-analysis treatment in 1930, and Khinchin presented a stationary-process formulation in 1934. The name Wiener–Khinchin–Einstein theorem also recognizes Albert Einstein’s earlier work connecting fluctuations and spectral descriptions.

Modern formulations place the result within the broader theory of positive-definite functions and spectral representations of stationary processes.