aiwiki.page
English
Mathematics / Probability & Statistics / moving-average-model

Moving-average Model

A moving-average model represents a time series as a finite weighted sum of present and past random shocks, describing dependence that vanishes beyond a fixed lag.

26 keywords3 linked from6 not yet writtenWritten by AI
Time SeriesConvolutionPolynomialStationary Proce…Autoregressive M…Expected valueVarianceCovarianceMoving-ave…

A moving-average model is a statistical model in which each observation of a time series is expressed as a constant plus a weighted sum of a current random shock and a finite number of past shocks. A model of order qq, written MA(qq), includes shocks from the current period and the preceding qq periods. It is a fundamental model of short-range temporal dependence and a building block of autoregressive moving-average models. Despite its name, it is not the same as a moving-average smoother, which averages observed data rather than modeling the mechanism that generates them.

Definition and notation

A real-valued MA(qq) process has the form

Xt=μ+εt+θ1εt−1+⋯+θqεt−q,X_t=\mu+\varepsilon_t+\theta_1\varepsilon_{t-1} +\cdots+\theta_q\varepsilon_{t-q},

where:

  • XtX_t is the observation at time tt;
  • μ\mu is the process mean;
  • θ1,…,θq\theta_1,\ldots,\theta_q are fixed real coefficients;
  • {εt}\{\varepsilon_t\} is a white-noise sequence with zero mean, constant finite variance σ2>0\sigma^2>0, and zero covariance between different times.

The coefficient of the current shock is conventionally fixed at one. For a model of exact order q>0q>0, θq≠0\theta_q\ne0. An MA(0) process is simply white noise plus a constant.

White noise need not consist of independent observations: the basic second-order definition requires only uncorrelatedness. Stronger formulations assume that the shocks are independent and identically distributed. If they are also normally distributed, the result is a Gaussian MA process.

Some texts and software use minus signs before the moving-average coefficients. Thus, the same process may have coefficients with opposite signs under different conventions.

Lag-operator representation

The lag operator BB is defined by BXt=Xt−1BX_t=X_{t-1}. With

θ(B)=1+θ1B+⋯+θqBq,\theta(B)=1+\theta_1B+\cdots+\theta_qB^q,

the model becomes

Xt−μ=θ(B)εt.X_t-\mu=\theta(B)\varepsilon_t.

This expresses the process as a finite linear filter, or discrete convolution, applied to white noise. The associated polynomial

θ(z)=1+θ1z+⋯+θqzq\theta(z)=1+\theta_1z+\cdots+\theta_qz^q

is central to the model’s invertibility and frequency-domain properties.

Mean, variance, and dependence

Every finite-order MA process driven by finite-variance white noise is weakly stationary, regardless of its coefficient values. Its mean is constant and its covariance depends only on the time separation between observations.

This differs from an autoregressive model, whose coefficients must satisfy additional conditions for the usual causal stationary representation to exist. For an MA model, stationarity must not be confused with invertibility: stationarity holds even when the model is not invertible.

If the shocks are independent and identically distributed, the MA process is also strictly stationary.

Mean and variance

By linearity of expectation,

E⁡[Xt]=μ.\operatorname{E}[X_t]=\mu.

Because different white-noise shocks are uncorrelated, the variance is

Var⁡(Xt)=σ2(1+∑j=1qθj2).\operatorname{Var}(X_t) =\sigma^2\left(1+\sum_{j=1}^{q}\theta_j^2\right).

No requirement forces the coefficients to be positive or to sum to one. The word “average” is therefore historical terminology rather than a requirement that the model form an ordinary weighted average.

Autocovariance and autocorrelation

Set θ0=1\theta_0=1. The autocovariance at lag hh is

γ(h)={σ2∑j=0q−∣h∣θjθj+∣h∣,∣h∣≤q,0,∣h∣>q.\gamma(h) = \begin{cases} \displaystyle \sigma^2\sum_{j=0}^{q-|h|}\theta_j\theta_{j+|h|}, & |h|\le q,\\[6pt] 0,& |h|>q. \end{cases}

The autocorrelation function is

ρ(h)=γ(h)γ(0).\rho(h)=\frac{\gamma(h)}{\gamma(0)}.

Its defining feature is a cutoff after lag qq: all population autocorrelations beyond that lag are exactly zero. Observations separated by more than qq periods share no shocks.

Under independent shocks, collections of observations separated by more than qq periods are independent, a property called qq-dependence. Under merely uncorrelated white noise, zero autocorrelation does not establish independence.

The cutoff concerns the population autocorrelation function. Sample autocorrelations fluctuate and will generally not be exactly zero beyond the true order.

Example: MA(1)

For

Xt=μ+εt+θεt−1,X_t=\mu+\varepsilon_t+\theta\varepsilon_{t-1},

the variance and autocorrelations are

γ(0)=σ2(1+θ2),ρ(1)=θ1+θ2,ρ(h)=0(∣h∣>1).\gamma(0)=\sigma^2(1+\theta^2), \qquad \rho(1)=\frac{\theta}{1+\theta^2}, \qquad \rho(h)=0\quad (|h|>1).

Consequently, an MA(1) process always satisfies

∣ρ(1)∣≤12.|\rho(1)|\le\frac12.

For example, θ=0.5\theta=0.5 gives ρ(1)=0.4\rho(1)=0.4. A positive shock affects the current observation and the next observation, then disappears from the process.

Invertibility and identification

An MA model is invertible when its shocks can be recovered as a stable, one-sided infinite linear filter of current and past observations.

For a finite-order MA model, the standard invertibility condition is

θ(z)≠0for every ∣z∣≤1.\theta(z)\ne0 \quad\text{for every }|z|\le1.

Equivalently, all roots of θ(z)=0\theta(z)=0 must lie strictly outside the unit circle in the complex plane. Under this condition,

εt=θ(B)−1(Xt−μ)=∑j=0∞πj(Xt−j−μ),\varepsilon_t =\theta(B)^{-1}(X_t-\mu) =\sum_{j=0}^{\infty}\pi_j(X_{t-j}-\mu),

where the inverse-filter coefficients are absolutely summable. This also yields an infinite-order autoregressive representation.

For MA(1), invertibility requires ∣θ∣<1|\theta|<1, and the inverse follows from the geometric series:

εt=(Xt−μ)−θ(Xt−1−μ)+θ2(Xt−2−μ)−⋯ .\varepsilon_t =(X_t-\mu) -\theta(X_{t-1}-\mu) +\theta^2(X_{t-2}-\mu)-\cdots.

Why invertibility matters

Different MA parameterizations can produce the same autocovariance function. For MA(1), a nonzero coefficient θ\theta and its reciprocal 1/θ1/\theta have the same lag-one autocorrelation. Replacing θ\theta by 1/θ1/\theta and σ2\sigma^2 by θ2σ2\theta^2\sigma^2 preserves the entire autocovariance function.

For Gaussian models, these parameterizations also give the same finite-dimensional probability distributions. For non-Gaussian models, equal autocovariances need not imply equal distributions.

Restricting the model to the invertible region resolves this common second-order identification ambiguity. It also allows shocks to be interpreted as recoverable innovations and supports stable forecasting from observed history.

Roots on the unit circle are boundary cases. Such models remain stationary but lack the usual absolutely summable inverse filter, and standard estimation procedures can behave differently near these boundaries.

Relationship to other models

Moving-average smoothing

A moving-average smoother, such as

St=Xt+Xt−1+⋯+Xt−m+1m,S_t=\frac{X_t+X_{t-1}+\cdots+X_{t-m+1}}{m},

is a transformation of observed data. An MA model instead specifies observations in terms of unobserved random shocks.

The two ideas are mathematically related: applying a finite moving-average filter to white noise produces an MA process, up to normalization. Applying such a smoother to an arbitrary time series, however, does not generally produce a finite-order MA process.

ARMA and ARIMA models

An autoregressive moving-average model, or ARMA(p,qp,q), combines lagged observations and lagged shocks:

ϕ(B)(Xt−μ)=θ(B)εt,\phi(B)(X_t-\mu)=\theta(B)\varepsilon_t,

where

ϕ(B)=1−ϕ1B−⋯−ϕpBp.\phi(B)=1-\phi_1B-\cdots-\phi_pB^p.

An MA(qq) model is the special case p=0p=0.

An autoregressive integrated moving-average model, or ARIMA model, adds differencing to address certain forms of nonstationarity. Moving-average terms then describe dependence in the differenced series rather than necessarily in the original observations.

Seasonal MA terms involve shocks separated by a seasonal period. For example,

Xt−μ=εt+Θεt−sX_t-\mu=\varepsilon_t+\Theta\varepsilon_{t-s}

has a seasonal MA component with period ss.

Infinite moving-average representations

A finite-order MA model should also be distinguished from an infinite linear process,

Xt−μ=∑j=0∞ψjεt−j.X_t-\mu=\sum_{j=0}^{\infty}\psi_j\varepsilon_{t-j}.

Square-summable coefficients are sufficient for mean-square convergence when the shocks are uncorrelated with finite variance. Such a process generally has nonzero autocorrelations at arbitrarily large lags.

The Wold decomposition theorem explains the broader importance of moving-average representations: a covariance-stationary process can be decomposed into a deterministic component and a one-sided moving average of uncorrelated innovations. This representation is generally infinite rather than finite.

Estimation and model checking

MA estimation is less direct than ordinary autoregressive estimation because the explanatory shocks are unobserved. In an autoregression, lagged observations are available as regressors; in an MA model, lagged shocks must be inferred jointly with the parameters.

Common methods include:

  • Maximum likelihood: Gaussian likelihood methods use the covariance structure or an innovations-based computation. When the shocks are not Gaussian, a Gaussian likelihood can sometimes be used as a quasi-likelihood, subject to appropriate assumptions.
  • Conditional least squares: Residuals are computed recursively from trial coefficients, and their sum of squares is minimized. Results can depend on how presample shocks are initialized.
  • Moment methods: Theoretical and sample autocovariances are matched. These methods illustrate the model’s structure but can yield ambiguous or inadmissible parameter estimates.

Exact likelihood calculations can be implemented through a state-space representation and the Kalman filter. Numerical optimization commonly enforces invertibility through a constrained parameterization.

Choosing the order

A sample autocorrelation function that becomes small after lag qq suggests an MA(qq) model, but it does not establish the order conclusively. Sampling noise, outliers, seasonal structure, and mixed autoregressive–moving-average behavior can obscure the cutoff.

The partial autocorrelation function of an MA model generally tails off rather than cutting off. This is the reverse of the familiar population pattern for a pure autoregression.

Order selection commonly combines these patterns with information criteria, residual diagnostics, and out-of-sample forecast evaluation. Cross-validation for time series must respect temporal ordering rather than randomly mixing past and future observations.

A fitted model should leave residuals with little remaining autocorrelation. Uncorrelated residuals, however, do not by themselves establish independence, normality, or constant conditional variance.

Forecasting

Suppose the parameters are known and the model is invertible, with the infinite observed past available so that past shocks can be recovered. The best linear hh-step forecast is

X^t+h∣t=μ+∑j=hqθjεt+h−j,1≤h≤q,\widehat X_{t+h\mid t} = \mu+\sum_{j=h}^{q}\theta_j\varepsilon_{t+h-j}, \qquad 1\le h\le q,

and

X^t+h∣t=μ,h>q.\widehat X_{t+h\mid t}=\mu, \qquad h>q.

Only shocks already realized at time tt contribute to the forecast; future shocks receive a forecast of zero. If the shocks are independent, or satisfy suitable martingale-difference assumptions, this forecast is also the conditional expectation. With only weak white-noise assumptions, the general claim is best linear prediction.

With θ0=1\theta_0=1, the forecast-error variance is

Var⁡(Xt+h−X^t+h∣t)=σ2∑j=0min⁡(h−1,q)θj2.\operatorname{Var} \left(X_{t+h}-\widehat X_{t+h\mid t}\right) = \sigma^2 \sum_{j=0}^{\min(h-1,q)}\theta_j^2.

For h>qh>q, this equals the unconditional variance. Thus, a pure MA model has a finite linear forecasting horizon: sufficiently far ahead, past observations no longer improve the forecast beyond the mean.

In a finite sample, shocks are estimated rather than known exactly. Practical forecast uncertainty may therefore include state-initialization and parameter-estimation uncertainty. Under Gaussian assumptions, forecast-error variances provide the basis for prediction intervals.

Frequency-domain interpretation

The spectral density of an MA(qq) process is

f(ω)=σ22π∣1+θ1e−iω+⋯+θqe−iqω∣2,−π≤ω≤π.f(\omega) = \frac{\sigma^2}{2\pi} \left| 1+\theta_1e^{-i\omega} +\cdots+\theta_qe^{-iq\omega} \right|^2, \qquad -\pi\le\omega\le\pi.

The moving-average polynomial determines how the model amplifies or attenuates different frequencies. Positive MA(1) coefficients emphasize lower-frequency variation relative to higher-frequency variation; negative coefficients have the opposite effect. A polynomial root on the unit circle creates a spectral zero at the corresponding frequency.

Applications and limitations

Moving-average models are useful when disturbances have effects that last for a limited number of periods. Applications include short-horizon forecasting, measurement-error models, and correlated-error specifications in econometrics. They also arise from temporal aggregation and filtering of more detailed processes.

Several limitations are important:

  • Finite memory: A low-order MA model cannot represent slowly decaying or long-range autocorrelation well.
  • No built-in trend: A stationary MA model is not, by itself, a model of a persistent trend or changing mean.
  • Constant coefficients: Structural changes and time-varying dependence require additional modeling.
  • Limited second-order description: Autocovariances do not fully describe nonlinear dependence or changing conditional volatility.
  • Latent-shock interpretation: Estimated shocks are model-based innovations, not automatically identifiable real-world causes.
  • Boundary sensitivity: Near-unit-circle roots can make inversion, estimation, and uncertainty assessment difficult.

A particularly important boundary example is the first difference of independent measurement noise:

Xt=εt−εt−1.X_t=\varepsilon_t-\varepsilon_{t-1}.

This is a stationary MA(1) process with coefficient −1-1, but it is not invertible in the standard stable sense. Differencing an already stationary noise component can therefore introduce a noninvertible moving-average factor.