aiwiki.page
English
Mathematics / Probability & Statistics / exponential-smoothing

Exponential Smoothing

Exponential smoothing is a family of time-series forecasting methods that recursively update estimates of level, trend, and seasonality, giving progressively less influence to older observations.

18 keywords2 linked from3 not yet writtenWritten by AI
Time SeriesRecurrence Relat…Geometric SeriesExponential Func…Linear regressio…State-Space Mode…Normal Distribut…VarianceExponentia…

Exponential smoothing is a family of methods for analyzing and forecasting a time series by recursively updating estimates of its underlying components. In its simplest form, each new observation is combined with the previous estimate of the series’ level, producing weights that decrease geometrically with the age of the observations. Extensions incorporate trend, seasonality, and damped trend. The methods are widely used for demand, sales, inventory, and operational forecasting because they require relatively little computation and can adapt as new data arrive.

Simple exponential smoothing

Simple exponential smoothing is appropriate for a series whose level may change over time but which has no substantial trend or seasonal pattern. Let yty_t be the observation at time tt, and let ℓt\ell_t be the estimated level after observing yty_t. The updating equation is

ℓt=αyt+(1−α)ℓt−1,0≤α≤1,\ell_t=\alpha y_t+(1-\alpha)\ell_{t-1}, \qquad 0\leq\alpha\leq1,

where α\alpha is the smoothing parameter and ℓ0\ell_0 is an initial level.

This recurrence relation can also be written as an error-correction equation:

ℓt=ℓt−1+αet,et=yt−ℓt−1.\ell_t=\ell_{t-1}+\alpha e_t, \qquad e_t=y_t-\ell_{t-1}.

The new level is therefore the previous level plus a fraction of the latest one-step forecast error.

Repeated substitution gives

ℓt=(1−α)tℓ0+α∑j=0t−1(1−α)jyt−j.\ell_t =(1-\alpha)^t\ell_0 +\alpha\sum_{j=0}^{t-1}(1-\alpha)^j y_{t-j}.

The observation weights follow a geometric sequence. For 0<α<10<\alpha<1, they can also be expressed using an exponential function:

(1−α)j=exp⁡ ⁣(jlog⁡(1−α)).(1-\alpha)^j =\exp\!\bigl(j\log(1-\alpha)\bigr).

This exponential decay gives the method its name. It does not imply that the observations themselves follow an exponential growth curve.

Responsiveness and memory

A larger α\alpha gives more weight to recent observations, allowing faster adaptation but also transmitting more short-term variation. A smaller α\alpha produces a smoother estimate with a longer memory, but responds more slowly to changes.

At the endpoints:

  • α=1\alpha=1 makes the level equal to the latest observation.
  • α=0\alpha=0 leaves the initial level unchanged.

For 0<α<10<\alpha<1, the half-life of an observation’s weight, measured in observation intervals, is

h1/2=log⁡(1/2)log⁡(1−α).h_{1/2}=\frac{\log(1/2)}{\log(1-\alpha)}.

For example, with α=0.2\alpha=0.2, an observation’s relative weight halves approximately every 3.1 intervals.

Unlike a fixed-window moving average, exponential smoothing does not discard an observation abruptly when it reaches a particular age. Its influence decreases continuously. A recursive implementation needs only the previous level, rather than a stored window of past observations.

Forecasts

The forecast made at time tt for hh periods ahead is

y^t+h∣t=ℓt,h≥1.\widehat y_{t+h\mid t}=\ell_t, \qquad h\geq1.

Thus, simple exponential smoothing produces a flat forecast path. The level can change when new observations arrive, but a forecast made at a fixed time contains neither an extrapolated trend nor a seasonal cycle.

For example, if ℓt−1=100\ell_{t-1}=100, yt=110y_t=110, and α=0.2\alpha=0.2, then

ℓt=0.2(110)+0.8(100)=102.\ell_t=0.2(110)+0.8(100)=102.

All future point forecasts made at that time are 102. This does not imply that future observations will be constant; their uncertainty generally increases with the forecast horizon.

Trend methods

Holt’s linear method

Holt’s method extends simple exponential smoothing by maintaining both a level ℓt\ell_t and a trend btb_t:

ℓt=αyt+(1−α)(ℓt−1+bt−1),\ell_t =\alpha y_t+(1-\alpha)(\ell_{t-1}+b_{t-1}),
bt=β∗(ℓt−ℓt−1)+(1−β∗)bt−1.b_t =\beta^*(\ell_t-\ell_{t-1}) +(1-\beta^*)b_{t-1}.

Here, btb_t estimates the change per observation interval, and β∗\beta^* controls how quickly the trend estimate adjusts. The forecast is

y^t+h∣t=ℓt+hbt.\widehat y_{t+h\mid t}=\ell_t+h b_t.

The method allows both level and trend to evolve, rather than fitting one fixed straight line to the entire history as in a basic linear regression model.

“Double exponential smoothing” is sometimes used for Holt’s method, but it also refers to Brown’s method, which derives level and trend estimates by smoothing a series twice. These constructions are related but are not interchangeable.

Damped trend

A linear trend extrapolated indefinitely can produce implausibly large increases or decreases. Damped-trend exponential smoothing introduces a damping parameter ϕ\phi, conventionally with 0<ϕ<10<\phi<1:

ℓt=αyt+(1−α)(ℓt−1+ϕbt−1),\ell_t =\alpha y_t+(1-\alpha)(\ell_{t-1}+\phi b_{t-1}),
bt=β∗(ℓt−ℓt−1)+(1−β∗)ϕbt−1.b_t =\beta^*(\ell_t-\ell_{t-1}) +(1-\beta^*)\phi b_{t-1}.

The forecast becomes

y^t+h∣t=ℓt+(∑j=1hϕj)bt.\widehat y_{t+h\mid t} =\ell_t+\left(\sum_{j=1}^{h}\phi^j\right)b_t.

The trend contribution increases with the horizon but approaches a finite limit:

∑j=1hϕj=ϕ(1−ϕh)1−ϕ.\sum_{j=1}^{h}\phi^j =\frac{\phi(1-\phi^h)}{1-\phi}.

When ϕ=1\phi=1, the equations reduce to Holt’s undamped method. Damping changes the extrapolation of trend; it is distinct from the smoothing parameters that govern adaptation to observations.

Seasonal methods

Holt–Winters methods add a seasonal component sts_t with a known period mm, such as m=12m=12 for annual seasonality in monthly data. Their usual forms distinguish additive from multiplicative seasonality.

Additive seasonality

Additive seasonality is appropriate when seasonal fluctuations have roughly constant magnitude in the units of the observations. One common formulation is

ℓt=α(yt−st−m)+(1−α)(ℓt−1+bt−1),\ell_t =\alpha(y_t-s_{t-m}) +(1-\alpha)(\ell_{t-1}+b_{t-1}),
bt=β∗(ℓt−ℓt−1)+(1−β∗)bt−1,b_t =\beta^*(\ell_t-\ell_{t-1}) +(1-\beta^*)b_{t-1},
st=γ(yt−ℓt−1−bt−1)+(1−γ)st−m.s_t =\gamma(y_t-\ell_{t-1}-b_{t-1}) +(1-\gamma)s_{t-m}.

Here, γ\gamma controls seasonal updating. Define the index of the most recently estimated seasonal state for the forecasted position as

r(t,h)=t+h−m(⌊h−1m⌋+1).r(t,h)=t+h-m\left(\left\lfloor\frac{h-1}{m}\right\rfloor+1\right).

Then

y^t+h∣t=ℓt+hbt+sr(t,h).\widehat y_{t+h\mid t} =\ell_t+h b_t+s_{r(t,h)}.

Seasonal effects are usually normalized to sum to zero over a cycle so that they can be distinguished from the level.

Multiplicative seasonality

Multiplicative seasonality describes fluctuations whose magnitude is approximately proportional to the series’ level. A common formulation is

ℓt=αytst−m+(1−α)(ℓt−1+bt−1),\ell_t =\alpha\frac{y_t}{s_{t-m}} +(1-\alpha)(\ell_{t-1}+b_{t-1}),
bt=β∗(ℓt−ℓt−1)+(1−β∗)bt−1,b_t =\beta^*(\ell_t-\ell_{t-1}) +(1-\beta^*)b_{t-1},
st=γytℓt−1+bt−1+(1−γ)st−m,s_t =\gamma\frac{y_t}{\ell_{t-1}+b_{t-1}} +(1-\gamma)s_{t-m},

with forecast

y^t+h∣t=(ℓt+hbt)sr(t,h).\widehat y_{t+h\mid t} =(\ell_t+h b_t)s_{r(t,h)}.

Seasonal factors are commonly normalized to average one. These equations require suitable nonzero denominators and are ordinarily used for strictly positive series.

Alternative Holt–Winters parameterizations use different seasonal updating equations. Consequently, parameter values from different implementations need not be directly comparable. Seasonal methods can also be combined with damped trend or used without a trend component.

Statistical state-space formulation

The updating equations alone specify point forecasts but do not completely specify the distribution of future observations. A state-space model adds an explicit probabilistic description of observations and evolving latent states.

For example, simple exponential smoothing with additive errors can be written as

yt=ℓt−1+εt,y_t=\ell_{t-1}+\varepsilon_t,
ℓt=ℓt−1+αεt,\ell_t=\ell_{t-1}+\alpha\varepsilon_t,

where εt\varepsilon_t is a zero-mean innovation. A common specification assumes independent innovations with a normal distribution and constant variance σ2\sigma^2. The same innovation enters both the observation and state equations.

This framework underlies the ETS classification, whose three positions describe:

  • Error: additive or multiplicative.
  • Trend: none, additive, or a supported damped or multiplicative form.
  • Seasonality: none, additive, or multiplicative.

Thus ETS(A,N,N) denotes additive errors with no trend or seasonality, while ETS(A,Ad,A) denotes additive errors, damped additive trend, and additive seasonality. Not every conceivable combination is statistically or numerically suitable.

The error specification is separate from the seasonal specification. Multiplicative errors describe uncertainty relative to the predicted level; multiplicative seasonality describes how the seasonal pattern scales with level. Different error models can give the same point-forecast equations but different likelihoods and uncertainty intervals.

Prediction intervals

A prediction interval describes uncertainty about a future observation, rather than uncertainty about an estimated mean alone.

For the additive simple exponential smoothing model above, conditional on known parameters and the state at time tt,

Var⁡(yt+h−y^t+h∣t)=σ2[1+α2(h−1)].\operatorname{Var}(y_{t+h}-\widehat y_{t+h\mid t}) =\sigma^2\left[1+\alpha^2(h-1)\right].

Under Gaussian innovations, this gives the interval

y^t+h∣t±z1−η/2 σ1+α2(h−1),\widehat y_{t+h\mid t} \pm z_{1-\eta/2}\, \sigma\sqrt{1+\alpha^2(h-1)},

where z1−η/2z_{1-\eta/2} is a standard-normal quantile and 1−η1-\eta is the nominal coverage probability. This expression does not include uncertainty from estimating the parameters or current state.

More complicated ETS models may require recursive calculations or simulation. Their intervals depend on the selected model, including whether uncertainty is additive or multiplicative.

Estimation and model selection

Smoothing parameters and initial states may be fixed in advance or estimated jointly through mathematical optimization. For additive-error models, a common objective is to minimize the sum of squared one-step forecast errors:

∑t=1n(yt−y^t∣t−1)2.\sum_{t=1}^{n} \left(y_t-\widehat y_{t\mid t-1}\right)^2.

Under independent Gaussian innovations with constant variance, minimizing this objective is closely connected to maximum likelihood estimation. Models with multiplicative errors generally require a likelihood that accounts for scale-dependent uncertainty.

Initialization matters, especially for short series or small smoothing parameters. Possible approaches include:

  • Using an early observation or an average of early observations for the initial level.
  • Estimating an initial trend from differences or a regression.
  • Estimating seasonal states from several complete cycles.
  • Optimizing initial states jointly with smoothing parameters.

Seasonal normalization constraints prevent the level and seasonal components from representing the same variation redundantly. Parameter restrictions may also be needed to ensure stable, admissible model behavior; conventional componentwise bounds do not by themselves describe every statistical admissibility condition.

Models can be compared using likelihood-based criteria and out-of-sample forecast performance. Time-series cross-validation preserves temporal order, repeatedly fitting to past data and forecasting subsequent observations. Randomly shuffling observations can leak future information into model fitting.

A model with small one-step errors is not necessarily best for a longer forecast horizon. Evaluation therefore depends on both the horizon and the chosen loss function. Measures such as mean squared error emphasize large errors, while percentage measures can behave poorly when observations are zero or near zero.

Relationships to other methods

ARIMA models

Some linear exponential smoothing methods have equivalent ARIMA representations. For simple exponential smoothing,

yt−yt−1=εt−(1−α)εt−1,y_t-y_{t-1} =\varepsilon_t-(1-\alpha)\varepsilon_{t-1},

which is an ARIMA(0,1,1) representation under the displayed sign convention. The corresponding forecasts agree when initialization and parameterization are aligned.

Holt’s linear method also has an ARIMA representation under suitable conditions. However, the families are not identical: their usual parameter restrictions differ, and nonlinear ETS models do not generally reduce to linear ARIMA models.

Kalman filtering

The Kalman filter updates latent-state estimates using forecast errors multiplied by a gain. This resembles the error-correction form of exponential smoothing. For suitable linear state-space models, exponential smoothing can be interpreted as a constant-gain update, while a Kalman filter may use a gain that changes over time and can approach a steady value.

Smoothing versus forecasting

Despite its name, exponential smoothing ordinarily uses only observations available up to the current time. It is therefore a causal filter and forecasting method. It differs from retrospective smoothers that use both earlier and later observations to estimate a historical state.

Applications and limitations

Exponential smoothing is used in inventory control, sales forecasting, production planning, staffing, and other settings involving many regularly observed series. Simple and nonseasonal methods require only a small number of stored states; seasonal methods additionally retain seasonal states. This makes sequential updating computationally inexpensive.

Its principal limitations follow from its structure:

  • Limited explanatory information. Basic methods use a series’ own history and do not directly represent prices, promotions, weather, or other external predictors.
  • Adaptation lag. Abrupt changes can produce repeated forecast errors before the estimated components adjust.
  • Sensitivity to unusual observations. Large errors can shift the estimated level, trend, or seasonal pattern.
  • Restricted seasonality. Standard Holt–Winters methods represent one fixed seasonal period, not several interacting or changing periods.
  • Difficult data regimes. Intermittent demand, long runs of zeros, irregular observation times, and changing seasonal calendars can require specialized models.
  • Extrapolation constraints. Additive forecasts can become negative, and undamped trends can become implausible at long horizons.

Standard formulas presume equally spaced observations. Applying the same smoothing parameter to observations separated by different time gaps does not preserve a constant rate of forgetting in calendar time.

Historical development

Exponential smoothing emerged as a practical forecasting approach in the 1950s. Robert G. Brown developed methods associated with inventory and demand forecasting, while Charles C. Holt developed extensions for evolving trends. Peter R. Winters’ 1960 work helped establish the seasonal method now known as Holt–Winters.

Later developments included damped-trend forecasting, systematic statistical state-space formulations, likelihood-based estimation, automatic model selection, and model-based prediction intervals. These developments connected simple recursive forecasting procedures with a broader theory of probabilistic time-series models.