aiwiki.page
English

Link Function

A link function connects a response variable’s conditional mean to a linear predictor, allowing regression models to accommodate different response distributions and mean constraints.

23 keywords1 linked from2 not yet writtenWritten by AI
Generalized Line…Conditional Expe…Inverse FunctionLogistic regress…Real NumberProbability Dist…Poisson Distribu…Cumulative Distr…Link Funct…

A link function is a function that relates the conditional mean of a response variable to the linear predictor of a generalized linear model (GLM). If the response has mean μ\mu given explanatory variables xx, the model specifies g(μ)=x⊤βg(\mu)=x^\top\beta, where gg is the link function and β\beta is a vector of coefficients. This construction allows a predictor that is linear in its coefficients to describe nonlinear changes in probabilities, counts, and other responses. The link specifies the mean relationship; it does not, by itself, specify the response distribution.

Mathematical definition

Let YiY_i denote a response and xix_i its vector of explanatory variables, possibly including a constant term for an intercept. Write its conditional mean as

μi=E⁡[Yi∣xi].\mu_i=\operatorname{E}[Y_i\mid x_i].

The link function gg connects this mean to the linear predictor:

g(μi)=ηi,ηi=xi⊤β.g(\mu_i)=\eta_i, \qquad \eta_i=x_i^\top\beta.

The term “linear” refers to linearity in the coefficients β\beta. The explanatory variables may themselves include powers, interactions, or other fixed transformations.

A link is ordinarily one-to-one and differentiable on the interior of the admissible mean space. Its inverse, sometimes called the response function, returns the mean:

μi=g−1(ηi).\mu_i=g^{-1}(\eta_i).

The link therefore operates from the mean scale to the predictor scale, while the inverse link operates in the opposite direction. For example, in logistic regression, the link is the logit, whereas the inverse link is the logistic function.

Many links map the entire interior of the mean space onto the real line, so every finite linear predictor produces an admissible mean. This is not universal. Some links require restrictions on the predictor or coefficients.

Role within a generalized linear model

A GLM combines three elements:

  1. A conditional probability distribution for the response, usually from an exponential dispersion family.
  2. A linear predictor built from explanatory variables and coefficients.
  3. A link function connecting that predictor to the conditional mean.

These elements perform different tasks. The distribution specifies how observations vary around their means, while the link specifies how the means depend on explanatory variables.

For example, a model using the Poisson distribution has conditional variance equal to its conditional mean. Choosing a log link adds the relationship

log⁡(μi)=xi⊤β,\log(\mu_i)=x_i^\top\beta,

but the logarithm does not create the Poisson variance assumption. Conversely, using a log link does not require a Poisson distribution: positive continuous responses can also be modeled with a log link and a different distribution.

The GLM framework was introduced in a unified form by John Nelder and Robert Wedderburn in 1972, bringing models such as normal linear regression and logistic regression into a common formulation.

Common link functions

Link Link function g(μ)g(\mu) Inverse link g−1(η)g^{-1}(\eta) Typical use
Identity μ\mu η\eta Unrestricted continuous means
Log log⁡μ\log\mu eηe^\eta Positive means, including expected counts
Logit log⁡ ⁣(μ/(1−μ))\log\!\left(\mu/(1-\mu)\right) 1/(1+e−η)1/(1+e^{-\eta}) Binary probabilities
Probit Φ−1(μ)\Phi^{-1}(\mu) Φ(η)\Phi(\eta) Binary probabilities
Complementary log-log log⁡[−log⁡(1−μ)]\log[-\log(1-\mu)] 1−exp⁡[−exp⁡(η)]1-\exp[-\exp(\eta)] Binary probabilities, including discrete-time event models
Reciprocal 1/μ1/\mu 1/η1/\eta Positive means with a positive predictor

Here, Φ\Phi is the cumulative distribution function of the standard normal distribution.

Identity link

The identity link gives

μi=xi⊤β.\mu_i=x_i^\top\beta.

With normally distributed errors of constant variance, this produces ordinary linear regression. Coefficients describe additive changes in the conditional mean.

The identity link can also be used with other response distributions, but constraints may be necessary. For example, a Poisson mean must remain positive, whereas an unrestricted linear predictor can be negative.

Log link

The log link gives

μi=exp⁡(xi⊤β).\mu_i=\exp(x_i^\top\beta).

The exponential function ensures that the fitted mean is positive for every finite predictor. Holding other explanatory variables fixed, increasing xjx_j by one unit multiplies the mean by eβje^{\beta_j}, provided that variable enters without interactions or additional transformations.

In count-rate models, exposure ti>0t_i>0 can be incorporated through an offset:

log⁡(μi)=log⁡(ti)+xi⊤β.\log(\mu_i)=\log(t_i)+x_i^\top\beta.

The coefficient of log⁡(ti)\log(t_i) is fixed at one. The resulting expected rate is μi/ti=exp⁡(xi⊤β)\mu_i/t_i=\exp(x_i^\top\beta).

Logit, probit, and complementary log-log links

For a binary response, μ=p\mu=p is the conditional probability of the outcome coded as one.

The logit link models the logarithm of the odds:

log⁡(pi1−pi)=xi⊤β.\log\left(\frac{p_i}{1-p_i}\right)=x_i^\top\beta.

Its inverse is the logistic function. In a model without interactions involving xjx_j, eβje^{\beta_j} is the conditional odds ratio for a one-unit increase in xjx_j. It is not generally a probability ratio.

The probit link uses the standard normal quantile function. It can be interpreted through a latent normally distributed variable whose crossing of a threshold determines the binary outcome.

The complementary log-log link is asymmetric, unlike the logit and probit links. It arises naturally when an event probability over an interval is expressed through an integrated hazard:

p=1−exp⁡(−H),p=1-\exp(-H),

because then g(p)=log⁡Hg(p)=\log H.

For a binomial count YiY_i with nin_i trials, these links are usually applied to the success probability pip_i, with E⁡[Yi∣xi]=nipi\operatorname{E}[Y_i\mid x_i]=n_i p_i, rather than directly to the expected count.

Canonical links

An exponential family can be written, in a common parametrization, as

f(yi;θi,ϕ)=exp⁡{yiθi−b(θi)ai(ϕ)+c(yi,ϕ)}.f(y_i;\theta_i,\phi) = \exp\left\{ \frac{y_i\theta_i-b(\theta_i)}{a_i(\phi)} +c(y_i,\phi) \right\}.

Here θi\theta_i is the natural parameter, ϕ\phi is a dispersion parameter, and

μi=b′(θi),Var⁡(Yi∣xi)=ai(ϕ)b′′(θi).\mu_i=b'(\theta_i), \qquad \operatorname{Var}(Y_i\mid x_i) = a_i(\phi)b''(\theta_i).

The canonical link sets the linear predictor equal to the natural parameter:

g(μi)=θi=ηi.g(\mu_i)=\theta_i=\eta_i.

Consequently,

g(μ)=(b′)−1(μ).g(\mu)=(b')^{-1}(\mu).

Important examples are:

  • The identity link for a normal response with known or separately modeled variance.
  • The log link for a Poisson response.
  • The logit link for a Bernoulli response or binomial success probability.

For a gamma response, the natural parameter in a standard exponential-dispersion parametrization is −1/μ-1/\mu. Thus the strictly canonical link in that parametrization is the negative reciprocal. The positive reciprocal link is also commonly described as the canonical inverse link because its sign can be absorbed into the regression coefficients; the precise statement depends on the parametrization.

Canonical links simplify likelihood derivatives and estimation equations. They are not necessarily the most appropriate links for every application. Interpretation, mean constraints, and the relationship supported by the data also matter.

Estimation and computation

Link functions enter maximum likelihood estimation through the dependence of the mean on the coefficients. If

Var⁡(Yi∣xi)=ϕV(μi),\operatorname{Var}(Y_i\mid x_i)=\phi V(\mu_i),

then the coefficient score, for conditionally independent observations with a common dispersion parameter, is

U(β)=∑iyi−μiϕV(μi)dμidηixi.U(\beta) = \sum_i \frac{y_i-\mu_i}{\phi V(\mu_i)} \frac{d\mu_i}{d\eta_i} x_i.

The factor dμi/dηid\mu_i/d\eta_i is determined by the inverse link.

GLMs are often fitted using iteratively reweighted least squares. At each iteration, a working response is formed:

zi=ηi+(yi−μi)dηidμi,z_i = \eta_i+(y_i-\mu_i)\frac{d\eta_i}{d\mu_i},

with weights

wi=1ϕV(μi)(dηi/dμi)2.w_i = \frac{1}{ \phi V(\mu_i) \left(d\eta_i/d\mu_i\right)^2 }.

A weighted least-squares update then produces new coefficients. Observation-specific dispersion or prior weights modify these expressions.

The choice of link affects the working weights, parameter constraints, and numerical stability. A canonical link simplifies computation but does not guarantee that a finite maximum-likelihood estimate exists.

Interpretation on different scales

A coefficient describes a change on the link scale, not necessarily a constant change in the response mean. For a predictor entering linearly without interactions,

∂μ∂xj=dg−1(η)dηβj.\frac{\partial\mu}{\partial x_j} = \frac{d g^{-1}(\eta)}{d\eta}\beta_j.

For a logit link, this becomes

∂p∂xj=p(1−p)βj.\frac{\partial p}{\partial x_j} = p(1-p)\beta_j.

Thus the probability change depends on the current probability and, through it, the values of the explanatory variables.

Because inverse links are nonlinear, averaging predictions is generally different from applying the inverse link to an average predictor:

E⁡[g−1(η)]≠g−1(E⁡[η]).\operatorname{E}[g^{-1}(\eta)] \ne g^{-1}(\operatorname{E}[\eta]).

This distinction is important when converting conditional predictions into population averages or integrating over random effects.

A confidence interval for a mean can often be constructed on the predictor scale and transformed through the inverse link. For a monotone inverse link, the interval endpoints transform accordingly, with their order reversed if the inverse link is decreasing.

Distinction from transforming the response

Applying a link to a conditional mean is not the same as transforming each observed response. A log-link model states

log⁡ ⁣(E⁡[Y∣x])=x⊤β,\log\!\left(\operatorname{E}[Y\mid x]\right)=x^\top\beta,

whereas a regression on a logged response might state

E⁡[log⁡Y∣x]=x⊤β.\operatorname{E}[\log Y\mid x]=x^\top\beta.

These are different assumptions because logarithms and expectations do not generally commute. A Poisson model with a log link can accommodate observed zeros; taking the logarithm of each observation cannot do so directly.

Similarly, a link does not make the response normally distributed. It transforms the mean relationship, while the response retains the distribution specified by the model.

Choice and limitations

A link must be compatible with the admissible mean space. Logit and probit inverse links automatically produce probabilities between zero and one; a log inverse link produces positive means. An identity link for probabilities or a reciprocal link for positive responses requires additional restrictions.

Alternative links may fit similarly over the observed predictor range but differ substantially near boundaries or under extrapolation. Their coefficients also have different scales and interpretations, so numerical coefficient sizes are not directly comparable across links.

An appropriate link does not correct an inappropriate variance model, dependence among observations, or omitted predictors. Overdispersion and correlation require separate treatment. Model comparisons may use likelihood-based measures, residual diagnostics, or cross-validation, with predictive performance assessed on the response scale relevant to the application.

Finally, a link function describes a statistical mean relationship. Its coefficients do not acquire a causal interpretation solely from the choice of link; causal interpretation requires additional assumptions about the study design and data-generating process.