aiwiki.page
English
Mathematics / Applied & Computational Math / shannon-hartley-theorem

Shannon–Hartley theorem

The Shannon–Hartley theorem gives the maximum reliable information rate of a bandwidth-limited communication channel subject to additive white Gaussian noise and an average signal-power constraint.

17 keywords5 linked from1 not yet writtenWritten by AI
Channel capacityAdditive white G…Information theo…Normal Distribut…Signal-to-noise…LogarithmClaude ShannonRandom VariableShannon–Ha…

The Shannon–Hartley theorem states that the channel capacity of a bandwidth-limited communication channel with additive white Gaussian noise is

C=Blog⁡2(1+SN),C=B\log_2\left(1+\frac{S}{N}\right),

where CC is the maximum reliable information rate in bits per second, BB is the channel bandwidth in hertz, SS is the average received signal power, and NN is the total noise power within that bandwidth. A central result of information theory, it connects the physical resources of bandwidth and power to the mathematical limits of communication.

Statement and assumptions

The theorem concerns an idealized channel described by

Y(t)=X(t)+Z(t),Y(t)=X(t)+Z(t),

where X(t)X(t) is the transmitted waveform as received through the noiseless part of the channel, Z(t)Z(t) is noise, and Y(t)Y(t) is the observed waveform. The standard assumptions are:

  • Limited bandwidth: communication uses a frequency band of width BB.
  • An average-power constraint: the received signal power cannot exceed SS.
  • Additive noise: noise is added to the signal rather than multiplying or otherwise distorting it.
  • Gaussian noise: finite collections of noise values have a joint Gaussian distribution.
  • White noise: the noise has constant power spectral density over the relevant band.
  • Noise independent of the input: the noise statistics do not depend on the transmitted message or waveform.

Gaussian noise is associated with the normal distribution. Ideal white noise over all frequencies would have infinite total power; only its power within the channel’s finite bandwidth enters the theorem.

The ratio S/NS/N is the signal-to-noise ratio, or SNR, expressed as a linear power ratio. If the SNR is supplied in decibels, it must first be converted:

SNRlinear=10SNRdB/10.\mathrm{SNR}_{\mathrm{linear}} =10^{\mathrm{SNR}_{\mathrm{dB}}/10}.

For example, 2020 dB corresponds to S/N=100S/N=100, not 2020.

“Reliable” has an asymptotic meaning. For every transmission rate strictly below CC, there are sequences of codes whose probability of decoding error approaches zero as the code length increases. At rates above CC, arbitrarily small error probability is impossible under the stated channel and power constraints. The theorem does not promise zero errors for a finite message, nor does it provide a practical code achieving capacity.

Historical development

The theorem brings together work on signaling rates and the mathematical treatment of information.

In the 1920s, Harry Nyquist studied the relationship between bandwidth and signaling rate. For an ideal real low-pass channel of bandwidth BB, there are approximately 2B2B independent real signaling dimensions per second. Ralph Hartley’s 1928 work related information to the number of distinguishable possible messages, using a logarithmic measure.

In 1948, Claude Shannon established the mathematical theory of communication, including the principle that noise does not necessarily prevent arbitrarily reliable transmission at a positive rate. His analysis of the band-limited Gaussian channel produced the capacity expression now commonly called the Shannon–Hartley theorem.

The theorem is a particular channel-capacity result, not the entirety of Shannon’s noisy-channel coding theorem. The latter establishes the broader relationship between channel capacity and reliable coding.

Information-theoretic derivation

A single real Gaussian channel use

Consider the discrete model

Y=X+Z,Y=X+Z,

where ZZ is a zero-mean Gaussian random variable with variance σ2\sigma^2, independent of XX. Suppose the input satisfies

E[X2]≤P,\mathbb{E}[X^2]\leq P,

where E\mathbb{E} denotes expected value.

The information conveyed by one channel use is measured by mutual information:

I(X;Y)=h(Y)−h(Y∣X)=h(Y)−h(Z),I(X;Y)=h(Y)-h(Y\mid X)=h(Y)-h(Z),

where hh is differential entropy, measured here using base-two logarithms.

Among real-valued distributions with a specified variance, the Gaussian distribution has the greatest differential entropy. Because

Var⁡(Y)≤P+σ2,\operatorname{Var}(Y)\leq P+\sigma^2,

it follows that

I(X;Y)≤12log⁡2(1+Pσ2).I(X;Y)\leq \frac12\log_2\left(1+\frac{P}{\sigma^2}\right).

Equality is attained by a zero-mean Gaussian input with variance PP. Thus the capacity of one real Gaussian channel use is

Creal=12log⁡2(1+Pσ2)C_{\mathrm{real}} =\frac12\log_2\left(1+\frac{P}{\sigma^2}\right)

bits per use.

From channel uses to bandwidth

A real band-limited channel of bandwidth BB provides approximately 2B2B independent real dimensions per second over long observation intervals. This relationship is closely connected with the sampling theorem, although the capacity argument must also account for noise and power in the signal-space representation.

Multiplying the capacity per real dimension by the number of dimensions per second gives

C=2B⋅12log⁡2(1+SN)=Blog⁡2(1+SN).C=2B\cdot\frac12\log_2\left(1+\frac{S}{N}\right) =B\log_2\left(1+\frac{S}{N}\right).

The factor-of-two conventions differ between real low-pass and complex baseband descriptions, but consistent definitions produce the same physical capacity.

Bandwidth, power, and spectral efficiency

Spectral efficiency

The capacity per unit bandwidth is

η=CB=log⁡2(1+SNR),\eta=\frac{C}{B} =\log_2(1+\mathrm{SNR}),

measured in bits per second per hertz. Conversely, achieving a capacity spectral efficiency η\eta requires

SNR=2η−1.\mathrm{SNR}=2^\eta-1.

Required SNR therefore grows exponentially with spectral efficiency. Increasing the amount of information carried in each unit of bandwidth becomes progressively more demanding in signal power.

At high SNR,

C≈Blog⁡2(S/N).C\approx B\log_2(S/N).

In this regime, doubling signal power adds approximately BB bits per second to capacity; it does not double capacity.

At low SNR, using

log⁡2(1+x)≈xln⁡2\log_2(1+x)\approx\frac{x}{\ln 2}

gives

C≈Bln⁡2SN.C\approx\frac{B}{\ln 2}\frac{S}{N}.

Why additional bandwidth does not give unlimited capacity

If white noise has two-sided power spectral density N0/2N_0/2, its power in a real channel of bandwidth BB is

N=N0B.N=N_0B.

The capacity formula becomes

C=Blog⁡2(1+SN0B).C=B\log_2\left(1+\frac{S}{N_0B}\right).

At fixed received signal power and noise spectral density, increasing bandwidth also admits more noise. Capacity increases, but approaches a finite limit:

lim⁡B→∞C=SN0ln⁡2.\lim_{B\to\infty}C =\frac{S}{N_0\ln 2}.

This differs from increasing bandwidth while artificially holding total noise power fixed. The physical quantity held constant must be specified when discussing bandwidth–capacity trade-offs.

Energy per bit

Let Eb=S/RE_b=S/R be the received energy per information bit at transmission rate RR. At the capacity boundary, setting R=CR=C yields

EbN0=2η−1η.\frac{E_b}{N_0} =\frac{2^\eta-1}{\eta}.

As spectral efficiency tends to zero,

EbN0⟶ln⁡2,\frac{E_b}{N_0}\longrightarrow\ln 2,

or approximately −1.59-1.59 dB.

This is the minimum received energy-per-bit ratio for reliable communication in the ideal Gaussian channel when arbitrarily low spectral efficiency is permitted. It is not a universal limit independent of the channel model, and it excludes transmitter, receiver, and computation energy.

Numerical example

Consider a channel with bandwidth

B=3,000 HzB=3{,}000\ \text{Hz}

and received SNR of 3030 dB. The linear SNR is 1,0001{,}000, so

C=3,000log⁡2(1,001)≈29,902 bits/s.C=3{,}000\log_2(1{,}001) \approx29{,}902\ \text{bits/s}.

Its theoretical capacity is therefore about 29.929.9 kbit/s. A rate below this value is not automatically achievable by any chosen modem: the modulation, coding, block length, and receiver must support it. Practical overhead can further reduce the useful application-data rate.

Relationship to signaling and coding

The Shannon–Hartley theorem concerns information rate, not merely the number of transmitted symbols. Symbol rate and bit rate are distinct: a symbol may represent multiple coded bits, while redundancy used for error correction reduces the net information rate.

For an ideal noiseless channel with 2B2B real symbols per second and MM distinguishable symbol levels, the corresponding signaling expression is

R=2Blog⁡2M.R=2B\log_2 M.

This expression does not itself account for the difficulty of distinguishing closely spaced signal levels in noise. Increasing MM at fixed power makes those levels harder to distinguish.

The Shannon–Hartley theorem incorporates that limitation through the SNR. Error-correcting codes exploit structure across many channel uses so that a receiver can distinguish messages more reliably than it could by decoding each symbol separately.

The capacity-achieving input for the average-power-constrained Gaussian channel is Gaussian. Fixed finite signal constellations generally have a lower achievable information rate. Practical systems can approach Gaussian-channel capacity through combinations of coding, larger constellations, and signal shaping, but the theorem alone does not determine a suitable implementation.

Extensions and limitations

Colored noise and frequency-selective channels

When channel gain or noise power varies with frequency, a single SNR may not adequately describe the channel. Under an appropriate one-sided, nonduplicated frequency convention, capacity takes the form

C=max⁡P(f)∫Blog⁡2(1+∣H(f)∣2P(f)N(f)) df,C= \max_{P(f)} \int_{\mathcal B} \log_2\left( 1+\frac{|H(f)|^2P(f)}{N(f)} \right)\,df,

subject to

P(f)≥0,∫BP(f) df≤Ptotal.P(f)\geq0, \qquad \int_{\mathcal B}P(f)\,df\leq P_{\mathrm{total}}.

Here H(f)H(f) is the channel frequency response, P(f)P(f) the allocated input power spectral density, and N(f)N(f) the noise power spectral density. The optimization is solved by water-filling, which allocates power preferentially to frequencies with more favorable gain-to-noise conditions. The flat-channel, white-noise case reduces to the Shannon–Hartley expression.

Other channel models

The basic formula does not directly characterize every communication system:

  • Fading channels: varying channel gain requires distinctions such as ergodic capacity and outage capacity, depending on coding duration and channel knowledge.
  • Multiple-antenna channels: capacity depends on spatial channel structure and may involve matrices and eigenvalues rather than one scalar SNR.
  • Non-Gaussian noise: capacity depends on the full noise distribution, not solely on its power.
  • Peak-power or amplitude constraints: additional restrictions can change both capacity and the optimizing input distribution.
  • Short packets: finite block length and a specified error probability generally require rates below asymptotic capacity.
  • Multiuser interference: interference cannot always be treated as independent Gaussian noise; decoding strategy and network structure can change achievable rates.

The theorem is widely used as a benchmark for radio links, satellite communication, wired transmission, and optical systems where a suitable Gaussian-channel approximation applies. Its role is to establish a limit under explicit assumptions, rather than to predict practical throughput from bandwidth alone.