Skip to content
Signal Theory

Random processes, power spectral density and thermal noise

Stationarity and ergodicity, power spectral density, passing through a filter with proof, white noise, thermal noise, kT/C noise with proof, shot and 1/f noise.

4 min read 0 comments

A random variable describes an uncertain number; a random process describes an entire uncertain signal, a random value at every instant. The noise of a circuit, speech, transmitted data are random processes. Their waveform cannot be predicted, but one can compute their power, how it is spread over frequency and how a filter changes it. In this lesson we cover these tools and apply them to thermal noise.

Random processes

A process X(t) can be thought of as the set of all the waveforms an experiment could produce (the realisations). At a fixed instant, X(t_1) is a random variable. The main quantities are

    \[ \mu_X(t) = E[X(t)],\qquad R_X(t_1, t_2) = E[X(t_1)\,X(t_2)] \]

the mean and the autocorrelation, which says how closely the values at two different instants are related.

Stationarity. A process is wide-sense stationary if its mean is constant and its autocorrelation depends only on the distance between the instants: R_X(t + \tau, t) = R_X(\tau). Its statistical properties do not change over time. R_X(0) = E[X^2] is the average power.

Ergodicity. A process is ergodic if time averages over a single long realisation coincide with ensemble averages. This is the assumption always made in practice: you measure the noise of one amplifier for a long time, not of a thousand amplifiers at the same instant.

Power spectral density

Definition (Wiener-Khinchin). For a stationary process, the power spectral density is the Fourier transform of the autocorrelation:

    \[ S_X(f) = \int_{-\infty}^{+\infty} R_X(\tau)\,e^{-j2\pi f\tau}\,d\tau \]

Inverting at \tau = 0: E[X^2] = R_X(0) = \int S_X(f)\,df. The total power is the area under S_X, and the power in a band is the area over that band: S_X is measured in W/Hz (or V²/Hz).

Passing through a filter

Theorem. If a stationary process X passes through a stable LTI system with frequency response H(f), the output Y is stationary and

    \[ S_Y(f) = |H(f)|^2\,S_X(f) \]

Proof. The output is Y(t) = \int h(\alpha)X(t - \alpha)\,d\alpha. Then

    \[ R_Y(\tau) = E[Y(t + \tau)Y(t)] = \iint h(\alpha)h(\beta)\,E[X(t + \tau - \alpha)X(t - \beta)]\,d\alpha\,d\beta = \iint h(\alpha)h(\beta)\,R_X(\tau - \alpha + \beta)\,d\alpha\,d\beta \]

which depends only on \tau. The expression is the convolution of R_X with h(\tau) and with h(-\tau). Transforming, convolution becomes a product and the transform of h(-\tau) is H^*(f) (real response): S_Y = H\,H^*\,S_X = |H|^2 S_X. ∎

The filter’s phase does not matter: for power only the magnitude counts.

White noise

A process with constant spectral density,

    \[ S_N(f) = \frac{N_0}{2} \]

is called white noise, by analogy with white light, which contains all colours. Its autocorrelation is \frac{N_0}{2}\delta(\tau): two values at different instants, however close, are uncorrelated. It is an idealisation (it would have infinite power), but an excellent model as long as the noise bandwidth is much wider than that of the system receiving it. If it is also Gaussian it is called additive white Gaussian noise (AWGN), the standard model of communication channels.

Filtered white noise: input spectral density, filter response and output spectral density
On the left the constant spectral density of white noise. In the middle the squared magnitude of a low-pass filter’s response. On the right their product, the spectral density of the output noise: the power is now finite and concentrated in the filter’s band.

At the output of a filter the noise power is \frac{N_0}{2}\int|H(f)|^2\,df. The noise equivalent bandwidth B_N is defined as the bandwidth of the ideal filter with the same peak gain that passes the same power: for an RC low-pass, B_N = \frac{\pi}{2}f_{-3dB}.

Thermal noise

The electrons of a resistor jiggle because of temperature, and a noise voltage appears across it. Johnson and Nyquist’s result (1928) is that, up to very high frequencies, it is white Gaussian noise with

    \[ \overline{v_n^2} = 4kTRB \]

in a bandwidth B, where k = 1.38\cdot 10^{-23} J/K is Boltzmann’s constant and T the absolute temperature.

Example. A 1 kΩ resistor at 300 K, over the 20 kHz audio band: v_n = \sqrt{4\cdot 1.38\cdot 10^{-23}\cdot 300\cdot 1000\cdot 20\,000} \approx 0.58 µV. It seems small, but it is the floor below which no microphone or sensor connected to that resistance can be amplified without noise.

Theorem (kT/C noise). The noise voltage across the capacitor of an RC circuit has power

    \[ \overline{v_C^2} = \frac{kT}{C} \]

independent of R.

Proof. The resistor noise, with density 4kTR over positive frequencies only, passes through the RC divider: |H(f)|^2 = 1/(1 + (2\pi fRC)^2). Integrating:

    \[ \overline{v_C^2} = \int_0^\infty \frac{4kTR}{1 + (2\pi fRC)^2}\,df = 4kTR\cdot\frac{1}{2\pi RC}\cdot\frac{\pi}{2} = \frac{kT}{C} \]

using \int_0^\infty \frac{du}{1 + u^2} = \frac{\pi}{2}. ∎

A larger resistance makes more noise, but it also narrows the bandwidth in the same proportion. With C = 1 pF, \sqrt{kT/C} \approx 64 µV: it is the fundamental noise of switched-capacitor circuits and of image sensor pixels, and the reason the sampling capacitors of precision converters cannot be too small.

Other noises

  • Shot noise: the current in a junction is made of discrete charges crossing it at random; it is white with \overline{i_n^2} = 2qIB.
  • 1/f noise (flicker): a spectral density that grows at low frequencies, due to defects in materials; it dominates in MOSFETs below a few kilohertz.
  • Markov processes: in many systems (packet queues, channels with burst errors) the future depends only on the present state; Markov chains are the basic model to describe them.

Related articles

Subscribe
Notify of
guest
0 Comments
Oldest
Newest Most Voted
0
Would love your thoughts, please comment.x
()
x