Characteristic Functions and Moments
~6 min read · Tier toggle changes depth (T1 plain · T2 standard · T3 rigour)
You should be comfortable with the definition of expectation \( \mathrm{E}[X] \) for discrete and continuous random variables, and know how to differentiate standard functions like \( e^{ax} \) and \( \cos(x) \).
Why this matters
Characteristic functions provide a unified algebraic tool for handling sums of random variables, which is the core challenge in experimental data analysis.
In experimental physics, we rarely measure a single quantity in isolation; we accumulate counts, sum voltages, or integrate signals over time. The distribution of a sum is notoriously difficult to compute directly from probability density functions, requiring complex convolution integrals. The characteristic function transforms this difficult operation in the probability space into a simple multiplication in the frequency domain. This lecture introduces this powerful transform, establishes its link to the moments of a distribution (mean, variance, etc.), and demonstrates why it is the natural precursor to the Central Limit Theorem.
The Idea in Plain Language
Think of the characteristic function as a "fingerprint" of a probability distribution. Just as a musical chord can be decomposed into its constituent frequencies using a Fourier transform, a probability distribution can be decomposed into its oscillatory components. The characteristic function \( \phi_X(t) \) is essentially the Fourier transform of the probability density function, but with a sign convention that makes it particularly convenient for probability theory. The key insight is that while adding two random variables \( X \) and \( Y \) corresponds to convolving their distributions (a messy integral), it corresponds to simply multiplying their characteristic functions. This algebraic simplicity is the primary reason physicists use this tool: it turns the hard problem of "what is the distribution of the sum?" into the easy problem of "what is the product of the transforms?"
Definition and Basic Properties
For a real random variable \( X \), the characteristic function is defined as the expected value of the complex exponential \( e^{itX} \):
Here \( t \) is a real parameter. This definition is robust because \( |e^{itX}| = 1 \) for all real \( X \) and \( t \), ensuring the expectation always exists, even if the moments of \( X \) do not. Two immediate properties follow from this definition. First, at \( t=0 \), \( e^{i0 \cdot X} = 1 \), so \( \phi_X(0) = \mathrm{E}[1] = 1 \). Second, by the triangle inequality for expectations, \( |\phi_X(t)| = |\mathrm{E}[e^{itX}]| \leq \mathrm{E}[|e^{itX}|] = \mathrm{E}[1] = 1 \). Thus, the characteristic function is a bounded function with magnitude at most 1.
Generating Moments
One of the most useful features of the characteristic function is its ability to generate the moments of the distribution. If the \( n \)-th absolute moment \( \mathrm{E}[|X|^n] \) is finite, then \( \phi_X(t) \) is \( n \)-times differentiable. The derivatives at the origin \( t=0 \) are directly related to the raw moments \( \mathrm{E}[X^k] \).
Specifically, for \( k \leq n \):
Let us verify this for the first two moments. Differentiating the definition once:
Evaluating at \( t=0 \):
Thus, \( \mathrm{E}[X] = -i \phi_X'(0) \). Differentiating a second time:
Evaluating at \( t=0 \):
Thus, \( \mathrm{E}[X^2] = -\phi_X''(0) \). This relationship allows us to calculate the mean and variance (since \( \mathrm{Var}(X) = \mathrm{E}[X^2] - (\mathrm{E}[X])^2 \)) directly from the derivatives of the characteristic function at the origin.
Independence and Sums
The most powerful property for experimental physics is the behavior of characteristic functions under addition of independent variables. Let \( X \) and \( Y \) be independent random variables. The characteristic function of their sum \( Z = X + Y \) is:
Because \( X \) and \( Y \) are independent, the expectation of the product is the product of the expectations:
This factorization property means that if we have \( N \) independent and identically distributed (i.i.d.) random variables \( X_1, \dots, X_N \), the characteristic function of their sum \( S_N = \sum_{i=1}^N X_i \) is simply:
This is the algebraic engine that drives the Central Limit Theorem. By analyzing the behavior of \( \left( \phi_X(t) \right)^N \) as \( N \to \infty \), we can determine the limiting distribution of the sum, regardless of the specific shape of the original distribution \( X \), provided the variance is finite.
The derivation
Characteristic Functions and Moments
For a real random variable X the characteristic function is φX(t) = E[eitX], defined for every t ∈ ℝ. We derive three properties: (i) φX(0) = 1 and |φX(t)| ≤ 1; (ii) if E[|X|n] < ∞ then φX is n-times differentiable and its derivatives at the origin generate the moments, φX(k)(0) = ik E[Xk] for k ≤ n; (iii) for independent X, Y the characteristic function factorises, φX+Y(t) = φX(t)φY(t).
A common pitfall is assuming that the characteristic function uniquely determines the distribution for all random variables. While it is true that the characteristic function is a unique identifier for the distribution (unlike the moment generating function, which may not exist), one must be careful with the inverse transform. The inversion formula requires the characteristic function to be integrable. For distributions with heavy tails (like the Cauchy distribution), the characteristic function may not decay fast enough for simple inversion techniques, and the moments may not exist. Always check the existence of moments before using the derivative method to calculate them. C
Worked example
Problem. A detector records a count rate that follows a Poisson distribution with mean \( \lambda = 10 \). We take \( N = 5 \) independent measurements. Calculate the mean and variance of the total count \( S = X_1 + \dots + X_5 \) using the characteristic function properties.
- Identify the characteristic function of a single Poisson variable \( X \). For a Poisson distribution with parameter \( \lambda \), the probability mass function is \( P(X=k) = \frac{e^{-\lambda} \lambda^k}{k!} \). The characteristic function is \( \phi_X(t) = \exp(\lambda(e^{it} - 1)) \).
- Use the independence property for the sum of \( N=5 \) variables: \( \phi_S(t) = (\phi_X(t))^5 = \exp(5\lambda(e^{it} - 1)) \).
- Recognize that this is the characteristic function of a Poisson distribution with parameter \( \lambda' = 5\lambda \).
- Calculate the mean using the first derivative at \( t=0 \). \( \phi_S'(t) = \phi_S(t) \cdot 5\lambda i e^{it} \). At \( t=0 \), \( \phi_S'(0) = 1 \cdot 5\lambda i = i(5\lambda) \). Thus, \( \mathrm{E}[S] = -i \phi_S'(0) = 5\lambda \).
- Calculate the variance. For a Poisson distribution, variance equals the mean. Alternatively, use the second derivative: \( \phi_S''(0) = -\mathrm{E}[S^2] \). From the Poisson form, \( \mathrm{Var}(S) = 5\lambda \).
Units check. The mean and variance are dimensionless counts, consistent with the input parameter \( \lambda \) being a count rate integrated over time (or simply a count parameter). The factor of 5 correctly scales both the mean and the variance linearly with the number of independent trials.
Three quick tests
- What is the value of \( \phi_X(0) \) for any random variable \( X \)?
Answer
1 - If \( X \) and \( Y \) are independent, what is \( \phi_{X-Y}(t) \) in terms of \( \phi_X \) and \( \phi_Y \)?
Answer
\( \phi_X(t) \phi_Y(-t) \) - How is the variance \( \mathrm{Var}(X) \) related to the second derivative of the characteristic function at the origin?
Answer
\( \mathrm{Var}(X) = -\phi_X''(0) + (\phi_X'(0))^2 \) (using \( \mathrm{E}[X^2] - (\mathrm{E}[X])^2 \))
Problem set for PU-106 →
Each derivation in this unit ends with five graded problems with full solutions.
Lecture notes drafted with a local model and checked line by line by the editor; every number in the worked example was recomputed.
Derivations cited: D-117 · Threads: chance