Random Variables, Distributions, Expectation and Variance
~6 min read · Tier toggle changes depth (T1 plain · T2 standard · T3 rigour)
You should be comfortable with the axioms of probability and conditional probability from the previous lecture. Familiarity with basic calculus (summation and integration) is required.
Why this matters
Random variables translate physical measurements into mathematical objects we can manipulate.
In experimental physics, we rarely measure a single fixed value; instead, we observe a distribution of outcomes due to noise, environmental fluctuations, or intrinsic quantum uncertainty. To make sense of this data, we need a rigorous framework that defines what the "average" value is and how much the data scatters around that average. This lecture introduces random variables, their probability distributions, and the key statistical moments—expectation and variance—that quantify the central tendency and spread of physical measurements.
The Idea in Plain Language
Imagine you are measuring the length of a metal rod using a caliper. If you repeat the measurement ten times, you will not get ten identical numbers; you will get a cluster of values close to the true length. A random variable is simply a rule that assigns a numerical value to each possible outcome of this experiment. The distribution tells you how likely each value is. The expectation (or mean) is the "center of gravity" of this distribution—if you repeated the experiment infinitely many times, this is where the average would settle. The variance measures how "spread out" the data is; a small variance means your measurements are precise and clustered tightly, while a large variance indicates high noise or imprecision. These two numbers, the mean and the variance, are the fundamental descriptors of any physical measurement process.
Discrete and Continuous Distributions
A random variable \( X \) can be discrete, taking a countable set of values, or continuous, taking any value in an interval. For a discrete variable, the probability mass function (PMF) \( p(x) \) gives the probability that \( X \) equals a specific value \( x \). The sum of all probabilities must be one: \( \sum_{x} p(x) = 1 \). For a continuous variable, we use a probability density function (PDF) \( f(x) \). Here, \( f(x) \) is not a probability but a density; the probability that \( X \) falls within an interval \( [a, b] \) is the area under the curve: \( P(a \le X \le b) = \int_{a}^{b} f(x) \, dx \). In both cases, the total probability over the entire sample space is unity.
Expectation: The Weighted Average
The expectation \( E[X] \) generalizes the concept of an average. For a discrete variable, it is the sum of each value weighted by its probability: \( E[X] = \sum_{x} x p(x) \). For a continuous variable, it is the integral of the value weighted by its density: \( E[X] = \int_{-\infty}^{\infty} x f(x) \, dx \). This is the first moment of the distribution. A crucial property, derived in D-108, is linearity. For any constants \( a \) and \( b \), and random variables \( X \) and \( Y \), the expectation of a linear combination is the linear combination of the expectations: \( E[aX + bY] = aE[X] + bE[Y] \). This linearity holds regardless of whether \( X \) and \( Y \) are independent, making it a powerful tool for analyzing complex experimental setups where multiple sources of error combine.
Variance: Quantifying Uncertainty
While expectation tells us the center, variance tells us the width. The variance \( \mathrm{Var}(X) \) is defined as the expected value of the squared deviation from the mean: \( \mathrm{Var}(X) = E[(X - E[X])^2] \). This definition is intuitive but computationally inconvenient. Using the linearity of expectation, we can expand this into a more useful form: \( \mathrm{Var}(X) = E[X^2] - (E[X])^2 \). This is the second moment minus the square of the first moment. The standard deviation \( \sigma = \sqrt{\mathrm{Var}(X)} \) has the same units as \( X \) and is often reported as the uncertainty in a measurement. From D-108, we also know how variance behaves under affine transformations: \( \mathrm{Var}(aX + b) = a^2 \mathrm{Var}(X) \). Note that adding a constant \( b \) shifts the distribution but does not change its spread, while scaling by \( a \) scales the spread by \( |a| \).
The derivation
Expectation, Variance and Their Algebra
For random variables X, Y on a common probability space and constants a, b ∈ ℝ, expectation is linear, E[aX + bY] = aE[X] + bE[Y], the variance obeys Var(X) = E[X²] − (E[X])², and under an affine map the moments scale as E[aX + b] = aE[X] + b and Var(aX + b) = a²Var(X) — each following directly from the definition of expectation as a probability-weighted sum or integral.
A common misconception is that the variance of a sum is the sum of the variances. This is only true if the variables are uncorrelated. In general, \( \mathrm{Var}(X + Y) = \mathrm{Var}(X) + \mathrm{Var}(Y) + 2\mathrm{Cov}(X, Y) \). If your experimental errors are correlated (e.g., using the same faulty clock to time two events), you must account for the covariance term, or your uncertainty estimate will be significantly underestimated. C
Worked example
Problem. A student measures the period of a pendulum 5 times, obtaining the values (in seconds): 1.02, 1.05, 1.01, 1.04, 1.03. Calculate the sample mean and the sample variance (using the unbiased estimator with \( N-1 \) in the denominator).
- Calculate the mean \( \bar{x} \): \( \bar{x} = \frac{1}{5}(1.02 + 1.05 + 1.01 + 1.04 + 1.03) = \frac{5.15}{5} = 1.03 \ \mathrm{s} \).
- Calculate the squared deviations from the mean: \( (1.02-1.03)^2 = 0.0001 \), \( (1.05-1.03)^2 = 0.0004 \), \( (1.01-1.03)^2 = 0.0004 \), \( (1.04-1.03)^2 = 0.0001 \), \( (1.03-1.03)^2 = 0 \).
- Sum the squared deviations: \( 0.0001 + 0.0004 + 0.0004 + 0.0001 + 0 = 0.0010 \).
- Divide by \( N-1 = 4 \) to get the unbiased sample variance: \( s^2 = \frac{0.0010}{4} = 0.00025 \ \mathrm{s^2} \).
Units check. The mean has units of seconds, consistent with the data. The variance has units of seconds squared, which is correct for the square of a deviation. The standard deviation would be \( \sqrt{2.5 \times 10^{-4}} \approx 0.016 \ \mathrm{s} \), which is a reasonable uncertainty for this dataset.
Three quick tests
- If \( E[X] = 2 \) and \( \mathrm{Var}(X) = 4 \), what is \( E[3X + 5] \)?
Answer
Using linearity, \( E[3X + 5] = 3E[X] + 5 = 3(2) + 5 = 11 \). - What is the variance of a constant random variable \( X = c \)?
Answer
The variance is 0, because there is no spread around the mean. - Why do we use \( N-1 \) instead of \( N \) in the sample variance formula?
Answer
To provide an unbiased estimator of the population variance, correcting for the fact that the sample mean is estimated from the same data.
Problem set for PU-106 →
Each derivation in this unit ends with five graded problems with full solutions.
Lecture notes drafted with a local model and checked line by line by the editor; every number in the worked example was recomputed.
Derivations cited: D-108 · Threads: chance