Functions of Random Variables and Variance of Combinations

The Intuition of Combining Uncertainties

Imagine you are measuring the length of a table with a ruler that has a small wobble, and the width with a different ruler that also wobbles. If you want the area, you multiply the two. Intuitively, if the length measurement is high, the area is high. If the width is high, the area is high. The errors tend to "add up" in their effect on the final result. However, if you were measuring the difference between two lengths, and the first ruler wobbled high, the difference would be small, but if the second ruler wobbled high, the difference would be large. The errors work in opposite directions. The key insight is that variance is not simply additive in a naive way; it depends on how the variables relate to each other. If two measurements are independent, their random wobbles are uncorrelated, and the variances add in a specific quadratic way. If they are correlated, the cross-terms become significant, and the total uncertainty can be larger or smaller than the simple sum of individual uncertainties.

Variance of Sums and Differences

Let \( X \) and \( Y \) be two random variables. We first consider the variance of their sum, \( S = X + Y \). By the definition of variance, \( \mathrm{Var}(S) = \mathrm{E}[(S - \mathrm{E}[S])^2] \). Expanding this square leads to terms involving the variances of \( X \) and \( Y \), as well as a cross-term involving the covariance. The covariance, \( \mathrm{Cov}(X, Y) \), measures how much \( X \) and \( Y \) vary together. If \( X \) and \( Y \) are independent, their covariance is zero. In this common case, the variance of the sum is simply the sum of the variances.

If \( X \) and \( Y \) are uncorrelated, \( \mathrm{Cov}(X, Y) = 0 \), and we recover the familiar rule: \( \mathrm{Var}(X + Y) = \mathrm{Var}(X) + \mathrm{Var}(Y) \). This is often called the "root-sum-square" rule when taking the square root to get the standard deviation. Note that for the difference \( D = X - Y \), the variance is \( \mathrm{Var}(X - Y) = \mathrm{Var}(X) + \mathrm{Var}(Y) - 2\mathrm{Cov}(X, Y) \). For independent variables, the variance of the difference is also the sum of the variances. This is a frequent source of confusion: subtracting two noisy measurements does not reduce the noise; it adds the variances.

Scaling and Linear Combinations

What happens if we scale a random variable? If \( Y = aX \) where \( a \) is a constant, the spread of the distribution is scaled by \( |a| \). Therefore, the variance, which is the mean squared deviation, is scaled by \( a^2 \). This is a crucial distinction: variance is quadratic in the scaling factor, while standard deviation is linear. For a general linear combination \( S = aX + bY \), the variance is derived by expanding the square of the deviation. The result includes the variances of \( X \) and \( Y \) weighted by the squares of their coefficients, plus a covariance term weighted by the product of the coefficients.

This formula generalizes to any number of variables. If we have \( n \) random variables \( X_1, \dots, X_n \) and constants \( a_1, \dots, a_n \), the variance of the sum \( S = \sum_{i=1}^n a_i X_i \) is given by the double sum of the covariances weighted by the products of the coefficients. This is the general result cited in the derivation card D-109. When the variables are pairwise uncorrelated, all off-diagonal covariance terms vanish, and the formula simplifies to the sum of the weighted variances.

The Role of Covariance

The term \( \mathrm{Cov}(X, Y) \) is the bridge between the individual uncertainties and the combined uncertainty. It is defined as \( \mathrm{Cov}(X, Y) = \mathrm{E}[(X - \mu_X)(Y - \mu_Y)] \). If \( X \) and \( Y \) tend to be above their means simultaneously, the covariance is positive. If one tends to be above its mean when the other is below, the covariance is negative. In experimental physics, positive covariance often arises from systematic errors that affect multiple measurements in the same direction. For example, if a temperature sensor is biased high, it will bias all temperature readings high, creating a positive covariance between readings taken at different times. Ignoring this covariance leads to an underestimation of the total uncertainty. Conversely, if two measurements are designed to cancel out a specific systematic error, they may exhibit negative covariance, reducing the total variance of the difference. Always ask: are my measurements truly independent, or do they share a common source of error?