maths2u
PhysicsMathsChemistryBiology
Tier
⌕ Search ⌘K
Theorem

The strong law of large numbers

T-132Home MU-404Threads chance
Statement

Let \((\Omega,\mathcal{F},\mathbb{P})\) be a probability space and let \((X_n)_{n\ge1}\) be a sequence of independent, identically distributed (i.i.d.) real random variables on it, with partial sums \(S_n = X_1+X_2+\cdots+X_n\). Then there exists a constant \(\mu\in\mathbb{R}\) such that \(\dfrac{S_n}{n}\to\mu\) almost surely as \(n\to\infty\) if and only if \(\mathbb{E}|X_1|\lt\infty\); and when this holds, \(\mu=\mathbb{E}[X_1]\). Almost sure convergence means \(\mathbb{P}\!\left(\left\{\omega\in\Omega:\lim_{n\to\infty} S_n(\omega)/n = \mu\right\}\right)=1\).

Why it matters

This is the theorem that makes "probability" and "long-run frequency" the same object on a single sample path, not merely on average over many hypothetical repetitions. The weak law only guarantees \(S_n/n\) is probably close to \(\mu\) for each fixed large \(n\); the strong law guarantees that for almost every realisation of the whole infinite sequence, the running average eventually settles at \(\mu\) and stays there. This distinction is what licenses Monte Carlo simulation, empirical estimation of probabilities and moments, and the frequentist interpretation of probability itself.

It is also the prototype of a genuinely measure-theoretic result: its proof cannot be done with characteristic functions or elementary Chebyshev bounds alone, and it forces the machinery of martingale-like maximal inequalities, tail-event zero-one laws, and summability lemmas that recur throughout the rest of probability theory (martingale convergence, ergodic theorems).

Hypotheses
Independence of the full sequence.If \(X_1=X_2=\cdots=X\) for a single non-degenerate random variable \(X\), the variables are identically distributed but totally dependent; then \(S_n/n = X\) for every \(n\), which converges a.s. to the random variable \(X\), not to any constant \(\mathbb{E}[X]\), unless \(X\) is a.s. constant. Independence is what forces the fluctuations to average out.
Identical distribution (or a summability substitute).Drop this and keep only independence: Kolmogorov's more general theorem for independent (not iid) \(X_k\) requires \(\sum_k \operatorname{Var}(X_k)/k^2 \lt \infty\). Without either hypothesis, take independent \(X_k\) with \(\mathbb{E}[X_k]=0\) and \(\operatorname{Var}(X_k)\) growing fast enough that this sum diverges (Problem 5 constructs one): \(S_n/n\) fails to converge a.s.
Finiteness of \(\mathbb{E}|X_1|\).If \(X_1\) is standard Cauchy, \(\mathbb{E}|X_1|=\infty\) and, by the scaling property of the Cauchy law, \(S_n/n\) has exactly the standard Cauchy distribution for every \(n\) — it never concentrates, and in fact \(\limsup_n |S_n|/n=\infty\) a.s. (Problem 1's companion fact). Finite mean is not a technical convenience; it is exactly the borderline for the theorem to be true at all, which is why the statement above is an "if and only if".
Completeness / a.s.-sense of the underlying \(\sigma\)-algebra.The convergence set \(\{\omega:\lim_n S_n(\omega)/n=\mu\}\) is only guaranteed to be measurable because it can be written using countably many operations (\(\limsup\), \(\liminf\), countable unions/intersections over rational \(\varepsilon\)) on the measurable functions \(S_n\); dropping measurability of the \(X_n\) themselves, or working outside a probability space with a genuine \(\sigma\)-algebra, makes the statement "\(\mathbb{P}(\cdot)=1\)" meaningless.
Proof
1
\text{WLOG } \mu=\mathbb{E}[X_1]=0 \text{: replace } X_k \text{ by } X_k-\mu.
Almost sure convergence is preserved under adding the constant \(\mu\) to every term (continuity of translation); it suffices to prove \(S_n/n\to 0\) a.s. for the centred sequence. A
2
\sum_{k=1}^\infty \mathbb{P}(|X_k|\gt k) = \sum_{k=1}^\infty \mathbb{P}(|X_1|\gt k) \lt \infty
Identical distribution gives \(\mathbb{P}(|X_k|\gt k)=\mathbb{P}(|X_1|\gt k)\) for every \(k\); the tail-sum identity \(\sum_{k=1}^\infty \mathbb{P}(|X_1|\gt k) \le \mathbb{E}|X_1| \lt \infty\) follows from Tonelli's theorem applied to \(|X_1| = \int_0^{|X_1|} dt \le \sum_{k=0}^\infty \mathbf{1}_{\{|X_1|\gt k\}}\). B
3
\mathbb{P}(X_k \ne Y_k \text{ infinitely often}) = 0, \qquad Y_k := X_k\,\mathbf{1}_{\{|X_k|\le k\}}
The First Borel–Cantelli Lemma: since \(\sum_k \mathbb{P}(X_k\ne Y_k)=\sum_k\mathbb{P}(|X_k|\gt k)\lt\infty\) by Step 2, almost every \(\omega\) has \(X_k(\omega)=Y_k(\omega)\) for all but finitely many \(k\). Hence \((S_n-T_n)/n \to 0\) a.s., where \(T_n=\sum_{k=1}^n Y_k\), because a finite (random) number of discrepancies divided by \(n\) vanishes. It suffices to show \(T_n/n\to 0\) a.s. A
4
b_k := \mathbb{E}[Y_k] = \mathbb{E}\!\left[X_1\,\mathbf{1}_{\{|X_1|\le k\}}\right] \longrightarrow \mathbb{E}[X_1] = 0 \quad (k\to\infty)
\(Y_k\) has the same law as \(X_1\mathbf{1}_{\{|X_1|\le k\}}\) since \(X_k\) and \(X_1\) are identically distributed; \(X_1\mathbf{1}_{\{|X_1|\le k\}}\to X_1\) pointwise and is dominated by the integrable \(|X_1|\), so the Dominated Convergence Theorem gives the limit. A
5
\frac{1}{n}\sum_{k=1}^n b_k \longrightarrow 0 \quad (n\to\infty)
Cesàro's Lemma: the Cesàro mean of a sequence converging to a limit \(L\) converges to the same \(L\); here \(L=0\) by Step 4. A
6
\sum_{k=1}^\infty \frac{\operatorname{Var}(Y_k)}{k^2} \;\le\; 2\,\mathbb{E}|X_1| \;\lt\; \infty
\(\operatorname{Var}(Y_k)\le \mathbb{E}[Y_k^2]=\mathbb{E}\big[X_1^2\mathbf{1}_{\{|X_1|\le k\}}\big]\) (identical distribution again). Summing over \(k\) and applying Tonelli's theorem to interchange sum and expectation (all terms nonnegative): \(\sum_{k=1}^\infty \frac{1}{k^2}\mathbb{E}\big[X_1^2\mathbf{1}_{\{|X_1|\le k\}}\big] = \mathbb{E}\!\left[X_1^2\sum_{k\ge \max(1,|X_1|)} \frac{1}{k^2}\right]\le \mathbb{E}\!\left[X_1^2 \cdot \frac{2}{\max(1,|X_1|)}\right] \le 2\,\mathbb{E}|X_1|\), using the standard bound \(\sum_{k\ge m} k^{-2}\le 2/m\) for \(m\ge1\) and \(x^2/\max(1,x)\le |x|+1\) with care on the constant absorbed into the factor \(2\). C
7
\sum_{k=1}^\infty \frac{Y_k-b_k}{k} \text{ converges almost surely.}
The summands \((Y_k-b_k)/k\) are independent (functions of the independent \(X_k\)) with mean zero and \(\sum_k \operatorname{Var}\!\left(\frac{Y_k-b_k}{k}\right)=\sum_k \frac{\operatorname{Var}(Y_k)}{k^2}\lt\infty\) by Step 6. Kolmogorov's Convergence Criterion (an instance of the Kolmogorov Three-Series Theorem, itself proved via Kolmogorov's Maximal Inequality \(\mathbb{P}\big(\max_{1\le j\le n}|\textstyle\sum_{k\le j} a_k|\ge\varepsilon\big)\le \varepsilon^{-2}\operatorname{Var}(\sum_{k\le n}a_k)\) and a Cauchy-criterion argument) states that summable variances for independent mean-zero terms imply a.s. convergence of the series. C
8
\frac{1}{n}\sum_{k=1}^n (Y_k-b_k) \longrightarrow 0 \quad \text{a.s.}
Kronecker's Lemma: if \(\sum_{k=1}^\infty a_k/k\) converges (here \(a_k=Y_k-b_k\), a convergent series by Step 7), then \(\frac{1}{n}\sum_{k=1}^n a_k \to 0\). This is a deterministic summation-by-parts fact applied pathwise on the a.s. event where the series of Step 7 converges. B
9
\frac{T_n}{n} = \frac{1}{n}\sum_{k=1}^n (Y_k-b_k) + \frac{1}{n}\sum_{k=1}^n b_k \longrightarrow 0+0 = 0 \quad \text{a.s.}
Add the a.s. limit of Step 8 to the deterministic limit of Step 5. Combined with Step 3, \(S_n/n = (S_n-T_n)/n + T_n/n \to 0\) a.s., which is the centred claim; undoing Step 1's shift gives \(S_n/n\to\mu\) a.s. for the original sequence. This proves sufficiency of \(\mathbb{E}|X_1|\lt\infty\). A
10
\mathbb{E}|X_1|=\infty \;\Longrightarrow\; S_n/n \text{ does not converge a.s. to any finite constant.}
If \(\mathbb{E}|X_1|=\infty\) then \(\sum_k \mathbb{P}(|X_k|\gt k)=\sum_k\mathbb{P}(|X_1|\gt k)=\infty\) (same tail-sum identity as Step 2, reversed inequality), and the \(X_k\) are independent, so the Second Borel–Cantelli Lemma gives \(\mathbb{P}(|X_k|\gt k \text{ i.o.})=1\). If \(S_n/n\to L\) a.s. for some finite \(L\), then \(X_n/n = S_n/n - \frac{n-1}{n}\cdot\frac{S_{n-1}}{n-1} \to L-L=0\) a.s., contradicting \(|X_k|/k\gt1\) infinitely often a.s. Hence no finite a.s. limit exists, establishing necessity. C
Result
\mathbb{E}|X_1|\lt\infty \iff \dfrac{X_1+\cdots+X_n}{n}\xrightarrow{\text{a.s.}}\mathbb{E}[X_1]

Reading. For an i.i.d. sequence, almost every infinite sample path of running averages converges, and converges to exactly one number, the common mean — provided that mean is finite in the first place. If the mean is not finite (or does not exist by symmetry-cancellation, as for Cauchy), no such almost-sure stabilisation happens.

Scope. Stated here for real-valued i.i.d. sequences on an arbitrary probability space; it extends verbatim to \(\mathbb{R}^d\)-valued or separable Banach-valued i.i.d. sequences with \(\mathbb{E}\|X_1\|\lt\infty\) (apply the scalar result coordinatewise, or component-wise against a countable norming set). It does not by itself give a rate of convergence — that is the content of the law of the iterated logarithm and of finite-sample concentration inequalities, which require more than a finite first moment.

Corollaries & converses
  • Borel's normal number theorem is the special case \(X_k=\) \(k\)-th binary digit of a Lebesgue-uniform point in \([0,1]\): digit frequencies converge a.s. to \(1/2\) (Problem 2).
  • Strong consistency of the sample mean as an estimator: for any i.i.d. sample with finite mean, \(\bar X_n\to\mu\) a.s., and more generally \(\frac1n\sum_k g(X_k)\to\mathbb{E}[g(X_1)]\) a.s. for any measurable \(g\) with \(\mathbb{E}|g(X_1)|\lt\infty\), by applying the theorem to the i.i.d. sequence \(g(X_k)\).
  • Glivenko–Cantelli (uniform a.s. convergence of the empirical distribution function) follows from applying this theorem to indicator sequences \(\mathbf{1}_{\{X_k\le t\}}\) at each rational \(t\) and a monotonicity/continuity patching argument.
  • Converse direction. The theorem as stated already is an "if and only if" (Step 10 supplies the converse: a.s. convergence to a finite limit forces \(\mathbb{E}|X_1|\lt\infty\)), so there is no separate converse to check — this is one of the few classical theorems that is sharp in both directions.
  • The weak law of large numbers is strictly implied by the strong law (a.s. convergence implies convergence in probability), but not conversely: there exist independent, non-identically-distributed sequences obeying a weak law with no strong law (Problem 5 sketches the mechanism).
Fails without
  • Drop independence: \(X_k=X\) for all \(k\), \(X\) non-degenerate. \(S_n/n=X\) a.s. for every \(n\); the sequence converges a.s. to the random variable \(X\), not to the constant \(\mathbb{E}[X]\) — the "limit" is not even deterministic.
  • Drop finite mean: \(X_k\) i.i.d. standard Cauchy. \(S_n\) has exactly the same Cauchy law as \(nX_1\) for every \(n\) (stability of the Cauchy family under summation and rescaling), so \(S_n/n\) has the standard Cauchy law for every \(n\) — it never settles down, and \(\limsup_n |S_n/n|=\infty\) a.s.
  • Drop identical distribution, keep only independence, without Kolmogorov's variance-summability condition: take independent \(X_k\) with \(\mathbb{E}[X_k]=0\) and \(\operatorname{Var}(X_k)\) chosen so \(\sum_k \operatorname{Var}(X_k)/k^2=\infty\) (Problem 5); \(S_n/n\) need not converge a.s. even though each \(X_k\) has mean zero.
Common errors
  • Citing the strong law when only the weak law (convergence in probability) has actually been established, or treating them as interchangeable — they differ in the underlying mode of convergence and the strong law needs a genuinely stronger argument (Kolmogorov's inequality, not just Chebyshev's).
  • Forgetting the finite-mean hypothesis and applying the theorem to heavy-tailed data (e.g. certain financial return models, or Cauchy-like ratios) where the sample mean simply does not stabilise no matter how large \(n\) is.
  • Believing the theorem gives a convergence rate ("after \(n\) samples the error is about such-and-such") — it asserts eventual almost-sure convergence with no quantitative bound; rates require finite variance and the CLT/LIL or Hoeffding-type inequalities.
  • Confusing "almost sure" with "for all \(\omega\)": the exceptional null set of paths on which convergence fails is genuinely nonempty in general (only excluded because it has probability zero), not vacuous.
  • Applying the truncation lemma (Step 6) with the wrong exponent, e.g. assuming \(\mathbb{E}[X_1^2]\lt\infty\) is needed — it is not; the whole point of the truncation argument is to make do with only \(\mathbb{E}|X_1|\lt\infty\).
Discussion

The theorem is due to Kolmogorov (1930s), sharpening earlier weak laws of Bernoulli, Chebyshev, and Khinchin, and earlier strong-law special cases due to Cantelli and Borel (who proved the normal-number case). Kolmogorov's contribution was to isolate the precise moment condition — finite first moment, nothing more — as necessary and sufficient, using the maximal inequality that now bears his name together with a summability trick (Kronecker's Lemma) that converts a.s. convergence of a weighted series into a.s. convergence of a Cesàro-type average.

A different, shorter proof due to Etemadi (1981) proves the same theorem for merely pairwise independent, identically distributed sequences, using only a truncation and the Borel–Cantelli lemmas plus a subsequence-and-interpolation argument, avoiding Kolmogorov's inequality entirely. That the conclusion survives weakening independence to pairwise independence is a genuinely deeper fact about the structure of the proof (it shows full independence was never really needed for the iid case), and is a standard illustration that the "obvious" proof strategy is not always the most economical one.

The strong law is best understood as the ergodic theorem for the shift map on sequence space with the product measure induced by an iid law: Birkhoff's Pointwise Ergodic Theorem, applied to the measure-preserving shift \(T(x_1,x_2,\ldots)=(x_2,x_3,\ldots)\) and the coordinate function \(f(x)=x_1\), gives exactly \(\frac1n\sum_{k=0}^{n-1} f(T^k x)=S_n/n\to\mathbb{E}[X_1]\) a.s., because the shift on an iid product measure is ergodic. Kolmogorov's SLLN is thus the oldest special case of a far larger family of pointwise ergodic theorems.

A common misconception is that the strong law says something about a single large \(n\) — it does not. It is a statement about the tail behaviour of the entire infinite sequence \((S_n/n)_{n\ge1})\) as a path; two sample paths can have wildly different values of \(S_{1000}/1000\) while both eventually converging to \(\mu\). Statements about a single finite \(n\) belong to concentration inequalities (Hoeffding, Bernstein, Chebyshev), which are a logically separate topic even though they are often taught alongside the law of large numbers.

Worked examples
1
\text{Monte Carlo estimate of } I=\int_0^1 x^2\,dx = \tfrac{1}{3}.
Let \(U_1,U_2,\ldots\) be i.i.d. Uniform\((0,1)\) and set \(X_k=U_k^2\). Then \(X_k\) are i.i.d. with \(\mathbb{E}|X_1|=\mathbb{E}[U_1^2]=\int_0^1 u^2\,du=\tfrac13\lt\infty\), so the hypotheses of the theorem hold for the sequence \((X_k)\). A
2
\hat I_n := \frac{1}{n}\sum_{k=1}^n U_k^2 \xrightarrow{\text{a.s.}} \mathbb{E}[U_1^2] = \int_0^1 u^2\,du
Direct application of the theorem to \((X_k)=(U_k^2)\), identifying \(\mu=\mathbb{E}[X_1]\) with the integral by the definition of expectation for a function of a Uniform\((0,1)\) variable. A
\hat I_n \xrightarrow{\text{a.s.}} \tfrac{1}{3}

Reading. Averaging \(U_k^2\) over an ever-larger simulated sample computes the integral almost surely in the limit — this is the theoretical justification for basic Monte Carlo integration, and it needs nothing more than \(\mathbb{E}|X_1|\lt\infty\), which is automatic here since \(U_1^2\in[0,1]\) is bounded.

1
\text{Strong consistency of the sample variance } S_n^2 = \frac1n\sum_{k=1}^n (X_k-\bar X_n)^2.
Let \(X_1,X_2,\ldots\) be i.i.d. with \(\mathbb{E}[X_1^2]\lt\infty\) (which implies \(\mathbb{E}|X_1|\lt\infty\) by Cauchy–Schwarz), mean \(\mu\), variance \(\sigma^2\). Apply the theorem twice: to \((X_k)\) directly, giving \(\bar X_n\to\mu\) a.s.; and to \((X_k^2)\), an i.i.d. sequence with \(\mathbb{E}|X_1^2|=\mathbb{E}[X_1^2]\lt\infty\), giving \(\frac1n\sum_{k=1}^n X_k^2 \to \mathbb{E}[X_1^2]\) a.s. A
2
S_n^2 = \frac1n\sum_{k=1}^n X_k^2 - \bar X_n^2 \xrightarrow{\text{a.s.}} \mathbb{E}[X_1^2] - \mu^2 = \sigma^2
Algebraic identity \(\frac1n\sum(X_k-\bar X_n)^2=\frac1n\sum X_k^2-\bar X_n^2\), then the Continuous Mapping principle for a.s. convergence (if \(a_n\to a\) and \(b_n\to b\) a.s. on the same probability-one event, then \(a_n-b_n^2\to a-b^2\) on that same event, since subtraction and squaring are continuous) applied to the two a.s. limits from Step 1. B
S_n^2 \xrightarrow{\text{a.s.}} \sigma^2

Reading. The plug-in sample-variance estimator converges almost surely to the true variance whenever the true second moment is finite — this is why sample variance is called a strongly consistent estimator, and the proof is nothing more than two applications of the strong law glued together with continuity.

Problems
  1. Let \(X_1,X_2,\ldots\) be i.i.d. Poisson\((\lambda)\). Show \(\bar X_n\to\lambda\) a.s. and explain why this justifies estimating an unknown Poisson rate by a long-run count average.
    SolutionPoisson\((\lambda)\) has \(\mathbb{E}|X_1|=\mathbb{E}[X_1]=\lambda\lt\infty\) since \(X_1\ge0\). The hypotheses of the theorem hold (i.i.d., finite mean), so \(\bar X_n = S_n/n \to \mathbb{E}[X_1]=\lambda\) almost surely. Practically: if arrivals in disjoint unit windows are i.i.d. Poisson\((\lambda)\) counts, the observed long-run average count per window converges, with probability one, to the true rate \(\lambda\), which is exactly the guarantee needed to justify rate estimation from a single long observed record rather than from many independent repetitions of the whole experiment.
  2. (Borel's normal number theorem) For \(x\in[0,1]\) with binary expansion \(x=\sum_{k=1}^\infty d_k(x) 2^{-k}\), let \(D_n(x)=\frac1n\sum_{k=1}^n d_k(x)\) be the frequency of \(1\)-digits among the first \(n\). Show \(D_n(x)\to\tfrac12\) for Lebesgue-almost every \(x\in[0,1]\).
    SolutionEquip \([0,1]\) with Lebesgue measure \(\lambda\), which is a probability measure. The digit maps \(X_k(x):=d_k(x)\) are, under \(\lambda\), i.i.d. Bernoulli\((1/2)\) random variables (a standard fact: the binary digits of a Lebesgue-uniform point are independent fair coin flips — this can itself be checked by verifying the joint distribution of any finite digit-block is uniform on \(\{0,1\}^m\), matching independence). Since \(X_k\in\{0,1\}\), \(\mathbb{E}|X_1|=\mathbb{E}[X_1]=\tfrac12\lt\infty\). By the theorem, \(D_n(x)=\frac1n\sum_{k=1}^n X_k(x) \to \mathbb{E}[X_1]=\tfrac12\) for \(\lambda\)-almost every \(x\), which is exactly Borel's statement (with "almost every" meaning "off a Lebesgue-null exceptional set").
  3. Let \(X_1,X_2,\ldots\) be i.i.d., nonnegative, with \(\mathbb{E}[X_1]=\infty\) (e.g. \(\mathbb{P}(X_1\gt t)=1/t\) for \(t\ge1\)). Show \(S_n/n \to \infty\) almost surely.
    SolutionFor each \(M\gt0\), let \(X_k^{(M)} = \min(X_k,M)\), an i.i.d. sequence bounded by \(M\), hence with \(\mathbb{E}|X_1^{(M)}|\le M\lt\infty\). By the theorem, \(\frac1n\sum_{k=1}^n X_k^{(M)} \to \mathbb{E}[X_1^{(M)}]\) a.s. Since \(X_k\ge X_k^{(M)}\) pointwise for every \(k\) (as \(X_k\ge0\)), \(S_n/n \ge \frac1n\sum_{k=1}^n X_k^{(M)} \to \mathbb{E}[X_1^{(M)}]\) a.s., so \(\liminf_n S_n/n \ge \mathbb{E}[X_1^{(M)}]\) a.s. for every fixed \(M\). By Monotone Convergence, \(\mathbb{E}[X_1^{(M)}]\uparrow \mathbb{E}[X_1]=\infty\) as \(M\to\infty\), so \(\liminf_n S_n/n=\infty\) a.s., i.e. \(S_n/n\to\infty\) a.s.
  4. Prove the key inequality used in Step 6 of the proof: for \(X_1\ge0\) with \(\mathbb{E}[X_1]\lt\infty\), \(\sum_{k=1}^\infty \frac{1}{k^2}\mathbb{E}\big[X_1^2\mathbf{1}_{\{X_1\le k\}}\big] \le 2\,\mathbb{E}[X_1]\).
    SolutionBy Tonelli's theorem (all terms nonnegative, so sum and expectation may be exchanged freely): \(\sum_{k=1}^\infty \frac1{k^2}\mathbb{E}\big[X_1^2\mathbf 1_{\{X_1\le k\}}\big] = \mathbb{E}\Big[X_1^2\sum_{k\ge \max(1,\lceil X_1\rceil)} \frac{1}{k^2}\Big]\). For any real \(m\ge1\), comparing the tail sum to an integral gives \(\sum_{k\ge m} k^{-2} \le \int_{m-1}^\infty x^{-2}\,dx \le \frac{2}{m}\) for \(m\ge1\) (since \(\int_{m-1}^\infty x^{-2}dx = \frac1{m-1}\le\frac2m\) for \(m\ge2\), and directly bounded by \(2\) for \(m=1\)). Applying this with \(m=\max(1,\lceil X_1\rceil) \ge \max(1,X_1)\) gives \(X_1^2 \sum_{k\ge m} k^{-2} \le X_1^2 \cdot \frac{2}{\max(1,X_1)} \le 2X_1\) (since \(X_1^2/\max(1,X_1)\le X_1\) always, whether \(X_1\le1\) or \(X_1\gt1\)). Taking expectations, \(\sum_{k=1}^\infty \frac1{k^2}\mathbb{E}[X_1^2\mathbf1_{\{X_1\le k\}}] \le 2\,\mathbb{E}[X_1]\), as required.
  5. Construct independent, mean-zero random variables \(X_1,X_2,\ldots\) (not identically distributed) for which \(\sum_k \operatorname{Var}(X_k)/k^2=\infty\) and for which \(S_n/n\) does not converge to \(0\) almost surely, illustrating that Kolmogorov's variance-summability condition cannot be dropped when identical distribution is dropped.
    SolutionLet \(X_k\) be independent with \(\mathbb{P}(X_k=k)=\mathbb{P}(X_k=-k)=\tfrac12\) for each \(k\), so \(\mathbb{E}[X_k]=0\) and \(\operatorname{Var}(X_k)=k^2\). Then \(\operatorname{Var}(X_k)/k^2=1\) for every \(k\), so \(\sum_k \operatorname{Var}(X_k)/k^2=\sum_k 1=\infty\): the summability condition fails badly. Heuristically \(S_n\) is a sum of \(n\) independent terms each of typical size \(\Theta(n)\) (since \(|X_n|=n\) exactly), so \(S_n\) is dominated by its last few terms and does not average down: \(X_n/n=\pm1\) exactly, which does not tend to \(0\), so \(S_n/n = \frac{n-1}{n}\cdot\frac{S_{n-1}}{n-1} + \frac{X_n}{n}\) cannot converge to any limit \(L\) unless \(X_n/n\to0\) — which it manifestly does not, since \(|X_n|/n\equiv1\). Hence \(S_n/n\) does not converge a.s. (its fluctuation of size \(\pm1/n \times n=\pm1\) from the last term alone never damps out), confirming that identical distribution (or its replacement, Kolmogorov's summability condition) is essential and not merely a convenience in the proof.