The law of total probability

Theorem

The law of total probability

T-034Home MU-106Threads chance

Statement

Let \( (\Omega, \mathcal{F}, \mathbb{P}) \) be a probability space and let \( \{B_i\}_{i \in I} \) be a countable partition of \( \Omega \): the index set \( I \) is countable (finite or countably infinite), each \( B_i \in \mathcal{F} \), the sets are pairwise disjoint (\( B_i \cap B_j = \emptyset \) for \( i \neq j \)), and \( \bigcup_{i \in I} B_i = \Omega \). Then for every event \( A \in \mathcal{F} \), \[ \mathbb{P}(A) \;=\; \sum_{i \in I} \mathbb{P}(A \cap B_i) \;=\; \sum_{i \,:\, \mathbb{P}(B_i) \gt 0} \mathbb{P}(A \mid B_i)\, \mathbb{P}(B_i), \] where \( \mathbb{P}(A \mid B_i) = \mathbb{P}(A \cap B_i)/\mathbb{P}(B_i) \) is the conditional probability, defined whenever \( \mathbb{P}(B_i) \gt 0 \). The series converges absolutely, and the terms with \( \mathbb{P}(B_i) = 0 \) contribute nothing. The hypothesis \( \bigcup_i B_i = \Omega \) may be weakened to \( \mathbb{P}\left( \bigcup_i B_i \right) = 1 \).

Why it matters

The law of total probability is the fundamental divide-and-conquer principle of probability theory: to compute the probability of a complicated event, split the sample space into cases you understand, compute the probability of the event within each case, and reassemble with the case weights. Almost every serious probability computation — Bayes' theorem, first-step analysis of Markov chains, gambler's-ruin recursions, mixture models, filtering and prediction — is an application of this identity or of its conditional-expectation generalisation.

It is also the precise statement that probability is a weighted average over scenarios: \( \mathbb{P}(A) \) is the average of the conditional probabilities \( \mathbb{P}(A \mid B_i) \), weighted by how likely each scenario \( B_i \) is. This single sentence is behind base-rate reasoning, diagnostic testing, and the whole Bayesian machinery.

Hypotheses

Hypothesis. Each \( B_i \) is measurable, \( B_i \in \mathcal{F} \). Otherwise \( \mathbb{P}(A \cap B_i) \) is undefined. On \( ([0,1], \mathcal{B}, \lambda) \) take \( B_1 \) a Vitali set and \( B_2 \) its complement: the two sides of the identity cannot even be written down, since a Vitali set carries no Lebesgue measure.

Hypothesis. The \( B_i \) are pairwise disjoint. Overlap causes double counting. On \( \Omega = \{1,2\} \) with the uniform measure, take \( B_1 = \{1,2\} \), \( B_2 = \{1\} \) and \( A = \{1\} \): then \( \sum_i \mathbb{P}(A \cap B_i) = \tfrac12 + \tfrac12 = 1 \neq \tfrac12 = \mathbb{P}(A) \).

Hypothesis. The \( B_i \) exhaust \( \Omega \) (at least up to a null set: \( \mathbb{P}(\bigcup_i B_i) = 1 \)). If mass escapes, the sum undercounts. On \( \Omega = \{1,2,3\} \) uniform, take \( B_1 = \{1\} \), \( B_2 = \{2\} \), \( A = \{3\} \): the right side is \( 0 \) but \( \mathbb{P}(A) = \tfrac13 \).

Hypothesis. The index set \( I \) is countable. Countable additivity is exactly countable; there is no uncountable analogue. Partition \( [0,1] \) with Lebesgue measure into its singletons \( B_x = \{x\} \), \( x \in [0,1] \): every \( \mathbb{P}(B_x) = 0 \), so any reasonable "sum" of \( \mathbb{P}(A \cap B_x) \) is \( 0 \), yet \( \mathbb{P}([0,1]) = 1 \). (Indeed an uncountable sum of positive terms always diverges, so an "uncountable additivity" axiom would force all but countably many atoms to be null and would contradict this example.)

Hypothesis. \( \mathbb{P}(B_i) \gt 0 \) for the terms written in conditional form. \( \mathbb{P}(A \mid B_i) \) is undefined by the elementary ratio definition when \( \mathbb{P}(B_i) = 0 \). The identity survives because such terms satisfy \( \mathbb{P}(A \cap B_i) \leq \mathbb{P}(B_i) = 0 \) and may be dropped — but writing \( \mathbb{P}(A \mid B_i)\mathbb{P}(B_i) \) for them, as if it meant something, invites Borel–Kolmogorov-type paradoxes when a "natural value" of the conditional is smuggled in.

Proof

\[ A \;=\; A \cap \Omega \;=\; A \cap \Big( \bigcup_{i \in I} B_i \Big) \;=\; \bigcup_{i \in I} \big( A \cap B_i \big). \]

Distributivity of intersection over arbitrary unions (pure set algebra), using the hypothesis \( \bigcup_i B_i = \Omega \). A

Each \( A \cap B_i \in \mathcal{F} \).

A \( \sigma \)-algebra is closed under complements and countable unions, hence under finite intersections via \( A \cap B_i = \left( A^{c} \cup B_i^{c} \right)^{c} \) (De Morgan). Both \( A \) and \( B_i \) lie in \( \mathcal{F} \) by hypothesis. A

The family \( \{ A \cap B_i \}_{i \in I} \) is pairwise disjoint: for \( i \neq j \), \[ (A \cap B_i) \cap (A \cap B_j) \;\subseteq\; B_i \cap B_j \;=\; \emptyset. \]

Monotonicity of intersection plus the disjointness hypothesis on the partition. This is the step that licenses additivity in Step 4. B

\[ \mathbb{P}(A) \;=\; \mathbb{P}\Big( \bigcup_{i \in I} (A \cap B_i) \Big) \;=\; \sum_{i \in I} \mathbb{P}(A \cap B_i). \]

The axiom of countable additivity (\( \sigma \)-additivity) of \( \mathbb{P} \), applicable because by Steps 2–3 we have a countable, pairwise disjoint family in \( \mathcal{F} \). This is the unique point where countability of \( I \) is used, and it is indispensable. The series has nonnegative terms and partial sums bounded by \( 1 \), so it converges absolutely and its value is independent of the enumeration of \( I \) (rearrangement theorem for nonnegative series). C

For every \( i \) with \( \mathbb{P}(B_i) \gt 0 \): \[ \mathbb{P}(A \cap B_i) \;=\; \mathbb{P}(A \mid B_i)\, \mathbb{P}(B_i). \]

Definition of conditional probability, \( \mathbb{P}(A \mid B_i) = \mathbb{P}(A \cap B_i)/\mathbb{P}(B_i) \), multiplied through by \( \mathbb{P}(B_i) \gt 0 \). A

For every \( i \) with \( \mathbb{P}(B_i) = 0 \): \[ 0 \;\leq\; \mathbb{P}(A \cap B_i) \;\leq\; \mathbb{P}(B_i) \;=\; 0, \] so these terms vanish from the sum in Step 4.

Monotonicity of measures (\( E \subseteq F \Rightarrow \mathbb{P}(E) \leq \mathbb{P}(F) \)), itself a consequence of finite additivity and nonnegativity applied to \( F = E \cup (F \setminus E) \). Hence discarding null cells loses nothing. C

\[ \mathbb{P}(A) \;=\; \sum_{i \in I} \mathbb{P}(A \cap B_i) \;=\; \sum_{i \,:\, \mathbb{P}(B_i) \gt 0} \mathbb{P}(A \mid B_i)\, \mathbb{P}(B_i). \]

Substitute Step 5 into Step 4 and drop the null terms by Step 6. For the weakened hypothesis \( \mathbb{P}(\bigcup_i B_i) = 1 \): write \( N = \Omega \setminus \bigcup_i B_i \); then \( \mathbb{P}(N) = 0 \) by complementation, \( \mathbb{P}(A \cap N) = 0 \) by monotonicity, and finite additivity of \( \mathbb{P} \) over \( A = (A \cap N) \cup \bigcup_i (A \cap B_i) \) gives the same conclusion. \( \blacksquare \) B

Result

\[ \mathbb{P}(A) \;=\; \sum_{i} \mathbb{P}(A \mid B_i)\, \mathbb{P}(B_i) \qquad \text{for any countable partition } \{B_i\} \text{ of } \Omega \text{ with } B_i \in \mathcal{F}. \]

Reading. To find the probability of \( A \), split the world into countably many mutually exclusive, exhaustive scenarios; find how likely \( A \) is within each scenario; then average those conditional probabilities, weighting each by the probability of its scenario. Probability of an event = expected conditional probability over cases.

Scope. Any probability space whatsoever — discrete, continuous, or mixed — provided the partition is countable, measurable, disjoint, and exhaustive up to a null set. For uncountable "partitions" (e.g. conditioning on the value of a continuous random variable) the sum must be replaced by an integral, \( \mathbb{P}(A) = \int \mathbb{P}(A \mid X = x)\, f_X(x)\, dx \), whose rigorous meaning requires conditional expectation with respect to a \( \sigma \)-algebra, not this theorem.

Corollaries & converses

Bayes' theorem. For \( \mathbb{P}(A) \gt 0 \) and \( \mathbb{P}(B_j) \gt 0 \): \( \mathbb{P}(B_j \mid A) = \dfrac{\mathbb{P}(A \mid B_j)\mathbb{P}(B_j)}{\sum_i \mathbb{P}(A \mid B_i)\mathbb{P}(B_i)} \) — the denominator is exactly the law of total probability.
Averaging bounds. \( \inf_i \mathbb{P}(A \mid B_i) \leq \mathbb{P}(A) \leq \sup_i \mathbb{P}(A \mid B_i) \) (infimum and supremum over cells with \( \mathbb{P}(B_i) \gt 0 \)): a weighted average lies between its extreme entries.
Complement splitting. The two-cell case \( \mathbb{P}(A) = \mathbb{P}(A \mid B)\mathbb{P}(B) + \mathbb{P}(A \mid B^{c})\mathbb{P}(B^{c}) \) for \( 0 \lt \mathbb{P}(B) \lt 1 \) is the workhorse special case.
Tower property (generalisation, not corollary). \( \mathbb{P}(A) = \mathbb{E}\left[ \mathbb{P}(A \mid \mathcal{G}) \right] \) for any sub-\( \sigma \)-algebra \( \mathcal{G} \subseteq \mathcal{F} \); the present theorem is the case \( \mathcal{G} = \sigma(\{B_i\}) \).
Converse fails. Validity of \( \mathbb{P}(A) = \sum_i \mathbb{P}(A \cap B_i) \) for all \( A \in \mathcal{F} \) does not force \( \{B_i\} \) to be a genuine partition — only a partition up to null sets. Counterexample: on \( ([0,1], \mathcal{B}, \lambda) \) take \( B_1 = [0, \tfrac12] \), \( B_2 = [\tfrac12, 1] \), which overlap at the null point \( \tfrac12 \); the identity holds for every Borel \( A \). Even weaker: taking \( A = \Omega \) forces only \( \sum_i \mathbb{P}(B_i) = 1 \), which alone implies nothing about disjointness or coverage pointwise.

Fails without

Without countability: \( \Omega = [0,1] \) with Lebesgue measure, partitioned into the uncountably many singletons \( B_x = \{x\} \). Each \( \mathbb{P}(A \cap B_x) = 0 \), so every countable subfamily sums to \( 0 \), yet \( \mathbb{P}([0,1]) = 1 \). Step 4 (countable additivity) is the step with no uncountable counterpart.
Without disjointness: \( \Omega = \{1,2\} \) uniform, \( B_1 = \{1,2\} \), \( B_2 = \{1\} \), \( A = \{1\} \). Then \( \sum_i \mathbb{P}(A \mid B_i)\mathbb{P}(B_i) = \tfrac12 \cdot 1 + 1 \cdot \tfrac12 = 1 \), but \( \mathbb{P}(A) = \tfrac12 \): the overlap \( \{1\} \) is counted twice.
Without exhaustiveness: \( \Omega = \{1,2,3\} \) uniform, \( B_1 = \{1\} \), \( B_2 = \{2\} \), \( A = \{2,3\} \). The formula gives \( 0 \cdot \tfrac13 + 1 \cdot \tfrac13 = \tfrac13 \), but \( \mathbb{P}(A) = \tfrac23 \): the mass of \( A \) inside the uncovered region \( \{3\} \) is lost.
Without measurability: with a non-measurable cell (Vitali set in \( [0,1] \)), the quantities \( \mathbb{P}(B_i) \) and \( \mathbb{P}(A \cap B_i) \) simply do not exist; no version of the identity can be stated.

Common errors

The unweighted average. Computing \( \mathbb{P}(A) \) as \( \frac{1}{n}\sum_i \mathbb{P}(A \mid B_i) \), forgetting the weights \( \mathbb{P}(B_i) \). Correct only when the cells are equiprobable.
Transposed conditional. Substituting \( \mathbb{P}(B_i \mid A) \) where \( \mathbb{P}(A \mid B_i) \) is required (the prosecutor's fallacy). The two differ by the factor \( \mathbb{P}(B_i)/\mathbb{P}(A) \).
Leaky case analysis. Using cases that overlap or fail to exhaust \( \Omega \) — e.g. conditioning on "first card is an ace" vs "second card is an ace", which are neither disjoint nor exhaustive.
Weights that don't sum to 1. A quick sanity check: if your \( \mathbb{P}(B_i) \) do not sum to \( 1 \), your "partition" is not one, and the answer will be wrong by exactly the missing or doubled mass.
Conditioning on null events pointwise. Writing \( \mathbb{P}(A \mid X = x) \) via the ratio definition when \( \mathbb{P}(X = x) = 0 \); the elementary theorem does not apply, and naive limiting arguments can give different answers depending on the limiting family (Borel–Kolmogorov).

Discussion

The law of total probability is often called the partition theorem or the law of alternatives. Its two-cell germ is implicit in the earliest probability calculations of Pascal and Fermat (1654), where problems were routinely solved by splitting on the outcome of the next trial; the systematic statement and use belong to the era of Bayes (posthumously, 1763) and above all Laplace, whose Théorie analytique des probabilités (1812) made "reasoning by cases with weights" the engine of inverse probability. In the modern axiomatisation of Kolmogorov (1933) the theorem is a two-line consequence of countable additivity — which is precisely the point: countable additivity was designed so that arguments of this shape work.

Structurally, the theorem says that the map \( A \mapsto \mathbb{P}(A) \) is recoverable from its restrictions to the cells of any countable measurable partition: \( \mathbb{P} = \sum_i \mathbb{P}(B_i)\, \mathbb{P}(\cdot \mid B_i) \) exhibits \( \mathbb{P} \) as a countable convex combination (mixture) of the conditional measures \( \mathbb{P}(\cdot \mid B_i) \). This is the mathematical skeleton of every mixture model in statistics: a population is a weighted blend of subpopulations, and any event's probability is the corresponding blend of within-group probabilities. Read dynamically, with the partition given by the first step of a random process, it becomes first-step analysis, the standard technique for hitting probabilities and ruin problems in Markov chains.

The deep generalisation replaces the partition by an arbitrary sub-\( \sigma \)-algebra \( \mathcal{G} \subseteq \mathcal{F} \): the conditional probability \( \mathbb{P}(A \mid \mathcal{G}) \) is defined (via the Radon–Nikodym theorem) as the a.s.-unique \( \mathcal{G} \)-measurable random variable satisfying \( \int_G \mathbb{P}(A \mid \mathcal{G})\, d\mathbb{P} = \mathbb{P}(A \cap G) \) for all \( G \in \mathcal{G} \), and taking \( G = \Omega \) yields the tower identity \( \mathbb{E}\left[ \mathbb{P}(A \mid \mathcal{G}) \right] = \mathbb{P}(A) \). The elementary theorem is the special case \( \mathcal{G} = \sigma(\{B_i\}) \), where \( \mathbb{P}(A \mid \mathcal{G}) \) is the step function taking value \( \mathbb{P}(A \mid B_i) \) on each cell with \( \mathbb{P}(B_i) \gt 0 \). This is also the honest reading of the continuous version \( \mathbb{P}(A) = \int_{\mathbb{R}} \mathbb{P}(A \mid X = x)\, f_X(x)\, dx \): the integrand is a version of a conditional probability, unique only up to \( f_X \)-null sets, which is why pointwise manipulation of \( \mathbb{P}(A \mid X = x) \) without care leads to the Borel–Kolmogorov paradox.

Common misconceptions. The theorem does not require the cells to have positive probability (null cells are simply dropped), does not require finitely many cells, and does not say \( \mathbb{P}(A) \) is the plain average of the \( \mathbb{P}(A \mid B_i) \) — it is the weighted average, and the weights matter enormously (base-rate neglect is exactly the error of ignoring them). Nor is the identity a definition of \( \mathbb{P}(A) \): it is a theorem whose proof consumes one full axiom, countable additivity, and it genuinely fails for merely finitely additive set functions when the partition is infinite.

Worked examples

Example 1 (finite partition: factory defect rate). A component is made on machine \( M_1 \), \( M_2 \), or \( M_3 \) with probabilities \( 0.5 \), \( 0.3 \), \( 0.2 \); the machines produce defective components at rates \( 1\% \), \( 2\% \), \( 3\% \) respectively. Find the probability that a randomly chosen component is defective.

Let \( B_i = \{ \text{made on } M_i \} \), \( i = 1,2,3 \), and \( D = \{ \text{defective} \} \). The \( B_i \) are pairwise disjoint (a component is made on exactly one machine), exhaust \( \Omega \), and \( \mathbb{P}(B_1) + \mathbb{P}(B_2) + \mathbb{P}(B_3) = 0.5 + 0.3 + 0.2 = 1 \).

Verify the partition hypotheses before invoking the theorem — this check is part of the solution. A

\[ \mathbb{P}(D) = \sum_{i=1}^{3} \mathbb{P}(D \mid B_i)\,\mathbb{P}(B_i) = (0.01)(0.5) + (0.02)(0.3) + (0.03)(0.2). \]

Law of total probability with the finite partition \( \{B_1, B_2, B_3\} \); each \( \mathbb{P}(B_i) \gt 0 \), so every term is in legitimate conditional form. A

\[ \mathbb{P}(D) = 0.005 + 0.006 + 0.006 = 0.017. \]

Arithmetic. Sanity check: \( 0.017 \) lies between the smallest (\(0.01\)) and largest (\(0.03\)) conditional rates, as the averaging-bounds corollary demands. A

\[ \mathbb{P}(\text{defective}) = 0.017 = 1.7\%. \]

Reading. The overall defect rate is the machine-usage-weighted average of the three machines' defect rates — closer to \( 1\% \) than to \( 3\% \) because the cleanest machine does half the work.

Scope. Any finite mixture of sources with known source shares and known within-source rates.

Example 2 (countably infinite partition: Poisson thinning). An insect lays \( N \) eggs, where \( N \sim \mathrm{Poisson}(\lambda) \), \( \lambda \gt 0 \), and each egg hatches independently with probability \( p \in (0,1) \), independently of \( N \). Find the probability that no egg hatches.

Let \( A = \{ \text{no egg hatches} \} \) and \( B_n = \{ N = n \} \) for \( n \in \mathbb{N} \cup \{0\} \). The \( B_n \) are disjoint, exhaust \( \Omega \), and form a countably infinite partition with \( \mathbb{P}(B_n) = e^{-\lambda} \lambda^{n} / n! \gt 0 \) for all \( n \geq 0 \).

The Poisson distribution assigns positive mass to every \( n \geq 0 \) and total mass \( 1 \); partition hypotheses verified. A

\[ \mathbb{P}(A \mid B_n) = (1-p)^{n}. \]

Given \( N = n \), the hatching indicators are \( n \) independent Bernoulli\((p)\) trials, independent of \( \{N=n\} \) by hypothesis; all \( n \) fail with probability \( (1-p)^n \). (For \( n = 0 \) the empty product is \( 1 \): no eggs, vacuously none hatch.) B

\[ \mathbb{P}(A) = \sum_{n=0}^{\infty} \mathbb{P}(A \mid B_n)\, \mathbb{P}(B_n) = \sum_{n=0}^{\infty} (1-p)^{n} \, e^{-\lambda} \frac{\lambda^{n}}{n!}. \]

Law of total probability over the countably infinite partition \( \{B_n\}_{n \geq 0} \) — this is exactly where the countable (not merely finite) form of the theorem is needed. B

\[ \mathbb{P}(A) = e^{-\lambda} \sum_{n=0}^{\infty} \frac{ \big( \lambda(1-p) \big)^{n} }{n!} = e^{-\lambda} \, e^{\lambda(1-p)} = e^{-\lambda p}. \]

The exponential series \( \sum_{n \geq 0} x^{n}/n! = e^{x} \), absolutely convergent for all \( x \in \mathbb{R} \), applied at \( x = \lambda(1-p) \gt 0 \). A

\[ \mathbb{P}(\text{no egg hatches}) = e^{-\lambda p}. \]

Reading. The answer is as if the number of hatched eggs were itself Poisson with mean \( \lambda p \) — and indeed it is: this computation is the first glimpse of the Poisson thinning theorem.

Scope. Any random-number-of-trials setup where the trial count has a known distribution and outcomes are conditionally i.i.d. given the count.

Problems

Urn I contains 3 red and 2 blue balls; Urn II contains 1 red and 4 blue. A fair coin is flipped: heads selects Urn I, tails Urn II, and one ball is drawn uniformly at random from the selected urn. Find the probability the ball is red.
Solution
Let \( B_1 = \{\text{Urn I chosen}\} \), \( B_2 = \{\text{Urn II chosen}\} \), a partition with \( \mathbb{P}(B_1) = \mathbb{P}(B_2) = \tfrac12 \), and let \( R = \{\text{red drawn}\} \). Conditional probabilities: \( \mathbb{P}(R \mid B_1) = \tfrac35 \), \( \mathbb{P}(R \mid B_2) = \tfrac15 \). By the law of total probability, \[ \mathbb{P}(R) = \tfrac12 \cdot \tfrac35 + \tfrac12 \cdot \tfrac15 = \tfrac{3}{10} + \tfrac{1}{10} = \tfrac{2}{5}. \] Note this equals the unweighted average \( \tfrac12(\tfrac35 + \tfrac15) \) only because the urns are equiprobable; had the coin been biased, the weights would matter.
A disease has prevalence \( 1\% \). A test detects it with sensitivity \( \mathbb{P}(+ \mid \text{ill}) = 0.99 \) and has false-positive rate \( \mathbb{P}(+ \mid \text{healthy}) = 0.05 \). (a) Find \( \mathbb{P}(+) \). (b) Deduce \( \mathbb{P}(\text{ill} \mid +) \).
Solution
(a) Partition on health status: \( B_1 = \{\text{ill}\} \), \( B_2 = \{\text{healthy}\} \), with \( \mathbb{P}(B_1) = 0.01 \), \( \mathbb{P}(B_2) = 0.99 \). Then \[ \mathbb{P}(+) = (0.99)(0.01) + (0.05)(0.99) = 0.0099 + 0.0495 = 0.0594. \] (b) By Bayes' theorem, whose denominator is exactly part (a), \[ \mathbb{P}(\text{ill} \mid +) = \frac{(0.99)(0.01)}{0.0594} = \frac{0.0099}{0.0594} = \frac{1}{6} \approx 0.167. \] Despite the accurate test, five of every six positives are false — the small weight \( \mathbb{P}(B_1) = 0.01 \) suppresses the true-positive term. Ignoring that weight is base-rate neglect, the canonical misuse of this theorem.
A fair die is rolled, showing \( N \in \{1, \dots, 6\} \); a fair coin is then tossed \( N \) times. Find the probability that every toss lands heads.
Solution
Partition on \( B_n = \{N = n\} \), \( n = 1, \dots, 6 \), each with \( \mathbb{P}(B_n) = \tfrac16 \). Given \( N = n \), all-heads has probability \( (\tfrac12)^n \). By the law of total probability, \[ \mathbb{P}(\text{all heads}) = \sum_{n=1}^{6} \frac{1}{6} \left( \frac{1}{2} \right)^{n} = \frac{1}{6} \left( \frac{1}{2} + \frac{1}{4} + \cdots + \frac{1}{64} \right) = \frac{1}{6} \left( 1 - \frac{1}{64} \right) = \frac{1}{6} \cdot \frac{63}{64} = \frac{21}{128}. \] Here the geometric partial sum \( \sum_{n=1}^{6} (1/2)^n = 1 - (1/2)^6 \) was used. Answer: \( \tfrac{21}{128} \approx 0.164 \).
Let \( \{B_i\}_{i \in I} \) be a countable partition of \( \Omega \) with all \( \mathbb{P}(B_i) \gt 0 \), and let \( A \in \mathcal{F} \). Prove \[ \inf_{i \in I} \mathbb{P}(A \mid B_i) \;\leq\; \mathbb{P}(A) \;\leq\; \sup_{i \in I} \mathbb{P}(A \mid B_i), \] and show by example that both inequalities can be strict.
Solution
Write \( m = \inf_i \mathbb{P}(A \mid B_i) \) and \( M = \sup_i \mathbb{P}(A \mid B_i) \); both exist in \( [0,1] \) since each conditional probability lies in \( [0,1] \). By the law of total probability and \( m \leq \mathbb{P}(A \mid B_i) \leq M \) for every \( i \), \[ \mathbb{P}(A) = \sum_i \mathbb{P}(A \mid B_i)\, \mathbb{P}(B_i) \;\leq\; \sum_i M\, \mathbb{P}(B_i) = M \sum_i \mathbb{P}(B_i) = M, \] using term-by-term comparison of nonnegative series (legitimate since both converge) and \( \sum_i \mathbb{P}(B_i) = \mathbb{P}(\Omega) = 1 \) by countable additivity over the partition. The lower bound is identical with \( m \) in place of \( M \). Strictness: in Problem 1, \( m = \tfrac15 \lt \mathbb{P}(R) = \tfrac25 \lt \tfrac35 = M \). (Equality can occur, e.g. when all conditional probabilities coincide, in which case \( \mathbb{P}(A) \) equals that common value — a useful "conditional probabilities constant" trick.)
(a) Let \( \{B_i\}_{i \in I} \) be countable, \( B_i \in \mathcal{F} \), pairwise disjoint, with \( \mathbb{P}\left( \bigcup_{i} B_i \right) = 1 \) but \( \bigcup_i B_i \neq \Omega \). Prove that the law of total probability still holds for every \( A \in \mathcal{F} \). (b) Explain precisely which step of the main proof fails for the uncountable partition of \( ([0,1], \mathcal{B}, \lambda) \) into singletons, and why no repair is possible.
Solution
(a) Let \( U = \bigcup_i B_i \in \mathcal{F} \) (countable union of measurable sets) and \( N = \Omega \setminus U \in \mathcal{F} \). Then \( \mathbb{P}(N) = 1 - \mathbb{P}(U) = 0 \) by finite additivity on \( \Omega = U \cup N \). For \( A \in \mathcal{F} \), decompose \[ A = (A \cap N) \cup \bigcup_{i \in I} (A \cap B_i), \] a countable union of pairwise disjoint measurable sets (disjointness exactly as in Step 3 of the main proof, plus \( (A \cap N) \cap (A \cap B_i) \subseteq N \cap B_i = \emptyset \)). Countable additivity gives \[ \mathbb{P}(A) = \mathbb{P}(A \cap N) + \sum_{i} \mathbb{P}(A \cap B_i), \] and \( \mathbb{P}(A \cap N) \leq \mathbb{P}(N) = 0 \) by monotonicity, so \( \mathbb{P}(A) = \sum_i \mathbb{P}(A \cap B_i) \). Converting the terms with \( \mathbb{P}(B_i) \gt 0 \) via the definition of conditional probability, and discarding null terms as in Step 6, completes the proof. (b) Steps 1–3 survive verbatim for the singleton partition \( \{ \{x\} \}_{x \in [0,1]} \): the decomposition \( A = \bigcup_{x} (A \cap \{x\}) \) is a valid disjoint union of measurable sets. The failure is Step 4: countable additivity applies only to countable disjoint families, and \( [0,1] \) is uncountable (Cantor). No repair is possible, because any extension of the additivity axiom to uncountable families is inconsistent with this very space: uncountable additivity would give \( 1 = \lambda([0,1]) = \sum_{x \in [0,1]} \lambda(\{x\}) = \sum_{x} 0 = 0 \). (More generally, an uncountable family of strictly positive terms cannot have a finite sum — some \( \varepsilon \)-level set \( \{ x : \lambda(\{x\}) \gt 1/k \} \) would be infinite — so uncountable "sums" of probabilities of null cells can only ever be \( 0 \).) This is why conditioning on a continuum of cases must instead be routed through densities and conditional expectation.