Boltzmann Entropy from Microstate Counting

Derivation

Boltzmann Entropy from Microstate Counting

D-231 Home PU-302 Threads chance · energy Depends on —

Statement

For an isolated system in equilibrium, whose accessible microstates are all equally probable and number \(\Omega\), the thermodynamic entropy is \(S = k_B \ln \Omega\), where \(k_B\) is Boltzmann's constant. This is fixed uniquely (up to the choice of \(k_B\)) by demanding that entropy be additive over statistically independent subsystems whose microstate counts multiply.

Why it matters

This single relation is the bridge from mechanics to thermodynamics: it defines entropy microscopically in terms of counting, turning the abstract second-law quantity into something one can in principle compute from a Hamiltonian. Every equilibrium result of statistical mechanics — the Sackur–Tetrode equation, the canonical ensemble, the equation of state of an ideal gas — descends from it.

It also explains why entropy is extensive and why it increases: extensivity is the logarithm converting products of independent counts into sums, and increase is the overwhelming statistical weight of high-\(\Omega\) macrostates. The derivation matters because it shows \(S = k_B \ln \Omega\) is not a definition plucked from air but the only function consistent with additivity.

Assumptions

Equal a priori probabilities.Every accessible microstate of the isolated system is equally likely; drop this and \(\Omega\) is no longer the right measure — one must weight states, and the correct object becomes the Gibbs entropy \(-k_B\sum_i p_i \ln p_i\), which only reduces to \(k_B\ln\Omega\) for the uniform distribution.

Isolated system with sharp conserved quantities.Fixed \(N\), \(V\), and energy in a thin shell \([E, E+\delta E]\). Drop energy isolation and the microcanonical count \(\Omega\) is not well defined; one must pass to the canonical ensemble and the free energy.

Statistical independence of the two subsystems.The subsystems are weakly coupled: their interaction energy is negligible compared with each subsystem's energy, so \(\Omega = \Omega_1 \Omega_2\) exactly. If the coupling is strong (long-range forces, surface energy comparable to bulk) the counts do not factorize, additivity fails, and entropy ceases to be extensive.

Entropy is a smooth function of the count alone.We assume \(S\) depends on the macrostate only through \(\Omega\), via a differentiable function \(S=f(\Omega)\). Drop smoothness and the functional equation admits pathological non-logarithmic solutions (non-measurable Hamel-basis constructions); differentiability, or even monotonicity plus measurability, is what forces the logarithm.

Derivation

\[ S = f(\Omega) \]

Posit that entropy is some universal function of the number of accessible microstates alone — the only intensive-independent information the microcanonical macrostate carries. A

\[ \Omega = \Omega_1\,\Omega_2 \]

For two independent subsystems, each microstate of the whole is a pair (one state from each), so counts multiply — the fundamental counting principle. A

\[ f(\Omega_1\,\Omega_2) = f(\Omega_1) + f(\Omega_2) \]

Impose additivity of entropy over independent subsystems (thermodynamic extensivity). This is the Cauchy functional equation for a logarithm. B

\[ \frac{\partial}{\partial \Omega_1}\Big[f(\Omega_1\Omega_2)\Big] = \Omega_2\, f'(\Omega_1\Omega_2) = f'(\Omega_1) \]

Differentiate both sides of the additive relation with respect to \(\Omega_1\) at fixed \(\Omega_2\); the right side's \(f(\Omega_2)\) term vanishes. Legal because \(f\) is assumed differentiable. BC

\[ \Omega_2\, f'(\Omega_1\Omega_2)\Big|_{\Omega_2=1} = f'(\Omega_1) \;\Rightarrow\; f'(\Omega_1) = \frac{f'(1)}{\Omega_1} \]

Evaluate at \(\Omega_2 = 1\) (a subsystem with one microstate adds nothing). Writing \(f'(1)\equiv k\), the derivative is fixed to be \(k/\Omega\). B

\[ f(\Omega) = \int_1^{\Omega} \frac{k}{\Omega'}\,d\Omega' = k\ln\Omega + C \]

Integrate. The additivity condition at \(\Omega=1\) gives \(f(1)=2f(1)\Rightarrow f(1)=0\), so the constant \(C=0\): a system with a single microstate has zero entropy. C

\[ k \equiv k_B \]

The constant \(k\) is undetermined by counting alone; it sets the entropy unit. Matching \(1/T = \partial S/\partial E\) to the ideal-gas thermometer fixes \(k = k_B = 1.380649\times10^{-23}\ \mathrm{J\,K^{-1}}\). A

Result

\[ \boxed{\,S = k_B \ln \Omega\,} \]

Reading. Entropy is the logarithm of the number of equally likely microscopic configurations consistent with the macrostate, scaled by Boltzmann's constant. The logarithm is not decorative: it is the unique map that turns the multiplicative combinatorics of independent systems into the additive bookkeeping thermodynamics demands. Doubling the system size squares \(\Omega\) but only doubles \(S\).

Units check. \(\Omega\) is a pure count, so \(\ln\Omega\) is dimensionless; \(k_B\) carries \(\mathrm{J\,K^{-1}}\). Hence \(S\) has units \(\mathrm{J\,K^{-1}}\), matching the classical thermodynamic entropy defined through \(dS = \delta Q_{\text{rev}}/T\).

Limiting cases

\(\Omega = 1\): a unique microstate (perfect order) gives \(S = 0\) — the Nernst / third-law ground state.
\(\Omega_1\Omega_2\) for independent halves: \(S = k_B\ln\Omega_1 + k_B\ln\Omega_2\), recovering extensivity \(S \propto N\).
Uniform distribution limit of Gibbs: \(p_i = 1/\Omega\) turns \(-k_B\sum_i p_i\ln p_i\) into \(k_B\ln\Omega\), so Boltzmann is the microcanonical special case of Gibbs.
Large \(N\): \(\ln\Omega \sim N\ln(\cdots)\) so relative fluctuations \(\sim N^{-1/2}\to 0\) and the macrostate is sharp.

Breaks when

Long-range or strongly coupled systems. Gravitating systems, unscreened Coulomb systems, and small clusters have interaction energy comparable to subsystem energy, so \(\Omega \ne \Omega_1\Omega_2\). Additivity fails, entropy is non-extensive, and \(S=k_B\ln\Omega\) no longer combines correctly — Tsallis-type or explicitly non-additive treatments are needed.
Non-uniform microstate probabilities. Away from equilibrium, or for a system in contact with a reservoir, the microstates are not equiprobable; \(k_B\ln\Omega\) overcounts and one must use the Gibbs/Shannon form \(-k_B\sum_i p_i\ln p_i\).
Ill-defined counts (classical continuum without \(h\)). In a classical phase space \(\Omega\) is a volume with dimensions, so \(\ln\Omega\) is dimensionally inconsistent and shifts under coordinate rescaling. A phase-space cell \(h^{3N}\) (and Gibbs' \(1/N!\)) is required to make \(\Omega\) a dimensionless, permutation-correct count; omitting \(N!\) reproduces the Gibbs mixing paradox.

Failure modes

Adding counts instead of multiplying. Writing \(\Omega = \Omega_1 + \Omega_2\) for a composite system — this contradicts the counting principle and would make entropy non-additive.
Forgetting Gibbs' \(1/N!\). Treating identical particles as distinguishable inflates \(\Omega\) by \(N!\), giving a non-extensive entropy and the spurious entropy of mixing of a gas with itself.
Using \(\log_{10}\) or dropping \(k_B\). The base is absorbed into \(k_B\); mixing bases or omitting \(k_B\) yields wrong numerical entropies and broken units.
Applying \(k_B\ln\Omega\) out of equilibrium. Using the microcanonical formula when microstates are not equiprobable — the correct object is the Gibbs entropy.
Confusing \(\Omega\) (shell count) with the total phase-space volume \(\Phi(E)\). For large \(N\) they give the same entropy to \(O(\ln N)\), but conflating them without that justification is an error.

Discussion

The derivation's power is that it fixes the form of \(S\) from a symmetry-like requirement rather than from any specific microscopic model. Additivity of an extensive quantity plus multiplicativity of independent counts is a Cauchy functional equation, and — under mild regularity — the logarithm is its only continuous solution. Everything model-specific (the ideal gas, a spin lattice, a black-body cavity) enters only through how one computes \(\Omega\); the map from \(\Omega\) to \(S\) is universal.

Physically, \(k_B\ln\Omega\) explains the arrow of time as a statement about phase-space volumes. A macrostate is overwhelmingly likely to evolve toward one of larger \(\Omega\) simply because such macrostates occupy exponentially more of phase space; the "increase of entropy" is the logarithm of that ratio. The constant \(k_B\) is a units conversion between the natural information measure \(\ln\Omega\) (nats) and joules per kelvin — a historical accident of defining temperature before microstates were understood.

The relation connects thermodynamics to information theory: \(\ln\Omega\) is, up to \(k_B\), the Shannon entropy of a uniform distribution over \(\Omega\) outcomes. This is why entropy, Landauer's principle, and the thermodynamic cost of erasing information are all governed by the same constant. It also situates the third law: as \(T\to 0\) a system settles into its (possibly degenerate) ground manifold, \(\Omega\to g_0\), and \(S\to k_B\ln g_0\), zero for a non-degenerate ground state.

At a deeper level, the choice \(S=f(\Omega)\) with \(f\) a function of the count alone is itself an assumption that additivity partly enforces: if one instead demands only that \(S\) be an additive extensive function and that thermodynamic equilibrium maximize it under exchange of energy, one recovers the same logarithm because \(1/T=\partial S/\partial E\) must be intensive and equal across subsystems in contact. The Boltzmann form and the equilibrium condition \(T_1=T_2\) are two faces of the same additivity requirement — the maximization of \(S_1(E_1)+S_2(E-E_1)\) over \(E_1\) yields \(f'\!\) matching, which is exactly the shared-temperature statement.

Common misconceptions. "Entropy is disorder" is a heuristic, not a definition — \(\Omega\) counts accessible microstates, and some "ordered-looking" states (e.g. a crystal at low \(T\)) can have large \(\Omega\) through phonon modes. Also, \(S=k_B\ln\Omega\) is not more fundamental than the Gibbs entropy; it is the equiprobable special case of it.

Worked examples

\[ \text{Two-level paramagnet: } N \text{ independent spins, each up or down} \]

Each spin has 2 accessible states and spins are independent, so counts multiply. A

\[ \Omega = 2^N \]

Product of \(N\) factors of 2 (fundamental counting principle). A

\[ S = k_B\ln\Omega = k_B\, N\ln 2 \]

Apply the boxed result; \(\ln 2^N = N\ln 2\). A

\[ N = N_A = 6.022\times10^{23},\quad k_B = 1.381\times10^{-23}\ \mathrm{J\,K^{-1}} \]

Insert one mole and constants (numbers only after the symbolic result). A

\[ S = (1.381\times10^{-23})(6.022\times10^{23})(0.6931)\approx 5.76\ \mathrm{J\,K^{-1}} \]

Reading. A mole of free two-state spins at infinite temperature (both orientations equally likely) carries \(S = R\ln 2 \approx 5.76\ \mathrm{J\,K^{-1}}\); \(R=N_Ak_B\) is the gas constant.

Units check. \(k_B\) in \(\mathrm{J\,K^{-1}}\) times two dimensionless factors gives \(\mathrm{J\,K^{-1}}\).

\[ \text{Vacancy entropy: } n \text{ empty sites among } N \text{ lattice sites} \]

The microstates are the distinct placements of \(n\) indistinguishable vacancies on \(N\) sites — a combination. A

\[ \Omega = \binom{N}{n} = \frac{N!}{n!\,(N-n)!} \]

Number of ways to choose \(n\) sites out of \(N\). A

\[ S = k_B\ln\binom{N}{n} \approx -k_B N\Big[x\ln x + (1-x)\ln(1-x)\Big],\ \ x=\tfrac{n}{N} \]

Apply the boxed result and Stirling's approximation \(\ln M!\approx M\ln M - M\) for large \(N,n\). B

\[ N = 1.000\ \mathrm{mol},\quad x = 10^{-4} \]

Insert numbers: a dilute vacancy fraction of \(10^{-4}\). A

\[ S \approx -R\big[10^{-4}\ln 10^{-4} + (1-10^{-4})\ln(1-10^{-4})\big]\approx 9.2\times10^{-3}\,R \approx 0.077\ \mathrm{J\,K^{-1}} \]

Reading. A tiny equilibrium vacancy concentration already contributes measurable configurational entropy; this term drives the thermodynamic stability of point defects at finite \(T\).

Units check. \(R = N_A k_B\) in \(\mathrm{J\,K^{-1}}\) multiplied by a dimensionless bracket gives \(\mathrm{J\,K^{-1}}\).

Problems

A system has \(\Omega = 4\) accessible microstates. Compute \(S\) in \(\mathrm{J\,K^{-1}}\).
Solution
\(S = k_B\ln 4 = (1.381\times10^{-23})(1.386) = 1.91\times10^{-23}\ \mathrm{J\,K^{-1}}\).
Two independent subsystems have \(\Omega_1 = 8\) and \(\Omega_2 = 16\). Find the total entropy and verify additivity.
Solution
\(\Omega = 8\times16 = 128 = 2^7\). \(S = k_B\ln 128 = 7k_B\ln 2 = 6.70\times10^{-23}\ \mathrm{J\,K^{-1}}\). Check: \(S_1 = 3k_B\ln 2\), \(S_2 = 4k_B\ln 2\), sum \(=7k_B\ln 2\). Additive. ✓
By how much does the entropy of a mole of two-state spins change when \(\Omega\) goes from \(2^{N_A}\) (fully random) to \(1\) (fully aligned)?
Solution
\(\Delta S = k_B\ln 1 - k_B\ln 2^{N_A} = -N_A k_B\ln 2 = -R\ln 2 = -5.76\ \mathrm{J\,K^{-1}}\). Entropy decreases by \(R\ln 2\).
An Einstein solid of \(N=3\) oscillators holds \(q=3\) quanta of energy. Using \(\Omega = \binom{q+N-1}{q}\), find \(S\).
Solution
\(\Omega = \binom{3+3-1}{3} = \binom{5}{3} = 10\). \(S = k_B\ln 10 = (1.381\times10^{-23})(2.303) = 3.18\times10^{-23}\ \mathrm{J\,K^{-1}}\).
Estimate the configurational entropy per mole of a binary alloy \(A_{0.5}B_{0.5}\) (random mixing) and compare to \(R\ln 2\).
Solution
\(\Omega = \binom{N}{N/2}\); Stirling gives \(S = -R[\,0.5\ln0.5 + 0.5\ln0.5\,] = R\ln 2 = 5.76\ \mathrm{J\,K^{-1}\,mol^{-1}}\). The equimolar mixing entropy is exactly \(R\ln 2\), identical to the fully random two-state spin case — both realize maximum entropy over two equally weighted alternatives per particle.