The statistical definition of entropy

Result

The statistical definition of entropy

T-085Home CU-303Threads energy · quantum

Statement

Boltzmann's S = k ln W.

Why it matters

boltzmann-populations and molecular-partition-function together give a fully microscopic description of how molecules occupy their available quantum energy levels; this result is the bridge that turns that microscopic bookkeeping into the macroscopic thermodynamic quantity entropy, replacing the vague, purely empirical notion of "disorder" carried over from classical thermodynamics with a precise, countable definition.

Because entropy in this statistical form is defined directly from a count of accessible microstates, it also explains phenomena classical thermodynamics alone cannot — most notably residual entropy, the non-zero entropy some substances retain even as temperature approaches absolute zero — and it is the conceptual foundation on which equipartition-theorem and heat-capacity-from-levels are both ultimately built.

Hypotheses

Every microstate consistent with a system's fixed macroscopic constraints (particle number, volume, total energy) is equally probable.This is the fundamental postulate of statistical mechanics: without it, there would be no principled reason to identify entropy with a simple count of microstates rather than some weighted or otherwise unequal accounting of them. The system's particles, where genuinely identical, must be treated as indistinguishable when microstates are counted.Treating identical particles as if they were distinguishable overcounts the true number of physically distinct microstates by a factor of \(N!\), producing a non-extensive, unphysical entropy of mixing for identical gases (the Gibbs paradox); molecular-partition-function's translational treatment builds in the corresponding \(N!\) correction for exactly this reason.

Proof

W = \text{number of accessible microstates (complexions) consistent with a system's macrostate.}

A macrostate (specified, for instance, by fixed total energy, particle number, and volume) is generally consistent with an enormous number of distinct microscopic arrangements of individual particles among the available quantum states; \(W\) counts exactly how many. A

S = k \ln W

Boltzmann's foundational postulate defines statistical entropy directly in terms of this microstate count, with \(k\) (Boltzmann's constant) fixing the numerical scale connecting the dimensionless count \(W\) to entropy's ordinary thermodynamic units of \(\text{J/K}\). A

W_{\text{total}} = W_A \times W_B \quad\Rightarrow\quad S_{\text{total}} = k\ln(W_AW_B) = S_A+S_B

For two independent subsystems, every microstate of the combined system corresponds to one independent choice of microstate in each subsystem, so the total microstate count multiplies; Step 2's logarithm converts this multiplicative combination directly into an additive one, recovering the extensivity (additivity) required of any genuine thermodynamic entropy. A

W = N!\prod_i\frac{g_i^{\,n_i}}{n_i!}

Enumerating how \(N\) particles can be distributed among energy levels \(\varepsilon_i\) (with degeneracies \(g_i\) and populations \(n_i\)) via the standard combinatorial weight formula connects this page's microstate count directly to boltzmann-populations' level populations and molecular-partition-function's level-by-level bookkeeping. B

\text{If } W>1 \text{ persists as } T\to0 \text{ (residual degeneracy), then } S\not\to0 \text{ even at absolute zero.}

If a substance's ground state itself is not unique — for instance, a molecular crystal in which molecules can sit in more than one energetically indistinguishable orientation — Step 2 applied directly to that ground-state degeneracy predicts a genuine, non-zero "residual" entropy persisting even in the limit of absolute zero, a direct, quantitative and testable consequence of treating \(S=k\ln W\) as literal, not merely a convenient bookkeeping device. B

Result

S = k\ln W

Reading. Entropy is, precisely and literally, Boltzmann's constant multiplied by the logarithm of the number of microscopic arrangements consistent with a system's observed macroscopic state — not a vague measure of disorder, but a specific, countable quantity.

Scope. Fully general, applying to any system for which the microstate count can be enumerated — ideal gases, crystals, spin systems — and connects, via the partition-function machinery of molecular-partition-function, to the ordinary macroscopic thermodynamic entropy of classical thermodynamics.

Corollaries & converses

equipartition-theorem and heat-capacity-from-levels both ultimately trace back to this same statistical foundation, since heat capacity is a derivative of internal energy, itself computed from the same level populations (Step 4) that determine \(W\).
boltzmann-populations' population formula, \(n_i\propto g_ie^{-\varepsilon_i/kT}\), is precisely the population distribution that maximises \(W\) (Step 4) subject to fixed total energy and particle number — this page names the quantity being maximised, and boltzmann-populations shows what that maximisation yields.
Measured residual entropy in real substances (molecular crystals showing orientational disorder frozen in on cooling) is direct experimental confirmation that \(S=k\ln W\) genuinely counts degenerate microstates, not merely a convenient theoretical bookkeeping device (Step 5).

Fails without

Drop the equal-probability postulate (Hypotheses): without every accessible microstate being equally likely, there would be no principled reason to identify entropy with a simple logarithm of the microstate count rather than some other, unequally weighted accounting.
Treat identical particles as distinguishable when counting \(W\): this overcounts the true number of physically distinct microstates by a factor of \(N!\), producing the unphysical, non-extensive Gibbs-paradox entropy of "mixing" for two samples of an identical gas.

Common errors

Treating "entropy as disorder" as the actual physical content of the formula, rather than as a loose analogy for what is, precisely, a specific countable multiplicity \(W\) (Discussion).
Forgetting that \(k\) (per-particle) and \(R\) (per-mole) are related by \(R=kN_A\), and mixing up molar and per-particle statistical entropy calculations as a result.
Assuming \(S=k\ln W\) applies only to ideal gases; it is fully general and applies equally to crystals, spin systems, or any other system whose microstates can be enumerated (Result, Scope).
Assuming the third law's "zero entropy at absolute zero" holds universally without qualification; it holds only for a perfect, fully non-degenerate crystal, and genuine residual entropy (Step 5) is a real, measurable exception, not merely a theoretical curiosity.

Discussion

Ludwig Boltzmann developed the statistical interpretation of entropy through the 1870s; the formula \(S=k\ln W\) is so closely associated with him that it is carved, in this exact form, on his tombstone in Vienna. Its significance goes beyond a single new formula: it re-founded entropy from a purely macroscopic, empirically measured quantity in classical thermodynamics into a direct, first-principles measure of microscopic multiplicity and probability.

A related subtlety, the Gibbs paradox, arises if identical gas particles are (incorrectly) treated as distinguishable when counting \(W\): doing so predicts a spurious entropy increase upon "mixing" two samples of the identical gas, a physically absurd result resolved only by dividing the naive microstate count by \(N!\) to properly account for particle indistinguishability (Hypotheses), a correction built directly into molecular-partition-function's treatment of translational motion.

Common misconception: that entropy simply is disorder, as an intuitive, freestanding concept. The formula defines entropy as the logarithm of a specific, countable number of microstates; this often correlates with an intuitive sense of "disorder" for many everyday systems, but the physical content of the theory is the count \(W\) itself, not the informal word used to describe it.

Worked examples

4\text{ distinguishable particles, 2 levels, evenly split (}n_1=n_2=2\text{): } W=\frac{4!}{2!\,2!}=6

Applying Step 4's combinatorial weight formula (with equal degeneracies, so the \(g_i^{n_i}\) factors drop out) to this small, explicit system gives exactly six distinct microscopic arrangements consistent with the same macroscopic \(2\)-\(2\) population split. A

S = k\ln6 \approx (1.381\times10^{-23})(1.792) \approx 2.47\times10^{-23}\,\text{J/K}

Direct substitution into Step 2 converts this small system's microstate count into a numerical statistical entropy, illustrating the formula's mechanics on a system simple enough to enumerate completely by hand. A

\text{Residual entropy estimate for one mole of a two-fold orientationally disordered solid: } S_{\text{res}}=R\ln2\approx5.76\,\text{J K}^{-1}\text{mol}^{-1}

Reading. If each of \(N_A\) molecules in a mole of solid can independently sit in either of two energetically indistinguishable orientations at the crystal's ground state, \(W=2^{N_A}\), and \(S_{\text{res}}=k\ln(2^{N_A})=kN_A\ln2=R\ln2\), a standard, purely formula-derived theoretical estimate of residual molar entropy.

Scope. The identical logic, with \(2\) replaced by however many equally likely orientations are available per particle, gives the general theoretical estimate \(S_{\text{res}}=R\ln(\text{number of orientations})\) for any substance showing this type of ground-state orientational disorder.

Problems

Six distinguishable particles are distributed among two equally accessible levels with \(n_1=4\), \(n_2=2\). Compute \(W\) and \(S\).
Solution
\(W=\dfrac{6!}{4!\,2!}=15\). \(S=k\ln15\approx(1.381\times10^{-23})(2.708)\approx3.74\times10^{-23}\,\text{J/K}\).
Estimate the theoretical residual molar entropy of a hypothetical solid in which each molecule can independently adopt any one of three energetically indistinguishable ground-state orientations.
Solution
By the same logic as the Worked example (with \(3\) orientations instead of \(2\)), \(W=3^{N_A}\), so \(S_{\text{res}}=k\ln(3^{N_A})=R\ln3\approx(8.314)(1.099)\approx9.13\,\text{J K}^{-1}\text{mol}^{-1}\).
Using Step 3, explain why entropy is an extensive property (i.e. why the entropy of a combined system equals the sum of its parts' individual entropies), starting purely from the multiplicative combination of independent subsystems' microstate counts.
Solution
For two independent subsystems, every microstate of the joint system is one independent pairing of a microstate of \(A\) with a microstate of \(B\), so \(W_{\text{total}}=W_AW_B\). Taking the logarithm (Step 2) converts this product into a sum: \(S_{\text{total}}=k\ln(W_AW_B)=k\ln W_A+k\ln W_B=S_A+S_B\), showing additivity follows directly and only from the logarithmic form of the definition combined with the multiplicative combination rule for independent subsystems' microstate counts.