The Rayleigh Quotient and Variational Eigenvalues
~6 min read · Tier toggle changes depth (T1 plain · T2 standard · T3 rigour)
You should be comfortable with the definition of a self-adjoint operator, the concept of an inner product in a Hilbert space, and the basic properties of eigenvectors and eigenvalues. Familiarity with the spectral theorem for compact self-adjoint operators is assumed.
Why this matters
The Rayleigh quotient provides a scalar function whose stationary points directly reveal the eigenvalues and eigenvectors of an operator.
In quantum mechanics and classical mechanics, we often cannot solve the eigenvalue equation \(\hat{A}\psi = \lambda \psi\) analytically. However, we can always approximate the ground state or low-lying excited states by minimizing a specific ratio of expectation values. This variational approach transforms a difficult differential equation problem into a constrained optimization problem, allowing us to bound eigenvalues from above with arbitrary precision using trial functions.
The Intuition of the Rayleigh Quotient
Imagine you have a physical system described by a self-adjoint operator \(\hat{A}\), such as the Hamiltonian. The eigenvalues represent the allowed energy levels. The Rayleigh quotient, defined for any non-zero vector \(\psi\), is simply the ratio of the expectation value of the operator to the norm squared of the vector. If \(\psi\) happens to be an eigenvector, this ratio is exactly the corresponding eigenvalue. If \(\psi\) is a mixture of eigenvectors, the ratio lies somewhere between the smallest and largest eigenvalues present in that mixture. Crucially, if you perturb \(\psi\) slightly, the Rayleigh quotient changes only to second order if \(\psi\) is an eigenvector. This means eigenvectors are the "flat spots" or stationary points of this landscape. By finding the lowest point in this landscape, you find the ground state energy. By finding the next local minimum constrained to be orthogonal to the ground state, you find the first excited state.
Stationary Points and Eigenvectors
To find the stationary points of the Rayleigh quotient \(R(\psi) = \dfrac{\langle \psi, \hat{A}\psi\rangle}{\langle \psi,\psi\rangle}\), we consider a small variation \(\psi \to \psi + \epsilon \delta\psi\). The condition for a stationary point is that the first-order change in \(R\) vanishes. This leads directly to the eigenvalue equation.
Thus, the stationary points of the Rayleigh quotient are precisely the eigenvectors of the operator. The value of the quotient at these points is the corresponding eigenvalue. This establishes a deep link between optimization and spectral theory: finding eigenvalues is equivalent to finding the critical values of this scalar functional.
The Variational Principle for the Ground State
For a self-adjoint operator with a discrete spectrum bounded from below, the smallest eigenvalue \(\lambda_1\) is the global minimum of the Rayleigh quotient over all non-zero vectors in the Hilbert space.
This result is powerful because it provides an upper bound for the ground state energy. If we choose any trial function \(\phi\), the value \(R(\phi)\) will always be greater than or equal to \(\lambda_1\). By choosing better and better trial functions, we can drive \(R(\phi)\) closer to the true ground state energy from above. This is the foundation of the variational method used in quantum chemistry and condensed matter physics.
Extending to Excited States: The Min-Max Principle
While the ground state is found by a simple global minimum, excited states require more care. The \(n\)-th eigenvalue \(\lambda_n\) is characterized by a min-max principle. We consider all subspaces \(S\) of dimension \(n\). Within each subspace, we find the maximum value of the Rayleigh quotient. Then, we minimize this maximum over all possible \(n\)-dimensional subspaces.
Equivalently, \(\lambda_n\) is the minimum of the Rayleigh quotient over all vectors orthogonal to the first \(n-1\) eigenvectors. This orthogonal constraint ensures that we are not simply recovering the ground state but are forced to explore the next available mode of the system. This principle generalizes the variational method to any excited state, providing rigorous upper bounds for \(\lambda_n\).
The derivation
Variational Characterization of Eigenvalues
Let \(\hat{A}\) be a self-adjoint operator on a Hilbert space \(\mathcal{H}\) with a discrete, bounded-below spectrum and eigenvalues ordered \(\lambda_1 \le \lambda_2 \le \lambda_3 \le \cdots\). Define the Rayleigh quotient \(R(\psi) = \dfrac{\langle \psi, \hat{A}\psi\rangle}{\langle \psi,\psi\rangle}\) for \(\psi \neq 0\). Then the stationary points of \(R\) are exactly the eigenvectors of \(\hat{A}\), the stationary values are the eigenvalues, and the ordered eigenvalues obey the Courant–Fischer min–max principle \(\lambda_n = \min_{\dim S = n}\ \max_{\psi \in S \setminus \{0\}} R(\psi)\).
A common pitfall is assuming that the Rayleigh quotient is linear in \(\psi\). It is not; it is homogeneous of degree zero, meaning \(R(c\psi) = R(\psi)\) for any non-zero scalar \(c\). This scale invariance is why we can normalize \(\psi\) to unit length without loss of generality, but it also means that the "direction" of the vector in Hilbert space is what matters, not its magnitude. Confusing the quotient with the expectation value \(\langle \psi, \hat{A}\psi\rangle\) (which depends on normalization) leads to incorrect variational bounds. C
Worked example
Problem. Consider a two-level quantum system with Hamiltonian \(\hat{H} = \begin{pmatrix} 2 & 1 \\ 1 & 2 \end{pmatrix}\) in units where \(\hbar = 1\). Use the variational principle to estimate the ground state energy using the trial vector \(\phi = \begin{pmatrix} 1 \\ 1 \end{pmatrix}\). Then, calculate the exact ground state energy and compare.
- Compute the numerator \(\langle \phi, \hat{H}\phi\rangle\). First, \(\hat{H}\phi = \begin{pmatrix} 2 & 1 \\ 1 & 2 \end{pmatrix} \begin{pmatrix} 1 \\ 1 \end{pmatrix} = \begin{pmatrix} 3 \\ 3 \end{pmatrix}\). Then, \(\langle \phi, \hat{H}\phi\rangle = \begin{pmatrix} 1 & 1 \end{pmatrix} \begin{pmatrix} 3 \\ 3 \end{pmatrix} = 6\).
- Compute the denominator \(\langle \phi, \phi\rangle\). \(\langle \phi, \phi\rangle = \begin{pmatrix} 1 & 1 \end{pmatrix} \begin{pmatrix} 1 \\ 1 \end{pmatrix} = 2\).
- Calculate the Rayleigh quotient \(R(\phi) = \frac{6}{2} = 3\). This is our variational estimate for the ground state energy.
- Find the exact eigenvalues by solving \(\det(\hat{H} - \lambda I) = 0\). \(\det \begin{pmatrix} 2-\lambda & 1 \\ 1 & 2-\lambda \end{pmatrix} = (2-\lambda)^2 - 1 = 0\). Thus, \((2-\lambda)^2 = 1\), so \(2-\lambda = \pm 1\). The eigenvalues are \(\lambda_1 = 1\) and \(\lambda_2 = 3\).
- Compare the estimate \(3\) with the exact ground state energy \(\lambda_1 = 1\). The estimate is an upper bound, as expected, but it is actually equal to the excited state energy because the trial vector \(\begin{pmatrix} 1 \\ 1 \end{pmatrix}\) is an eigenvector corresponding to \(\lambda_2 = 3\). To estimate the ground state, we should have chosen a vector orthogonal to this, such as \(\begin{pmatrix} 1 \\ -1 \end{pmatrix}\), which would yield \(R = 1\).
Units check. The Hamiltonian is given in energy units (Joules, assuming \(\hbar=1\) absorbs time units appropriately for the context, or simply arbitrary energy units). The Rayleigh quotient is a ratio of two quantities with the same dimensions (energy), so the result is in energy units. The comparison is consistent.
Three quick tests
- Is the Rayleigh quotient invariant under scaling of the vector \(\psi\) by a non-zero constant?
Answer
Yes, because both the numerator and denominator scale by the square of the constant, which cancels out. - If \(\hat{A}\) is self-adjoint, is the Rayleigh quotient always real?
Answer
Yes, because \(\langle \psi, \hat{A}\psi\rangle\) is real for self-adjoint \(\hat{A}\), and \(\langle \psi, \psi\rangle\) is real and positive. - Does the variational principle provide a lower or upper bound for the ground state energy?
Answer
An upper bound. The minimum of the Rayleigh quotient is the ground state energy, so any trial value is greater than or equal to the true minimum.
Problem set for PU-104 →
Each derivation in this unit ends with five graded problems with full solutions.
Lecture notes drafted with a local model and checked line by line by the editor; every number in the worked example was recomputed.