Eigenvalues and Eigenvectors

Most vectors, when you hit them with a matrix, get spun around and stretched into some new direction. But a special few refuse to turn. They come out pointing exactly the way they went in — only longer, shorter, or flipped. Those stubborn directions are eigenvectors, and the factor by which they stretch is the eigenvalue. This one idea — "which directions does a transformation leave alone?" — turns out to be one of the deepest and most useful in all of mathematics. It is how Google once ranked the web, how engineers predict whether a bridge will shake itself apart, and how data scientists compress thousand-dimensional data into a picture you can actually look at.

Learning Objectives

• Define eigenvalues and eigenvectors geometrically and algebraically from $Av = \lambda v$.
• Derive and solve the characteristic equation $\det(A - \lambda I) = 0$.
• Find eigenvectors once eigenvalues are known, and interpret them as invariant directions.
• Diagonalize a matrix and explain why $A = PDP^{-1}$ makes powers of $A$ trivial.
• Connect eigen-analysis to real systems: PageRank, mechanical vibration, and Principal Component Analysis.

Quick Answer

An eigenvector of a square matrix $A$ is a nonzero vector $v$ whose direction is unchanged by $A$: multiplying gives back a scaled copy, $Av = \lambda v$. The scalar $\lambda$ is the corresponding eigenvalue. To find the eigenvalues you solve the characteristic equation $\det(A - \lambda I) = 0$, a polynomial in $\lambda$. For each root you solve $(A - \lambda I)v = 0$ to get its eigenvectors. If an $n \times n$ matrix has $n$ independent eigenvectors, it can be diagonalized as $A = PDP^{-1}$, where $D$ holds the eigenvalues — which makes repeated application ($A^k$) and long-term behaviour easy to compute.

Where It Came From

Eigenvalues were born not from abstract algebra but from a very physical question in the 1700s: how does a spinning body behave? Leonhard Euler, studying the rotation of rigid bodies, and later Lagrange, discovered that every solid object has three special perpendicular axes — the principal axes — about which it spins cleanly without wobbling. Finding those axes meant finding directions that a certain "inertia" matrix left fixed. That is an eigenvector problem, though nobody called it that yet.

The same mathematics erupted again in the study of planetary orbits and, crucially, in vibrations. When Joseph Fourier, Jean d'Alembert, and Daniel Bernoulli analyzed vibrating strings and heat flow, they kept finding that complicated motion decomposes into simple "modes," each vibrating at its own natural frequency. Those frequencies were, once more, eigenvalues. In the 1800s Augustin-Louis Cauchy connected these threads, proving that symmetric matrices arising from physics always have real principal-axis directions — a landmark result we now call the spectral theorem.

The word itself is a translation hybrid. The German mathematician David Hilbert, working around 1904 on integral equations, used eigen — meaning "own" or "characteristic," as in "one's own value" — to name these quantities. English adopted the term half-translated as eigenvalue rather than fully anglicizing it to "characteristic value," and the hybrid stuck. So the motivation was never algebra for its own sake: people needed to know the natural, self-owned behaviours of physical systems, and eigenvalues were the language those behaviours spoke.

The Core Equation: Directions a Matrix Leaves Alone

A matrix $A$ acts on vectors by multiplication, generally rotating and stretching them. We hunt for the rare vectors that only stretch:

$Av = \lambda v, \quad v \neq 0.$

Read this as: "applying $A$ to $v$ does the same thing as simply scaling $v$ by the number $\lambda$." Geometrically, $v$ points along a direction that $A$ preserves. If $\lambda > 1$ the vector lengthens; if $0 < \lambda < 1$ it shrinks; if $\lambda < 0$ it flips to the opposite side; if $\lambda = 0$ the vector collapses onto the origin (meaning $A$ is not invertible).

To turn this into something solvable, move everything to one side. Using the identity matrix $I$ so the scalar $\lambda$ can be subtracted from a matrix:

$Av - \lambda I v = 0 \implies (A - \lambda I)v = 0.$

This is a homogeneous system. It has a nonzero solution $v$ only if the matrix $A - \lambda I$ is singular — that is, only if its determinant is zero. That condition is the characteristic equation:

$\det(A - \lambda I) = 0.$

Solving the Characteristic Equation — A Full Worked Example

Take

$A = \begin{pmatrix} 2 & 1 \\ 1 & 2 \end{pmatrix}.$

Step 1 — Form $A - \lambda I$:

$A - \lambda I = \begin{pmatrix} 2 - \lambda & 1 \\ 1 & 2 - \lambda \end{pmatrix}.$

Step 2 — Take the determinant and set it to zero:

$(2 - \lambda)(2 - \lambda) - (1)(1) = 0.$

Expanding: $\lambda^2 - 4\lambda + 4 - 1 = \lambda^2 - 4\lambda + 3 = 0$.

Step 3 — Solve the quadratic: $(\lambda - 1)(\lambda - 3) = 0$, so $\lambda_1 = 1$ and $\lambda_2 = 3$.

Step 4 — Find each eigenvector by plugging back into $(A - \lambda I)v = 0$.

For $\lambda_1 = 1$:

$\begin{pmatrix} 1 & 1 \\ 1 & 1 \end{pmatrix}\begin{pmatrix} x \\ y \end{pmatrix} = 0 \implies x + y = 0 \implies v_1 = \begin{pmatrix} 1 \\ -1 \end{pmatrix}.$

For $\lambda_2 = 3$:

$\begin{pmatrix} -1 & 1 \\ 1 & -1 \end{pmatrix}\begin{pmatrix} x \\ y \end{pmatrix} = 0 \implies -x + y = 0 \implies v_2 = \begin{pmatrix} 1 \\ 1 \end{pmatrix}.$

Check: $Av_2 = \begin{pmatrix} 2 & 1 \\ 1 & 2 \end{pmatrix}\begin{pmatrix} 1 \\ 1 \end{pmatrix} = \begin{pmatrix} 3 \\ 3 \end{pmatrix} = 3v_2$. ✓

Notice the eigenvectors are perpendicular — a guaranteed bonus for symmetric matrices. Two quick sanity tools: the eigenvalues sum to the trace ($1 + 3 = 4 = 2 + 2$) and multiply to the determinant ($1 \times 3 = 3 = 2\cdot 2 - 1\cdot 1$).

Diagonalization: The Payoff

Suppose an $n \times n$ matrix has $n$ linearly independent eigenvectors. Stack them as columns of a matrix $P$, and put the eigenvalues on the diagonal of $D$. Then

$A = PDP^{-1}.$

This says: to apply $A$, first switch to "eigen-coordinates" ($P^{-1}$), where the action is just independent scaling ($D$), then switch back ($P$). The magic appears when you take powers:

$A^k = PD^kP^{-1},$

because all the middle $P^{-1}P$ pairs cancel. Raising a diagonal matrix to a power just raises each entry to that power — trivial. Computing $A^{100}$ directly means 99 matrix multiplications; via diagonalization it is one $D^{100}$ plus two multiplications.

Worked example. For the $A$ above,

$P = \begin{pmatrix} 1 & 1 \\ -1 & 1 \end{pmatrix}, \quad D = \begin{pmatrix} 1 & 0 \\ 0 & 3 \end{pmatrix}.$

Then $A^{10} = P\begin{pmatrix} 1 & 0 \\ 0 & 3^{10} \end{pmatrix}P^{-1}$. Since $3^{10} = 59049$, the largest eigenvalue utterly dominates — the long-term behaviour of repeatedly applying $A$ is governed almost entirely by $\lambda = 3$ and its eigenvector $\begin{pmatrix} 1 \\ 1 \end{pmatrix}$. This "the biggest eigenvalue wins in the long run" principle is exactly what makes eigenvalues predict the fate of dynamic systems.

Why Eigenvalues Matter: Three Real Engines

PageRank. Google's original algorithm modelled the web as a giant matrix where entry $(i,j)$ describes the chance a random surfer moves from page $j$ to page $i$. The ranking of every page is the dominant eigenvector (eigenvalue $\lambda = 1$) of that matrix — the steady-state distribution the surfer settles into. Repeatedly multiplying any starting guess by the matrix converges to it, precisely because the top eigenvalue dominates.

Vibrations and stability. Every bridge, building, or aircraft wing has natural frequencies of oscillation. These are eigenvalues of the system's stiffness-and-mass equations; the eigenvectors are the mode shapes. The 1940 Tacoma Narrows Bridge collapse is the classic warning: wind drove the structure near an eigen-frequency and the oscillation grew without bound. Engineers compute eigenvalues to guarantee real-world forcing stays away from them.

Principal Component Analysis (PCA). Given high-dimensional data, PCA finds the covariance matrix and computes its eigenvectors. The eigenvector with the largest eigenvalue points along the direction of greatest variation in the data; the eigenvalue measures how much variance that direction captures. Keeping the top few eigenvectors compresses data while losing minimal information — the backbone of dimensionality reduction, facial recognition, and genomics.

Real-World Applications

• Search and networks: PageRank, recommendation systems, and detecting communities in social graphs via the graph Laplacian's eigenvalues.
• Structural and mechanical engineering: modal analysis of bridges, cars, and turbines to avoid resonant failure.
• Quantum mechanics: measurable quantities (energy levels) are eigenvalues of operators — the Schrödinger equation is an eigenvalue equation.
• Data science and ML: PCA, spectral clustering, and stability analysis of training dynamics.
• Population and economics: long-term growth rates of populations (Leslie matrices) and Markov steady states are dominant eigenvalues.

Common Mistakes

1. Thinking the zero vector is an eigenvector. Since $A\mathbf{0} = \lambda \mathbf{0}$ holds for every $\lambda$, allowing $\mathbf{0}$ would make eigenvalues meaningless. Correction: eigenvectors must be nonzero by definition; only eigenvalues can be zero.

2. Confusing which is scaled by which. Students write $Av = v\lambda$ and then try to "divide by $v$." You cannot divide by a vector. Correction: treat $Av = \lambda v$ as a system and always solve via $\det(A - \lambda I) = 0$, never by division.

3. Assuming every matrix diagonalizes. A matrix with a repeated eigenvalue may not have enough independent eigenvectors — for example $\begin{pmatrix} 1 & 1 \\ 0 & 1 \end{pmatrix}$ has only $\lambda = 1$ with a single eigenvector direction. Correction: diagonalization requires $n$ independent eigenvectors; otherwise you need the Jordan form. Also remember real matrices can have complex eigenvalues (rotations do).

Comparison and Connections

Eigenvalues sit alongside several related tools; keeping them distinct prevents confusion.

Concept	What it captures	Requires
Eigenvalue $\lambda$	Stretch factor along an invariant direction	Square matrix
Eigenvector $v$	The invariant direction itself	$Av = \lambda v$, $v \neq 0$
Determinant	Product of all eigenvalues; volume scaling	Square matrix
Trace	Sum of all eigenvalues	Square matrix
Singular values (SVD)	Stretch factors of any matrix	Works for non-square too

Singular values generalize the idea to rectangular matrices and are the eigenvalues of $A^{T}A$ (square-rooted). The determinant and trace are just compressed summaries of the eigenvalue spectrum, which is why they serve as quick checks.

Practice Questions

Recall

State the equation defining an eigenvector, and name the equation you solve to find eigenvalues. Answer: $Av = \lambda v$ with $v \neq 0$; eigenvalues come from the characteristic equation $\det(A - \lambda I) = 0$.

Understanding

Why must $A - \lambda I$ be singular for an eigenvalue to exist? Guidance: $(A - \lambda I)v = 0$ needs a nonzero solution. A matrix maps a nonzero vector to zero only when it is singular, i.e. its determinant is zero.

Application

Find the eigenvalues of $A = \begin{pmatrix} 4 & 0 \\ 1 & 3 \end{pmatrix}$. Answer: It is triangular, so eigenvalues are the diagonal entries: $\lambda = 4$ and $\lambda = 3$. (Check: trace $= 7$, det $= 12 = 4 \times 3$.)

Analysis

A Markov transition matrix always has $\lambda = 1$ as an eigenvalue. Why, and what does its eigenvector represent? Guidance: Columns sum to 1, so the all-ones vector is a left eigenvector for $\lambda = 1$; the corresponding right eigenvector (normalized) is the steady-state distribution the system converges to — the basis of PageRank.

FAQ

Can eigenvalues be negative or complex? Yes. Negative eigenvalues flip a vector's direction. Complex eigenvalues appear for rotation-like matrices; they encode rotation plus scaling and come in conjugate pairs for real matrices.

How many eigenvalues does a matrix have? An $n \times n$ matrix has exactly $n$ eigenvalues counted with multiplicity (some may repeat or be complex), because the characteristic polynomial has degree $n$.

Is an eigenvector unique? No. Any nonzero scalar multiple of an eigenvector is also an eigenvector for the same eigenvalue — it is really a whole invariant line (or subspace). We usually pick a convenient or unit-length representative.

What if two eigenvalues are equal? They share an eigenspace that may be one- or higher-dimensional. If it has full dimension, the matrix still diagonalizes; if not, it is "defective" and needs the Jordan normal form.

Do I always need the characteristic polynomial? For hand calculations of small matrices, yes. For large matrices, numerical methods like the QR algorithm or power iteration are used instead — power iteration is literally repeated multiplication converging to the dominant eigenvector, exactly like PageRank.

Quick Revision

• Definition: $Av = \lambda v$, $v \neq 0$. Eigenvector = invariant direction; eigenvalue = stretch factor.
• Find eigenvalues: solve $\det(A - \lambda I) = 0$.
• Find eigenvectors: solve $(A - \lambda I)v = 0$ for each $\lambda$.
• Sum of eigenvalues = trace; product = determinant (fast checks).
• Diagonalization: $A = PDP^{-1}$, so $A^k = PD^kP^{-1}$.
• Largest-magnitude eigenvalue dominates long-term behaviour.
• Symmetric matrices: real eigenvalues, orthogonal eigenvectors.

Related Topics

Prerequisites

• Matrices and Matrix Operations — the Linear Algebra overview covers matrix multiplication and the identity matrix.
• Determinants — needed to form and solve the characteristic equation.

Related Topics

• Diagonalization and matrix powers.
• Singular Value Decomposition (SVD) — the rectangular-matrix generalization.

Next Topics

• Principal Component Analysis and applications in Statistics (see Statistics and Probability).
• Systems of differential equations, where eigenvalues determine stability (see Differential Equations).