Linear Algebra

Linear algebra is the mathematics of straight-line structure: vectors, matrices, and the transformations that stretch, rotate, and reshape them. At first glance it can look like an endless bookkeeping of numbers in grids, but underneath it is a deeply geometric subject. Every list of numbers is a point or an arrow in space, every matrix is a machine that moves those arrows around, and almost every quantitative question — "how do I best fit a line to data?", "how does this 3D model rotate on screen?", "which direction does this system naturally settle into?" — becomes a clean question about vectors and the operations we perform on them.

Why does it matter so much? Because linearity is the one kind of complexity we can fully solve. Nonlinear problems are usually attacked by approximating them with linear ones, so linear algebra sits at the heart of calculus, statistics, physics, economics, and computer science. It is the language of computer graphics, the engine of machine learning and neural networks, the backbone of search engines and recommendation systems, and the standard tool for solving the enormous systems of equations that model everything from bridges to weather. Learn it well and you gain a lens that makes an astonishing range of modern technology suddenly legible.

Learning Objectives

• Understand vectors as both geometric arrows and lists of numbers, and operate on them (addition, scaling, dot product).
• Perform matrix arithmetic fluently — multiplication, transpose, inverse — and interpret each operation geometrically.
• Compute and interpret determinants as signed measures of area, volume, and invertibility.
• Solve systems of linear equations using elimination, matrix methods, and an understanding of when solutions exist.
• Find eigenvalues and eigenvectors, and explain why they reveal a transformation's natural axes.
• Grasp the abstract idea of a vector space, including basis, dimension, and span.
• Recognize matrices as linear transformations and connect algebraic operations to geometric change.

Quick Answer

Linear algebra studies vectors (quantities with direction and magnitude) and the linear transformations that act on them, most often represented by matrices. Vectors can be added and scaled; matrices multiply vectors to produce new vectors, encoding operations like rotation, scaling, and projection. A system of linear equations asks which vector, when transformed, lands on a given target — and matrix methods solve these systems efficiently even at massive scale. The determinant of a matrix tells you whether a transformation is invertible and how much it scales areas or volumes. Eigenvalues and eigenvectors identify the special directions a transformation leaves unchanged except for stretching, exposing the "natural behavior" of the underlying process. Stepping up in abstraction, a vector space is any collection of objects that can be added and scaled consistently, and a basis is the minimal set of directions needed to describe everything in it. Together these ideas form a toolkit that turns geometry into algebra and back again, which is exactly why linear algebra underpins graphics, data science, and scientific computing.

Where It Came From

The oldest roots of linear algebra lie in solving simultaneous equations. Ancient Chinese mathematicians recorded elimination methods in The Nine Chapters on the Mathematical Art (around 200 BCE), essentially performing row operations on coefficient arrays two millennia before matrices had a name. In 17th-century Japan, Seki Takakazu, and independently in Europe, Gottfried Leibniz, developed early notions of the determinant to characterize when systems could be solved.

The modern subject crystallized in the 19th century. Carl Friedrich Gauss formalized the elimination technique now bearing his name while computing planetary orbits. Arthur Cayley introduced matrices as objects in their own right in the 1850s and defined how to multiply them, while James Joseph Sylvester coined the term "matrix." Hermann Grassmann and Giuseppe Peano pushed toward the abstract axioms of a vector space, freeing the ideas from specific lists of numbers. The 20th century then supplied the payoff: as computers arrived, linear algebra became the computational workhorse of science and engineering, and the rise of machine learning and computer graphics made it, arguably, the most economically important branch of mathematics alive today.

Topics at a Glance

Topic	What You'll Learn	Key Concepts
Vectors	Represent quantities with direction and magnitude, and combine them	Components, addition, scalar multiplication, dot product, norm
Matrices and Matrix Operations	Store and transform data in rectangular arrays	Multiplication, transpose, identity, inverse
Determinants	Measure how a transformation scales space and whether it is invertible	Cofactor expansion, area/volume scaling, singularity
Systems of Linear Equations	Solve many equations in many unknowns systematically	Gaussian elimination, row reduction, consistency
Eigenvalues and Eigenvectors	Find the natural axes and scaling factors of a transformation	Characteristic equation, diagonalization, invariant directions
Vector Spaces	Understand the abstract setting where all this lives	Span, basis, dimension, subspaces, linear independence
Linear Transformations	See matrices as functions that move space linearly	Mapping, kernel, image, composition, geometry of change

Learning Path

Real-World Applications

• Computer graphics and games: every rotation, scaling, and camera projection of a 3D scene is a matrix multiplying vertex vectors — matrices are literally what move objects on screen.
• Machine learning: data sets are matrices, model parameters are vectors, and training is a cascade of matrix operations; techniques like principal component analysis are eigenvalue problems in disguise.
• Search and recommendations: Google's original PageRank algorithm finds the dominant eigenvector of a giant web-link matrix, and recommendation engines factor huge user-item matrices.
• Engineering and physics: structural analysis, electrical circuits, and quantum mechanics all reduce to solving large systems of linear equations or eigenvalue problems.
• Economics and operations: input-output models, optimization, and equilibrium calculations rely on matrix inversion and linear systems.
• Data compression: images and signals are compressed by approximating their matrices with a few dominant components.

Key Terms

Term	Definition	Related Concept
Vector	An object with magnitude and direction, written as an ordered list of numbers	Components, norm
Matrix	A rectangular array of numbers representing data or a transformation	Matrix multiplication
Determinant	A scalar measuring how a matrix scales area/volume; zero means non-invertible	Invertibility
Eigenvector	A nonzero vector whose direction is unchanged by a transformation	Eigenvalue
Eigenvalue	The scalar factor by which an eigenvector is stretched	Characteristic equation
Span	The set of all vectors reachable by combining a given set	Basis
Basis	A minimal set of independent vectors that spans a space	Dimension
Linear transformation	A function preserving addition and scalar multiplication	Matrix
Rank	The number of independent directions a matrix actually reaches	Kernel, image

Quick Revision

• A vector is both an arrow in space and a list of numbers; you can add vectors and scale them.
• Matrix multiplication applies a transformation; order matters — $AB$ is generally not $BA$.
• The dot product measures alignment: $0$ means perpendicular, positive means similar direction.
• A matrix is invertible exactly when its determinant is nonzero and its columns are linearly independent.
• Solving $Ax = b$ means finding the input vector $x$ that a transformation maps onto $b$.
• Eigenvectors satisfy $Av = \lambda v$: the transformation only stretches them by the factor $\lambda$.
• A basis gives coordinates; the dimension is how many basis vectors you need.
• Almost every nonlinear problem is solved by locally approximating it with a linear one.

Related Topics

Prerequisites

• Algebra — comfort with variables, equations, and functions.
• Geometry — points, lines, and coordinates in space.

Related Topics

• Calculus — multivariable calculus leans heavily on vectors and matrices.
• Statistics and Probability — regression and data analysis are built on linear algebra.

Next Topics

• Differential Equations — systems of differential equations are solved with eigenvalues and matrices.
• Discrete Mathematics — graphs and networks are often studied through their matrices.