Matrices and Matrix Operations

A matrix is, at first glance, just a rectangular grid of numbers. But that plain description hides one of the most powerful ideas in all of mathematics: a single matrix can encode an entire transformation — a rule that takes vectors and reshapes them, rotates them, stretches them, or collapses them. Once you see matrices this way, their strange multiplication rule stops looking arbitrary and starts looking inevitable.

In this page we build matrices from the ground up: how to write them, how to add and scale them, and — the crucial part — how and why we multiply them the way we do. Matrix operations are the language behind computer graphics, machine learning, physics simulations, economics, and the solution of linear systems. Master them here and the rest of linear algebra opens up.

Learning Objectives

• Read and write matrix notation, including dimensions, entries, and indexing conventions.
• Perform matrix addition, subtraction, and scalar multiplication fluently.
• Compute matrix products by hand and state exactly when a product is defined.
• Explain why matrix multiplication uses the row-by-column rule rather than entrywise multiplication.
• Recognize the identity matrix and understand what a matrix inverse is and when it exists.
• Connect matrix operations to the composition of linear transformations.

Quick Answer

A matrix is a rectangular array of numbers with $m$ rows and $n$ columns, written as an $m \times n$ matrix. You add or subtract two matrices of the same size entry by entry, and you multiply a matrix by a scalar by scaling every entry. Matrix multiplication is different: to multiply an $m \times n$ matrix by an $n \times p$ matrix, each entry of the product is the dot product of a row from the first with a column from the second — so the inner dimensions must match, and the result is $m \times p$. The identity matrix $I$ acts like the number 1 (it leaves matrices unchanged), and the inverse $A^{-1}$, when it exists, satisfies $A A^{-1} = I$. Matrix multiplication is defined this way because it exactly captures the composition of linear transformations.

Where It Came From

Matrices were born from a very practical need: solving systems of linear equations. For centuries, mathematicians manipulated the coefficients of equations directly, and the arrays of numbers that appeared were treated as bookkeeping, not as objects in their own right. Chinese mathematicians in the Nine Chapters on the Mathematical Art (around 200 BCE) already used grid arrangements of coefficients and something very close to Gaussian elimination — but they had no notion of a matrix as a single algebraic thing.

The decisive leap came in the mid-nineteenth century in England. James Joseph Sylvester coined the word matrix in 1850 (from the Latin for "womb," because he saw it as the array from which determinants — minors — are "born"). His friend and collaborator Arthur Cayley then did something revolutionary: in his 1858 Memoir on the Theory of Matrices, he treated matrices as objects you can add, subtract, and — crucially — multiply, with their own algebra. Cayley noticed that when you substitute one linear change of variables into another, the resulting coefficients combine by exactly the row-by-column rule. In other words, he defined matrix multiplication so that it would represent the composition of two linear substitutions. This is the whole reason matrix multiplication looks the way it does; it was reverse-engineered to match how transformations chain together. Cayley also observed that a matrix satisfies its own characteristic equation (the Cayley–Hamilton theorem), cementing matrices as genuine algebraic objects. Later, quantum mechanics (Heisenberg's 1925 "matrix mechanics") and modern computing turned this once-abstract theory into an everyday tool.

Matrix Notation and Basic Structure

A matrix is a rectangular array of numbers arranged in rows and columns. We usually name matrices with capital letters. The matrix

$A = \begin{bmatrix} 2 & -1 & 0 \\ 3 & 5 & 4 \end{bmatrix}$

has 2 rows and 3 columns, so we say $A$ is a $2 \times 3$ matrix (rows always come first). The individual numbers are called entries or elements. We refer to the entry in row $i$ and column $j$ as $a_{ij}$. For the matrix above, $a_{12} = -1$ (row 1, column 2) and $a_{23} = 4$ (row 2, column 3).

Some special shapes get their own names:

• A square matrix has the same number of rows and columns ($n \times n$).
• A row vector is a $1 \times n$ matrix; a column vector is an $m \times 1$ matrix.
• The main diagonal of a square matrix consists of the entries $a_{11}, a_{22}, a_{33}, \dots$

Two matrices are equal only if they have the same dimensions and every corresponding entry matches.

Addition, Subtraction, and Scalar Multiplication

These operations are the "easy" ones because they work entry by entry.

To add (or subtract) two matrices, they must be the same size, and you simply combine corresponding entries:

$\begin{bmatrix} 1 & 2 \\ 3 & 4 \end{bmatrix} + \begin{bmatrix} 5 & 6 \\ 7 & 8 \end{bmatrix} = \begin{bmatrix} 6 & 8 \\ 10 & 12 \end{bmatrix}$

Here $1+5 = 6$, $2+6 = 8$, $3+7 = 10$, and $4+8 = 12$. If the sizes differ, the sum is simply undefined — there is no meaningful way to line the entries up.

Scalar multiplication means multiplying every entry by the same number (a "scalar"):

$3 \cdot \begin{bmatrix} 2 & -1 \\ 0 & 4 \end{bmatrix} = \begin{bmatrix} 6 & -3 \\ 0 & 12 \end{bmatrix}$

Each entry is tripled. Subtraction is just adding a scalar multiple of $-1$: $A - B = A + (-1)B$.

These operations obey the comfortable rules you already expect from ordinary numbers: addition is commutative ($A + B = B + A$) and associative, and scalar multiplication distributes over addition ($k(A+B) = kA + kB$).

Matrix Multiplication: The Row-by-Column Rule

Here the story gets interesting. To multiply matrix $A$ by matrix $B$, the number of columns of $A$ must equal the number of rows of $B$. If $A$ is $m \times n$ and $B$ is $n \times p$, then the product $AB$ is $m \times p$. The inner dimensions ($n$ and $n$) must match and then vanish; the outer dimensions ($m$ and $p$) survive.

The entry in row $i$, column $j$ of $AB$ is the dot product of row $i$ of $A$ with column $j$ of $B$:

$(AB)_{ij} = \sum_{k=1}^{n} a_{ik} b_{kj}$

Worked example. Let

$A = \begin{bmatrix} 1 & 2 \\ 3 & 4 \end{bmatrix}, \qquad B = \begin{bmatrix} 5 & 6 \\ 7 & 8 \end{bmatrix}$

Both are $2 \times 2$, so $AB$ is $2 \times 2$. Compute each entry:

• Row 1 · Column 1: $(1)(5) + (2)(7) = 5 + 14 = 19$
• Row 1 · Column 2: $(1)(6) + (2)(8) = 6 + 16 = 22$
• Row 2 · Column 1: $(3)(5) + (4)(7) = 15 + 28 = 43$
• Row 2 · Column 2: $(3)(6) + (4)(8) = 18 + 32 = 50$

$AB = \begin{bmatrix} 19 & 22 \\ 43 & 50 \end{bmatrix}$

A critical warning: matrix multiplication is not commutative. In general $AB \neq BA$. For the matrices above,

$BA = \begin{bmatrix} 23 & 34 \\ 31 & 46 \end{bmatrix}$

which is completely different from $AB$. The order matters, always.

Why Multiplication Is Defined This Way

The row-by-column rule feels bizarre until you know its purpose. A matrix represents a linear transformation — a function that takes an input vector $\mathbf{x}$ and produces an output vector $A\mathbf{x}$. For example, applying $A$ to the column vector $\begin{bmatrix} x \\ y \end{bmatrix}$ produces a new vector whose components are linear combinations of $x$ and $y$.

Now suppose you apply transformation $B$ first, then transformation $A$. The combined effect on a vector $\mathbf{x}$ is $A(B\mathbf{x})$. The remarkable fact is that this composition is itself a linear transformation, and the single matrix representing it is exactly the product $AB$. Cayley defined multiplication precisely so that "do $B$, then do $A$" equals "do $AB$."

This also explains, at a stroke, two things that puzzle beginners. First, why the inner dimensions must match: the output of $B$ must have the right number of components to feed into $A$. Second, why order matters: putting on your socks then your shoes is not the same as shoes then socks — composing transformations depends on which you do first. If multiplication were merely entrywise, it would carry none of this meaning and would be useless for the problems matrices were invented to solve.

The Identity Matrix and the Inverse

Among square matrices, one plays the role that the number 1 plays for ordinary numbers. The identity matrix $I_n$ is the $n \times n$ matrix with 1s on the main diagonal and 0s everywhere else:

$I_3 = \begin{bmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{bmatrix}$

For any compatible matrix $A$, we have $AI = A$ and $IA = A$. The identity represents the transformation that does nothing — it leaves every vector where it is.

The inverse of a square matrix $A$ is the matrix $A^{-1}$ that "undoes" it:

$A A^{-1} = A^{-1} A = I$

Not every matrix has an inverse. A matrix is invertible (or nonsingular) exactly when its determinant is nonzero; if the determinant is zero, the transformation collapses space in a way that cannot be reversed. For a $2 \times 2$ matrix there is a clean formula:

$A = \begin{bmatrix} a & b \\ c & d \end{bmatrix} \implies A^{-1} = \frac{1}{ad - bc} \begin{bmatrix} d & -b \\ -c & a \end{bmatrix}$

Worked example. Let $A = \begin{bmatrix} 4 & 7 \\ 2 & 6 \end{bmatrix}$. The determinant is $ad - bc = (4)(6) - (7)(2) = 24 - 14 = 10$. Since this is nonzero, $A$ is invertible:

$A^{-1} = \frac{1}{10} \begin{bmatrix} 6 & -7 \\ -2 & 4 \end{bmatrix} = \begin{bmatrix} 0.6 & -0.7 \\ -0.2 & 0.4 \end{bmatrix}$

Check: $A A^{-1} = \begin{bmatrix} (4)(0.6)+(7)(-0.2) & (4)(-0.7)+(7)(0.4) \\ (2)(0.6)+(6)(-0.2) & (2)(-0.7)+(6)(0.4) \end{bmatrix} = \begin{bmatrix} 2.4 - 1.4 & -2.8 + 2.8 \\ 1.2 - 1.2 & -1.4 + 2.4 \end{bmatrix} = \begin{bmatrix} 1 & 0 \\ 0 & 1 \end{bmatrix}$. It works.

Real-World Applications

• Computer graphics and games. Every rotation, scaling, and translation of a 3D model is a matrix. Composing them by multiplication lets a GPU transform millions of vertices per frame.
• Machine learning and neural networks. A layer of a neural network multiplies an input vector by a weight matrix; training adjusts those entries. The heaviest computation in deep learning is matrix multiplication.
• Solving linear systems. A system $A\mathbf{x} = \mathbf{b}$ is solved (when possible) by $\mathbf{x} = A^{-1}\mathbf{b}$, or more practically by elimination — used in engineering, circuit analysis, and economics.
• Google's PageRank and networks. The link structure of the web is a giant matrix; repeated multiplication reveals which pages are most central.
• Cryptography and error correction. Hill ciphers and coding schemes encode messages by multiplying blocks of data by matrices.

Common Mistakes

Mistake 1: Multiplying matrices entrywise. Students often try to compute $AB$ by multiplying corresponding entries (like addition). Why it's wrong: that operation (the Hadamard product) exists but is not the standard matrix product and does not represent composition of transformations. Correction: always use the row-by-column dot-product rule.

Mistake 2: Assuming $AB = BA$. Because ordinary numbers commute, beginners expect matrices to as well. Why it's wrong: matrix multiplication represents ordered composition, and order changes the result; often one product isn't even defined while the other is. Correction: preserve order carefully and never swap factors.

Mistake 3: Thinking every square matrix has an inverse. Why it's wrong: if the determinant is zero (a singular matrix), the transformation loses information — like projecting 3D onto a plane — and cannot be undone. Correction: always check that $ad - bc \neq 0$ (or the determinant more generally) before inverting, and remember non-square matrices never have a two-sided inverse.

Comparison and Connections

Matrix multiplication behaves very differently from ordinary number multiplication, and keeping the differences straight prevents most errors.

Property	Real numbers	Matrix multiplication
Commutative	Yes: $ab = ba$	No: usually $AB \neq BA$
Associative	Yes	Yes: $(AB)C = A(BC)$
Identity element	The number 1	The identity matrix $I$
Inverse always exists?	Yes, except 0	No — only if determinant $\neq 0$
$XY = 0 \Rightarrow X=0$ or $Y=0$?	Yes	No — nonzero matrices can multiply to zero

Matrix addition, by contrast, behaves just like number addition (commutative and associative). Matrices also connect directly to vectors (which are just single-row or single-column matrices), to determinants (which measure how a matrix scales area or volume and detect invertibility), and to linear transformations (the geometric meaning behind every operation on this page).

Practice Questions

Recall

What are the dimensions of a matrix with 4 rows and 2 columns, and what condition must two matrices satisfy to be added?

Answer: It is a $4 \times 2$ matrix. Two matrices can be added only if they have identical dimensions (same number of rows and same number of columns).

Understanding

If $A$ is $3 \times 5$ and $B$ is $5 \times 2$, is $AB$ defined? What about $BA$? Give the sizes of any defined products.

Answer: $AB$ is defined because the inner dimensions match (5 = 5), giving a $3 \times 2$ matrix. $BA$ is not defined, because the inner dimensions (2 and 3) do not match.

Application

Compute $CD$ where $C = \begin{bmatrix} 2 & 0 \\ 1 & 3 \end{bmatrix}$ and $D = \begin{bmatrix} 1 & 4 \\ 2 & 5 \end{bmatrix}$.

Answer: Row 1: $(2)(1)+(0)(2)=2$ and $(2)(4)+(0)(5)=8$. Row 2: $(1)(1)+(3)(2)=7$ and $(1)(4)+(3)(5)=19$. So $CD = \begin{bmatrix} 2 & 8 \\ 7 & 19 \end{bmatrix}$.

Analysis

Explain, using the idea of transformations, why the inverse of a matrix with determinant 0 cannot exist.

Answer: A determinant of 0 means the transformation squashes space into a lower dimension (for example, mapping a plane onto a line). Multiple distinct inputs then map to the same output, so there is no way to recover the original input uniquely. An inverse would have to reverse the map, but reversing a many-to-one collapse is impossible, so no inverse exists.

FAQ

Is a vector just a special matrix? Yes. A column vector is an $m \times 1$ matrix and a row vector is a $1 \times n$ matrix. Everything you learn about matrix operations applies directly to vectors.

Why does the order of multiplication matter so much? Because $AB$ means "apply $B$ first, then $A$." Composing transformations in a different order generally gives a different result — just as rotating then reflecting differs from reflecting then rotating.

Can I multiply a $2 \times 3$ matrix by another $2 \times 3$ matrix? Not with standard multiplication — the columns of the first (3) do not match the rows of the second (2). You could multiply a $2 \times 3$ by a $3 \times 2$, though.

What is the difference between the inverse and the transpose? The transpose $A^T$ flips a matrix across its diagonal (rows become columns) and always exists. The inverse $A^{-1}$ undoes the transformation and exists only for invertible square matrices. They are unrelated in general.

Do I always need the inverse to solve $A\mathbf{x} = \mathbf{b}$? No, and usually you shouldn't compute it. Gaussian elimination is faster and more numerically stable. The inverse is mainly a conceptual tool and is useful when you must apply the same "undo" to many different vectors.

Quick Revision

• An $m \times n$ matrix has $m$ rows, $n$ columns; entry $a_{ij}$ is row $i$, column $j$.
• Add/subtract: same size only, entry by entry. Scalar multiply: scale every entry.
• Multiply $A_{m\times n} B_{n\times p} = C_{m\times p}$: inner dims must match; $(AB)_{ij} = \sum_k a_{ik}b_{kj}$ (row · column).
• Matrix multiplication is associative but not commutative ($AB \neq BA$ in general).
• Identity $I$: 1s on the diagonal, acts like the number 1: $AI = IA = A$.
• Inverse $A^{-1}$: satisfies $AA^{-1} = I$; exists only when determinant $\neq 0$.
• $2 \times 2$ inverse: $\frac{1}{ad-bc}\begin{bmatrix} d & -b \\ -c & a \end{bmatrix}$.
• Multiplication is defined by the row-by-column rule because it represents composition of linear transformations (Cayley, 1858).

Related Topics

Prerequisites

• Linear Algebra Overview

Related Topics

• Vectors and vector operations
• Determinants and their geometric meaning
• Linear transformations

Next Topics

• Systems of linear equations and Gaussian elimination
• Eigenvalues and eigenvectors