Linear Transformations

Learning Objectives

By the end of this page, you should be able to:

• Define a linear transformation and state the two properties it must satisfy
• Represent a linear transformation as a matrix and apply it to a vector
• Distinguish between injective, surjective, and isomorphic transformations
• Compute the result of common transformations like scaling, rotation, and reflection
• Explain why linear transformations are the mathematical backbone of graphics and machine learning
• Identify common mistakes students make when reasoning about linearity

Quick Answer

A linear transformation is a function that maps vectors from one space to another while preserving vector addition and scalar multiplication — in plain terms, straight lines stay straight, the origin stays fixed, and scaling a vector before or after the transformation gives the same result. Every linear transformation between finite-dimensional spaces can be written as a matrix, so applying the transformation is just matrix-vector multiplication. This matters in computer science because rotating a 3D model, resizing an image, compressing data with PCA, and passing data through a neural network layer are all linear transformations in disguise. Once you can spot "this is just $T(v) = Av$," a huge chunk of graphics and machine learning becomes far less mysterious.

What Is a Linear Transformation?

A linear transformation is a function $T: V \to W$ between two vector spaces that preserves the two operations that make vector spaces useful: addition and scalar multiplication.

Formally, $T$ is linear if for all vectors $u, v$ and scalar $c$:

$T(u + v) = T(u) + T(v) \qquad \text{(additivity)}$ $T(cu) = cT(u) \qquad \text{(homogeneity)}$

Why it matters: These two rules guarantee predictability. If you know how $T$ acts on a small set of building-block vectors, you automatically know how it acts on every combination of them. That's what makes a transformation representable as a single matrix instead of an arbitrary black-box function.

Common misunderstanding: Students often assume any "nice-looking" function is linear — for example, $T(x) = 2x + 3$ looks simple, but it fails the test because $T(0) \neq 0$ (a linear transformation must always send the zero vector to the zero vector). Adding a constant, taking a square, or applying a trigonometric function all break linearity.

Representing a Transformation as a Matrix

Every linear transformation between finite-dimensional vector spaces can be written as $T(v) = Av$, where $A$ is a matrix. Once you know $A$, applying $T$ to any vector is just matrix-vector multiplication.

Worked example — scaling:

Suppose you want to scale every point in 2D space by 2 along the x-axis and 3 along the y-axis. The transformation matrix is:

$A = \begin{pmatrix} 2 & 0 \\ 0 & 3 \end{pmatrix}$

Applying it to the vector $v = \begin{pmatrix} 4 \\ 5 \end{pmatrix}$:

$Av = \begin{pmatrix} 2 & 0 \\ 0 & 3 \end{pmatrix}\begin{pmatrix} 4 \\ 5 \end{pmatrix} = \begin{pmatrix} 2(4) + 0(5) \\ 0(4) + 3(5) \end{pmatrix} = \begin{pmatrix} 8 \\ 15 \end{pmatrix}$

The point $(4, 5)$ moved to $(8, 15)$ — stretched horizontally by 2 and vertically by 3, exactly as the matrix describes.

Worked example — rotation:

A rotation by angle $\theta$ (counterclockwise) around the origin uses:

$R(\theta) = \begin{pmatrix} \cos\theta & -\sin\theta \\ \sin\theta & \cos\theta \end{pmatrix}$

Rotating the point $(1, 0)$ by $90°$($\theta = \pi/2$, so$\cos\theta = 0$,$\sin\theta = 1$):

$R(90°)\begin{pmatrix}1\\0\end{pmatrix} = \begin{pmatrix} 0 & -1 \\ 1 & 0 \end{pmatrix}\begin{pmatrix}1\\0\end{pmatrix} = \begin{pmatrix}0\\1\end{pmatrix}$

The point moves from $(1,0)$ to $(0,1)$ — a quarter turn counterclockwise, which matches what you'd expect visually.

Real-world example: Every time a video game rotates a character or a photo-editing app resizes an image, it's multiplying pixel coordinates by a transformation matrix like the ones above.

Why it matters: Representing transformations as matrices means you can combine multiple transformations (rotate, then scale, then reflect) by simply multiplying their matrices together, producing one matrix that does all three steps at once. This is exactly how graphics pipelines chain operations efficiently.

Key Properties of Linear Transformations

• Dimensionality: The dimension of the image (range) of $T$ can never exceed the dimension of the domain — a transformation cannot "create" extra independent directions out of nowhere.
• Injective (one-to-one): $T$ is injective if the only vector it sends to the zero vector is the zero vector itself (its null space is trivial). This means no two distinct inputs produce the same output.
• Surjective (onto): $T$ is surjective if its range covers the entire codomain — every possible output vector is actually achieved by some input.
• Isomorphism: A transformation that is both injective and surjective is an isomorphism — it's reversible, meaning an inverse transformation exists that perfectly undoes it.

Common misunderstanding: Students sometimes think "injective" and "invertible" mean the same thing regardless of context. A transformation must be both injective and surjective (i.e., an isomorphism) for a true inverse to exist across the full codomain; being injective alone only guarantees no information is lost, not that every output is reachable.

How Linear Transformations Are Used in Computer Science

Computer graphics: Rotating, scaling, and projecting 3D models onto a 2D screen are all linear transformations. GPUs are optimized specifically to perform millions of these matrix-vector multiplications per second.

Machine learning: A fully connected neural network layer computes $T(x) = Wx + b$. The $Wx$ part is a linear transformation; the $+b$ makes the whole thing "affine" rather than strictly linear, but the transformation logic — mapping input features to a new space — is the same idea.

import numpy as np

# A linear transformation matrix that scales x by 2 and shears y
A = np.array([[2, 0],
              [1, 1]])

v = np.array([3, 4])
result = A @ v          # matrix-vector multiplication
print(result)           # [6, 7]

Data compression (PCA): Principal Component Analysis finds a linear transformation that rotates data into a new coordinate system where the first few axes capture most of the variance, letting you discard the rest with minimal information loss.

Why it matters: Because linear transformations compose (chain together) through matrix multiplication, complex pipelines — like a neural network with many layers, or a graphics pipeline with rotation-then-projection — reduce to repeated applications of the same simple operation: multiply a vector by a matrix.

Key Terms

Term	Definition
Linear transformation	A function $T: V \to W$ that preserves vector addition and scalar multiplication
Domain	The vector space that inputs to $T$ come from
Codomain	The vector space that outputs of $T$ land in
Range (image)	The actual set of outputs $T$ produces, a subset of the codomain
Null space (kernel)	The set of vectors that $T$ maps to the zero vector
Injective	A transformation where distinct inputs always produce distinct outputs
Surjective	A transformation whose range covers the entire codomain
Isomorphism	A transformation that is both injective and surjective, and therefore invertible
Affine transformation	A linear transformation plus a constant shift, e.g., $T(x) = Ax + b$

Common Mistakes

1. Misconception: "Any function that looks like a straight line on a graph is a linear transformation." Why it's wrong: A function like $f(x) = 2x + 3$ graphs as a straight line but is not linear in the vector-space sense because it doesn't send the zero vector to zero. Correct: A linear transformation must satisfy $T(0) = 0$ and both additivity and homogeneity. Functions with an added constant are "affine," not linear.

2. Misconception: "If a transformation is injective, it must also be invertible." Why it's wrong: Injectivity only guarantees that no two inputs collapse to the same output — it says nothing about whether every element of the codomain is reached. Correct: A transformation is only truly invertible (an isomorphism) when it is both injective and surjective. A map from a small space into a much larger one can be injective without covering the whole codomain.

3. Misconception: "Combining two transformations means adding their matrices together." Why it's wrong: Adding matrices produces a transformation that applies both effects "at once" in a blended way, not one transformation followed by another. Correct: Applying transformation $A$ and then transformation $B$ corresponds to matrix multiplication $BA$ (note the order — $B$ is applied to the result of $A$), not matrix addition.

Comparison and Connections

Concept	Linear Transformation	Affine Transformation	Matrix Multiplication
Sends origin to origin?	Always	Not necessarily	N/A (it's the operation itself)
Formula	$T(v) = Av$	$T(v) = Av + b$	Combines two transformations
Preserves vector addition/scaling?	Yes	No (because of the shift $b$)	Preserves linearity of the result
Typical CS use	Rotation, scaling, PCA	Neural network layers, graphics translations	Composing multiple transformations into one

Practice Questions

Recall

1. What are the two defining properties a function must satisfy to be a linear transformation? Answer: Additivity, $T(u+v) = T(u) + T(v)$, and homogeneity, $T(cu) = cT(u)$.
2. What is the null space of a linear transformation? Answer: The set of all input vectors that map to the zero vector in the output space.

Understanding 3. Why must every linear transformation send the zero vector to the zero vector? Answer guidance: By homogeneity, $T(0 \cdot v) = 0 \cdot T(v) = 0$ for any vector $v$, so plugging in the zero vector always forces the output to be zero — this is a direct consequence of the linearity rules, not an extra assumption. 4. Why does composing two linear transformations correspond to multiplying their matrices rather than adding them? Answer guidance: Composition means feeding the output of one transformation into the next. Multiplying matrices $BA$ produces exactly the matrix that reproduces "apply $A$, then apply $B$" as a single step, while addition would just blend the two effects on the same input rather than chaining them.

Application 5. A graphics engine needs to rotate an object by 90° and then scale it by 2 in both directions. Write the matrix that performs both operations in a single multiplication. Answer guidance: Rotation matrix $R(90°) = \begin{pmatrix}0 & -1\\1 & 0\end{pmatrix}$, scaling matrix $S = \begin{pmatrix}2 & 0\\0 & 2\end{pmatrix}$. Combined transformation is $S \cdot R(90°) = \begin{pmatrix}0 & -2\\2 & 0\end{pmatrix}$, applied as a single matrix-vector multiplication. 6. A neural network layer computes $T(x) = Wx + b$. Is this a linear transformation? Justify your answer. Answer: No — it's affine, not linear, because the added bias vector $b$ means $T(0) = b \neq 0$ in general, violating the requirement that a linear transformation must map the zero vector to zero.

Analysis 7. Compare an injective-but-not-surjective transformation to an isomorphism. Give an example of each in terms of dimension. Answer guidance: An injective-but-not-surjective transformation maps a smaller space into a larger one without collapsing any information, e.g., $T: \mathbb{R}^2 \to \mathbb{R}^3$ embedding a plane inside 3D space — every output is reachable within a 2D slice of $\mathbb{R}^3$, but not the full space. An isomorphism requires equal dimensions on both sides and full coverage, e.g., a rotation $T: \mathbb{R}^2 \to \mathbb{R}^2$. 8. Why is it useful in PCA that the transformation applied to the data is linear rather than some arbitrary nonlinear function? Answer guidance: Linearity guarantees the new axes (principal components) are just a rotation of the original coordinate system, preserving distances and relationships between data points in a mathematically well-understood way — a nonlinear transformation could distort the data unpredictably, making "variance along an axis" meaningless.

FAQ

Q: Is every matrix a linear transformation? A: Yes — any matrix $A$ defines a linear transformation $T(v) = Av$, and conversely, every linear transformation between finite-dimensional spaces can be written as some matrix. They're two views of the same idea.

Q: What's the difference between "linear" and "affine"? A: A linear transformation must send the zero vector to zero and satisfies $T(v) = Av$. An affine transformation allows a shift, $T(v) = Av + b$, so it doesn't need to fix the origin. Most real-world graphics transformations (like moving an object across the screen) are technically affine.

Q: Why do we care whether a transformation is invertible? A: Invertibility tells you whether you can recover the original input from the output. This matters in compression (can you decompress losslessly?), cryptography (can you decrypt?), and solving equations (does a unique solution exist?).

Q: How is a linear transformation different from just multiplying numbers? A: Ordinary multiplication acts on single numbers; a linear transformation acts on entire vectors (or higher-dimensional data) at once, and the matrix that represents it can stretch, rotate, reflect, or project — effects that a single number can't produce.

Q: Do linear transformations always preserve angles and lengths? A: No. Scaling and shearing transformations are linear but change lengths and angles. Only special linear transformations, like rotations and reflections, preserve both — these are called orthogonal transformations.

Quick Revision

• A linear transformation $T: V \to W$ preserves vector addition and scalar multiplication.
• $T(0) = 0$ always holds for a linear transformation.
• Every linear transformation between finite-dimensional spaces can be written as $T(v) = Av$.
• Injective = no two inputs map to the same output; surjective = every output is reached; isomorphism = both.
• Composing transformations corresponds to matrix multiplication, not addition — order matters.
• Rotation matrices use $\cos\theta$ and $\sin\theta$; scaling matrices are diagonal.
• Affine transformations add a shift term ($Av + b$) and don't fix the origin.
• Graphics engines chain rotate/scale/project operations by multiplying their matrices together.
• Neural network layers are affine transformations followed by a nonlinear activation function.
• PCA relies on a linear transformation (rotation) to find axes of maximum variance in data.
• The null space (kernel) of a transformation tells you what information gets collapsed to zero.

Related Topics

Prerequisites: Matrices and Determinants, vectors and vector spaces, basic function notation

Related Topics: Eigenvalues and eigenvectors, systems of linear equations, matrix decomposition (SVD, PCA)

Next Topics: Probability Theory, Random Variables