A matrix is a transformation

A table of numbers that moves every point of the plane. Read it by its columns: they say where the two basic arrows land, and everything else follows.

10 min Prerequisites: Module 1 (vectors, linear combinations)

Symbols in this lessonX capital letter (matrix)𝐱 bold lowercase (vector)∘ composition

1. The idea

A matrix is a table of numbers, and in machine learning it often just stores data, one row per example. But a matrix is also a machine: multiply it by a vector and out comes another vector. A 2 × 2 matrix moves every point of the plane, and you can watch it happen.

The way to read one is by its columns. The first column is where the arrow î = (1, 0) lands; the second is where ĵ = (0, 1) lands. Every other point follows, because (x, y) = x·î + y·ĵ and the matrix keeps that recipe: A(x, y) = x·(first column) + y·(second column), a linear combination of the columns, as in the last lesson. Grid lines stay straight, parallel and evenly spaced. That is what “linear” means.

Every dense layer in a network is one of these, in hundreds of dimensions: W𝐱 rotates, stretches and squashes its input before the activation function bends it. Stack two linear layers with nothing in between and you get just another matrix, W₂W₁, which is why that bend matters so much.

2. Try it: warp the plane

Try the example matrices, then move the four entries yourself. Watch where î and ĵ land: they are the columns. The shaded square shows what happens to area, which is the next lesson but one.

3. The math

Matrix times vector

\begin{pmatrix} a & b \\ c & d \end{pmatrix}\begin{pmatrix} x \\ y \end{pmatrix} = \begin{pmatrix} ax + by \\ cx + dy \end{pmatrix}

Each output entry is a row of the matrix dotted with the vector.

The columns are where î and ĵ land

A\hat{\imath} = \begin{pmatrix} a \\ c \end{pmatrix} \qquad A\hat{\jmath} = \begin{pmatrix} b \\ d \end{pmatrix}

î = (1, 0) and ĵ = (0, 1), the unit arrows along the axes.

Linear means it keeps combinations

A(x\,\hat{\imath} + y\,\hat{\jmath}) = x\,A\hat{\imath} + y\,A\hat{\jmath}

So knowing the two columns is knowing the whole transformation.

A rotation

R_\theta = \begin{pmatrix} \cos\theta & -\sin\theta \\ \sin\theta & \cos\theta \end{pmatrix}

Turns the plane by θ counter-clockwise.

Two linear layers are one

W_2(W_1\mathbf{x}) = (W_2W_1)\,\mathbf{x}

Without an activation in between, depth adds nothing.

4. The code (python)

mm2_transform.py

import numpy as np

A = np.array([[1.0, 1.0],      # a shear
              [0.0, 1.0]])
print(A @ np.array([1.0, 0.0]))   # [1. 0.]
print(A @ np.array([0.0, 1.0]))   # [1. 1.]
print(A @ np.array([2.0, 3.0]))   # [5. 3.]

t = np.radians(90)                # a quarter turn
R = np.array([[np.cos(t), -np.sin(t)],
              [np.sin(t),  np.cos(t)]])
print((R @ np.array([1.0, 0.0])).round(3))   # [0. 1.]

W1 = np.array([[2.0, 0.0], [1.0, 1.0]])
W2 = np.array([[0.5, 1.0], [0.0, 3.0]])
x = np.array([1.0, 2.0])
print(W2 @ (W1 @ x), (W2 @ W1) @ x)          # [4. 9.] [4. 9.]

5. You’ll see this in

Core Course chapter 8: XOR why a network needs the bend between linear layers
The multi-layer perceptron in the catalog
Reference notes: linear algebra coordinate transformations

6. Check yourself

Answer all 4 questions correctly to complete the chapter · 0 / 4 done

Q1/4 Where does A = [[2, 0], [0, 3]] send the point (1, 1)?

Q2/4 What does the second column of a 2 × 2 matrix tell you?

Q3/4 Two linear layers with no activation between them are equivalent to…

Q4/4 Which matrix turns the plane a quarter turn counter-clockwise?

Finished this chapter?

Weekly newsletter

What changed in ML architectures this week

New models worth knowing, benchmark moves, and one architecture explained in plain words. Free, once a week, unsubscribe anytime.

Learn machine learning from scratch