Learn machine learning from scratch

Start at chapter 1 and build up: what a model is, how it measures its own mistakes, and how it learns. Everything is worked out in plain Python, with the maths shown rather than assumed. Two shorter tracks hold reference notes on the maths and on modern architectures.

Track Progress: 0 Completed
0%
Maths for MLLEVEL 1 · BEGINNERModule 0 · Reading maths

Sets, ℝ and shapes: what W ∈ ℝ^(m×n) tells you

∈ means “is a member of”, ℝ is the real numbers, and ℝ with a raised size is the fastest way a paper tells you a tensor's shape.

10 min Prerequisites: Lesson 2 (scalars, vectors and matrices)
Symbols in this lesson∈ element ofℝ real numbersℝⁿ real vector / matrix space{ | } set-builder notation⊆ subset

1. The idea

A set is a collection of things, written with curly braces: {1, 2, 3}. The symbol ∈ means “is a member of”: 2 ∈ {1, 2, 3}, and 7 ∉ {1, 2, 3}. Small sets like these are rare in machine learning papers. What you'll see all the time is ∈ used to say what kind of thing a variable is.

ℝ is the set of all real numbers. ℝⁿ is the set of lists of n real numbers, and ℝ^(m×n) is the set of tables with m rows and n columns. So W ∈ ℝ^(768×3072) is the paper telling you W's shape: 768 rows, 3072 columns. In NumPy that is W.shape == (768, 3072). Whenever you meet a new symbol in a paper, look for the line that says which ℝ it lives in.

Set-builder notation describes a set by a rule: {x ∈ ℝ | x > 0} is “every real number x such that x is positive”. The bar reads “such that”. In Python it is a list comprehension: [x for x in xs if x > 0].

2. Try it: read a shape

Choose a matrix or a vector and set its size. The grid shows one square per number, with the maths and the NumPy shape underneath.

A grid of m rows by n columns of squares, one per number. Sliders set m and n, and the text below reads W ∈ ℝ^(m×n), W.shape == (m, n), and the count m times n. A vector option shows a single row of n squares, written x ∈ ℝⁿ with shape (n,).

3. The math

Membership
2∈{1,2,3}7∉{1,2,3}2 \in \{1, 2, 3\} \qquad 7 \notin \{1, 2, 3\}
∈ is “is in”; a slash through it is “is not in”.
Shapes
x∈RdW∈Rm×n\mathbf{x} \in \mathbb{R}^{d} \qquad W \in \mathbb{R}^{m \times n}
A vector of d real numbers; a matrix of m rows and n columns.
Set-builder notation
{ x∈R∣x>0 }\{\, x \in \mathbb{R} \mid x > 0 \,\}
Everything before the bar, filtered by the rule after it.
A batch of images
X∈RB×C×H×WX \in \mathbb{R}^{B \times C \times H \times W}
Batch size, channels, height, width. Note that W here means width, not weights: letters get reused, so read the paper's definitions.
Number systems inside each other
N⊂Z⊂R\mathbb{N} \subset \mathbb{Z} \subset \mathbb{R}
Counting numbers, then all integers, then all real numbers.

4. The code (python)

mm0_sets.py
import numpy as np

W = np.zeros((768, 3072))             # W in R^(768x3072)
print(W.shape, W.size)                 # (768, 3072) 2359296

x = np.array([0.5, -1.0, 2.0])         # x in R^3
print(x.shape)                         # (3,)

xs = [-2.0, 0.5, 3.0, -0.1]
print([v for v in xs if v > 0])        # [0.5, 3.0]

images = np.zeros((32, 3, 64, 64))     # a batch: B x C x H x W
print(images.ndim, images.shape)       # 4 (32, 3, 64, 64)

5. You’ll see this in

6. Check yourself

Answer all 4 questions correctly to complete the chapter · 0 / 4 done
Q1/4 A paper says W ∈ ℝ^(512×64). What is W.shape in NumPy?
Q2/4 How many numbers does 𝐱 ∈ ℝ¹⁰ hold?
Q3/4 What is the set {x ∈ ℝ | x² < 4}?
Q4/4 In X ∈ ℝ^(B×C×H×W) for a batch of images, what does C count?

Finished this chapter?

Weekly newsletter

What changed in ML architectures this week

New models worth knowing, benchmark moves, and one architecture explained in plain words. Free, once a week, unsubscribe anytime.