Maths for ML / Notation

x\sqrt{x}

square root

Arithmetic and relationsBeginner

Say it “root of”

The number that, multiplied by itself, gives x.

Example

dk\sqrt{d_k}

Attention divides scores by this to keep them from growing with dimension.

In code

np.sqrt(x)
torch.sqrt(x)

Where you’ll see it in ML

Scaling in attention, standard deviation (√variance), L2 norms.

Lessons that use it

← All 94 symbols