Maths for ML / Notation

\lambda

lambda

Greek letters and conventionsBeginner

Say it “lambda”

Conventionally a regularization strength — or an eigenvalue in linear algebra.

Example

L + \lambda\lVert\mathbf{w}\rVert_2^2

Weight decay: penalize large weights.

In code

weight_decay = 1e-2

torch.optim.AdamW(params, weight_decay=1e-2)

Where you’ll see it in ML

Ridge/L2, weight decay, Lagrange multipliers, eigenvalues.

Don’t confuse it with

$\mathbf{A}\mathbf{v} = \lambda\mathbf{v}$ eigenvalue / eigenvector An eigenvector is a direction A only stretches (doesn't rotate); λ is how much it stretches.

Lessons that use it

Norms: how big is a vector?

← All 94 symbols