Maths for ML / Notation

\eta

eta (learning rate)

Greek letters and conventionsBeginner

Say it “eta”

Conventionally the learning rate — how big a step to take along the gradient.

Example

\theta \leftarrow \theta - \eta\,\nabla L

In code

lr = 1e-3

torch.optim.SGD(params, lr=1e-3)

Where you’ll see it in ML

Every optimizer has one; schedules change it over training.

Don’t confuse it with

$\nabla_{\theta} L$ nabla / gradient The vector of all partial derivatives. It points in the direction the function increases fastest.
$\alpha$ alpha Generic coefficient. Often a learning rate, a mixing weight, or attention weights.

Lessons that use it

← All 94 symbols