epsilon
Say it “epsilon”
A tiny positive number — or, in models, a noise term.
Example
Avoids divide-by-zero in normalization.
In code
eps = 1e-8torch.nn.LayerNorm(d, eps=1e-5)Where you’ll see it in ML
Numerical stability in Adam/LayerNorm; ε-greedy in RL; noise in diffusion.