Maths for ML / Notation

\mathbf{a}\otimes\mathbf{b}

tensor / outer / Kronecker product

Linear algebraAdvanced

Say it “a tensor b”

Combines two vectors into a matrix of all pairwise products (outer product), or two matrices into a bigger block matrix (Kronecker).

Example

\mathbf{a}\otimes\mathbf{b} = \mathbf{a}\mathbf{b}^{\top}

For vectors, this is the outer product.

In code

np.outer(a, b)  # vectors: outer product
np.kron(A, B)   # matrices: Kronecker product

torch.outer(a, b)

Where you’ll see it in ML

Low-rank updates, second-order optimizers (K-FAC, Shampoo).

Don’t confuse it with

$\mathbf{a}\odot\mathbf{b}$ Hadamard (element-wise) product Multiply matching entries and keep them as a vector/matrix — no summing.
$\mathbf{a}\cdot\mathbf{b}$ dot product Multiply matching entries and add them up. Gives one number that's large when the vectors point the same way.

← All 94 symbols