Transpose, identity, inverse and determinant

1. The idea

Four operations come up all the time. The transpose Aᵀ flips a matrix over its diagonal: rows become columns, so an m × n matrix becomes n × m. It is the usual fix for a shape mismatch, and it is why attention writes QKᵀ: every query dotted with every key.

The identity matrix I has 1s on the diagonal and 0s everywhere else; multiplying by it changes nothing, like multiplying a number by 1. The inverse A⁻¹ undoes A, so A⁻¹A = I. In the widget, apply A and then A⁻¹ and the grid comes back where it started.

The determinant says how much A scales area (volume, in more dimensions). det A = 2 doubles every area; a negative determinant also flips the plane over. det A = 0 means A squashes the plane onto a line, and nothing can un-squash it: A has no inverse. Code rarely computes an inverse itself. To find 𝐱 with A𝐱 = 𝐛, np.linalg.solve(A, b) is faster and more accurate than inv(A) @ b.

2. Try it: undo a matrix

Set a matrix, then press “Apply A, then A⁻¹”. Now try Squash flat, whose determinant is 0, and press it again.

3. The math

4. The code (python)

import numpy as np

A = np.array([[3.0, 1.0],
              [2.0, 1.0]])
print(A.T.tolist())                       # [[3.0, 2.0], [1.0, 1.0]]
print(round(np.linalg.det(A), 6))         # 1.0
A_inv = np.linalg.inv(A)
print(A_inv.round(6).tolist())            # [[1.0, -1.0], [-2.0, 3.0]]
print((A_inv @ A).round(6).tolist())      # [[1.0, 0.0], [0.0, 1.0]]

b = np.array([4.0, 3.0])
print(np.linalg.solve(A, b))              # [1. 1.]

S = np.array([[1.0, 2.0],
              [0.5, 1.0]])                # det = 1*1 - 2*0.5 = 0
print(round(abs(np.linalg.det(S)), 6))    # 0.0

5. You’ll see this in

• Modern Architectures: attention QKᵀ, every query against every key
• Linear regression in the catalog its exact solution uses (XᵀX)⁻¹
• Reference notes: linear algebra

6. Check yourself