Functions and arrows: f: ℝⁿ → ℝ, ↦ and f ∘ g

1. The idea

f: ℝⁿ → ℝ is a function's type signature: f takes a list of n numbers and gives back one number. A loss function has exactly this type: it takes all of a model's parameters and returns one number to make small. The arrow → is about what kind of thing goes in and comes out; it says nothing about what f does.

The other arrow, ↦ (“maps to”), says what happens to a single input. x ↦ x² is the function that squares its input, the same thing as Python's lambda x: x**2.

f ∘ g, read “f after g”, means: apply g first, then apply f to the result, so (f ∘ g)(x) = f(g(x)). Read it right to left. A neural network is exactly this: layer 1, then layer 2, then layer 3, written f₃ ∘ f₂ ∘ f₁. Order matters: g(f(x)) is usually a different function, as the widget shows.

2. Try it: chain two functions

g(x) = 2x − 1 is a tiny linear layer and f(x) = max(0, x) is ReLU. Feed in x and follow it through both, then swap the order and watch the graph change.

3. The math

4. The code (python)

def g(x):                     # g: R -> R, x |-> 2x - 1 (a tiny linear layer)
    return 2 * x - 1

def f(x):                     # f: R -> R, x |-> max(0, x) (ReLU)
    return max(0.0, x)

def compose(outer, inner):    # outer after inner
    return lambda x: outer(inner(x))

f_after_g = compose(f, g)
g_after_f = compose(g, f)
print(f_after_g(0.0), g_after_f(0.0))               # 0.0 -1.0
print([f_after_g(x) for x in (-1.0, 0.5, 2.0)])     # [0.0, 0.0, 3.0]

5. You’ll see this in

• Core Course chapter 8: XOR two layers composed solve what one cannot
• Core Course chapter 5: autograd the chain rule runs backwards through a composition

6. Check yourself