Probability, and probability given something

1. The idea

A probability is a number between 0 and 1 that says how likely an event is: P(A) = 0.3 means A happens about 30% of the time. A random variable is a quantity whose value depends on chance, like tomorrow's temperature or the class of the next image, and P(X = x) is the probability that it takes the value x.

The bar in P(A | B) reads “given”: the probability of A once you know B happened. You get it by counting only inside B: P(A | B) = P(A and B) / P(B). Knowing B can change the odds a lot (umbrellas are far more common when it rains) or not at all, in which case A and B are independent and P(A | B) = P(A).

Almost every model in machine learning is a conditional probability. A classifier outputs P(class | image); a language model outputs P(next token | the tokens so far). A lowercase p(x) usually means a density or a whole distribution rather than a single probability, which lesson 25 comes back to.

2. Try it: count inside B

A hundred people: blue ones carry an umbrella (A), red rings are standing in the rain (B). Switch to counting only where it rains and compare P(A | B) with P(A). Then set the link to 0 and see when they agree.

3. The math

4. The code (python)

import numpy as np

rng = np.random.default_rng(0)
n = 100_000
rain = rng.random(n) < 0.25                                   # P(B) = 0.25
umbrella = np.where(rain, rng.random(n) < 0.8, rng.random(n) < 0.1)

print(round(umbrella.mean(), 3))                              # 0.276
print(round(umbrella[rain].mean(), 3))                        # 0.801
print(round((umbrella & rain).mean() / rain.mean(), 3))       # 0.801

5. You’ll see this in

• Core Course chapter 9: softmax a classifier's P(class | 𝐱)
• Reference notes: probability distributions and maximum likelihood

6. Check yourself