Events, Independence and Mutual Exclusivity

Intuition first

Two words get confused more than any other pair in probability, and the confusion is costly because they mean almost opposite things.

Mutually exclusive means the two events cannot happen together. A single card cannot be both a heart and a spade.

Independent means knowing one happened tells you nothing about whether the other happened. The first coin toss landing heads tells you nothing about the second toss.

Notice these are not two flavours of the same idea. Mutually exclusive events are maximally informative about each other: if I tell you the card is a spade, you now know with certainty it is not a heart. That is the opposite of telling you nothing.

Mutually exclusive events

Events $A$ and $B$ are mutually exclusive (or disjoint) when they share no outcomes:

$$A \cap B = \varnothing \quad\Longrightarrow\quad \Prob(A \cap B) = 0$$

This is what licenses plain addition, from Axiom 3:

$$\Prob(A \cup B) = \Prob(A) + \Prob(B) \qquad \text{(only if disjoint)}$$

For one die roll, $A = \{2,4,6\}$ (even) and $B = \{1,3,5\}$ (odd) are mutually exclusive. $A = \{2,4,6\}$ and $C = \{4,5,6\}$ are not — they share $4$ and $6$.

Independent events

Events $A$ and $B$ are independent when the probability of both occurring factorises:

$$\Prob(A \cap B) = \Prob(A)\,\Prob(B)$$

This is a definition, not something you can eyeball. You verify independence by checking whether the product rule actually holds numerically.

The two are almost never the same thing

Suppose $\Prob(A) > 0$ and $\Prob(B) > 0$, and $A$ and $B$ are mutually exclusive. Then:

$$\Prob(A \cap B) = 0 \qquad\text{but}\qquad \Prob(A)\Prob(B) > 0$$

These cannot be equal, so mutually exclusive events with non-zero probability are never independent.

The only case where both hold at onceAdvanced

Both conditions require

$$\Prob(A \cap B) = 0 \quad\text{and}\quad \Prob(A \cap B) = \Prob(A)\Prob(B)$$

Together these force $\Prob(A)\Prob(B) = 0$, so at least one of the two events has probability zero. That is the sole degenerate case: an event that essentially never happens is trivially independent of everything, including events it cannot co-occur with.

So in every interesting case, the two properties are mutually exclusive properties — which is a pleasing irony.

Independence and conditional probability

Independence has an equivalent formulation that matches the intuition more directly. If $\Prob(B) > 0$:

$$A \ind B \quad\Longleftrightarrow\quad \Prob(A \mid B) = \Prob(A)$$

In words: learning that $B$ happened leaves your assessment of $A$ unchanged.

Why the two definitions agreeAdvanced

The conditional probability is defined as

$$\Prob(A \mid B) = \frac{\Prob(A \cap B)}{\Prob(B)}, \qquad \Prob(B) > 0$$

If $A$ and $B$ are independent, substitute $\Prob(A \cap B) = \Prob(A)\Prob(B)$:

$$\Prob(A \mid B) = \frac{\Prob(A)\Prob(B)}{\Prob(B)} = \Prob(A)$$

Conversely, if $\Prob(A \mid B) = \Prob(A)$, multiply both sides by $\Prob(B)$:

$$\Prob(A \cap B) = \Prob(A)\Prob(B)$$

So each statement implies the other. The product form is usually preferred as the definition because it is symmetric in $A$ and $B$ and does not require $\Prob(B) > 0$.

Solved problem 1 · Testing independence numerically

A single card is drawn from a standard 52-card deck. Let $K$ be "the card is a king" and $H$ be "the card is a heart". Are $K$ and $H$ independent? Are they mutually exclusive?

Step 1 — compute the individual probabilities

There are 4 kings and 13 hearts in 52 cards:

$$\Prob(K) = \frac{4}{52} = \frac{1}{13}, \qquad \Prob(H) = \frac{13}{52} = \frac{1}{4}$$

Step 2 — compute the joint probability directly

Exactly one card is both a king and a heart:

$$\Prob(K \cap H) = \frac{1}{52}$$

Since this is not zero, $K$ and $H$ are not mutually exclusive.

Step 3 — test the product rule

$$\Prob(K)\,\Prob(H) = \frac{1}{13} \times \frac{1}{4} = \frac{1}{52}$$

Compare with the actual joint probability:

$$\Prob(K \cap H) = \frac{1}{52} = \Prob(K)\Prob(H) \;\checkmark$$

The product rule holds exactly, so the events are independent.

Step 4 — sanity-check with the conditional form

$$\Prob(K \mid H) = \frac{\Prob(K \cap H)}{\Prob(H)} = \frac{1/52}{1/4} = \frac{4}{52} = \frac{1}{13} = \Prob(K)$$

Knowing the card is a heart does not change the chance it is a king — among the 13 hearts, exactly one is a king, the same $1/13$ rate as the full deck.

Answer

$K$ and $H$ are independent but not mutually exclusive. A deck is structured as a 4 × 13 grid, and suit tells you nothing about rank.

Solved problem 2 · Independence is destroyed by a small change

Remove the two red jacks from a standard deck, leaving 50 cards. Are $K$ (king) and $H$ (heart) still independent?

Step 1 — recount

Removing two jacks removes no kings and one heart:

$$\Prob(K) = \frac{4}{50}, \qquad \Prob(H) = \frac{12}{50}, \qquad \Prob(K \cap H) = \frac{1}{50}$$

Step 2 — test the product rule

$$\Prob(K)\Prob(H) = \frac{4}{50} \times \frac{12}{50} = \frac{48}{2500} = 0.0192$$$$\Prob(K \cap H) = \frac{1}{50} = 0.02$$

$0.02 \neq 0.0192$, so independence fails.

Step 3 — interpret the failure

$$\Prob(K \mid H) = \frac{1/50}{12/50} = \frac{1}{12} \approx 0.0833$$$$\Prob(K) = \frac{4}{50} = 0.08$$

Being told the card is a heart now nudges the chance of a king upward, because the hearts suit lost a non-king card while the deck as a whole lost two.

Answer

No. Independence is a numerical coincidence of the structure, not a property of the words "rank" and "suit" — perturbing the deck breaks it.

Independence of several events

For three events, pairwise independence is not enough. Full (mutual) independence requires the product rule for every subset:

$$\Prob(A \cap B) = \Prob(A)\Prob(B)$$ $$\Prob(A \cap C) = \Prob(A)\Prob(C)$$ $$\Prob(B \cap C) = \Prob(B)\Prob(C)$$ $$\Prob(A \cap B \cap C) = \Prob(A)\Prob(B)\Prob(C)$$

All four must hold. The first three can be satisfied while the fourth fails.

A counterexample: pairwise but not mutually independentAdvanced

Toss two fair coins. Define:

• $A$ = first coin is heads
• $B$ = second coin is heads
• $C$ = the two coins match (both heads or both tails)

Sample space $\Omega = \{HH, HT, TH, TT\}$, each of probability $1/4$.

$$\Prob(A) = \Prob(B) = \Prob(C) = \tfrac{1}{2}$$

Check the pairs:

$$\Prob(A \cap B) = \Prob(\{HH\}) = \tfrac14 = \tfrac12 \cdot \tfrac12 \;\checkmark$$$$\Prob(A \cap C) = \Prob(\{HH\}) = \tfrac14 = \tfrac12 \cdot \tfrac12 \;\checkmark$$$$\Prob(B \cap C) = \Prob(\{HH\}) = \tfrac14 = \tfrac12 \cdot \tfrac12 \;\checkmark$$

All three pairs are independent. Now the triple:

$$\Prob(A \cap B \cap C) = \Prob(\{HH\}) = \tfrac14$$$$\Prob(A)\Prob(B)\Prob(C) = \tfrac18$$

$\tfrac14 \neq \tfrac18$, so the three events are not mutually independent.

The reason is structural: any two of $A$, $B$, $C$ determine the third. If the first coin is heads and the coins match, the second coin must be heads — no randomness is left.

Independent versus identically distributed

You will constantly meet the phrase i.i.d. — independent and identically distributed. These are two separate claims:

• Independent: each observation carries no information about the others.
• Identically distributed: each observation is drawn from the same distribution.

Either can fail alone. Time series data is typically identically distributed but not independent (today correlates with yesterday). A dataset assembled from two different populations may have independent draws that are not identically distributed.

Exercise 1

A fair die is rolled once. Let $A$ = "the result is even" and $B$ = "the result is greater than 4". Determine whether $A$ and $B$ are independent, mutually exclusive, both, or neither.

Show solutionHide solution

$A = \{2,4,6\}$, $B = \{5,6\}$, $A \cap B = \{6\}$.

$$\Prob(A) = \tfrac{3}{6} = \tfrac12, \qquad \Prob(B) = \tfrac{2}{6} = \tfrac13, \qquad \Prob(A \cap B) = \tfrac16$$

Not mutually exclusive, since $A \cap B \neq \varnothing$.

Test the product rule:

$$\Prob(A)\Prob(B) = \tfrac12 \cdot \tfrac13 = \tfrac16 = \Prob(A \cap B) \;\checkmark$$

So they are independent. This surprises most people — the events feel related — which is exactly why independence must be checked numerically rather than judged by intuition.

Exercise 2

Two events satisfy $\Prob(A) = 0.4$, $\Prob(B) = 0.5$ and $\Prob(A \cup B) = 0.7$. Are they independent?

Show solutionHide solution

First recover the joint probability from inclusion–exclusion:

$$\Prob(A \cap B) = \Prob(A) + \Prob(B) - \Prob(A \cup B) = 0.4 + 0.5 - 0.7 = 0.2$$

Now test:

$$\Prob(A)\Prob(B) = 0.4 \times 0.5 = 0.2 = \Prob(A \cap B) \;\checkmark$$

Independent.

Exercise 3

A student argues: "$A$ and $B$ are independent, so $\Prob(A \cup B) = \Prob(A) + \Prob(B)$." Give a counterexample and state the correct formula.

Show solutionHide solution

Take a fair coin tossed twice, $A$ = first is heads, $B$ = second is heads. These are independent with $\Prob(A) = \Prob(B) = \tfrac12$. The student's formula gives $\Prob(A \cup B) = 1$, claiming at least one head is certain — but $TT$ occurs with probability $\tfrac14$.

The correct calculation uses inclusion–exclusion, with the overlap supplied by independence:

$$\Prob(A \cup B) = \Prob(A) + \Prob(B) - \Prob(A)\Prob(B) = \tfrac12 + \tfrac12 - \tfrac14 = \tfrac34$$

Independence tells you how to compute the intersection; it never removes the need to subtract it.

---

Next: Joint, Marginal and Conditional Probability, where conditional probability gets a proper treatment and we start reading probabilities off two-way tables.