Beginner30 min

The Normal Distribution

The Gaussian density, standardisation, z-tables, and the 68–95–99.7 rule with worked lookups.

Assumes you know

Variance, Moments and Generating Functions

The Normal Distribution

Intuition first

The normal distribution is the bell curve, and it is everywhere for a specific reason rather than by convention: whenever a quantity is the sum of many small independent contributions, its distribution tends towards normal regardless of what the individual contributions look like. That is the Central Limit Theorem, and it is why measurement errors, heights, and sample means all end up approximately normal.

Two parameters fix it entirely. The mean $\mu$ says where the peak sits; the standard deviation $\sigma$ says how wide it is. Nothing else — no skew, no separate tail parameter. That is unusually restrictive for a distribution used so widely, and it is the source of most misuse: real data with heavy tails or asymmetry is routinely modelled as normal because the mathematics is convenient.

The practical skill is standardisation: converting any normal question into a question about the single standard normal $Z$ , which is what tables and software know about.

Definition

X \sim \mathcal{N}(\mu, \sigma^2), \qquad f(x) = \frac{1}{\sigma\sqrt{2\pi}}\exp\left(-\frac{(x-\mu)^2}{2\sigma^2}\right)

Notation used in this lesson
Symbol	Meaning	Read aloud
μ	Mean, median and mode — all coincide	mu
σ	Standard deviation, the distance to the inflection point	sigma
σ²	Variance — note N(μ, σ²) is parametrised by variance	sigma squared
Z	Standard normal, N(0, 1)	Z
Φ(z)	Standard normal CDF, P(Z ≤ z)	Phi of z
z_α	The value with α probability above it	z alpha

Key structural facts:

Symmetric about $\mu$ , so mean = median = mode.
Inflection points at $\mu \pm \sigma$ , which is how you read $\sigma$ off a plotted curve.
Support is all of $\mathbb{R}$ — a normal model always assigns non-zero probability to negative values, which is why it is wrong for strictly positive quantities like income or duration.

Standardisation

Z = \frac{X - \mu}{\sigma} \sim \mathcal{N}(0, 1)

Why standardisation preserves normalityAdvanced

Let $Z = (X - \mu)/\sigma$ . Compute its CDF directly:

F_Z(z) = \Prob(Z \leq z) = \Prob\!\left(\frac{X - \mu}{\sigma} \leq z\right) = \Prob\big(X \leq \mu + \sigma z\big) = F_X(\mu + \sigma z)

Differentiate with respect to $z$ , applying the chain rule:

f_Z(z) = \sigma\, f_X(\mu + \sigma z) = \sigma \cdot \frac{1}{\sigma\sqrt{2\pi}}\exp\left(-\frac{(\mu + \sigma z - \mu)^2}{2\sigma^2}\right)

= \frac{1}{\sqrt{2\pi}}\exp\left(-\frac{\sigma^2 z^2}{2\sigma^2}\right) = \frac{1}{\sqrt{2\pi}}e^{-z^2/2}

which is exactly the standard normal density. The $\sigma$ from the Jacobian cancels the $\sigma$ in the denominator, and the argument of the exponential simplifies to $-z^2/2$ .

This works because the normal family is a location–scale family: shifting and scaling a normal gives another normal. Most distributions are not closed under both operations, which is why this trick is specific rather than general.

The empirical rule

\Prob(\lvert X - \mu \rvert \leq \sigma) \approx 0.6827

\Prob(\lvert X - \mu \rvert \leq 2\sigma) \approx 0.9545

\Prob(\lvert X - \mu \rvert \leq 3\sigma) \approx 0.9973

The 68–95–99.7 rule. Nearly all probability lies within three standard deviations, which is why values beyond that are treated as outliers under a normal model.

Solved problem 1 · Standardising and reading the table

Adult male heights are approximately $\mathcal{N}(175, 8^2)$ cm. Find the probability that a randomly chosen man is (a) shorter than 185 cm, (b) between 165 and 185 cm, (c) taller than 190 cm. (d) What height is exceeded by only 5% of men?

Step 1 — part (a): standardise

z = \frac{185 - 175}{8} = \frac{10}{8} = 1.25

\Prob(X < 185) = \Prob(Z < 1.25) = \Phi(1.25) = 0.8944

Step 2 — part (b): two bounds

z_1 = \frac{165 - 175}{8} = \frac{-10}{8} = -1.25, \qquad z_2 = 1.25

\Prob(-1.25 < Z < 1.25) = \Phi(1.25) - \Phi(-1.25)

By symmetry $\Phi(-1.25) = 1 - \Phi(1.25) = 1 - 0.8944 = 0.1056$ :

= 0.8944 - 0.1056 = 0.7888

Step 3 — part (c): upper tail

z = \frac{190 - 175}{8} = \frac{15}{8} = 1.875

\Prob(X > 190) = 1 - \Phi(1.875) \approx 1 - 0.9696 = 0.0304

Step 4 — part (d): invert the problem

We need $h$ with $\Prob(X > h) = 0.05$ , so $\Prob(X \leq h) = 0.95$ . The standard normal value with 95% below it is $z_{0.05} = 1.645$ .

Un-standardise by rearranging $z = (h - \mu)/\sigma$ :

h = \mu + z\sigma = 175 + 1.645 \times 8 = 175 + 13.16 = 188.16 \text{ cm}

Step 5 — sanity checks

Part (b) covers $\mu \pm 1.25\sigma$ and gives 78.9%, sensibly between the 68.3% for $\pm 1\sigma$ and 95.5% for $\pm 2\sigma$ ✓.

Part (d): 188.16 cm is $1.645\sigma$ above the mean, and the answer must exceed the mean since we want an upper 5% cutoff ✓.

Answer

(a) $0.8944$ ; (b) $0.7888$ ; (c) $0.0304$ ; (d) $188.16$ cm.

Linear combinations stay normal

For independent $X \sim \mathcal{N}(\mu_X, \sigma_X^2)$ and $Y \sim \mathcal{N}(\mu_Y, \sigma_Y^2)$ :

aX + bY + c \sim \mathcal{N}\big(a\mu_X + b\mu_Y + c,\; a^2\sigma_X^2 + b^2\sigma_Y^2\big)

Proof via moment generating functionsAdvanced

The normal MGF is

M_X(t) = \exp\left(\mu t + \tfrac{1}{2}\sigma^2 t^2\right)

For independent variables, MGFs of sums multiply. Also $M_{aX}(t) = M_X(at)$ , since $\E[e^{t(aX)}] = \E[e^{(at)X}]$ . So

M_{aX + bY}(t) = M_X(at)\,M_Y(bt) = \exp\left(\mu_X at + \tfrac12\sigma_X^2a^2t^2\right)\exp\left(\mu_Y bt + \tfrac12\sigma_Y^2b^2t^2\right)

Combine the exponents:

= \exp\left((a\mu_X + b\mu_Y)t + \tfrac12(a^2\sigma_X^2 + b^2\sigma_Y^2)t^2\right)

This is precisely the normal MGF with mean $a\mu_X + b\mu_Y$ and variance $a^2\sigma_X^2 + b^2\sigma_Y^2$ . Since an MGF determines a distribution uniquely, the combination is normal.

Note the variances add with squared coefficients even for a difference: $X - Y$ has variance $\sigma_X^2 + \sigma_Y^2$ , not the difference. This closure property is special — sums of independent uniforms are not uniform, sums of exponentials are not exponential.

Normality is assumed far more often than it is checked

The normal distribution is symmetric with thin tails: the probability beyond $4\sigma$ is about $6 \times 10^{-5}$ . Real data frequently violates this.

Income, claim sizes, file sizes, city populations are right-skewed. A normal model gives non-zero probability to negative values, which is impossible.
Financial returns have excess kurtosis. Events a normal model calls once-in-10,000-years occur every few years.
Bounded quantities — proportions, percentages, scores out of 10 — cannot be normal, though a normal may approximate the middle well.

Check before assuming: a Q–Q plot, or the Shapiro–Wilk test at modest sample sizes. Where the data is positive and skewed, modelling $\log X$ as normal — a lognormal model — is often both better and equally tractable.

Standard normal reference values

Confidence	Two-sided $z$	One-sided $z$
90%	1.645	1.282
95%	1.960	1.645
99%	2.576	2.326

$z = 1.96$ for a two-sided 95% interval is worth memorising; it appears in every confidence interval and hypothesis test in the statistics module.

python

import numpy as np
from scipy import stats

X = stats.norm(loc=175, scale=8)

print(f"(a) P(X < 185)         {X.cdf(185):.4f}")
print(f"(b) P(165 < X < 185)   {X.cdf(185) - X.cdf(165):.4f}")
print(f"(c) P(X > 190)         {X.sf(190):.4f}")
print(f"(d) 95th percentile    {X.ppf(0.95):.2f} cm")

# The empirical rule, computed rather than quoted.
Z = stats.norm()
for k in (1, 2, 3):
    print(f"within {k}σ: {Z.cdf(k) - Z.cdf(-k):.4%}")

# Linear combinations stay normal — verify empirically.
rng = np.random.default_rng(0)
A = rng.normal(10, 3, 500_000)
B = rng.normal(4, 2, 500_000)
D = 2 * A - B
print(f"\n2A - B: mean {D.mean():.3f} (theory {2*10 - 4})  "
      f"var {D.var():.3f} (theory {4*9 + 4})")

# Critical values used throughout the statistics module.
for conf in (0.90, 0.95, 0.99):
    print(f"{conf:.0%} two-sided z = {Z.ppf(1 - (1-conf)/2):.4f}")

Exercise 1

Test scores are $\mathcal{N}(72, 12^2)$ . The top 10% receive a distinction. What is the cutoff?

Show solution

We need $c$ with $\Prob(X > c) = 0.10$ , equivalently $\Prob(X \leq c) = 0.90$ .

The standard normal 90th percentile is $z = 1.282$ .

c = \mu + z\sigma = 72 + 1.282 \times 12 = 72 + 15.384 = 87.38

So roughly 87.4 marks. Sanity check: the cutoff is above the mean, as it must be for a top 10% threshold, and $1.282\sigma$ is between $1\sigma$ (top 15.9%) and $1.645\sigma$ (top 5%) ✓.

Exercise 2

Two independent machines fill bottles: machine A with $\mathcal{N}(500, 4)$ ml, machine B with $\mathcal{N}(498, 9)$ ml. A crate holds one bottle from each. Find the distribution of the total, and the probability the total is below 995 ml.

Show solution

The variances are given as $\sigma^2 = 4$ and $9$ , so $\sigma_A = 2$ and $\sigma_B = 3$ .

Sum of independent normals is normal, with means and variances adding:

T = A + B \sim \mathcal{N}(500 + 498,\; 4 + 9) = \mathcal{N}(998, 13)

\sigma_T = \sqrt{13} \approx 3.6056

Now standardise:

z = \frac{995 - 998}{3.6056} = \frac{-3}{3.6056} \approx -0.8321

\Prob(T < 995) = \Phi(-0.8321) \approx 0.2027

About 20.3%.

A common error here is adding the standard deviations, $2 + 3 = 5$ , instead of the variances. That would give $z = -0.6$ and $0.2743$ — an error of 7 percentage points. Variances add; standard deviations do not.

Next: t, Chi-Squared and F Distributions, the three distributions derived from the normal that make statistical inference possible.