Intermediate28 min

t, Chi-Squared and F Distributions

The three sampling distributions behind inference: definitions, degrees of freedom and interrelations.

Assumes you know

The Normal Distribution

t, Chi-Squared and F Distributions

Intuition first

These three exist because of one practical problem: you almost never know the population standard deviation $\sigma$ .

The CLT says $(\bar{X} - \mu)/(\sigma/\sqrt{n})$ is standard normal. But you have to estimate $\sigma$ from the same sample, and that estimate is itself random. Substituting $s$ for $\sigma$ introduces extra variability, so the ratio has heavier tails than a normal. That heavier-tailed distribution is Student's t.

The chi-squared is what the sample variance follows, which is why it appears whenever variances or counts are being tested. And the F is a ratio of two chi-squareds, which is what you need when comparing two variances — or, equivalently, when comparing explained to unexplained variation, which is ANOVA.

All three are built from normal samples, and all three are relatives of the gamma.

Chi-squared

Sum of squared independent standard normals:

Q = Z_1^2 + Z_2^2 + \dots + Z_k^2 \sim \chi^2_k

\E[Q] = k, \qquad \Var(Q) = 2k

It is $\text{Gamma}(\alpha = k/2, \beta = 2)$ , which immediately gives those moments from the gamma formulas $\alpha\beta$ and $\alpha\beta^2$ .

Notation used in this lesson
Symbol	Meaning	Read aloud
k or ν	Degrees of freedom	nu
χ²ₖ	Chi-squared with k degrees of freedom	chi squared k
s²	Sample variance with divisor n − 1	s squared
tₙ₋₁	t distribution with n − 1 degrees of freedom	t n minus one
F(d₁, d₂)	F with numerator and denominator degrees of freedom	F d one d two

The sample variance

\frac{(n-1)s^2}{\sigma^2} \sim \chi^2_{n-1}, \qquad \text{where } s^2 = \frac{1}{n-1}\sum_{i=1}^n (X_i - \bar{X})^2

Why n − 1 degrees of freedom, and why s² divides by n − 1Advanced

The deviations $X_i - \bar{X}$ satisfy one exact linear constraint:

\sum_{i=1}^{n}(X_i - \bar{X}) = \sum X_i - n\bar{X} = n\bar{X} - n\bar{X} = 0

So knowing $n-1$ of the deviations determines the last one. There are only $n-1$ freely varying quantities, hence $n-1$ degrees of freedom.

That constraint is also why $s^2$ divides by $n-1$ rather than $n$ . Consider the naive estimator $\hat\sigma^2 = \frac{1}{n}\sum(X_i - \bar X)^2$ . Expand about the true mean $\mu$ :

\sum_i (X_i - \bar X)^2 = \sum_i (X_i - \mu)^2 - n(\bar X - \mu)^2

Take expectations term by term:

\E\left[\sum_i (X_i - \bar X)^2\right] = n\sigma^2 - n \cdot \frac{\sigma^2}{n} = (n-1)\sigma^2

using $\Var(\bar X) = \sigma^2/n$ . So dividing by $n$ gives

\E[\hat\sigma^2] = \frac{n-1}{n}\sigma^2 < \sigma^2

— biased downward, because deviations are measured from $\bar X$ , which sits closer to the data than $\mu$ does. Dividing by $n-1$ corrects exactly this, giving $\E[s^2] = \sigma^2$ . This is Bessel's correction.

Student's t

T = \frac{Z}{\sqrt{Q/k}} \sim t_k, \qquad Z \sim \mathcal{N}(0,1), \; Q \sim \chi^2_k, \; \text{independent}

In practice:

\frac{\bar{X} - \mu}{s/\sqrt{n}} \sim t_{n-1}

\E[T] = 0 \text{ for } k > 1, \qquad \Var(T) = \frac{k}{k-2} \text{ for } k > 2

$df$	$t$ critical (95%, two-sided)	Normal
5	2.571	1.960
10	2.228	1.960
30	2.042	1.960
100	1.984	1.960
$\infty$	1.960	1.960

The F distribution

Ratio of two independent chi-squareds, each divided by its degrees of freedom:

F = \frac{Q_1/d_1}{Q_2/d_2} \sim F(d_1, d_2)

Used to compare two variances, and — because ANOVA decomposes total variation into between-group and within-group sums of squares — to compare several group means at once.

Two useful facts:

F(1, d) = t_d^2, \qquad F(d_1, d_2) = \frac{1}{F(d_2, d_1)} \text{ in distribution}

Solved problem 1 · A t-interval, and why the multiplier matters

A sample of 12 measurements has $\bar{x} = 48.2$ and $s = 3.6$ . Construct a 95% confidence interval for $\mu$ , then compare with what the normal multiplier would give.

Step 1 — degrees of freedom and multiplier

df = n - 1 = 11

The two-sided 95% critical value is $t_{0.025, 11} = 2.201$ .

Step 2 — standard error

\text{SE} = \frac{s}{\sqrt{n}} = \frac{3.6}{\sqrt{12}} = \frac{3.6}{3.4641} = 1.0392

Step 3 — margin of error

\text{ME} = 2.201 \times 1.0392 = 2.2873

Step 4 — the interval

48.2 \pm 2.2873 = (45.913,\; 50.487)

Step 5 — compare with the normal multiplier

\text{ME}_z = 1.96 \times 1.0392 = 2.0369 \quad\Longrightarrow\quad (46.163,\; 50.237)

Width $4.074$ against the correct $4.575$ — about 11% too narrow. An interval claiming 95% coverage would actually cover roughly 92% of the time.

Answer

$(45.91,\; 50.49)$ using $t_{11} = 2.201$ . Using $z = 1.96$ understates the width by 11% and overstates confidence.

Solved problem 2 · A chi-squared interval for a variance

Using the same sample ( $n = 12$ , $s^2 = 12.96$ ), construct a 95% confidence interval for $\sigma^2$ .

Step 1 — the pivotal quantity

\frac{(n-1)s^2}{\sigma^2} \sim \chi^2_{11}

Step 2 — critical values, which are asymmetric

\chi^2_{0.975, 11} = 21.920 \quad \text{(upper tail 2.5\%)}

\chi^2_{0.025, 11} = 3.816 \quad \text{(lower tail 2.5\%)}

Unlike $t$ and $z$ , these are not symmetric — the chi-squared distribution is right-skewed.

Step 3 — invert the inequality

From $3.816 \leq \frac{(n-1)s^2}{\sigma^2} \leq 21.920$ , taking reciprocals reverses the order:

\frac{(n-1)s^2}{21.920} \;\leq\; \sigma^2 \;\leq\; \frac{(n-1)s^2}{3.816}

Step 4 — substitute

(n-1)s^2 = 11 \times 12.96 = 142.56

\frac{142.56}{21.920} = 6.504, \qquad \frac{142.56}{3.816} = 37.359

\sigma^2 \in (6.504,\; 37.359)

Taking square roots for the standard deviation:

\sigma \in (2.550,\; 6.112)

Step 5 — note how wide this is

The point estimate is $s = 3.6$ , and the interval runs from $2.55$ to $6.11$ — the upper end is 2.4 times the lower. Variance is estimated far less precisely than a mean at the same sample size, which is why "the variance is roughly stable" is a much weaker claim than it sounds with $n = 12$ .

Answer

$\sigma^2 \in (6.50,\; 37.36)$ , so $\sigma \in (2.55,\; 6.11)$ . Note the asymmetry: the interval extends much further above $s^2 = 12.96$ than below it.

python

import numpy as np
from scipy import stats

# t vs normal critical values.
print(f"{'df':>5}  {'t (95%)':>8}  {'z':>6}  {'% wider':>8}")
for df in (5, 10, 11, 30, 100, 1000):
    t = stats.t.ppf(0.975, df)
    print(f"{df:5d}  {t:8.4f}  {1.96:6.3f}  {100*(t/1.959964 - 1):7.1f}%")

# Worked example 1.
n, xbar, s = 12, 48.2, 3.6
se = s / np.sqrt(n)
tcrit = stats.t.ppf(0.975, n - 1)
print(f"\nt interval  ({xbar - tcrit*se:.3f}, {xbar + tcrit*se:.3f})  width {2*tcrit*se:.3f}")
print(f"z interval  ({xbar - 1.96*se:.3f}, {xbar + 1.96*se:.3f})  width {2*1.96*se:.3f}")

# Worked example 2: chi-squared interval for the variance.
lo = (n-1) * s**2 / stats.chi2.ppf(0.975, n-1)
hi = (n-1) * s**2 / stats.chi2.ppf(0.025, n-1)
print(f"\nvariance interval ({lo:.3f}, {hi:.3f})   sigma ({np.sqrt(lo):.3f}, {np.sqrt(hi):.3f})")

# Verify the sampling distributions by simulation.
rng = np.random.default_rng(0)
N, mu, sigma = 12, 50.0, 4.0
samples = rng.normal(mu, sigma, size=(200_000, N))
s2 = samples.var(axis=1, ddof=1)
chi = (N - 1) * s2 / sigma**2
t_stat = (samples.mean(axis=1) - mu) / (np.sqrt(s2) / np.sqrt(N))

print(f"\n(n-1)s²/σ²: mean {chi.mean():.3f} (theory {N-1})  var {chi.var():.3f} (theory {2*(N-1)})")
print(f"t statistic: var {t_stat.var():.4f}  theory k/(k-2) = {(N-1)/(N-3):.4f}")

# F(1, d) = t_d squared.
print(f"\nF(1,10) 95th pct {stats.f.ppf(0.95, 1, 10):.4f}  "
      f"t_10 97.5th pct squared {stats.t.ppf(0.975, 10)**2:.4f}")

The last two blocks are the checks worth running: the simulated $t$ -statistic variance matches $k/(k-2)$ rather than 1, and $F(1,d)$ really is $t_d^2$ .

Exercise 1

Why does a $t$ interval get wider as the sample gets smaller, beyond the $1/\sqrt{n}$ effect?

Show solution

Two separate effects compound.

The standard error grows. $\text{SE} = s/\sqrt{n}$ increases as $n$ falls — the familiar effect.

The multiplier also grows. With fewer observations, $s$ is a worse estimate of $\sigma$ . The $t$ distribution accounts for that extra uncertainty with heavier tails, so the critical value rises: $2.042$ at $df = 30$ , $2.228$ at $df = 10$ , $2.571$ at $df = 5$ , $12.706$ at $df = 1$ .

Going from $n = 30$ to $n = 5$ , the SE grows by a factor of $\sqrt{30/5} \approx 2.45$ and the multiplier by $2.571/2.042 \approx 1.26$ , so the interval widens by about $3.1\times$ rather than $2.45\times$ .

The intuition: with a small sample you are uncertain about the mean and uncertain about how uncertain you are. The $t$ distribution prices in the second-order uncertainty, which a normal interval ignores.

Exercise 2

A sample of 20 gives $s^2 = 45$ . Test $H_0: \sigma^2 = 30$ against $H_1: \sigma^2 > 30$ at the 5% level.

Show solution

Test statistic:

\chi^2 = \frac{(n-1)s^2}{\sigma_0^2} = \frac{19 \times 45}{30} = \frac{855}{30} = 28.5

Under $H_0$ this is $\chi^2_{19}$ . The one-sided 5% critical value is

\chi^2_{0.95, 19} = 30.144

Since $28.5 < 30.144$ , do not reject $H_0$ at the 5% level.

The p-value is $\Prob(\chi^2_{19} > 28.5) \approx 0.074$ — suggestive but above 0.05.

Worth noting how weak this test is. The sample variance is 50% larger than the hypothesised value and still not significant at 5%, because variance estimates are so noisy: $\Var(\chi^2_{19}) = 38$ , so a standard deviation of about 6.2 on a statistic with mean 19. Detecting moderate changes in variance requires substantially larger samples than detecting comparable changes in a mean.

Next: Transformations of Random Variables.