Advanced24 min

Sums of Random Variables and Convolution

Distribution of a sum via convolution and MGFs, and the closure properties of common families.

Assumes you know

Variance, Moments and Generating Functions

Sums of Random Variables and Convolution

Intuition first

Sums are everywhere: total claim cost, cumulative waiting time, aggregate demand, the numerator of every sample mean. So the distribution of $X + Y$ is one of the most frequently needed objects in probability.

Its mean and variance are easy — linearity and independence handle them. The distribution is harder, because $X + Y = s$ can happen in many ways: $X$ small and $Y$ large, or the reverse, or anything between. Summing over all those combinations is what convolution does.

There are two routes. Convolve directly, which is an integral (or a sum) and is often unpleasant. Or multiply moment generating functions, which turns the problem into algebra — and for the standard families lets you recognise the answer instantly. The second route is why MGFs earn their keep.

Convolution

Discrete:

\Prob(X + Y = s) = \sum_k \Prob(X = k)\,\Prob(Y = s - k)

Continuous:

f_{X+Y}(s) = \int_{-\infty}^{\infty} f_X(x)\,f_Y(s - x)\,dx

Both assume independence.

Solved problem 1 · Two dice, by discrete convolution

Two fair dice. Find the distribution of the sum by convolution.

Step 1 — set up

Each die has $\Prob(X = k) = 1/6$ for $k = 1, \dots, 6$ . For the sum $s$ :

\Prob(S = s) = \sum_{k} \Prob(X = k)\Prob(Y = s - k) = \sum_k \frac{1}{6}\cdot\frac{1}{6}

over all $k$ with both $1 \leq k \leq 6$ and $1 \leq s - k \leq 6$ .

Step 2 — determine the valid range of k

The two constraints give

\max(1,\, s-6) \leq k \leq \min(6,\, s-1)

The number of valid $k$ is the number of terms, each contributing $1/36$ .

Step 3 — count terms for a few values

s = 2: \; k \in \{1\} \quad\Longrightarrow\quad 1 \text{ term}, \; \Prob = 1/36

s = 5: \; k \in \{1,2,3,4\} \quad\Longrightarrow\quad 4 \text{ terms}, \; \Prob = 4/36

s = 7: \; k \in \{1,\dots,6\} \quad\Longrightarrow\quad 6 \text{ terms}, \; \Prob = 6/36

s = 10: \; k \in \{4,5,6\} \quad\Longrightarrow\quad 3 \text{ terms}, \; \Prob = 3/36

Step 4 — the general formula

\Prob(S = s) = \frac{6 - \lvert s - 7 \rvert}{36}, \qquad s = 2, \dots, 12

Step 5 — verify

\frac{1 + 2 + 3 + 4 + 5 + 6 + 5 + 4 + 3 + 2 + 1}{36} = \frac{36}{36} = 1 \;\checkmark

The triangular shape is the convolution of two rectangles — a general fact, and the reason sums of uniforms become bell-shaped surprisingly quickly.

Answer

$\Prob(S = s) = \dfrac{6 - \lvert s-7\rvert}{36}$ , peaking at $6/36$ for $s = 7$ .

Closure properties

Some families are closed under addition of independent members. These are worth memorising, because recognising one saves the whole convolution.

Sum of independent	Result	Condition
$\text{Bin}(n_1,p) + \text{Bin}(n_2,p)$	$\text{Bin}(n_1+n_2,\, p)$	Same $p$
$\text{Poisson}(\lambda_1) + \text{Poisson}(\lambda_2)$	$\text{Poisson}(\lambda_1+\lambda_2)$	Always
$\mathcal{N}(\mu_1,\sigma_1^2) + \mathcal{N}(\mu_2,\sigma_2^2)$	$\mathcal{N}(\mu_1+\mu_2,\, \sigma_1^2+\sigma_2^2)$	Always
$\text{Gamma}(\alpha_1,\beta) + \text{Gamma}(\alpha_2,\beta)$	$\text{Gamma}(\alpha_1+\alpha_2,\, \beta)$	Same $\beta$
$\chi^2_{k_1} + \chi^2_{k_2}$	$\chi^2_{k_1+k_2}$	Always
$\text{Exp}(\lambda) \times \alpha$ terms	$\text{Gamma}(\alpha,\, 1/\lambda)$	Same $\lambda$
$\text{Uniform} + \text{Uniform}$	Triangular, not uniform	—

Proving closure with MGFsAdvanced

For independent $X$ and $Y$ , $M_{X+Y}(t) = M_X(t)M_Y(t)$ . Since an MGF determines a distribution uniquely, recognising the product identifies the sum.

Poisson. With $M(t) = \exp(\lambda(e^t-1))$ :

\exp\!\big(\lambda_1(e^t-1)\big)\exp\!\big(\lambda_2(e^t-1)\big) = \exp\!\big((\lambda_1+\lambda_2)(e^t-1)\big)

which is Poisson $(\lambda_1+\lambda_2)$ .

Gamma. With $M(t) = (1-\beta t)^{-\alpha}$ :

(1-\beta t)^{-\alpha_1}(1-\beta t)^{-\alpha_2} = (1-\beta t)^{-(\alpha_1+\alpha_2)}

Closure holds because the bases match — which is exactly why the same scale $\beta$ is required. With different scales the product is not of gamma form, and the sum is not gamma.

Why binomial needs the same $p$ . With $M(t) = (1-p+pe^t)^n$ :

(1-p+pe^t)^{n_1}(1-p+pe^t)^{n_2} = (1-p+pe^t)^{n_1+n_2}

Again the bases must be identical. Different $p$ gives a product of two different bases, which is not a binomial MGF — the sum is then a Poisson-binomial distribution with no closed form.

Solved problem 2 · Sum of two independent uniforms

$X, Y \sim \text{Uniform}(0,1)$ independent. Find the density of $S = X + Y$ by convolution.

Step 1 — write the convolution integral

f_S(s) = \int_{-\infty}^{\infty} f_X(x)\,f_Y(s-x)\,dx

Both densities equal 1 on $(0,1)$ and 0 elsewhere, so the integrand is 1 exactly when

0 < x < 1 \quad\text{and}\quad 0 < s - x < 1

The second condition rearranges to $s - 1 < x < s$ .

Step 2 — intersect the constraints

\max(0,\, s-1) < x < \min(1,\, s)

The integral of 1 over this interval is simply its length.

Step 3 — case s in (0,1)

Here $\max(0, s-1) = 0$ and $\min(1,s) = s$ , so the interval is $(0, s)$ with length $s$ :

f_S(s) = s

Step 4 — case s in [1,2)

Here $\max(0,s-1) = s-1$ and $\min(1,s) = 1$ , so the interval is $(s-1, 1)$ with length $2 - s$ :

f_S(s) = 2 - s

Step 5 — assemble and check

f_S(s) = \begin{cases} s & 0 \leq s \leq 1 \\ 2 - s & 1 < s \leq 2 \\ 0 & \text{otherwise}\end{cases}

A triangle peaking at $s = 1$ . Normalisation is the area of a triangle with base 2 and height 1:

\tfrac12 \times 2 \times 1 = 1 \;\checkmark

Step 6 — moments, as an independent check

\E[S] = \E[X] + \E[Y] = 0.5 + 0.5 = 1 \;\checkmark \text{(the peak, by symmetry)}

\Var(S) = \tfrac{1}{12} + \tfrac{1}{12} = \tfrac{1}{6} \approx 0.1667

Answer

Triangular on $[0,2]$ : $f_S(s) = s$ for $s \leq 1$ and $2-s$ for $s > 1$ . Two flat distributions convolve into a peaked one — the first step of the CLT visible in one calculation.

Sums of dependent variables

Convolution requires independence. Without it, the mean still adds by linearity, but the variance needs the covariance term and the distribution requires the full joint:

\E[X+Y] = \E[X] + \E[Y] \qquad \text{(always)}

\Var(X+Y) = \Var(X) + \Var(Y) + 2\Cov(X,Y)

A random number of terms

For $S = \sum_{i=1}^{N} X_i$ with $N$ random and independent of the $X_i$ — a compound distribution — condition on $N$ :

\E[S] = \E[N]\,\E[X], \qquad \Var(S) = \E[N]\Var(X) + \Var(N)\big(\E[X]\big)^2

Deriving the compound varianceAdvanced

Apply the variance decomposition from the conditional expectation lesson. Given $N = n$ , the sum of $n$ i.i.d. terms has

\E[S \mid N] = N\,\E[X], \qquad \Var(S \mid N) = N\Var(X)

Now:

\E\big[\Var(S\mid N)\big] = \E[N]\Var(X)

\Var\big(\E[S\mid N]\big) = \Var\big(N\E[X]\big) = \big(\E[X]\big)^2\Var(N)

Adding gives the stated result. The first term is variability in the individual claim sizes, the second variability in how many claims arrive.

This is the foundation of insurance risk modelling: total annual claims is a compound distribution, usually with $N$ Poisson (giving $\Var(N) = \E[N]$ ) and $X$ gamma or lognormal.

python

import numpy as np
from scipy import stats

rng = np.random.default_rng(0)

# Two dice: convolution against simulation.
pmf = np.array([(6 - abs(s - 7)) / 36 for s in range(2, 13)])
rolls = rng.integers(1, 7, (500_000, 2)).sum(axis=1)
print("s   theory   empirical")
for i, s in enumerate(range(2, 13)):
    print(f"{s:2d}  {pmf[i]:.5f}  {np.mean(rolls == s):.5f}")

# Sum of two uniforms is triangular.
S = rng.random(500_000) + rng.random(500_000)
print(f"\nsum of 2 uniforms: mean {S.mean():.4f} (1.0)  var {S.var():.4f} ({1/6:.4f})")
print(f"density near s=1 is ~2x density near s=0.5:")
for s in (0.25, 0.5, 1.0):
    band = np.mean(np.abs(S - s) < 0.01) / 0.02
    print(f"  s={s}: empirical {band:.3f}   theory {s if s <= 1 else 2-s:.3f}")

# Closure: same p works, different p does not.
a = rng.binomial(10, 0.5, 400_000) + rng.binomial(10, 0.5, 400_000)
b = rng.binomial(10, 0.3, 400_000) + rng.binomial(10, 0.7, 400_000)
print(f"\nBin(10,.5)+Bin(10,.5): var {a.var():.3f}  Bin(20,.5) var {20*0.25:.3f}  <- matches")
print(f"Bin(10,.3)+Bin(10,.7): var {b.var():.3f}  Bin(20,.5) var {20*0.25:.3f}  <- does NOT")

# Compound: total claims with Poisson count and gamma severity.
N = rng.poisson(12, 400_000)
total = np.array([rng.gamma(2, 500, n).sum() if n else 0.0 for n in N[:20_000]])
EX, VarX, EN = 2*500, 2*500**2, 12
print(f"\ncompound: mean {total.mean():,.0f}  theory {EN*EX:,.0f}")
print(f"          var  {total.var():,.0f}  theory {EN*VarX + EN*EX**2:,.0f}")

Exercise 1

$X \sim \text{Poisson}(3)$ and $Y \sim \text{Poisson}(5)$ , independent. Find $\Prob(X + Y = 2)$ .

Show solution

By the closure property, $X + Y \sim \text{Poisson}(3 + 5) = \text{Poisson}(8)$ .

\Prob(X+Y = 2) = \frac{e^{-8}\,8^2}{2!} = \frac{e^{-8}\times 64}{2} = 32\,e^{-8}

e^{-8} = 0.00033546

= 32 \times 0.00033546 \approx 0.010735

About 1.07%.

Verifying by direct convolution — three terms, since $X$ and $Y$ must sum to 2:

\Prob(X=0)\Prob(Y=2) + \Prob(X=1)\Prob(Y=1) + \Prob(X=2)\Prob(Y=0)

= e^{-3}\cdot\frac{25e^{-5}}{2} + 3e^{-3}\cdot 5e^{-5} + \frac{9e^{-3}}{2}\cdot e^{-5}

= e^{-8}\left(12.5 + 15 + 4.5\right) = 32e^{-8} \;\checkmark

The closure property replaced three terms with one — and would replace 101 terms with one if we wanted $\Prob(X+Y = 100)$ .

Exercise 2

An insurer expects 20 claims per year (Poisson) with claim sizes averaging £4,000 and standard deviation £6,000. Find the mean and standard deviation of annual total claims.

Show solution

A compound Poisson. With $\E[N] = \Var(N) = 20$ (Poisson), $\E[X] = 4000$ , $\Var(X) = 6000^2 = 36{,}000{,}000$ :

Mean:

\E[S] = \E[N]\E[X] = 20 \times 4000 = £80{,}000

Variance:

\Var(S) = \E[N]\Var(X) + \Var(N)\big(\E[X]\big)^2

= 20(36{,}000{,}000) + 20(16{,}000{,}000)

= 720{,}000{,}000 + 320{,}000{,}000 = 1{,}040{,}000{,}000

\text{SD}(S) = \sqrt{1.04\times10^9} \approx £32{,}249

Two things worth noting. The coefficient of variation is $32{,}249/80{,}000 \approx 0.40$ , so year-to-year totals swing substantially — a bad year of £145,000 is under a 2-standard-deviation event.

And the decomposition is informative: 69% of the variance comes from uncertainty in claim sizes and 31% from uncertainty in the number of claims. Reducing severity variability — through policy limits or reinsurance — would cut total risk more than stabilising claim frequency.

For a Poisson count the formula simplifies to $\Var(S) = \E[N]\,\E[X^2]$ , which is a useful check: $20 \times (36{,}000{,}000 + 16{,}000{,}000) = 20 \times 52{,}000{,}000 = 1.04\times10^9$ ✓.

Next: The Multivariate Normal Distribution, the last distribution in the module.