Beginner26 min

PMF, PDF and CDF

The three ways to describe a distribution, how to convert between them, and why densities are not probabilities.

Assumes you know

Random Variables

PMF, PDF and CDF

Intuition first

There are three standard ways to describe how a random variable spreads its probability, and they carry the same information in different shapes.

For a discrete variable, list the probability of each value. That is the PMF.

For a continuous variable, listing values is hopeless — each has probability zero — so instead describe probability per unit length. That is the PDF, and it is a density, not a probability.

The third option works for both: report the accumulated probability up to each point, $\Prob(X \leq x)$ . That is the CDF, and because it never decreases and always runs from 0 to 1, it is the most convenient object for computing interval probabilities and quantiles.

Notation used in this lesson
Symbol	Meaning	Read aloud
p(x) or pₓ(x)	PMF — probability that X equals x (discrete)	p of x
f(x) or fₓ(x)	PDF — probability density at x (continuous)	f of x
F(x) or Fₓ(x)	CDF — P(X ≤ x), for either kind	big F of x
Q(p) or F⁻¹(p)	Quantile function — the inverse of the CDF	quantile

The probability mass function

For a discrete $X$ ,

p(x) = \Prob(X = x)

with two requirements:

p(x) \geq 0 \quad \text{for all } x, \qquad \sum_{x \in S_X} p(x) = 1

The probability density function

For a continuous $X$ , probability is obtained by integrating:

\Prob(a \leq X \leq b) = \int_a^b f(x)\, dx

with the analogous requirements:

f(x) \geq 0, \qquad \int_{-\infty}^{\infty} f(x)\, dx = 1

The cumulative distribution function

F(x) = \Prob(X \leq x)

Defined for every random variable, and always has these properties:

Non-decreasing — accumulating more can never reduce the total.
Limits — $F(-\infty) = 0$ and $F(+\infty) = 1$ .
Right-continuous — jumps at discrete atoms, with the value at the jump included.

Discrete versus continuous. The discrete CDF is a staircase whose jump heights are the PMF values; the continuous CDF is smooth, and its slope is the density.

Converting between them

From	To	Discrete	Continuous
PMF/PDF	CDF	$F(x) = \sum_{t \leq x} p(t)$	$F(x) = \int_{-\infty}^{x} f(t)\,dt$
CDF	PMF/PDF	$p(x) = F(x) - F(x^-)$	$f(x) = \dfrac{dF}{dx}$

Interval probabilities from the CDF

\Prob(a < X \leq b) = F(b) - F(a)

For continuous variables the endpoints are irrelevant. For discrete ones they matter:

\Prob(a \leq X \leq b) = F(b) - F(a^-) = F(b) - F(a) + p(a)

Solved problem 1 · A discrete variable, all three functions

$X$ has PMF $p(1) = 0.2$ , $p(2) = 0.5$ , $p(4) = 0.3$ . Find the CDF, then $\Prob(X \leq 2)$ , $\Prob(X < 2)$ , $\Prob(2 \leq X \leq 4)$ and $\Prob(X \geq 2)$ .

Step 1 — check it is a valid PMF

0.2 + 0.5 + 0.3 = 1.0 \;\checkmark, \quad \text{all values} \geq 0 \;\checkmark

Step 2 — accumulate to get the CDF

F(x) = \begin{cases} 0 & x < 1 \\ 0.2 & 1 \leq x < 2 \\ 0.7 & 2 \leq x < 4 \\ 1.0 & x \geq 4 \end{cases}

Note $F$ is constant on $[2, 4)$ because no probability mass lies strictly between 2 and 4.

Step 3 — the four probabilities

\Prob(X \leq 2) = F(2) = 0.7

\Prob(X < 2) = F(2) - p(2) = 0.7 - 0.5 = 0.2

\Prob(2 \leq X \leq 4) = F(4) - F(2) + p(2) = 1.0 - 0.7 + 0.5 = 0.8

\Prob(X \geq 2) = 1 - \Prob(X < 2) = 1 - 0.2 = 0.8

Step 4 — sanity check by direct summation

\Prob(2 \leq X \leq 4) = p(2) + p(4) = 0.5 + 0.3 = 0.8 \;\checkmark

\Prob(X < 2) = p(1) = 0.2 \;\checkmark

Answer

$\Prob(X \leq 2) = 0.7$ ; $\Prob(X < 2) = 0.2$ ; $\Prob(2 \leq X \leq 4) = 0.8$ ; $\Prob(X \geq 2) = 0.8$ .

Notice $\Prob(X \leq 2) \neq \Prob(X < 2)$ — they differ by exactly $p(2) = 0.5$ , which is why strictness of inequalities cannot be ignored for discrete variables.

Solved problem 2 · A continuous variable: normalising, CDF, and a median

$X$ has density $f(x) = cx^2$ on $[0, 3]$ and zero elsewhere. Find $c$ , the CDF, $\Prob(1 \leq X \leq 2)$ , and the median.

Step 1 — find c by normalising

\int_0^3 cx^2\, dx = c\left[\frac{x^3}{3}\right]_0^3 = c \cdot \frac{27}{3} = 9c

Setting this equal to 1:

9c = 1 \quad\Longrightarrow\quad c = \frac{1}{9}

So $f(x) = x^2/9$ on $[0,3]$ .

Step 2 — integrate for the CDF

For $0 \leq x \leq 3$ :

F(x) = \int_0^x \frac{t^2}{9}\, dt = \frac{1}{9}\cdot\frac{x^3}{3} = \frac{x^3}{27}

F(x) = \begin{cases} 0 & x < 0 \\ x^3/27 & 0 \leq x \leq 3 \\ 1 & x > 3 \end{cases}

Check the endpoints: $F(0) = 0$ and $F(3) = 27/27 = 1 \;\checkmark$

Step 3 — interval probability

\Prob(1 \leq X \leq 2) = F(2) - F(1) = \frac{8}{27} - \frac{1}{27} = \frac{7}{27} \approx 0.2593

Step 4 — the median

The median $m$ satisfies $F(m) = 0.5$ :

\frac{m^3}{27} = \frac{1}{2} \quad\Longrightarrow\quad m^3 = 13.5 \quad\Longrightarrow\quad m = 13.5^{1/3} \approx 2.381

Step 5 — interpret

The median sits at 2.381 out of a $[0,3]$ range — well above the midpoint 1.5, because the density $x^2/9$ increases with $x$ and concentrates mass towards the right end.

Verify by differentiating back: $\frac{d}{dx}\left(\frac{x^3}{27}\right) = \frac{3x^2}{27} = \frac{x^2}{9} = f(x) \;\checkmark$

Answer

$c = 1/9$ ; $F(x) = x^3/27$ on $[0,3]$ ; $\Prob(1 \leq X \leq 2) = 7/27 \approx 0.259$ ; median $\approx 2.381$ .

Quantiles

The quantile function inverts the CDF:

Q(p) = \min\{x : F(x) \geq p\}

The median is $Q(0.5)$ , quartiles are $Q(0.25)$ and $Q(0.75)$ , and the 95th percentile is $Q(0.95)$ . The min is needed because the discrete CDF is a staircase and may jump past $p$ rather than hitting it exactly.

python

import numpy as np
from scipy import stats

# Continuous: the worked example, f(x) = x²/9 on [0,3], F(x) = x³/27.
F = lambda x: np.clip(x, 0, 3) ** 3 / 27
print(f"P(1 ≤ X ≤ 2) = {F(2) - F(1):.4f}   theory {7/27:.4f}")
print(f"median       = {(27 * 0.5) ** (1/3):.4f}")

# Discrete: PMF, CDF, and the >= trap.
xs = np.array([1, 2, 4]); pmf = np.array([0.2, 0.5, 0.3])
cdf = np.cumsum(pmf)
print(f"\nCDF at {xs} = {cdf}")
print(f"P(X >= 2) = {1 - cdf[0]:.2f}   <- 1 - F(1), NOT 1 - F(2)")

# Same idea with a named distribution: sf is the survival function 1 - cdf.
binom = stats.binom(n=10, p=0.3)
print(f"\nP(X >= 4) = {binom.sf(3):.4f}   (sf(3) = 1 - F(3))")
print(f"wrong:      {binom.sf(4):.4f}   (excludes X = 4)")

Exercise 1

$X$ is continuous with $F(x) = 1 - e^{-2x}$ for $x \geq 0$ . Find the density, the median, and $\Prob(X > 1)$ .

Show solution

Density — differentiate the CDF:

f(x) = \frac{d}{dx}\big(1 - e^{-2x}\big) = 2e^{-2x}, \qquad x \geq 0

This is the exponential distribution with rate $\lambda = 2$ .

Median — solve $F(m) = 0.5$ :

1 - e^{-2m} = 0.5 \quad\Longrightarrow\quad e^{-2m} = 0.5 \quad\Longrightarrow\quad -2m = \ln 0.5

m = \frac{\ln 2}{2} \approx \frac{0.6931}{2} \approx 0.3466

Upper tail:

\Prob(X > 1) = 1 - F(1) = 1 - (1 - e^{-2}) = e^{-2} \approx 0.1353

Worth noting: the mean of this distribution is $1/\lambda = 0.5$ , larger than the median $0.347$ . Right-skewed distributions have mean above median.

Exercise 2

A student writes $\Prob(X = 2) = f(2) = 0.8$ for a continuous variable with density $f$ . Give two reasons this is wrong.

Show solution

First, $\Prob(X = 2) = 0$ for any continuous variable. Probability comes from integrating the density over an interval, and the integral over the degenerate interval $[2,2]$ is zero. The correct statement involves a width:

\Prob(2 - \tfrac{\epsilon}{2} \leq X \leq 2 + \tfrac{\epsilon}{2}) \approx f(2)\,\epsilon

Second, the value 0.8 is being misread as a probability. A density has units of "per unit $x$ ", so $f(2) = 0.8$ means probability accumulates at a rate of 0.8 per unit near $x = 2$ . It carries no meaning as a probability on its own, and densities can legitimately exceed 1 — a uniform distribution on $[0, 0.1]$ has $f = 10$ everywhere on that interval, which would be nonsense if densities were probabilities.

The correct way to answer "how likely is $X$ near 2" is to pick an interval and integrate, or to report $F(b) - F(a)$ .

Next: Expectation, the first summary number computed from these functions.