Beginner24 min

Uniform and Exponential Distributions

Continuous uniform sampling, the exponential distribution, and its memoryless property.

Assumes you know

PMF, PDF and CDF

Uniform and Exponential Distributions

Intuition first

The uniform distribution is the statement that nothing in an interval is more likely than anything else. Flat density, no preference. It is the distribution of a random number generator, and the starting point for simulating everything else.

The exponential distribution answers "how long until the next event?" when events arrive at a constant rate. It is the waiting-time partner of the Poisson: Poisson counts how many arrive in a window, exponential times the gap between arrivals.

The exponential's defining oddity is memorylessness. A bus that has been overdue for 20 minutes is no more likely to arrive in the next minute than one that just missed you. The distribution has no memory of elapsed time. That is often physically wrong — machines age, patients deteriorate — and knowing when the assumption fails is most of the skill in using it.

The continuous uniform

X \sim \text{Uniform}(a, b), \qquad f(x) = \frac{1}{b - a} \text{ for } a \leq x \leq b

F(x) = \frac{x - a}{b - a}, \qquad \E[X] = \frac{a+b}{2}, \qquad \Var(X) = \frac{(b-a)^2}{12}

Deriving the uniform momentsAdvanced

\E[X] = \int_a^b \frac{x}{b-a}\,dx = \frac{1}{b-a}\left[\frac{x^2}{2}\right]_a^b = \frac{b^2 - a^2}{2(b-a)} = \frac{(b-a)(b+a)}{2(b-a)} = \frac{a+b}{2}

For the variance, first get the second moment:

\E[X^2] = \int_a^b \frac{x^2}{b-a}\,dx = \frac{b^3 - a^3}{3(b-a)} = \frac{a^2 + ab + b^2}{3}

using $b^3 - a^3 = (b-a)(a^2+ab+b^2)$ . Then

\Var(X) = \frac{a^2+ab+b^2}{3} - \frac{(a+b)^2}{4} = \frac{4(a^2+ab+b^2) - 3(a+b)^2}{12}

= \frac{4a^2 + 4ab + 4b^2 - 3a^2 - 6ab - 3b^2}{12} = \frac{a^2 - 2ab + b^2}{12} = \frac{(b-a)^2}{12}

The variance depends only on the width, not on the location — as it must, since shifting a distribution cannot change its spread.

The exponential

X \sim \text{Exponential}(\lambda), \qquad f(x) = \lambda e^{-\lambda x} \text{ for } x \geq 0

F(x) = 1 - e^{-\lambda x}, \qquad \E[X] = \frac{1}{\lambda}, \qquad \Var(X) = \frac{1}{\lambda^2}

Notation used in this lesson
Symbol	Meaning	Read aloud
λ	Rate — events per unit time	lambda
1/λ	Mean waiting time between events	one over lambda
S(x) = 1 − F(x)	Survival function, P(X > x) = e^{−λx}	survival function

Memorylessness

\Prob(X > s + t \mid X > s) = \Prob(X > t)

Proof, and why the exponential is the only continuous distribution with this propertyAdvanced

Using $\Prob(X > x) = e^{-\lambda x}$ and the definition of conditional probability:

\Prob(X > s+t \mid X > s) = \frac{\Prob(X > s+t \;\text{and}\; X > s)}{\Prob(X > s)}

Since $s + t > s$ , the event $X > s+t$ already implies $X > s$ , so the intersection is just $X > s + t$ :

= \frac{\Prob(X > s+t)}{\Prob(X > s)} = \frac{e^{-\lambda(s+t)}}{e^{-\lambda s}} = e^{-\lambda t} = \Prob(X > t)

The elapsed time $s$ cancels completely.

Uniqueness. Suppose a continuous distribution on $[0,\infty)$ is memoryless. Writing $S(x) = \Prob(X > x)$ , the property says

S(s + t) = S(s)\,S(t) \quad \text{for all } s, t \geq 0

This is Cauchy's functional equation in multiplicative form. Its only measurable solutions with $S(0) = 1$ and $S$ decreasing are $S(x) = e^{-\lambda x}$ for some $\lambda > 0$ . So the exponential is the unique memoryless continuous distribution — the geometric is its discrete counterpart.

Solved problem 1 · Server requests, both directions

Requests arrive at a server at 12 per minute, as a Poisson process. (a) Find the probability the next request arrives within 3 seconds. (b) Find the mean and standard deviation of the gap. (c) Given 10 seconds have passed with no request, find the probability of waiting another 3 seconds. (d) Find the probability that exactly 2 requests arrive in the next 6 seconds.

Step 1 — set up consistent units

Work in seconds. A rate of 12 per minute is

\lambda = \frac{12}{60} = 0.2 \text{ per second}

The gap $T$ between requests is $\text{Exponential}(0.2)$ .

Step 2 — part (a)

\Prob(T \leq 3) = F(3) = 1 - e^{-0.2 \times 3} = 1 - e^{-0.6}

e^{-0.6} = 0.548812

\Prob(T \leq 3) = 1 - 0.548812 = 0.451188

Step 3 — part (b)

\E[T] = \frac{1}{\lambda} = \frac{1}{0.2} = 5 \text{ seconds}, \qquad \text{SD}(T) = 5 \text{ seconds}

Mean equals standard deviation, as always for the exponential.

Step 4 — part (c): memorylessness in action

\Prob(T > 13 \mid T > 10) = \Prob(T > 3) = e^{-0.6} = 0.548812

So the probability of waiting more than another 3 seconds is $0.5488$ , and the probability of a request arriving within 3 more seconds is $1 - 0.5488 = 0.4512$ — identical to part (a). The ten seconds of waiting carried no information.

Step 5 — part (d): switch to the Poisson view

Counts use the Poisson with $\lambda_{\text{window}} = \text{rate} \times \text{window}$ :

\lambda_{6\text{s}} = 0.2 \times 6 = 1.2

\Prob(N = 2) = \frac{e^{-1.2}(1.2)^2}{2!} = \frac{0.301194 \times 1.44}{2} = \frac{0.433720}{2} = 0.216860

Step 6 — check the two views agree

The event "no requests in 6 seconds" can be computed either way:

\text{Poisson}: \; \Prob(N = 0) = e^{-1.2} = 0.301194

\text{Exponential}: \; \Prob(T > 6) = e^{-0.2 \times 6} = e^{-1.2} = 0.301194 \;\checkmark

They must agree, because "no arrivals in $[0,t]$ " and "the first arrival is after $t$ " are the same event. That identity is the formal link between the two distributions.

Answer

(a) $\approx 0.4512$ ; (b) mean and SD both 5 seconds; (c) $\approx 0.4512$ — unchanged by the 10-second wait; (d) $\approx 0.2169$ .

Inverse transform sampling

The uniform is the raw material for simulating any distribution.

\text{If } U \sim \text{Uniform}(0,1) \text{ then } X = F^{-1}(U) \text{ has CDF } F

Why it works, and the exponential caseAdvanced

Assume $F$ is continuous and strictly increasing, so $F^{-1}$ exists. Then

\Prob(X \leq x) = \Prob\big(F^{-1}(U) \leq x\big) = \Prob\big(U \leq F(x)\big) = F(x)

The last step uses the defining property of the standard uniform: $\Prob(U \leq u) = u$ for $u \in [0,1]$ , and $F(x)$ always lies in $[0,1]$ .

For the exponential. Set $u = F(x) = 1 - e^{-\lambda x}$ and solve for $x$ :

e^{-\lambda x} = 1 - u \quad\Longrightarrow\quad -\lambda x = \ln(1-u) \quad\Longrightarrow\quad x = -\frac{\ln(1-u)}{\lambda}

Since $1 - U$ is also uniform on $(0,1)$ , the simpler form $x = -\ln(U)/\lambda$ works equally well and is what most implementations use.

This is how numpy.random.exponential works underneath, and it generalises: any distribution with an invertible CDF can be sampled from uniform draws alone.

python

import numpy as np
from scipy import stats

rng = np.random.default_rng(0)

# Worked example: rate 0.2 per second.
T = stats.expon(scale=1/0.2)          # NOTE: scale = 1/rate
print(f"(a) P(T <= 3)          {T.cdf(3):.6f}")
print(f"(b) mean {T.mean():.1f}s   sd {T.std():.1f}s")
print(f"(c) P(T>13 | T>10)     {T.sf(13)/T.sf(10):.6f}   = P(T>3) {T.sf(3):.6f}")
print(f"(d) P(N=2) in 6s       {stats.poisson(1.2).pmf(2):.6f}")

# The Poisson/exponential identity.
print(f"\nP(no arrivals in 6s): Poisson {stats.poisson(1.2).pmf(0):.6f}  "
      f"Exponential {T.sf(6):.6f}")

# Inverse transform sampling, by hand.
U = rng.random(500_000)
X = -np.log(U) / 0.2
print(f"\ninverse transform: mean {X.mean():.4f} (theory 5)  sd {X.std():.4f} (theory 5)")

# Memorylessness, empirically: gaps beyond 10s, minus 10, look like fresh gaps.
gaps = rng.exponential(5.0, 2_000_000)
residual = gaps[gaps > 10] - 10
print(f"residual after 10s: mean {residual.mean():.4f}  <- still ~5, no ageing")

Exercise 1

A bus arrives uniformly at random between 10:00 and 10:30. You arrive at 10:10. Find the probability you wait more than 10 minutes, and your expected wait.

Show solution

Let $X$ be the arrival time in minutes after 10:00, so $X \sim \text{Uniform}(0, 30)$ .

Arriving at 10:10, you catch the bus only if it has not already gone — but the question asks for the wait, so condition on $X > 10$ :

\Prob(X > 10) = \frac{30 - 10}{30} = \frac{20}{30} = \frac{2}{3}

Given $X > 10$ , the bus time is uniform on $(10, 30)$ by the flat density. Waiting more than 10 minutes means $X > 20$ :

\Prob(X > 20 \mid X > 10) = \frac{30 - 20}{30 - 10} = \frac{10}{20} = 0.5

Expected wait, given the bus has not gone:

\E[X - 10 \mid X > 10] = \frac{10 + 30}{2} - 10 = 20 - 10 = 10 \text{ minutes}

Note that the uniform is not memoryless: as time passes the remaining wait shrinks. At 10:10 the expected further wait is 10 minutes; at 10:20 it is 5. Contrast the exponential, where it would stay constant.

Exercise 2

A component has exponential lifetime with mean 5 years. It has already run for 4 years. A manager argues it is "due to fail soon". Assess the argument, and say when the manager would be right.

Show solution

Under the exponential model the manager is wrong. With $\lambda = 1/5 = 0.2$ per year:

\Prob(T > 4 + t \mid T > 4) = \Prob(T > t) = e^{-0.2t}

The expected remaining life is still 5 years, exactly as when new. The component has not aged in any statistical sense.

For example, the chance of surviving another 5 years is

e^{-0.2 \times 5} = e^{-1} \approx 0.368

whether the component is new or four years old.

When the manager would be right: if the true lifetime distribution has an increasing hazard rate, which is typical for mechanical wear. A Weibull with shape $k > 1$ has hazard $h(t) = \frac{k}{\beta}\left(\frac{t}{\beta}\right)^{k-1}$ , increasing in $t$ , so an older component genuinely is closer to failure.

So the manager's intuition is sound engineering and wrong mathematics given the stated model. The correct response is to challenge the model rather than the intuition: check whether the empirical hazard rate is constant by plotting failures per unit time among survivors at each age. If it rises, the exponential is the wrong distribution and the manager is right.

Next: Gamma and Beta Distributions, which generalise the exponential and introduce the conjugate prior for probabilities.