Beginner10 min

Mathematical Notation Guide

Every symbol used on this site, defined once: scalars, vectors, matrices, operators, probability and asymptotics.

Mathematical Notation Guide

Intuition first

Most of the difficulty in reading mathematics is not conceptual. It is that a symbol appears without introduction and you cannot even say it out loud, let alone reason about it.

This page defines every symbol used across the site, with how to pronounce it. Individual lessons repeat the relevant subset in a notation table before using it, so you should never need this page mid-lesson — it is here as a reference and for the cases where a lesson assumes something it should have stated.

Read it once to calibrate, then come back when something looks unfamiliar.

Conventions

Convention	Meaning	Example
Lowercase italic	Scalar	$x$ , $\alpha$ , $n$
Lowercase bold	Vector	$\mathbf{x}$ , $\mathbf{w}$
Uppercase italic	Matrix	$A$ , $X$ , $\Sigma$
Uppercase, probability context	Random variable	$X$ , $Y$
Hat	Estimate or prediction	$\hat{\theta}$ , $\hat{y}$
Bar	Sample average	$\bar{x}$
Tilde	"Is distributed as"	$X \sim \mathcal{N}(0,1)$
Star	Optimal value	$h^$ , $\pi^$
Subscript $i$	Index over examples	$x_i$ , $y_i$
Subscript $j$	Index over features	$w_j$

Sets and logic

Symbol	Meaning	Read as
$\in$	Is an element of	"in"
$\notin$	Is not an element of	"not in"
$\subseteq$	Is a subset of	"subset of"
$\cup$	Union	"union"
$\cap$	Intersection	"intersect"
$\varnothing$	Empty set	"empty set"
$A^c$	Complement of $A$	"A complement"
$\lvert A \rvert$	Size of a set, or absolute value	"size of A"
$\mathbb{R}$	Real numbers	"R"
$\mathbb{R}^d$	$d$ -dimensional real vectors	"R to the d"
$\mathbb{Z}$ , $\mathbb{N}$	Integers, natural numbers	"Z", "N"
$[a,b]$	Closed interval, endpoints included	"closed a b"
$(a,b)$	Open interval, endpoints excluded	"open a b"
$\forall$	For all	"for all"
$\exists$	There exists	"there exists"
$\implies$	Implies	"implies"
$\iff$	If and only if	"if and only if"
$\neg$ , $\wedge$ , $\vee$	Not, and, or	"not", "and", "or"
$\vdash$ , $\models$	Provable, entails	"proves", "entails"

Probability

Symbol	Meaning	Read as
$\Omega$	Sample space	"omega"
$\Prob(A)$	Probability of event $A$	"P of A"
$\Prob(A \mid B)$	Probability of $A$ given $B$	"P of A given B"
$p(x)$	Probability mass function	"p of x"
$f(x)$	Probability density function	"f of x"
$F(x)$	Cumulative distribution function	"big F of x"
$\E[X]$	Expected value	"E of X"
$\Var(X)$ , $\sigma^2$	Variance	"variance of X"
$\sigma$	Standard deviation	"sigma"
$\Cov(X,Y)$	Covariance	"covariance"
$\rho$	Correlation coefficient	"rho"
$X \ind Y$	$X$ is independent of $Y$	"independent of"
$\sim$	Is distributed as	"distributed as"
$\mathcal{N}(\mu, \sigma^2)$	Normal distribution	"normal mu sigma squared"
$\mathbb{1}[\cdot]$	Indicator: 1 if true, else 0	"indicator"
$\xrightarrow{d}$ , $\xrightarrow{P}$	Converges in distribution / probability	"converges to"
$\Phi(z)$	Standard normal CDF	"Phi of z"

Linear algebra

Symbol	Meaning	Read as
$\mathbf{x}\T$	Transpose	"x transpose"
$A^{-1}$	Matrix inverse	"A inverse"
$A^{+}$	Moore–Penrose pseudoinverse	"A plus"
$\mathbf{x}\T\mathbf{y}$	Inner product	"x dot y"
$\norm{\mathbf{x}}$	Norm, length of a vector	"norm of x"
$\norm{\mathbf{x}}_1$ , $\norm{\mathbf{x}}_2$	L1 and L2 norms	"L one norm"
$\det(A)$ , $\lvert A \rvert$	Determinant	"determinant"
$\operatorname{tr}(A)$	Trace, sum of the diagonal	"trace"
$\operatorname{rank}(A)$	Rank	"rank"
$\lambda$	Eigenvalue	"lambda"
$I$	Identity matrix	"identity"
$\Sigma$	Covariance matrix (or a sum)	"sigma"
$\odot$	Elementwise (Hadamard) product	"elementwise product"

Calculus and optimisation

Symbol	Meaning	Read as
$\frac{dy}{dx}$	Derivative	"dy by dx"
$\frac{\partial f}{\partial x}$	Partial derivative	"partial f partial x"
$\nabla f$	Gradient, vector of partials	"grad f" or "nabla f"
$\nabla^2 f$ , $H$	Hessian, matrix of second partials	"Hessian"
$J$	Jacobian	"Jacobian"
$\int$	Integral	"integral"
$\argmin_x f(x)$	The $x$ achieving the minimum	"arg min"
$\min_x f(x)$	The minimum value itself	"min"
$\propto$	Proportional to	"proportional to"
$\approx$	Approximately equal	"approximately"
$\to$	Tends to	"tends to"
$O(\cdot)$ , $\Theta(\cdot)$	Asymptotic upper bound, tight bound	"big O", "big theta"
$\epsilon$	Small positive quantity, or tolerance	"epsilon"
$\delta$	Small quantity, or failure probability	"delta"

Machine learning

Symbol	Meaning	Read as
$\mathcal{X}$ , $\mathcal{Y}$	Input and output spaces	"script X", "script Y"
$\mathcal{D}$	Data distribution	"script D"
$\mathcal{H}$	Hypothesis space	"script H"
$h$ , $f$	A hypothesis, the true function	"h", "f"
$\theta$ , $w$ , $b$	Parameters, weights, bias	"theta", "w", "b"
$L(y,\hat y)$	Loss on one example	"loss"
$J(\theta)$	Objective over the dataset	"J of theta"
$R(h)$ , $\hat R(h)$	True risk, empirical risk	"risk", "R hat"
$\lambda$	Regularisation strength	"lambda"
$\eta$ , $\alpha$	Learning rate	"eta", "alpha"
$n$ , $d$ , $K$	Examples, features, classes	—
$\sigma(z)$	Sigmoid function	"sigmoid"
$\odot$	Elementwise product	—

Greek letters

Worth being able to pronounce, since you will meet them constantly.

Letter	Name	Typical use
$\alpha$	alpha	Learning rate, significance level, shape
$\beta$	beta	Coefficients, scale, Type II error
$\gamma$	gamma	Discount factor, kernel width
$\delta$	delta	Small change, failure probability
$\epsilon$	epsilon	Small quantity, noise, tolerance
$\eta$	eta	Learning rate
$\theta$	theta	Parameters generally
$\lambda$	lambda	Regularisation, rate, eigenvalue
$\mu$	mu	Mean
$\pi$	pi	Policy, mixture weight, 3.14159
$\rho$	rho	Correlation
$\sigma$	sigma	Standard deviation, sigmoid
$\tau$	tau	Temperature, time constant
$\phi$	phi	Feature map, normal density
$\psi$	psi	Auxiliary function
$\omega$	omega	Outcome, frequency
$\Gamma$	capital gamma	Gamma function
$\Delta$	capital delta	Change or difference
$\Sigma$	capital sigma	Sum, covariance matrix
$\Phi$	capital phi	Normal CDF
$\Omega$	capital omega	Sample space, asymptotic lower bound

Custom shorthands used on this site

To keep expressions readable, a few macros are defined site-wide:

\E[X], \quad \Var(X), \quad \Cov(X,Y), \quad \Prob(A), \quad X \ind Y

\argmin_\theta J(\theta), \quad \argmax_\theta J(\theta), \quad \mathbf{w}\T\mathbf{x}, \quad \norm{\mathbf{w}}, \quad \abs{x}

These render identically to their long forms; the shorthand exists only to keep the source legible.

Exercise 1

Read the following aloud, then state what it says in plain English:

\hat{\theta} = \argmin_{\theta \in \Theta} \; \frac{1}{n}\sum_{i=1}^{n} L\big(y_i,\, h_\theta(x_i)\big) + \lambda\norm{\theta}_2^2

Show solution

Aloud: "theta hat equals arg min over theta in capital theta of one over n, sum from i equals one to n, of the loss of y-i and h-theta of x-i, plus lambda times the L-two norm of theta squared."

In plain English: the estimated parameters $\hat\theta$ are whichever parameters, from among all allowed parameters $\Theta$ , minimise the sum of two things: the average loss across the $n$ training examples, and a penalty $\lambda$ times the squared length of the parameter vector.

Reading the pieces:

$\argmin$ , not $\min$ — the answer is a parameter vector, not a number.
$\frac{1}{n}\sum$ — an average over examples, which is empirical risk.
$h_\theta(x_i)$ — the model's prediction for example $i$ , with the subscript showing it depends on the parameters being chosen.
$\lambda\norm{\theta}_2^2$ — L2 regularisation; the larger $\lambda$ , the more large parameters are penalised.

This single line is regularised empirical risk minimisation, which is what almost every supervised learning algorithm in the curriculum computes.

Next: How to Actually Learn This Material, or go straight to Sample Spaces and the Axioms of Probability.