Beginner22 min

What Machine Learning Actually Is

Learning from data versus explicit programming, the three paradigms, and an honest account of what ML cannot do.

What Machine Learning Actually Is

Intuition first

Ordinary programming is you writing the rules. You decide what counts as spam, you write the if statements, the computer executes them.

Machine learning inverts that. You supply examples — 10,000 emails already labelled spam or not — and the computer searches for rules that reproduce those labels. You write the search procedure, not the rules.

That inversion is the whole idea, and it buys you something specific: it works for problems where you can recognise the right answer but cannot articulate the rule. You know a cat when you see one. Try writing down the pixel conditions for "cat" and you will fail. Learning from examples sidesteps the need to ever state the rule explicitly.

The cost is equally specific. The rules you get back are only as good as the examples you gave, they carry whatever biases those examples carried, and nobody — including you — can fully explain them afterwards.

A usable definition

The standard formulation is Tom Mitchell's, and it is worth memorising because it forces you to name three things before you start:

A program learns from experience E with respect to task T and performance measure P if its performance at T, measured by P, improves with E.

For spam filtering: T is classifying emails, E is the labelled corpus, P is perhaps the false-positive rate at 95% recall.

The three paradigms

Notation used in this lesson
Symbol	Meaning	Read aloud
x	An input, also called a feature vector or instance	x
y	The target — the thing being predicted	y
(x, y)	One labelled example	x y pair
f	The unknown true relationship we are trying to recover	f
f̂	Our learned approximation to f	f hat
n	Number of training examples	n
d	Number of features per example	d

Supervised learning

Every example comes with its answer. The data is a set of pairs $(x_1, y_1), \dots, (x_n, y_n)$ , and the goal is a function $\hat{f}$ that predicts $y$ from $x$ for inputs never seen before.

Two sub-cases, distinguished only by what $y$ is:

Regression — $y$ is a continuous number. House price, tomorrow's temperature, expected revenue.
Classification — $y$ is one of a finite set of labels. Spam or not, digit 0 through 9, which of five diseases.

This is the bulk of deployed machine learning, and the bulk of this curriculum.

Unsupervised learning

No labels. You have $x_1, \dots, x_n$ and want structure: groups of similar customers, a lower-dimensional representation, which transactions look anomalous.

The hard part is that without labels there is no unambiguous notion of correct. Two different clusterings can both be defensible, which is why evaluating unsupervised methods is genuinely harder than evaluating supervised ones.

Reinforcement learning

An agent takes actions in an environment and receives rewards. No one tells it the right action; it must discover which actions pay off by trying them, while the consequences may only become clear much later.

This is the setting for game playing, robotics, and — relevant to this curriculum — the alignment stage of large language models.

The three paradigms differ in what feedback the learner receives: the exact answer, no answer, or a delayed score.

The one assumption everything rests on

Machine learning works only if the future resembles the past. Formally, we assume training and future data are drawn from the same distribution:

(x_i, y_i) \stackrel{\text{i.i.d.}}{\sim} \mathcal{D}

Every guarantee in this curriculum inherits from that line. When a model that tested well collapses in production, the first suspect is always that $\mathcal{D}$ changed — new user demographics, a changed upstream pipeline, a different season.

Learning is not memorising

A lookup table that stores every training example and reproduces its label perfectly has zero training error and is useless. It cannot answer anything new.

The actual goal is generalisation: performing well on data never seen during training. This distinction is the subject of the next several lessons and the reason we hold data back rather than evaluating on what we trained on.

When not to use machine learning

An honest list, because this is where judgement matters more than technique.

Use rules instead when the rule is known. Tax calculations, validation logic, business policy. A learned model would be less accurate, less auditable and more expensive than the if statement you already have.

Do not use ML when you cannot tolerate being wrong. Models are statistical; they will produce confident errors. If a single mistake is catastrophic and unreviewable, either keep a human in the loop or do not ship it.

Do not use ML without data. The quantity needed scales with problem complexity. A few hundred examples can support logistic regression on ten features and cannot support a deep network on images.

Do not use ML when the objective cannot be measured. "Improve user satisfaction" is not optimisable. Something concrete and countable has to stand in for it, and choosing that proxy badly is how recommendation systems end up optimising for outrage.

Solved problem 1 · Classifying four problems

For each, name the paradigm and the sub-type, and identify what could go wrong.

(a) Predict tomorrow's electricity demand in megawatts from weather and calendar data, given five years of history.

(b) Group 50,000 news articles into topics, with no topic labels available.

(c) Train a robot arm to stack blocks; it receives $+1$ when a tower stands for three seconds.

(d) Decide whether a transaction is fraudulent, given 2 million transactions of which 400 are labelled fraud.

(a) Supervised regression

The target is continuous, and historical demand provides labels. But the i.i.d. assumption fails: electricity demand is a time series with trend and seasonality, so consecutive days are strongly dependent. Random train/test splitting would leak future information into training. The split must respect time order.

(b) Unsupervised clustering

No labels, so this is clustering or topic modelling. The difficulty is evaluation — with no ground truth, "correct" number of topics is undefined, and results depend heavily on the text representation chosen.

A delayed, sparse reward from interaction. The difficulty is precisely that sparsity: the arm receives no signal for thousands of near-miss attempts, a condition known as the credit assignment problem.

(d) Supervised classification, severely imbalanced

Labels exist, so it is supervised. The fraud rate is

\frac{400}{2{,}000{,}000} = 0.0002 = 0.02\%

A model predicting "never fraud" achieves 99.98% accuracy and catches nothing. Accuracy is the wrong measure here; precision, recall and the precision–recall curve are required.

Answer

(a) supervised regression, with a temporal-dependence trap; (b) unsupervised clustering, with no objective evaluation; (c) reinforcement learning with sparse delayed reward; (d) supervised classification at 0.02% positive rate, where accuracy is meaningless.

The smallest possible example

Supervised learning in eight lines, with no library doing the thinking:

python

import numpy as np

# Five houses: size in square metres → price in thousands.
X = np.array([50, 62, 75, 88, 100], dtype=float)
y = np.array([152, 190, 226, 272, 305], dtype=float)

# "Learning" = choosing the slope and intercept that best fit these pairs.
slope, intercept = np.polyfit(X, y, deg=1)

# Prediction on a size never seen in training.
print(f"f(x) = {slope:.3f}x + {intercept:.3f}")
print(f"predicted price for 80 m²: {slope * 80 + intercept:.1f}k")

Everything in this module is a more careful version of those three steps: choose a family of candidate functions, define what "best fit" means, and search. The difficulty is never the search — it is deciding what "best" should mean so that the result generalises.

Exercise 1

A company wants to "use AI to reduce customer churn". Convert this into a well-posed learning problem by specifying T, E and P, then state one reason it might still fail.

Show solution

One defensible specification:

T: for each active subscriber, predict the probability they cancel within the next 30 days — supervised binary classification.
E: 24 months of subscriber records, each labelled by whether that account cancelled in the 30 days following a snapshot date, using only features observable at the snapshot.
P: precision at the top 5% of predicted risk, since the retention team can only contact a fixed number of customers.

Why it may still fail: the prediction is not the goal. Knowing who will churn does not reduce churn — the intervention does. A model can be accurate and worthless if the customers it flags are precisely those no discount would retain. The decision problem ("who should we contact for maximum retained revenue") is causal, not predictive.

The subtler trap: the label depends on the snapshot date, and features must be computed strictly before it. Including "number of support tickets in the final week" leaks the outcome.

Exercise 2

Explain why a model with 100% training accuracy might be worse than one with 85% training accuracy.

Show solution

Training accuracy measures fit to data already seen, which the model may simply have memorised. A sufficiently flexible model can fit every training point exactly, including the noise, and thereby encode patterns that do not exist in the wider population.

The 85% model may have been prevented from memorising — by being simpler, or regularised — and so captures only the reproducible structure. On new data it can easily be the more accurate of the two.

This gap is the difference between fitting and generalising, and it is the entire subject of the bias–variance trade-off.

Next: Formulating a Learning Problem, which turns T, E and P into the mathematical objects an algorithm can actually optimise.