Unsupervised Learning

What can be learned without labels, the evaluation problem, and a map of the methods ahead.

Euclidean, Manhattan, Minkowski, cosine, Jaccard, Hamming and Mahalanobis, and when each is appropriate.

Assumes: Norms and the Condition Number

Lloyd's algorithm iterated by hand, the within-cluster sum-of-squares objective, and convergence guarantees.

Assumes: Distance and Similarity Measures

Sensitivity to seeds, k-means++, the elbow method, silhouette scores and the gap statistic.

Assumes: k-Means Clustering

Medoids instead of means, the PAM algorithm, and robustness to outliers and arbitrary distances.

Assumes: k-Means Clustering

Bottom-up merging, the proximity matrix updated step by step, and reading a dendrogram.

Assumes: Distance and Similarity Measures

Single, complete, average, centroid and Ward linkage contrasted, with the chaining effect demonstrated.

Assumes: Agglomerative Hierarchical Clustering

Splitting rather than merging, DIANA, bisecting k-means, and the cost comparison with agglomerative.

Assumes: Linkage Criteria

Density-based clustering, core/border/noise points, eps and minPts selection, and OPTICS reachability plots.

Assumes: Initialisation and Choosing k

Kernel density gradient ascent, bandwidth selection, and mode-seeking behaviour.

Assumes: DBSCAN and OPTICS

Similarity graphs, the Laplacian eigenmap, and clustering in the spectral embedding.

Assumes: DBSCAN and OPTICS · Eigenvalues and Eigenvectors

Soft assignment via mixtures, the likelihood, and GMM as a probabilistic generalisation of k-means.

Assumes: The Multivariate Normal Distribution

The EM algorithm derived via the lower bound, applied step by step to a two-component mixture.

Assumes: Gaussian Mixture Models

Internal indices (silhouette, Davies–Bouldin, Calinski–Harabasz) and external ones (ARI, NMI, purity).

Assumes: Initialisation and Choosing k

Maximum-variance and minimum-reconstruction-error derivations, eigendecomposition of the covariance matrix, worked by hand.

Assumes: Eigenvalues and Eigenvectors · Covariance and Correlation

Why practitioners compute PCA through the SVD, numerical benefits, and whitening.

Assumes: Principal Component Analysis · Singular Value Decomposition

Nonlinear components via the kernel trick, centring in feature space, and the pre-image problem.

Assumes: PCA via SVD · Kernel Methods

Latent factors with explicit noise modelling, and how it differs from PCA in assumptions and output.

Assumes: Principal Component Analysis

Blind source separation, non-Gaussianity as the objective, and FastICA.

Assumes: Factor Analysis

Preserving pairwise distances, classical MDS via eigendecomposition, and geodesic distances on manifolds.

Assumes: Principal Component Analysis

Neighbour distributions, KL divergence, perplexity, the crowding problem, and how to read (and misread) a t-SNE plot.

Assumes: Multidimensional Scaling and Isomap

Fuzzy simplicial sets, the topological motivation, and the practical trade-offs against t-SNE.

Assumes: t-SNE

Support, confidence and lift; the Apriori property and candidate generation traced on a transaction database.

The FP-tree, conditional pattern bases, and why it beats Apriori on dense data.

Assumes: Association Rule Mining and Apriori

Statistical tests, Isolation Forest, Local Outlier Factor and one-class SVM, with evaluation under extreme imbalance.

Assumes: DBSCAN and OPTICS

Self-training, label propagation, co-training and consistency regularisation when labels are scarce.

Assumes: Cluster Validation