MATH3320: Foundation of Data Analytics
Regression and validation, sparse approximation, spectral and learned dimensionality reduction, matrix completion, quadratic optimization, and clustering from center objectives to density hierarchies.
1.1 Least squares and residuals
Three original practice papers, 21 questions and 72 parts, with complete worked solutions and a detailed regression chapter.
Browse chapters
Chapter 1
Regression
Fit observations, measure held-out prediction error, and derive regularized and sparse models.
- Read section
1.1 Least squares and residuals
Build a regression design matrix, derive the normal equations, and interpret the residual through orthogonal projection and weighting.
- Read section
1.2 Model validation and ridge regression
Separate fitting from model validation, compute held-out errors, and derive ridge regression with an explicit loss convention.
- Read section
1.3 LASSO and matching pursuit
Derive soft thresholding and coordinate conditions, then trace matching pursuit and distinguish it from orthogonal refitting.
- Read section
The complete regression prediction chain
Step-by-step validation, refitting, inverse scaling, and regularization.
Chapter 2
Dimensionality reduction
Build coordinates from data, distances, kernels, and class labels, with the objective of each projection made explicit.
- Read section
2.1 SVD and principal components
Center a data matrix, connect its singular values to covariance, and derive PCA from variance and reconstruction error.
- Read section
2.2 Classical and non-metric multidimensional scaling
Recover coordinates from pairwise dissimilarities using double centering, and distinguish classical strain from non-metric stress.
- Read section
2.3 Centered kernels and kernel PCA
Center a kernel Gram matrix, normalize its eigenvectors in feature space, and project new observations using fixed training means.
- Read section
2.4 Fisher discriminants and class scatter
Decompose class scatter and solve the Fisher ratio through whitening, generalized eigenvectors, and the within-class metric.
- Read section
2.5 Learned metrics and random projections
Learn a positive-semidefinite distance from pair constraints and derive finite-set distance preservation under Gaussian projection.
- Read section
2.6 Low-rank matrix completion
Model missing entries using observation masks and low rank, derive singular-value thresholding, and construct the corresponding proximal update.
Chapter 3
Optimization
Relate gradients, curvature, and step size to the behavior of iterative least-squares fitting.
- Read section
3.1 Gradient descent on quadratics
Derive gradient descent for quadratic losses, analyze the step-size interval, and connect convergence to the spectrum of the Hessian.
Chapter 4
Clustering
Derive center objectives, probabilistic memberships, hierarchical merges and density connectivity from explicit definitions and original examples.
- Read section
4.1 Voronoi cells and k-center
Derive nearest-site cells and their Delaunay dual, then prove the metric two-approximation for farthest-first k-center clustering.
- Read section
4.2 Centre objectives and Lloyd updates
Compare mean, median and medoid center objectives, derive Lloyd descent and finite termination, and interpret local minima and elbow plots.
- Read section
4.3 Gaussian mixtures and expectation-maximization
Derive Gaussian-mixture responsibilities and weighted parameter updates, prove likelihood ascent, and distinguish soft EM from hard assignments.
- Read section
4.4 Hierarchical linkages and dendrograms
Build and cut nested partitions, derive Ward merge costs, compare linkage updates, and connect single-link cuts to graph components.
- Read section
4.5 Density connectivity and mean shift
Construct DBSCAN core components and border labels, follow mutual-reachability hierarchies, and prove Gaussian mean-shift density ascent.