PRINCIPAL COMPONENT ANALYSIS
Rotate the coordinate system toward directions that preserve the most variance.
01 Overview
02 The Problem
Many features may repeat the same signal. PCA rotates the coordinate system to expose the directions where the data vary most, then keeps only the leading directions.
03 Why It Matters
Fewer informative dimensions can make data easier to visualise, store, denoise, and model while retaining a measured fraction of its variance.
04 Intuition
PCA finds the longest cloud direction first, then a perpendicular direction, and so on. Projection onto the first directions is a compact summary of the original data.
05 Mathematical Foundation
After centring the features, PCA eigendecomposes the covariance matrix. Eigenvectors give directions and eigenvalues give the variance captured along them.
06 The Equation
- \(\Sigma\) centred covariance matrix
- \(v_j\) principal direction
- \(\lambda_j\) variance explained by that direction
07 How It Learns
- Centre each feature.
- Compute the covariance matrix.
- Find eigenvectors and eigenvalues.
- Sort directions by descending eigenvalue.
08 Algorithm
09 Visual Explanation
A quick visual summary of how this model sees data and makes its prediction.
10 Worked Example
If PC1 explains 92% of the variance, a one-dimensional projection retains most of the cloud's structure with a substantial reduction in representation size.