SUPPORT VECTOR MACHINE
Find the widest possible margin between classes and let support vectors define the boundary.
01 Overview
02 The Problem
SVM seeks a boundary that separates classes while leaving the widest possible margin around that boundary. Points nearest the margin become the support vectors.
03 Why It Matters
Maximising margin controls model complexity and can generalise well in high-dimensional spaces, especially when a useful kernel or linear boundary exists.
04 Intuition
Imagine a street between two groups. The best street is as wide as possible; only the houses touching its edges determine its position.
05 Mathematical Foundation
The soft-margin objective balances a small weight norm against hinge-loss penalties for points inside the margin or on the wrong side.
06 The Equation
- \(w^Tx+b=0\) separating hyperplane
- \(C\) penalty for margin violations
- \(y_i\) class label in \(\{-1,+1\}\)
07 How It Learns
- Start with a separating score.
- Measure hinge-loss violations.
- Update weights by gradient steps.
- Stop when the margin and loss stabilise.
08 Algorithm
09 Visual Explanation
A quick visual summary of how this model sees data and makes its prediction.
10 Worked Example
A boundary with a larger geometric margin can be preferable even when several separating boundaries classify the training points correctly.