Unsupervised Learning: Clustering

ml1-1 named supervised and unsupervised learning as this course's own two branches — every chapter since has been supervised, working from a known, provided answer (price, left_company). This chapter is the other branch, made concrete rather than abstract: the exact same datasets, with every label deliberately removed.

What Changes When There's No Label

Every technique since ml1-3 learned by comparing its own predictions against a known, correct answer during training. Clustering has no such answer to learn from at all — it's asked only to group similar data points together, discovering whatever structure exists on its own, with nobody ever telling it what the "right" groups are.

K-Means — The Algorithm, Step by Step

A genuinely different fitting mechanism than anything earlier in this course: no least-squares fit against a known target (ml1-3), no entropy-driven split search against known labels (ml1-7) — k-means iteratively refines its own evolving guess about where the clusters are, with nothing external to check itself against.

Choosing k — The Elbow Method

Unlike supervised learning, where the number of real classes is simply given (ml1-5's own two — Yes/No), nothing hands k to this chapter directly. Plotting inertia (the total squared distance from each point to its own assigned centroid) against increasing values of k always trends downward — more clusters always let points sit closer to their own center — but with sharply diminishing returns past a certain point. The "elbow" — where adding another cluster stops meaningfully reducing inertia — is a reasonable, practical choice for k.

Running It for Real — ds1-9's Own Used-Car Data, Unlabeled

from sklearn.cluster import KMeans
from sklearn.preprocessing import StandardScaler

X = df[["mileage", "year", "price"]]
X_scaled = StandardScaler().fit_transform(X)

kmeans = KMeans(n_clusters=3, random_state=42)
clusters = kmeans.fit_predict(X_scaled)

Running It on ds1-10's Own Employees — A Genuinely New Finding

X = df[["age", "years_at_company", "salary"]]
X_scaled = StandardScaler().fit_transform(X)
clusters = KMeans(n_clusters=3, random_state=42).fit_predict(X_scaled)

Why Scaling Isn't Optional Here

ml1-3 standardized features so coefficient sizes could be compared fairly. K-means needs standardization for a stricter reason: the algorithm's own "nearest centroid" step is a literal distance calculation. Left unscaled, salary (tens of thousands) would swamp years_at_company (single digits) in every distance computation, and the resulting clusters would essentially just be salary bands wearing three other features' names. Scaling here isn't a nicety for readability — it determines whether the clustering result means anything at all.

Crisp Assignment — And a Preview of ch10

Every point in k-means belongs to exactly one cluster, fully — a crisp assignment, with no notion of "70% in this cluster, 30% in that one." ml1-10's own Fuzzy Logic chapter picks up exactly here, with a system built specifically around partial, graded membership instead.

Hands-On Exercises