Linear Regression

ds1-9's own used-car dataset returns. Its Step 7 hypothesis — "does mileage predict price more strongly than the car's year does?" — was raised from a heatmap and explicitly left untested. This chapter tests it for real.

What Linear Regression Actually Does

Linear regression fits a straight line (or, with more than one feature, a flat hyperplane) through data to predict a continuous numeric value — a price, not a category. With one feature: price = w × mileage + b. With several features at once (this chapter's own case): price = w₁ × mileage + w₂ × year + b. Each w is a learned coefficient; b is the intercept — what the model would predict if every feature were zero.

How "Fitting" Actually Works

ds1-6 already defined variance as the average squared distance between each value and the mean. Linear regression's own fitting process — least squares — minimizes a close cousin of that same idea: the total squared distance between each actual price and the price the line would have predicted for it. "Squared" matters for the same reason it did in ds1-6: it penalizes large errors disproportionately more than small ones, and keeps positive and negative errors from canceling out.

Coefficients Are Interpretable — With One Real Caveat

Each coefficient answers a direct question: how much does the predicted price change per one-unit change in that feature, holding the other features constant? This is the actual mechanism for testing this chapter's own hypothesis.

Fitting the Model — ds1-9's Own Dataset, For Real

from sklearn.linear_model import LinearRegression
from sklearn.preprocessing import StandardScaler
from sklearn.model_selection import train_test_split

X = df[["mileage", "year"]]
y = df["price"]

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

scaler = StandardScaler()
X_train_scaled = scaler.fit_transform(X_train)
X_test_scaled = scaler.transform(X_test)

model = LinearRegression()
model.fit(X_train_scaled, y_train)
model.coef_       # [coefficient for mileage, coefficient for year]
model.intercept_  # b

Why a Fitted Coefficient Is Stronger Evidence Than a Raw Correlation

ds1-9's own heatmap already showed mileage and price correlating. ds1-6's own confounding-variable warning applies at full strength to that pairwise number alone — a raw correlation between two variables says nothing about a third, unmeasured factor possibly driving both. Multiple linear regression is a genuine, practical (if partial) step beyond that: fitting mileage and year together means each coefficient already accounts for the other — "holding year constant" is quite literally controlling for one of the two candidate confounders directly in the model itself, rather than leaving it unaccounted for.

What This Chapter Doesn't Yet Tell You

Knowing mileage matters more than year is a real finding — but it says nothing about how good this model actually is at predicting price overall. A model could correctly rank mileage as the stronger factor while still being wildly inaccurate in its actual dollar predictions. ml1-4 covers exactly that next.

Hands-On Exercises