Logistic Regression & Classification Basics

ds1-10's own employee-attrition table returns. Two things it deliberately left open: "does lower salary predict a higher likelihood of leaving?", and a genuine, unresolved confounding-variable question — "does department matter, or is department's own apparent effect actually explained by department-level salary differences instead?" This chapter tests both, for real.

Why ml1-3's Own Tool Doesn't Fit Here

left_company is Yes/No — a category, not a continuous number. Fitting ml1-3's own plain linear regression directly against a 0/1-coded target would produce genuine nonsense: nothing stops the fitted line from predicting -0.3 or 1.4 for some employees — numbers with no sensible reading as "how likely is this person to leave."

Logistic Regression — Same Linear Core, a New Final Step

Logistic regression keeps ml1-3's own linear combination of weighted features underneath — this is genuinely still "regression" in that sense, which is exactly why the confusing name persists — but passes the result through the sigmoid function before treating it as an answer. Sigmoid takes any real number and squashes it into the range (0, 1): very negative inputs approach 0, very positive inputs approach 1, and an input of exactly 0 lands at 0.5. The output is now a genuine probability — "how likely is this employee to leave?" — never a nonsensical value like -0.3.

From Probability to a Decision — The Threshold

A probability alone isn't yet a Yes/No prediction. A threshold — commonly 0.5 — converts it: probability above the threshold predicts Yes, below predicts No. 0.5 is a genuine, adjustable choice, not a law of nature; ml1-6's own precision/recall material covers exactly why and when moving that threshold matters.

A New Wrinkle: Encoding department

ml1-3's own two features were already numeric. department is categorical text (Sales, Engineering, Support) — a model can't multiply a weight by a word. One-hot encoding converts one categorical column into several binary columns, one per category (is_sales, is_engineering, is_support, each 0 or 1), so each category gets its own learnable weight.

from sklearn.linear_model import LogisticRegression
from sklearn.preprocessing import StandardScaler
from sklearn.model_selection import train_test_split

X = pd.get_dummies(df[["department", "age", "years_at_company", "salary"]])
y = df["left_company"].map({"Yes": 1, "No": 0})

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

scaler = StandardScaler()
X_train_scaled = scaler.fit_transform(X_train)
X_test_scaled = scaler.transform(X_test)

model = LogisticRegression()
model.fit(X_train_scaled, y_train)
model.coef_

Testing ds1-10's Own Confounding Question — For Real

This is the payoff ds1-10 itself couldn't reach: because department's own one-hot columns and salary are now fit together, the model can show whether department carries any real, independent weight once salary is already accounted for.

What This Chapter Hasn't Checked Yet

Same structure as ml1-3 into ml1-4: interpreting coefficients answered which factors matter, not how good this model actually is at correctly predicting who leaves. ml1-6 covers that next — and, for this particular dataset, honestly: attrition is a genuinely imbalanced target (far more "No" than "Yes" in most real companies), which makes the choice of evaluation metric matter more here than it did for ml1-4's own regression case.

Hands-On Exercises