Capstone: Building and Training a Real Neural Network

Neural Networks & Deep Learning

Chapter 11 · Capstone: Building and Training a Real Neural Network

One real dataset, already familiar: ds1-10's own employee-attrition table, already fit with ml1-5's logistic regression and ml1-7's random forest. This capstone builds a real, working neural network on the exact same problem — and asks the honest question this course owes a real answer to: does going deeper actually help here?

The Full Pipeline

import torch
import torch.nn as nn
import torch.optim as optim

X = pd.get_dummies(df[["department", "age", "years_at_company", "salary"]])
y = df["left_company"].map({"Yes": 1, "No": 0})
# train/val/test split — ml1-2, nn1-6
# StandardScaler — ds1-2/ml1-3

class AttritionNet(nn.Module):
    def __init__(self, n_features):
        super().__init__()
        self.hidden1 = nn.Linear(n_features, 16)
        self.hidden2 = nn.Linear(16, 8)
        self.dropout = nn.Dropout(0.3)     # nn1-6
        self.output = nn.Linear(8, 1)
        self.relu = nn.ReLU()              # nn1-4

    def forward(self, x):
        x = self.relu(self.hidden1(x))
        x = self.dropout(x)
        x = self.relu(self.hidden2(x))
        return torch.sigmoid(self.output(x))   # nn1-1's own closing callback — see below

model = AttritionNet(n_features=X.shape[1])
loss_fn = nn.BCELoss()                                    # nn1-5
optimizer = optim.Adam(model.parameters(), lr=0.01)        # nn1-6

best_val_loss = float("inf")
for epoch in range(200):
    # forward pass, loss, backward(), step() — nn1-5/nn1-6
    # track train_loss and val_loss each epoch — nn1-6's own learning curve
    # early stopping — nn1-6: save weights when val_loss improves

nn1-1's own closing callback

That final torch.sigmoid(self.output(x)) line is, structurally, exactly nn1-1's own opening claim made real: the output layer of this entire network is one sigmoid-activated neuron — ml1-5's own logistic regression, unchanged, just fed a richer, network-transformed set of inputs instead of the raw features directly.

Evaluating — The Same Metrics, For a Fair Comparison

from sklearn.metrics import precision_score, recall_score, f1_score
preds = (model(X_test) > 0.5).int()   # ml1-6's own threshold, applied here too
precision_score(y_test, preds), recall_score(y_test, preds), f1_score(y_test, preds)

ml1-6's own precision/recall/F1 apply completely unchanged — the same evaluation vocabulary works identically regardless of which model produced the predictions.

The Honest Finding

Does the neural network actually win here?

On this small, simple tabular dataset, the neural network's own precision/recall/F1 plausibly come back roughly comparable to — not dramatically better than — ml1-5's logistic regression and ml1-7's random forest, and it required considerably more code, more hyperparameters, and more training time to get there.

This is a real, important, well-documented practical truth

Neural networks' genuine advantages show up on large datasets and complex, high-dimensional data — images (nn1-7's own AlexNet), long sequences (nn1-8/nn1-9) — not necessarily small, simple tabular problems like this one, where a handful of numeric and categorical features rarely benefit much from the kind of hierarchical feature transformation nn1-3 and nn1-7 made such a strong case for. This isn't a knock against neural networks — it's a genuine, useful piece of practical judgment: ml1's own simpler models remain the right first choice for a great many real problems, not merely "the old stuff before deep learning got invented."

Chapter Attribution

Capstone element	Drawn from
The neuron-as-logistic-regression closing callback	nn1-1
Hidden layers enabling nonlinear feature transformation	nn1-3
ReLU activation, dropout regularization	nn1-4 / nn1-6
Binary cross-entropy loss, backward()	nn1-5
Adam optimizer, epochs, early stopping	nn1-6
Real PyTorch syntax throughout	nn1-10
Precision/recall/F1 evaluation, direct comparison	ml1-6 / ml1-5 / ml1-7
The dataset itself	ds1-10

Honest Scope Note

What this capstone — and this course — deliberately doesn't attempt

No CNN, RNN, or transformer applied here. This capstone's own dataset is tabular — a genuinely different data shape from the images (nn1-7) and sequences (nn1-8/nn1-9) those architectures were specifically built for. Forcing one of them onto tabular data here would be architecturally dishonest, not a real demonstration.
No production deployment or MLOps. Matching ml1-11's own precedent — this capstone stops at a trained, evaluated model.
Full transformer/attention depth remains deferred. nn1-9 previewed the concept; llm1 delivers the real mechanism.
Hyperparameter tuning is only lightly touched. Systematic search over architecture size, learning rate, and dropout rate is a real, substantial practice this course doesn't attempt in depth.

Hands-On Exercises

Exercise 1

Explain, using this chapter's own tip-box, precisely how the network's own final output layer closes the loop back to nn1-1's own opening claim about a single neuron.

📄 View solution

Exercise 2

Explain why this chapter's own honest finding (comparable, not dramatically better, performance) is described as "a real, important, well-documented practical truth" rather than a disappointing result, and identify the kind of data where neural networks' own real advantage actually shows up.

📄 View solution

Exercise 3

Explain why this chapter's own scope note calls forcing a CNN or RNN onto this capstone's dataset "architecturally dishonest," using this chapter's own reasoning about data shape.

📄 View solution

Chapter 11 Quick Reference — Course Summary

A single neuron is ml1-5's own logistic regression (nn1-1); stacking hidden layers solves what a single neuron structurally can't (nn1-2/nn1-3)
Activation functions (nn1-4), backpropagation (nn1-5), and practical training/regularization (nn1-6) are the real mechanism behind every trained network
CNNs (nn1-7) and RNNs/LSTMs (nn1-8) are specialized architectures for spatial and sequential data respectively; transformers (nn1-9) fixed what LSTMs couldn't
Real code (nn1-10) maps every concept onto actual PyTorch syntax
The honest finding: deep learning isn't automatically the right tool for every problem — ml1's own simpler models remain genuinely competitive on small, tabular data
Next up in the Data Science & ML subject: nlp1 — building toward "why LLMs are different," the direct bridge into llm1's own full transformer coverage