From Logistic Regression to Neurons
Neural Networks & Deep Learning
Chapter 1 · From Logistic Regression to Neurons
You already built a neural network's most basic building block. Not metaphorically — literally. This chapter proves it, then shows exactly what's new once you stop stopping at one.
One Neuron Is ml1-5's Own Logistic Regression, Unchanged
ml1-5's own logistic regression: take a weighted sum of the inputs, add a bias, squash the result through sigmoid.
ml1-5's logistic regression: z = w1*x1 + w2*x2 + ... + wn*xn + b output = sigmoid(z)
A single artificial neuron computes exactly this — no more, no less:
a single neuron: z = w1*x1 + w2*x2 + ... + wn*xn + b output = activation(z)
activation function to sigmoid, and the formula is character-for-character identical to ml1-5's own logistic regression. You've already trained one of these — ml1-5's own coefficients were literally a single neuron's own learned weights.
So What's Actually New?
Two genuinely new ideas, both architectural rather than mathematical:
- Many neurons, side by side. Instead of one neuron producing one output, a layer runs several neurons in parallel against the same inputs — each with its own independently learned weights and bias, each producing its own output.
- Layers, stacked. One layer's outputs become the next layer's inputs. A layer sitting between the raw inputs and the final output is a hidden layer — inputs, hidden layers, and outputs together form the network.
ml1-5's logistic regression is precisely "a neural network with zero hidden layers and exactly one output neuron." Nothing about that description is a stretch — it's the honest, literal special case this entire course generalizes outward from.
Terminology, Before It's Needed
| Term | Meaning |
|---|---|
| Input layer | The raw features themselves — not neurons, just the data going in |
| Hidden layer | A layer of neurons sitting between input and output, visible only to the network itself |
| Output layer | The final layer, producing the network's own prediction |
| Weights / bias | Exactly ml1-5's own learned coefficients and intercept, one set per neuron |
| Fully connected / dense | Every neuron in one layer connects to every neuron in the next |
Why Bother Stacking Layers At All?
A single neuron — one weighted sum, one squash — can only ever draw a straight decision boundary through its inputs, a flat line or hyperplane. No amount of retuning its weights changes that basic geometric limit. nn1-2 shows a real, concrete problem a single neuron structurally cannot solve, no matter how it's trained. nn1-3 shows exactly why stacking a hidden layer underneath fixes it. This chapter deliberately doesn't resolve that mystery yet — it's worth sitting with the question first.
This Course's Own Roadmap
| Chapter | Delivers |
|---|---|
| nn1-2 / nn1-3 | The real historical limit a single neuron hit, and how depth fixed it |
| nn1-4 / nn1-5 | Activation functions and how a whole network actually learns (backpropagation) |
| nn1-6 | Training a network without it overfitting |
| nn1-7 | CNNs — the architecture behind real image recognition |
| nn1-8 / nn1-9 | RNNs, LSTMs, and a first look at what replaced them |
| nn1-10 / nn1-11 | Real tools, and a real trained network of your own |
Hands-On Exercises
Using this chapter's own side-by-side formulas, explain precisely why a single neuron with a sigmoid activation is not merely similar to ml1-5's logistic regression, but the identical computation.
📄 View solutionExplain the two genuinely new architectural ideas this chapter introduces beyond ml1-5's own single neuron, and explain why the chapter calls these "architectural rather than mathematical" changes.
📄 View solutionExplain why this chapter deliberately doesn't reveal what problem a single neuron can't solve, or how stacking layers fixes it, leaving both for nn1-2 and nn1-3 instead.
📄 View solutionChapter 1 Quick Reference
- A single neuron with sigmoid activation = ml1-5's own logistic regression, exactly
- Layer — several neurons in parallel, each with its own weights · Hidden layer — a layer between input and output
- Logistic regression = "a neural network with zero hidden layers and one output neuron"
- A single neuron can only draw a straight decision boundary — nn1-2/nn1-3 show why that's a real, hard limit, and how depth fixes it
- Next chapter: The Perceptron & the XOR Problem