A Framework Tour: PyTorch vs. TensorFlow

nn1-1–nn1-9 built the concepts. This chapter maps them onto real, working code — and covers the two dominant frameworks the rest of this course's own ecosystem (llm1 included) is built on.

Two Frameworks, Two Different Starting Philosophies

TensorFlow (Google) originally used define-then-run: build the entire computation graph first, as a fixed structure, then feed data through it. Efficient for deployment and optimization, but genuinely awkward for debugging — you couldn't simply inspect an intermediate value mid-computation the way you'd step through ordinary Python. PyTorch (Meta) used eager execution — define-by-run — from the start: the computation graph is built dynamically as code actually executes, line by line, exactly like ordinary Python. This genuinely mattered for research and experimentation, and is a real, documented reason PyTorch became — and largely remains — the dominant framework in academic and research settings specifically.

An Honest Update — This Gap Has Narrowed

The Current Practical Landscape

A Real Small Network — Every Concept, Real Code

import torch
import torch.nn as nn
import torch.optim as optim

class SmallNetwork(nn.Module):
    def __init__(self):
        super().__init__()
        self.hidden = nn.Linear(2, 4)   # nn1-1's own hidden layer
        self.output = nn.Linear(4, 1)   # nn1-1's own output layer
        self.relu = nn.ReLU()           # nn1-4's own activation function

    def forward(self, x):
        x = self.relu(self.hidden(x))   # nn1-1's own forward propagation
        return torch.sigmoid(self.output(x))   # ml1-5's own sigmoid, one more time

model = SmallNetwork()
loss_fn = nn.BCELoss()                          # cross-entropy for binary output (nn1-5)
optimizer = optim.SGD(model.parameters(), lr=0.1)   # nn1-6's own gradient descent, with a learning rate

for epoch in range(1000):                       # nn1-6's own epochs
    predictions = model(X_train)                # forward pass
    loss = loss_fn(predictions, y_train)
    optimizer.zero_grad()
    loss.backward()                              # nn1-5's own backpropagation, one line
    optimizer.step()                             # the weight update itself

Hands-On Exercises