Capstone: Building a Real NLP Pipeline & Why LLMs Are Different

NLP

Chapter 10 · Capstone: Building a Real NLP Pipeline & Why LLMs Are Different

Nine chapters, each solving one specific, honestly-diagnosed problem. This capstone assembles the pieces that actually survive into a real, working sentiment-classification pipeline — then closes with the question this whole course has been quietly building toward: what does an LLM actually do differently?

The Pipeline

class SentimentClassifier(nn.Module):
    def __init__(self, glove_vectors, hidden_dim):
        super().__init__()
        # nlp1-9: pretrained GloVe, frozen — this dataset is small, per nlp1-9's own guidance
        self.embedding = nn.Embedding.from_pretrained(glove_vectors, freeze=True)
        # nlp1-6: LSTM sequence model, resolving nlp1-4's order-and-meaning problem together
        self.lstm = nn.LSTM(glove_vectors.shape[1], hidden_dim, batch_first=True)
        # nlp1-6: classifier head reading only the final hidden state — one label per sequence
        self.output = nn.Linear(hidden_dim, 1)

    def forward(self, x):
        embedded = self.embedding(x)              # nlp1-1: x is already tokenized/cleaned text
        _, (hidden, _) = self.lstm(embedded)
        return torch.sigmoid(self.output(hidden[-1]))   # binary sentiment, per nlp1-6

Every line here has a specific origin. Text arrives already tokenized and cleaned (nlp1-1), is converted to vectors using pretrained, frozen GloVe embeddings rather than training from scratch on this project's own small dataset (nlp1-9, resolving the exact cold-start problem that chapter diagnosed), processed in order by an LSTM so word order genuinely affects the result (nlp1-6, resolving nlp1-4's own two-problem cliffhanger together with the embeddings), and classified from the final hidden state through a single sigmoid output (nlp1-6's own binary classifier head).

Chapter Attribution Table

ChapterWhat it contributed
nlp1-1Tokenization, stopword removal, lemmatization — the cleaning step every input passes through first
nlp1-2 / nlp1-3Bag-of-words and TF-IDF — early vectorization approaches, superseded in this pipeline per nlp1-4's own diagnosis, but the first real proof that text could be turned into numbers at all
nlp1-4Proved bag-of-words loses both word order and word meaning — the two-problem diagnosis this whole pipeline exists to resolve
nlp1-5Introduced word embeddings and word2vec — the meaning-aware vector idea this capstone uses via nlp1-9's pretrained version instead of training from scratch
nlp1-6The LSTM sequence-classification architecture used directly in this capstone's own model
nlp1-7Extended the same architecture to per-token labeling (NER/POS) — a genuinely different pipeline from this capstone's own sequence classifier, not used here
nlp1-8Attention and self-attention, motivated by machine translation — conceptual foundation only; not implemented in this capstone (see scope note)
nlp1-9Pretrained GloVe embeddings, used frozen in this capstone's own embedding layer, directly per that chapter's own small-dataset guidance

Why LLMs Are Different

Look honestly at what this course actually built. nlp1-6's sentiment classifier and nlp1-7's NER tagger share the same underlying LSTM mechanism — but they are genuinely different pipelines: different output layers, different training objectives, separately trained models. Every task this course covered got its own purpose-built, hand-assembled architecture. That's not a flaw in how the course was taught — it's an honest reflection of how NLP actually worked before LLMs.

Three real differences, not just "bigger"
  • Scale. nlp1-9's GloVe was trained on billions of words to produce a static lookup table of word vectors. An LLM is trained on comparably massive (often larger) corpora to learn an entire deep network end to end — not a fixed table of word meanings, but a full model of how language behaves in context.
  • Self-supervised pretraining, taken further. Word2vec and GloVe already used a self-supervised signal — predict a word from its context, with no hand-labeled data required (nlp1-5, nlp1-9). LLMs scale that exact idea up to predicting the next token across a massive corpus, and that single pretraining objective turns out to be enough to learn grammar, facts, sentiment, and style all at once — capabilities this course had to teach as separate chapters.
  • One flexible architecture vs. many task-specific pipelines. This capstone's own model cannot do NER; nlp1-7's own model cannot do sentiment classification the way this capstone does. An LLM, once pretrained, can be steered toward sentiment classification, NER, translation, and more — often through prompting alone, sometimes through the same kind of fine-tuning nlp1-9 introduced for embeddings, but applied to an entire pretrained network rather than just a lookup table.

Put plainly: this course taught how to hand-build the right tool for each job. An LLM is closer to one very large, very capable tool that has already seen enough language to be pointed at most jobs without being rebuilt from scratch each time. Neither approach makes the other obsolete to understand — knowing exactly what a purpose-built pipeline is doing, and why, is exactly what makes it possible to reason honestly about what a much larger, more opaque model might be doing instead.

Hands-On Exercises

Exercise 1

Using this capstone's own code, trace each line back to the specific chapter it came from, and explain why nlp1-2/nlp1-3's own bag-of-words and TF-IDF approaches are listed in the attribution table but not actually used in the final pipeline.

📄 View solution
Exercise 2

Explain why nlp1-6's own sentiment classifier and nlp1-7's own NER tagger count as genuinely different pipelines despite sharing the same underlying LSTM mechanism, and explain what this reveals about how NLP worked before LLMs.

📄 View solution
Exercise 3

Explain each of this chapter's own three named differences between this course's own techniques and LLMs (scale, self-supervised pretraining taken further, one flexible architecture vs. many task-specific pipelines), using a specific example from earlier in this course for each one.

📄 View solution

Scope Note — What This Capstone Deliberately Doesn't Do

Honestly out of scope
  • No self-attention or Transformer implementation — nlp1-8 covered the motivation only; the full mechanism is deliberately deferred to llm1.
  • No NER/POS pipeline — nlp1-7's own bidirectional, per-token architecture is a genuinely separate model, not folded into this sentiment classifier.
  • No fine-tuning of the GloVe embeddings — frozen only, per nlp1-9's own guidance for a small dataset.
  • No production deployment, serving, or monitoring — this capstone is a working pipeline, not a shipped product.

Chapter 10 Quick Reference — Course Summary

  • The pipeline: tokenize/clean (1) → pretrained frozen embeddings (9, resolving 5) → LSTM sequence model (6, resolving 4) → sigmoid classifier head (6)
  • What was diagnosed and fixed: word order and word meaning as two genuinely independent problems (4), solved separately by sequence modeling (6) and embeddings (5/9)
  • What generalizes beyond this capstone: the same LSTM idea, extended to per-token output for NER/POS (7); attention as NLP's own real motivation for what becomes the Transformer (8)
  • Why LLMs are different: far greater scale, self-supervised pretraining pushed to learn many capabilities at once, and one flexible architecture in place of many hand-built pipelines
  • This completes the NLP course (10 chapters) — the direct bridge into llm1's own full transformer/LLM coverage