Named Entity Recognition & Part-of-Speech Tagging

NLP

Chapter 7 · Named Entity Recognition & Part-of-Speech Tagging

nlp1-6's own model produced exactly one output for an entire sequence. This chapter needs one output per token instead — and the architectural change required is smaller than it sounds.

Two Real, Practical Tasks

Named Entity Recognition (NER) — identifying and classifying named entities in text:

Sarah	works	at	Google	in	London	.
PER	O	O	ORG	O	LOC	O

Part-of-Speech (POS) tagging — assigning each word its grammatical role:

Sarah	works	at	Google	in	London	.
NOUN	VERB	ADP	PROPN	ADP	PROPN	PUNCT

Both are sequence labeling tasks — a genuine, distinct category from nlp1-6's own sequence classification: one label for every token, not one label for the whole sequence.

The Architectural Change — Smaller Than It Sounds

nlp1-6's own model discarded every hidden state except the very last one, using it as a single summary of the entire sequence. Sequence labeling keeps every hidden state — one per time step — and feeds each one individually through the same small classifier head, producing one prediction per token instead of one prediction total.

class SequenceLabeler(nn.Module):
    def __init__(self, vocab_size, embed_dim, hidden_dim, num_tags):
        super().__init__()
        self.embedding = nn.Embedding(vocab_size, embed_dim)
        self.lstm = nn.LSTM(embed_dim, hidden_dim, batch_first=True, bidirectional=True)
        self.output = nn.Linear(hidden_dim * 2, num_tags)

    def forward(self, x):
        embedded = self.embedding(x)
        outputs, _ = self.lstm(embedded)     # every hidden state, not just the last (nlp1-6's own change)
        return self.output(outputs)           # one prediction per token, via softmax over num_tags

Softmax, not nlp1-6's own sigmoid

nlp1-6's sentiment task was binary — one sigmoid-activated neuron, exactly nn1-1's own logistic regression. NER and POS tagging are usually genuine multi-class problems (several possible entity types or grammatical roles per token) — the output layer here needs num_tags outputs with a softmax, not a single sigmoid. A real, honest distinction worth naming rather than silently reusing the wrong output layer.

Bidirectional LSTMs — Why "Later" Words Help Too

"Washington" could be a person's surname or a place — often only resolvable by what comes after it in the sentence, not before. nn1-8's own plain LSTM only ever sees what came earlier. A bidirectional LSTM (BiLSTM) runs two LSTMs over the same sequence — one forward, one backward — and concatenates both hidden states at every position, so each token's own final representation genuinely reflects the whole sentence, not just its own left-hand context.

Hands-On Exercises

Exercise 1

Explain the specific architectural difference between this chapter's own sequence-labeling model and nlp1-6's own sequence-classification model, using this chapter's own code and nlp1-6's own code to identify exactly what changed.

📄 View solution

Exercise 2

Explain why this chapter's own output layer needs softmax over multiple tags rather than nlp1-6's own single sigmoid neuron, and explain what would go wrong if the sigmoid approach were reused unchanged for NER.

📄 View solution

Exercise 3

Using this chapter's own "Washington" example, explain why a plain, forward-only LSTM (nn1-8) can genuinely struggle with NER specifically, and explain what a bidirectional LSTM actually adds to fix it.

📄 View solution

Chapter 7 Quick Reference

NER — classify named entities (PER/ORG/LOC/...) · POS tagging — classify grammatical role, both per-token
Sequence labeling (one label per token) vs. nlp1-6's own sequence classification (one label per sequence)
The change: keep every hidden state instead of discarding all but the last, and feed each one through the classifier head
Softmax over multiple tags, not nlp1-6's own binary sigmoid — a genuine, honest distinction
Bidirectional LSTM — forward + backward passes concatenated, so later context can disambiguate earlier tokens too
Next chapter: From RNNs to Attention — NLP's Own Motivation for Transformers