Text Toolkit

Regular Expressions in Practice

Chapter 10 · Capstone: A Text Toolkit

The earlier chapters each taught one idea. In real programs the ideas arrive together, so this last chapter builds six small helpers that you could drop into any project. Each one is short, has a clear name, and uses patterns compiled once at module level (Chapter 6), with the habits from Chapter 9 built in.

What was run

Every helper and every example below was executed, and the comments show exactly what Python printed.

Design rules for a toolkit

Compile once. Patterns that never change live in module-level constants with an upper-case name.
One job per function. A function takes text and returns a result. It does not print.
Name your groups. m["y"] explains itself; m[3] does not.
Escape outside text. Anything that did not come from you goes through re.escape.
Fail loudly when it matters. A parser should raise an error on a bad line, not skip it silently.

Helper 1: squeeze whitespace

The simplest and most useful helper. It collapses every run of spaces, tabs and newlines into one space and trims the ends (Chapter 4).

import re

def squeeze(text):
    """Collapse runs of whitespace and trim the ends."""
    return re.sub(r"\s+", " ", text).strip()

print(repr(squeeze("  too   many \n spaces\there  ")))
# 'too many spaces here'
    

Helper 2: make a slug

A slug is the tidy, lower-case form of a title used in a web address. Instead of describing what to remove, describe what to keep: runs of letters and digits, then join them (Chapter 1).

def slugify(text):
    """Lower-case, keep letters and digits, join words with hyphens."""
    return "-".join(re.findall(r"[a-z0-9]+", text.lower()))

print(slugify("  Regular Expressions: in Practice! (2026) "))
# regular-expressions-in-practice-2026
    

Helper 3: find dates in two formats

This one combines alternation, named groups, finditer and a little ordinary Python (Chapters 2, 3, 6 and 7). A single pattern recognises both 2026-11-02 and 5 Dec 2026. Because the two alternatives need different group names, the code checks which one matched. The pattern only finds the shape; the function then checks the month name against a dictionary, which is a job for code, not for a pattern.

MONTHS = {m: i for i, m in enumerate(
    ["jan", "feb", "mar", "apr", "may", "jun", "jul", "aug", "sep", "oct", "nov", "dec"], 1)}
DATE = re.compile(
    r"\b(?:(?P<y>\d{4})-(?P<m>\d{2})-(?P<d>\d{2})"
    r"|(?P<d2>\d{1,2})\s+(?P<mon>[A-Za-z]{3})[a-z]*\s+(?P<y2>\d{4}))\b")

def find_dates(text):
    """Return every date found, as YYYY-MM-DD, in order."""
    found = []
    for m in DATE.finditer(text):
        if m["y"]:
            y, mo, d = int(m["y"]), int(m["m"]), int(m["d"])
        else:
            mo = MONTHS.get(m["mon"].lower())
            if mo is None:
                continue
            y, d = int(m["y2"]), int(m["d2"])
        found.append(f"{y:04d}-{mo:02d}-{d:02d}")
    return found

print(find_dates("Sent 2026-11-02, due 5 Dec 2026 and 14 September 2027; bad: 3 Foo 2020."))
# ['2026-11-02', '2026-12-05', '2027-09-14']
    

The made-up month 3 Foo 2020 has the right shape, so the pattern matches it, but the dictionary lookup rejects it. Splitting the work this way keeps the pattern readable.

Helper 4: read key=value pairs

Values may be bare words or quoted text containing spaces. The alternation tries the quoted form first. The helper checks is not None rather than truthiness, so an empty quoted value is kept as an empty string instead of being lost.

PAIR = re.compile(r'(\w+)=(?:"([^"]*)"|(\S+))')

def parse_pairs(text):
    """Turn key=value pairs (values may be quoted) into a dict."""
    result = {}
    for m in PAIR.finditer(text):
        quoted, bare = m.group(2), m.group(3)
        result[m.group(1)] = quoted if quoted is not None else bare
    return result

print(parse_pairs('user=ann id=42 note="two words" empty=""'))
# {'user': 'ann', 'id': '42', 'note': 'two words', 'empty': ''}
    

Helper 5: censor whole words safely

The list of words comes from outside, so each is escaped. Longer words go first so that a short word cannot win inside a longer one (Chapter 9). The usual \b boundary breaks for a word that ends in a symbol, such as c++, because there is no word boundary between + and a space. Lookaround (Chapter 5) says what is wanted instead: not preceded and not followed by a letter, digit or underscore.

def censor(text, words):
    """Replace whole words, case-insensitively, with asterisks."""
    if not words:
        return text
    alt = "|".join(re.escape(w) for w in sorted(words, key=len, reverse=True))
    return re.sub(rf"(?<!\w)(?:{alt})(?!\w)", lambda m: "*" * len(m[0]), text, flags=re.I)

print(censor("The Cat sat; concatenate is safe, CAT again. C++ ok", ["cat", "c++"]))
# The *** sat; concatenate is safe, *** again. *** ok
    

Notice that concatenate is left alone, because cat inside it has letters on both sides. The empty-list check matters too: an empty alternation would build the pattern (?:), which matches everywhere.

Helper 6: mask long numbers

A replacement function (Chapter 4) can compute its output from the match. Here every run of six or more digits keeps only its last four digits; shorter numbers are untouched.

def mask_digits(text, keep=4):
    """Hide all but the last `keep` digits of every run of 6+ digits."""
    return re.sub(r"\d{6,}", lambda m: "#" * (len(m[0]) - keep) + m[0][-keep:], text)

print(mask_digits("card 1234567890123456 ref 12345 id 9876543"))
# card ############3456 ref 12345 id ###6543
    

Using the helpers together

Because each helper takes text and returns text (or a simple value), they chain naturally:

note = "  Meeting   moved to 5 Dec 2026\n(was 2026-11-30).  Card 1234567890123456; ref=\"A 12\" "
clean = squeeze(note)
print(clean)
print(find_dates(clean))
print(mask_digits(clean))
print(slugify(clean)[:30])
print(len(clean) < len(note))
# Meeting moved to 5 Dec 2026 (was 2026-11-30). Card 1234567890123456; ref="A 12"
# ['2026-12-05', '2026-11-30']
# Meeting moved to 5 Dec 2026 (was 2026-11-30). Card ############3456; ref="A 12"
# meeting-moved-to-5-dec-2026-wa
# True
    

Helper	Main tools from the course	Chapter
`squeeze`	`re.sub`, `\s+`	4
`slugify`	`findall`, character classes	1
`find_dates`	alternation, named groups, `finditer`	2, 3, 6, 7
`parse_pairs`	groups, alternation, `finditer`	2, 6
`censor`	`re.escape`, lookaround, replacement function, flags	3, 4, 5, 9
`mask_digits`	counted quantifier, replacement function	1, 4

Where to go next

Add a test for every helper, including awkward inputs: empty text, a long near-miss, text full of special characters.
Use re.VERBOSE and comments as soon as a pattern passes one line (Chapter 3).
When the input has nested structure, reach for a real parser (Chapter 9).
Read the standard library documentation for the re module. You now know enough to follow every section of it.

Challenges

Exercise 1

Write redact_emails(text) that replaces every email address with [email] and returns both the new text and the number of replacements. Test it on Write to ann@example.com or bob.smith+x@mail.co.uk today. and on Nothing to hide here, only a@ and @b.

📄 View solution

Exercise 2

Write top_words(text, n=3) that returns the n most common words, ignoring case, with words such as didn't kept whole. Test it on The cat and the hat. The CAT didn't sit; the hat didn't mind. with the default and with n=5.

📄 View solution

Exercise 3

Write parse_config(text) for a small settings format: [section] headers, key = value lines, blank lines and # comments (also allowed after a value). Return a dictionary of dictionaries. Use a VERBOSE pattern with named groups, and raise ValueError with the line number for an unreadable line or a key that appears before any section.

📄 View solution

Quick Reference

Compile once: module-level upper-case constants for fixed patterns
One job per function: text in, result out, no printing
Named groups: m["name"] over m[2]
Outside text: always re.escape, and guard against an empty list
Boundaries for symbols: (?<!\w) and (?!\w) when \b will not do
Pattern plus code: let the pattern find the shape and ordinary code check the meaning
Parsers: raise an error with the line number rather than skipping bad input

Course complete

That is the end of Regular Expressions in Practice. Keep the helpers from this chapter, add your own as you meet new problems, and always test with awkward input.