Validating Real Data

Regular Expressions in Practice

Chapter 7 · Validating Real Data

Validation is the job most people first reach for regular expressions to do: is this text a date, a phone number, an email address? The tools from the earlier chapters are enough, but there is a skill to using them well. This chapter shows the habits that make validators reliable, and is honest about the point where a pattern should hand over to ordinary code.

What was run

As in the earlier chapters, every snippet was executed and the comments show what Python printed.

Shape first, meaning second

A pattern is very good at checking shape: how many characters, of what kind, in what order. It is poor at checking meaning: whether month 13 exists, or whether 30 February is a real day. The reliable approach is to split the work. Let the pattern check the shape and capture the parts, then let normal Python check the meaning.

import re, datetime

date_re = re.compile(r"(\d{4})-(\d{2})-(\d{2})")

def valid_date(s):
    m = date_re.fullmatch(s)
    if not m:
        return False
    y, mo, d = map(int, m.groups())
    try:
        datetime.date(y, mo, d)
    except ValueError:
        return False
    return True

for s in ["2026-11-02", "2026-13-01", "2026-02-30", "2024-02-29", "26-1-2"]:
    print(repr(s), valid_date(s))
# '2026-11-02' True
# '2026-13-01' False
# '2026-02-30' False
# '2024-02-29' True
# '26-1-2' False
    

A single pattern that also knew about month lengths and leap years would be long and fragile. The standard library already knows the calendar, so the pattern's only task was to get the three numbers out safely.

Use fullmatch, not match with $

A validator must say yes only when the entire string is acceptable. There is a trap here: $ matches at the very end of the string and also just before a final newline. Input read from files or forms often has a stray newline, and it slips through:

print(re.match(r"\d+$", "42\n"))
# <re.Match object; span=(0, 2), match='42'>
print(re.fullmatch(r"\d+", "42\n"))
# None
    

fullmatch has no such gap, and it also removes the need to remember both anchors. Make it the default for validation. (The alternative is \Z, which matches only at the true end.)

Beware: \d accepts more than 0-9

For text (str) patterns, \d matches any Unicode decimal digit, not only the ASCII ones. If you mean exactly 0-9, say so, or use the re.ASCII flag.

n = "٣٤"   # two Arabic-Indic digits
print(bool(re.fullmatch(r"\d+", n)),
      bool(re.fullmatch(r"[0-9]+", n)),
      bool(re.fullmatch(r"\d+", n, re.A)))
# True False False
    

Python's int() happily converts such digits, so this is not always a problem, but a validator whose job is "ASCII digits only" should be explicit.

Phone numbers: accept messy input, store clean output

People type phone numbers with spaces, hyphens and different prefixes. A good validator tolerates the common variations and then normalises to one form. Here the pattern uses re.VERBOSE so each part is labelled, and captures just the digits it cares about.

phone_re = re.compile(r"""
    (?:\+44\s?|0)       # country code or trunk zero
    (\d{4})             # area
    [\s-]?
    (\d{6})
""", re.X)

def norm_phone(s):
    m = phone_re.fullmatch(s.strip())
    return "+44" + m[1] + m[2] if m else None

for s in ["+44 1314 960123", "01314-960123", "01314 96012", "(0131) 496 0123"]:
    print(repr(s), norm_phone(s))
# '+44 1314 960123' +441314960123
# '01314-960123' +441314960123
# '01314 96012' None
# '(0131) 496 0123' None
    

The last input is a perfectly real way to write a number, but this pattern does not allow brackets. That is a design decision: you either extend the pattern for every format you meet, or you clean the input first and validate the cleaned version. Stripping everything except digits is a one-liner:

print(re.sub(r"\D", "", "(0131) 496 0123"))
# 01314960123
    

Cleaning first and checking the digit count afterwards is often simpler than one heroic pattern that allows every punctuation style.

Email addresses: the honest version

The full rules for email addresses are enormously complicated, and a pattern that follows them exactly is famously huge. In practice you want a sanity check that catches obvious typos, not a proof that the mailbox exists. A modest pattern does that:

email_re = re.compile(r"[\w.+-]+@[A-Za-z0-9-]+(?:\.[A-Za-z0-9-]+)+")

for s in ["ann@example.com", "a.b+tag@mail.co.uk", "ann@@example.com",
          "ann@example", "@example.com", "ann@exa mple.com", "a..b@example.com"]:
    print(s, bool(email_re.fullmatch(s)))
# ann@example.com True
# a.b+tag@mail.co.uk True
# ann@@example.com False
# ann@example False
# @example.com False
# ann@exa mple.com False
# a..b@example.com True
    

It rejects the typical slips (double @, no domain dot, empty name, a space). It also accepts a..b@example.com, which strictly is not valid. We could tighten the pattern to forbid doubled dots, but each tightening makes it longer and still leaves gaps in the other direction. This is the central point of the chapter:

The limit of validation

A pattern can tell you an email address looks plausible. Only sending a message to it and getting a response proves it works. Use a loose pattern to catch typos early, then confirm by the real route (a confirmation message), rather than growing the pattern.

Numbers with a range: split the job

The same shape-then-meaning idea applies to anything with a numeric range. An IPv4 address is four numbers from 0 to 255. Writing "0 to 255" as a pattern is possible but ugly; checking int(g) <= 255 is clear.

ip_re = re.compile(r"(\d{1,3})\.(\d{1,3})\.(\d{1,3})\.(\d{1,3})")

def valid_ip(s):
    m = ip_re.fullmatch(s)
    return bool(m) and all(int(g) <= 255 for g in m.groups())

for s in ["192.168.1.1", "256.1.1.1", "1.2.3", "10.0.0.255"]:
    print(s, valid_ip(s))
# 192.168.1.1 True
# 256.1.1.1 False
# 1.2.3 False
# 10.0.0.255 True
    

Where the shape is genuinely tidy, a pattern alone is fine. A hex colour is either three or six hex digits after a #:

hex_re = re.compile(r"#(?:[0-9a-f]{3}|[0-9a-f]{6})", re.I)

for s in ["#fff", "#A1B2C3", "#ffff", "fff"]:
    print(s, bool(hex_re.fullmatch(s)))
# #fff True
# #A1B2C3 True
# #ffff False
# fff False
    

Check	Pattern alone?	Better approach
Hex colour, simple codes	Yes	`fullmatch` with a clear pattern
Calendar date	Shape only	Pattern for parts, `datetime.date` for meaning
Number in a range (time, IPv4)	Shape only	Pattern for parts, `int()` comparisons
Phone number	Shape, per format	Clean to digits, then check length and prefix
Email address	Plausibility only	Loose pattern plus a confirmation message

Checklist for a trustworthy validator

Use fullmatch, so the whole input must fit and a trailing newline is rejected.
Be explicit about character sets: [0-9] or re.A when you mean ASCII digits.
Capture the parts, then check ranges and rules in code.
Normalise on the way in so the rest of the program sees one format.
Test the failures as carefully as the successes: write the bad inputs down and run them.
Decide what "good enough" is. A validator that is slightly loose and understandable beats one that is exact and unreadable.

Challenges

Exercise 1

Write valid_time(s) for 24-hour times in the form HH:MM. Use a pattern for the shape and ordinary comparisons for the ranges (hours up to 23, minutes up to 59). Test it on 09:30, 23:59, 24:00, 12:60 and 9:30.

📄 View solution

Exercise 2

Write one compiled pattern that checks the shape of a postcode: one or two capital letters, a digit, an optional capital letter or digit, a single space, a digit, then two capital letters. Test it on EH1 1YZ, SW1A 1AA, M1 1AE, eh1 1yz, EH11 1YZ and EH1 1Y and print True or False for each.

📄 View solution

Exercise 3

A username must start with a letter, then have 2 to 15 more letters, digits or underscores. Write check(s) returning "ok" or "invalid" using fullmatch. Show that "ok\n" is rejected, and show that re.match(r"[A-Za-z]\w{2,15}$", "bob\n") wrongly accepts a trailing newline.

📄 View solution

Quick Reference

fullmatch: the validation default; whole string must match, trailing newline rejected
$ gap: $ also matches before a final newline, so match(...$) can accept "42\n"
\d: any Unicode digit for str; use [0-9] or re.A for ASCII only
Shape then meaning: capture parts with groups, then check ranges and calendars in Python
Normalise: strip or reformat input so later code sees one format
Email: a loose sanity check only; confirm by really contacting the address

Coming next

Regular Expressions in Practice 8 turns to parsing logs: pulling fields out of lines of text and building a small report from them.