Lookaround

Regular Expressions in Practice

Chapter 5 · Lookaround

Every piece of a pattern so far has either matched characters or failed. Lookaround is different: it checks what sits just before or just after the current position, then reports yes or no without using up any characters. Because the check has no width, the text it inspects is not part of the match, and the next part of the pattern starts from the same spot.

What was run

As in the earlier chapters, every snippet was executed and the comments show what Python printed.

Syntax	Name	Succeeds when the text...
`(?=...)`	Positive lookahead	after this point matches `...`
`(?!...)`	Negative lookahead	after this point does not match `...`
`(?<=...)`	Positive lookbehind	before this point matches `...`
`(?<!...)`	Negative lookbehind	before this point does not match `...`

Lookahead

A positive lookahead (?=...) demands that something follows, but leaves it out of the result.

import re

print(re.findall(r"\w+(?=:)", "key: value, other: 2"))
# ['key', 'other']

print(re.findall(r"\d+(?= dollars)", "5 dollars, 7 euros, 12 dollars"))
# ['5', '12']
    

The colon and the word "dollars" were checked but are not in the results, which hold only the key names and the numbers. Compare that with \w+:, which would have returned 'key:' with the colon attached.

A negative lookahead (?!...) demands that something does not follow.

print(re.findall(r"\bfoo(?!bar)\w*", "foobar foobaz foo"))
# ['foobaz', 'foo']

print(re.findall(r"\bcat\b(?! food)", "cat food, cat toy, cat"))
# ['cat', 'cat']
    

The first pattern reads "foo, as long as it is not immediately followed by bar, then the rest of the word". The second finds every cat except the one in cat food.

Lookbehind

Lookbehind looks at the text before the current position. (?<=...) requires it; (?<!...) forbids it.

print(re.findall(r"(?<=\$)\d+", "cost $30 or 40 or $5"))
# ['30', '5']

print(re.findall(r"(?<!\$)\b\d+", "cost $30 or 40 or $5"))
# ['40']

print(re.findall(r"(?<=<b>).*?(?=</b>)", "<b>one</b> and <b>two</b>"))
# ['one', 'two']
    

In the second pattern the \b matters. Without it, the engine could start in the middle of $30, at the 0, where the character before is a 3, not a dollar sign, and report a stray 0. The word boundary stops a match from starting in the middle of a number. The third pattern pairs a lookbehind and a lookahead to pull out the text between two tags, leaving the tags out.

Lookbehind must have a fixed width

Python needs to know how far back to step, so the text inside a lookbehind must have a fixed length. Quantifiers such as + and * are not allowed there.

try:
    re.compile(r"(?<=a+)b")
except re.error as e:
    print("error:", e)
# error: look-behind requires fixed-width pattern

print(re.findall(r"(?<=ab)c|(?<=x)c", "abc xc zc"))
# ['c', 'c']
    

Alternatives of different widths inside one lookbehind are rejected too. The workaround is the one on the last line: write a separate lookbehind for each width and join them with | outside. (Third-party modules such as regex lift this restriction; the standard re module does not.)

Zero-width means nothing is consumed

Because a lookahead does not move the position, it can be used to find overlapping matches, which a normal pattern cannot, since each match uses up its characters. The trick is to put the whole pattern inside a lookahead and capture it.

print(re.findall(r"\w\w", "abcd"))         # ['ab', 'cd']
print(re.findall(r"(?=(\w\w))", "abcd"))   # ['ab', 'bc', 'cd']
print(re.findall(r"(?=(aa))", "aaaa"))    # ['aa', 'aa', 'aa']
    

At each position the lookahead checks whether two word characters follow and captures them; nothing is consumed, so the engine steps forward one character at a time and every position gets its chance.

Lookaround in practice

Matching a position, not text

A pattern made only of lookarounds matches an empty string at a position. Combined with re.sub and re.split from the last chapter, that lets you insert or cut at a place without deleting anything.

# Insert thousands separators: a position that follows a digit and
# is followed by a whole number of three-digit groups up to a boundary.
t = "Sales: 1234567 units, 89 units, 1000000 units, 999 units"
print(re.sub(r"(?<=\d)(?=(?:\d{3})+\b)", ",", t))
# Sales: 1,234,567 units, 89 units, 1,000,000 units, 999 units

# camelCase to snake_case: insert "_" between a lowercase letter/digit
# and an uppercase letter.
for w in ["camelCaseWordsHere", "myVar2Name", "already_snake"]:
    print(w, "->", re.sub(r"(?<=[a-z0-9])(?=[A-Z])", "_", w).lower())
# camelCaseWordsHere -> camel_case_words_here
# myVar2Name -> my_var2_name
# already_snake -> already_snake

# Split after each comma but keep the comma with its piece.
print(re.split(r"(?<=,)", "a,b,c"))
# ['a,', 'b,', 'c']
    

In the thousands example the number 89 and 999 are untouched: from the position inside them, no whole number of three-digit groups reaches the end of the number. (This simple version assumes whole numbers; a decimal such as 1234.5678 would be mishandled, a reminder that every pattern has assumptions.)

Several conditions at once

Several lookaheads placed at the start of a pattern all test from the same position, so together they act as a checklist of independent requirements. This is the standard way to express "contains X and Y and not Z".

pw = r"^(?=.*\d)(?=.*[A-Z])(?!.*\s).{8,12}$"

for s in ["Abcdefg1", "abcdefg1", "Abcdefg 1", "Abcdefghijkl1", "Ab1"]:
    print(repr(s), bool(re.fullmatch(pw, s)))
# 'Abcdefg1' True
# 'abcdefg1' False        (no capital)
# 'Abcdefg 1' False       (contains a space)
# 'Abcdefghijkl1' False   (13 characters, too long)
# 'Ab1' False             (too short)
    

Each (?=.*X) means "somewhere ahead there is an X". They do not consume anything, so the final .{8,12} still gets to match the whole string.

Skipping lines

text = "# note\ncode\n  # x\nmore"
print(re.findall(r"^(?!#).+", text, flags=re.M))
# ['code', '  # x', 'more']
    

The pattern keeps lines that do not start with #. Note that ' # x' survives, because its first character is a space; a pattern is only as clever as what you wrote.

When lookaround is the wrong tool

If you only need the part you captured, a normal capturing group is simpler: re.findall(r"(\w+):", s) returns the same keys as \w+(?=:). Reach for lookaround when the surrounding text must not be consumed (overlaps, substitutions at a position) or when you need several independent conditions on the same text.

Challenges

Exercise 1

Given "Sales: 1234567 units, 89 units, 1000000 units, 999 units", use re.sub with only lookarounds in the pattern to insert commas. Then explain, in a sentence, why 89 and 999 are not changed.

📄 View solution

Exercise 2

Write one pattern, using lookaheads, that accepts a string only if it is 6 to 10 characters long, contains at least one lowercase letter and at least one digit, and contains no spaces. Test it on "abc123", "ABC123", "abcdef", "abc 123" and "a1".

📄 View solution

Exercise 3

From "a [one] b [two words] c [] d", extract the text inside square brackets without the brackets, using lookbehind and lookahead. Make the first version return the empty brackets as an empty string, then change it so that empty brackets are ignored.

📄 View solution

Quick Reference

(?=...) / (?!...): what follows must / must not match; nothing is consumed
(?<=...) / (?<!...): what precedes must / must not match; the inside needs a fixed width
Different widths: use separate lookbehinds joined with | outside them
Overlapping matches: (?=(pattern)) with findall
Position-only patterns: combine lookarounds with re.sub or re.split to insert or cut without deleting
Checklists: several (?=...) at the start all test from the same position

Coming next

Regular Expressions in Practice 6 covers compiling patterns with re.compile and walking through matches one at a time with finditer.