Words by the Numbers 6 asked who wrote a text. This chapter asks a plainer question: how hard is it to read? Two things are easy to count, how long the sentences are and how long the words are, and for about 75 years people have turned those two counts into a single score. The maths is a weighted sum, so it fits on one line, and the Python is runnable. The more interesting half of the chapter is testing what the score does not see.
What was run, and what to trust
Every Python snippet in this chapter and in the three exercise solutions was executed (Python 3.14), and the output in the comments is what Python printed. The four books are plain-text files from Project Gutenberg (eBook numbers 11, 35, 1342 and 1404), which the code downloads once and saves next to the script; you need an internet connection the first time. Everything between the Gutenberg start and end markers is counted, so title pages, contents lists, headings and footnotes are in the totals; headings without a full stop run into the sentence after them. The scores are therefore approximate, and a different tokeniser, sentence splitter or syllable counter gives different numbers (that is one of the points of the chapter). The sentences in the first snippet, the two passages at the end of the third snippet and the 100-word example are invented, illustrative examples. The syllable counts in the test list are my own hand counts, and a few words can be counted two ways. The history comes from a web search that returned summaries of web pages; such fetched pages are often truncated or model-summarised, so treat the wording as a paraphrase. The search supported the two formulas, the 1948 date for Flesch's test and the 1975 Navy study, and it showed two source conflicts, noted below. Statements marked (check) were not confirmed against a source.
A Yardstick from 1948
Rudolf Flesch published his Reading Ease test in 1948, according to the search summaries (one of them places it in his book The Art of Readable Writing; I could not confirm which publication first printed it, check). The idea was that a writer could measure a draft instead of guessing. The grade-level version came later: it was developed under a US Navy contract in 1975 by J. Peter Kincaid and colleagues, so that the Navy could judge whether its training manuals matched the reading skills of recruits. The sources disagree on two details. One dates the Navy study to 1976 rather than 1975. And one says Flesch himself made the grade-level conversion, while another credits the joint Flesch and Kincaid work. Other formulas exist (Gunning Fog, SMOG and Dale-Chall are names I know of, but I did not check their details here; check). All of them are built from the same kind of ingredients.
The Formula, Worked by Hand
Both Flesch-style scores use only two ratios: the average number of words per sentence, and the average number of syllables per word. Per the search, the Reading Ease formula is 206.835 − 1.015 × (words per sentence) − 84.6 × (syllables per word), and the Grade Level formula is 0.39 × (words per sentence) + 11.8 × (syllables per word) − 15.59. Try them on an invented sample of 100 words in 5 sentences with 140 syllables. That is 20 words per sentence and 1.4 syllables per word. Reading Ease: 206.835 − 1.015 × 20 − 84.6 × 1.4 = 206.835 − 20.3 − 118.44 = 68.1. Grade Level: 0.39 × 20 + 11.8 × 1.4 − 15.59 = 7.8 + 16.52 − 15.59 = 8.7. Exercise 1 explores how the score moves when you change either ratio.
Reading Ease
Grade Level
Weight on words per sentence
−1.015
+0.39
Weight on syllables per word
−84.6
+11.8
Constant
206.835
−15.59
Higher score means
easier to read
harder to read
Reads as
a score, often described as 0 to 100
a US school grade
Notice how lopsided the weights are. One more syllable per word on average matters about 83 times more than one more word per sentence in Reading Ease (84.6 against 1.015), though the two quantities are on different scales, so the sensible comparison is a realistic nudge: 0.1 more syllables per word costs 8.5 points, one more word in every sentence only 1.0. In practice the word-length term dominates. The formulas assume that long words are hard and long sentences are hard; both are statistical tendencies in ordinary prose and not laws of reading.
Counting Sentences, Words and Syllables
The formula needs three counts, and each is a judgement. A sentence is cut at ., ! or ? followed by a space. A word is a run of letters, with an inner apostrophe allowed (compare the tokenising in Words by the Numbers 1). A syllable is the hard one: English spelling does not say how many there are. The classic shortcut is to count runs of vowel letters (a e i o u y) and then fix the commonest exceptions, such as a silent final e and the -ed ending in walked. The snippet below does that, tests the counter on 28 words I counted by hand, and scores two invented passages.
import re
def syllables(word):
w = re.sub(r"[^a-z]", "", word.lower())
ifnot w:
return0
n = len(re.findall(r"[aeiouy]+", w)) # runs of vowels = syllables, roughlyif w.endswith("e") andnot w.endswith("le") and n > 1:
n -= 1# a silent final e, as in "make"if w.endswith("ed") andnot w.endswith(("ted", "ded")) and n > 1:
n -= 1# "walked" is one syllable, "wanted" tworeturnmax(n, 1)
# Counts made by hand (my own reading of the words); the code is tested against them
HAND = {"the": 1, "make": 1, "table": 2, "reading": 2, "sentence": 2, "people": 2,
"beautiful": 3, "readability": 5, "syllable": 3, "area": 3, "idea": 3,
"quiet": 2, "science": 2, "poem": 2, "created": 3, "walked": 1,
"little": 2, "different": 3, "something": 2, "wanted": 2,
"chocolate": 3, "queue": 1, "jumped": 1, "ocean": 2, "really": 2,
"average": 3, "bakery": 3, "today": 2}
wrong = {w: (syllables(w), n) for w, n in HAND.items() if syllables(w) != n}
print(f"{len(HAND) - len(wrong)} of {len(HAND)} words right")
for w, (got, want) in wrong.items():
print(f" {w}: counted {got}, hand count {want}")
def stats(text):
sentences = [s for s in re.split(r"(?<=[.!?])\s+", text.strip()) if s]
words = re.findall(r"[A-Za-z]+(?:'[A-Za-z]+)?", text)
syl = sum(syllables(w) for w in words)
returnlen(sentences), len(words), syl
def flesch(text):
s, w, y = stats(text)
ease = 206.835 - 1.015 * (w / s) - 84.6 * (y / w)
grade = 0.39 * (w / s) + 11.8 * (y / w) - 15.59return ease, grade
samples = { # invented, illustrative"plain": "The cat sat on the mat. It was a warm day. The dog ran to the gate.",
"dense": "Notwithstanding considerable organisational reservations, the committee unanimously authorised comprehensive reconsideration of administrative responsibilities.",
}
for name, text in samples.items():
s, w, y = stats(text)
ease, grade = flesch(text)
print(f"{name:6} sentences {s} words {w} syllables {y} ease {ease:6.1f} grade {grade:5.1f}")
# 21 of 28 words right# area: counted 2, hand count 3# idea: counted 2, hand count 3# quiet: counted 1, hand count 2# science: counted 1, hand count 2# poem: counted 1, hand count 2# created: counted 2, hand count 3# something: counted 3, hand count 2# plain sentences 3 words 17 syllables 17 ease 116.5 grade -1.6# dense sentences 1 words 13 syllables 52 ease -144.8 grade 36.7
The counter gets 21 of 28 test words right (75%), which is a sensible thing to report before using it. Its errors have a pattern: vowels that sit side by side but belong to different syllables (area, idea, quiet, science, poem, created) are merged into one, and a silent e in the middle of a word (something) is counted. Errors in both directions partly cancel over a long text, but they do not cancel exactly. The two invented passages show the scale. The plain one (17 words, 17 syllables) scores 116.5 for ease, above the 100 that people often quote as the top, and a grade of −1.6. The dense one has a single 13-word sentence of 52 counted syllables and scores −144.8, a grade of 36.7. The formulas are weighted sums, so nothing stops them leaving the range; scores far outside it mean the text is outside what the formula was designed for, and they should not be read as real grades.
Four Books on the Yardstick
Now real text. The next snippet downloads four books and prints, for each, the two raw ratios and the two scores. Seeing the raw ratios next to the scores shows what is driving each result.
import pathlib, re, urllib.request
BOOKS = {"Alice in Wonderland": 11, "The Time Machine": 35,
"Pride and Prejudice": 1342, "The Federalist Papers": 1404}
def load(number):
path = pathlib.Path(f"pg{number}.txt")
ifnot path.exists(): # download once, then reuse the saved copy
url = f"https://www.gutenberg.org/cache/epub/{number}/pg{number}.txt"
req = urllib.request.Request(url, headers={"User-Agent": "Mozilla/5.0"})
path.write_bytes(urllib.request.urlopen(req).read())
raw = path.read_text(encoding="utf-8-sig").replace("\r", "")
start = raw.index("*** START OF")
start = raw.index("\n", start) + 1
end = raw.index("*** END OF")
return raw[start:end]
def syllables(word):
w = re.sub(r"[^a-z]", "", word.lower())
ifnot w:
return0
n = len(re.findall(r"[aeiouy]+", w))
if w.endswith("e") andnot w.endswith("le") and n > 1:
n -= 1if w.endswith("ed") andnot w.endswith(("ted", "ded")) and n > 1:
n -= 1returnmax(n, 1)
def stats(text):
for fancy, plain in (("’", "'"), ("“", '"'), ("”", '"')):
text = text.replace(fancy, plain) # curly quotes to plain ones
text = re.sub(r"\s+", " ", text) # one long line, so line breaks do not matter
sentences = [s for s in re.split(r"(?<=[.!?])[\"')\]]*\s+", text.strip()) if s]
words = re.findall(r"[A-Za-z]+(?:'[A-Za-z]+)?", text)
returnlen(sentences), len(words), sum(syllables(w) for w in words)
def flesch(text):
s, w, y = stats(text)
return206.835 - 1.015 * (w / s) - 84.6 * (y / w), 0.39 * (w / s) + 11.8 * (y / w) - 15.59print(f"{'book':22} {'words':>7} {'w/sent':>7} {'syl/word':>9} {'ease':>6} {'grade':>6}")
for title, number in BOOKS.items():
text = load(number)
s, w, y = stats(text)
ease, grade = flesch(text)
print(f"{title:22} {w:7,d} {w / s:7.1f} {y / w:9.3f} {ease:6.1f} {grade:6.1f}")
# book words w/sent syl/word ease grade# Alice in Wonderland 26,776 16.5 1.283 81.6 6.0# The Time Machine 32,772 16.8 1.397 71.5 7.5# Pride and Prejudice 127,864 17.6 1.448 66.5 8.3# The Federalist Papers 192,302 31.9 1.638 35.9 16.2
Book
Words per sentence
Syllables per word
Ease
Grade
Alice in Wonderland
16.5
1.283
81.6
6.0
The Time Machine
16.8
1.397
71.5
7.5
Pride and Prejudice
17.6
1.448
66.5
8.3
The Federalist Papers
31.9
1.638
35.9
16.2
The first three books have almost the same sentence length (16.5 to 17.6 words), so their order is set by word length: 1.283, 1.397 and 1.448 syllables per word. The Federalist essays (the newspaper essays of 1787 onward from Words by the Numbers 6) are in a different league on both counts, with sentences nearly twice as long and the longest words, giving a grade of 16.2, roughly university level on this scale. That ordering matches what a reader would expect, which is the real justification for the method: it ranks well, even if the individual numbers are rough. Treat a grade of 8.3 as "about like Pride and Prejudice and unlike the Federalist", not as a measured fact about a school year.
What the Scores Miss
A readability score is a summary of two averages, so anything that does not change those averages is invisible to it. The last snippet runs four experiments on Pride and Prejudice: shuffling the words inside each sentence, changing how sentences are cut, looking at the spread inside one book, and scoring two invented passages that the formula misjudges.
import pathlib, re, urllib.request
BOOKS = {"Alice in Wonderland": 11, "The Time Machine": 35,
"Pride and Prejudice": 1342, "The Federalist Papers": 1404}
def load(number):
path = pathlib.Path(f"pg{number}.txt")
ifnot path.exists(): # download once, then reuse the saved copy
url = f"https://www.gutenberg.org/cache/epub/{number}/pg{number}.txt"
req = urllib.request.Request(url, headers={"User-Agent": "Mozilla/5.0"})
path.write_bytes(urllib.request.urlopen(req).read())
raw = path.read_text(encoding="utf-8-sig").replace("\r", "")
start = raw.index("*** START OF")
start = raw.index("\n", start) + 1
end = raw.index("*** END OF")
return raw[start:end]
def syllables(word):
w = re.sub(r"[^a-z]", "", word.lower())
ifnot w:
return0
n = len(re.findall(r"[aeiouy]+", w))
if w.endswith("e") andnot w.endswith("le") and n > 1:
n -= 1if w.endswith("ed") andnot w.endswith(("ted", "ded")) and n > 1:
n -= 1returnmax(n, 1)
def stats(text):
for fancy, plain in (("’", "'"), ("“", '"'), ("”", '"')):
text = text.replace(fancy, plain) # curly quotes to plain ones
text = re.sub(r"\s+", " ", text) # one long line, so line breaks do not matter
sentences = [s for s in re.split(r"(?<=[.!?])[\"')\]]*\s+", text.strip()) if s]
words = re.findall(r"[A-Za-z]+(?:'[A-Za-z]+)?", text)
returnlen(sentences), len(words), sum(syllables(w) for w in words)
def flesch(text):
s, w, y = stats(text)
return206.835 - 1.015 * (w / s) - 84.6 * (y / w), 0.39 * (w / s) + 11.8 * (y / w) - 15.59import random, statistics
text = load(1342) # Pride and Prejudice
ease, grade = flesch(text)
print(f"original text: ease {ease:.1f}, grade {grade:.1f}")
# 1. Shuffle the words inside every sentence: the score cannot change
random.seed(1)
def scramble(sentence):
ws = sentence.split()
random.shuffle(ws)
return" ".join(ws)
plain = re.sub(r"\s+", " ", text)
sents = re.split(r"(?<=[.!?])\s+", plain.strip())
mixed = " ".join(scramble(s) for s in sents)
ease2, grade2 = flesch(mixed)
print(f"words shuffled inside each sentence: ease {ease2:.1f}, grade {grade2:.1f}")
print("example:", scramble(sents[1000]))
# 2. The sentence splitter is a hidden choice: "Mr." and "Mrs." look like full stops
abbr = len(re.findall(r"\b(?:Mr|Mrs|Dr|Ms|St)\.", text))
guarded = re.sub(r"\b(Mr|Mrs|Dr|Ms|St)\.", r"\1", text)
ease3, grade3 = flesch(guarded)
print(f"abbreviation full stops found: {abbr}")
print(f"with them removed: ease {ease3:.1f}, grade {grade3:.1f}")
# 3. How much does the score move from place to place inside one book?
words = text.split()
chunks = [" ".join(words[i:i + 1000]) for i inrange(0, len(words) - 1000, 1000)]
grades = [flesch(c)[1] for c in chunks]
print(f"{len(chunks)} chunks of 1,000 words: grade from {min(grades):.1f} to {max(grades):.1f}, "f"mean {statistics.mean(grades):.1f}, sd {statistics.stdev(grades):.1f}")
# 4. Two invented passages with a similar score but very different difficulty
easy_words_hard_idea = "If the set is not the whole, the rest is not in it, so it can not have it."
long_words_easy_idea = "Wonderful holidays, delicious chocolate, remarkable fireworks, beautiful family gatherings."for label, t in (("short words, hard to follow", easy_words_hard_idea), ("long words, easy to follow", long_words_easy_idea)):
e, g = flesch(t)
print(f"{label:28} ease {e:6.1f} grade {g:5.1f}")
# original text: ease 66.5, grade 8.3# words shuffled inside each sentence: ease 66.5, grade 8.3# example: herself at week, receive them calculated Netherfield on the could before. with which her following remaining Bennet, exactly Tuesday, had Jane’s pleasure till who would to daughters not finish bring# abbreviation full stops found: 1167# with them removed: ease 63.1, grade 9.7# 127 chunks of 1,000 words: grade from 4.5 to 15.7, mean 8.6, sd 2.1# short words, hard to follow ease 98.5 grade 4.2# long words, easy to follow ease -65.5 grade 24.6
Pride and Prejudice, same text
Ease
Grade
As downloaded
66.5
8.3
Words shuffled inside each sentence
66.5
8.3
Full stops after Mr, Mrs and similar removed
63.1
9.7
1,000-word pieces (127 of them)
grade from 4.5 to 15.7, mean 8.6, sd 2.1
Four lessons. First, the shuffled text keeps exactly the same score, 66.5 and 8.3, though it has become nonsense: the formula never looks at word order, grammar or meaning. Second, the sentence splitter is a hidden choice. The book has 1,167 abbreviation full stops (Mr., Mrs. and so on), and each one is counted as a sentence end by the simple splitter. Removing them moves the grade from 8.3 to 9.7, a difference as big as the gap between Alice in Wonderland and The Time Machine in the table above. Third, one book is not one number: 1,000-word pieces of the same novel range from grade 4.5 to 15.7, a standard deviation of 2.1 grades, so a single average hides a lot (my guess is that dialogue pages score low and long reflective paragraphs high, but I did not check this). Fourth, the two invented passages disagree with how a reader would judge them. A sentence in short words that is hard to follow (If the set is not the whole, the rest is not in it, so it can not have it.) scores 98.5 for ease, a grade of 4.2, because every word is short; a list of long but familiar words (Wonderful holidays, delicious chocolate, remarkable fireworks, beautiful family gatherings.) scores −65.5, a grade of 24.6. Which passage is harder is my judgement and not a measurement, but I expect most readers to agree that the formula has it backwards. Familiar long words, an unfamiliar topic, layout, the reader's background and the structure of an argument all matter, and none of them is in the formula.
Common mistakes
Treating a score as a measurement of understanding rather than of surface length; comparing scores from different tools without noting that their sentence splitters and syllable counters differ; scoring a text with headings, lists, tables or code in it as if it were prose; scoring very short texts, where one sentence moves the average; reading a grade of 24 or an ease of −145 as a real level; rewriting a text to hit a number (chopping sentences into fragments and swapping words for shorter ones raises the score and can make the writing worse); trusting the syllable counter without testing it on words you counted yourself; and reporting one figure for a long document when its parts range widely.
Exercise 1
Write a function for the Reading Ease formula that takes the number of words, sentences and syllables. Check it on the 100-word example (5 sentences, 140 syllables), then print a small table of scores for 10, 20 and 30 words per sentence with 1.2, 1.5 and 1.8 syllables per word. Solve the formula for the syllables per word needed to score 60 at 15 words per sentence, and say which change moves the score more.
Compare two syllable counters on the four books: the careful one from this chapter and a naive one that only counts runs of vowel letters. Print the grade level from each, and the difference. Does the choice of counter change the order of the books? Does it change the size of the gaps between them?
Cut each book into 1,000-word pieces and score every piece. For each book, print the number of pieces, the mean and standard deviation of the grade, the lowest and highest, and the share of pieces above grade 10. Is the spread inside a book smaller or larger than the gap between the books?