Computational Linguistics 10, Exercise 1: word error rate by hand Reference: "the lamp is out" (4 words, so n = 4). 1. "the lamp was out": one substitution (is -> was). 1/4 = 0.25. 2. "the lamp out": one deletion (is missing). 1/4 = 0.25. 3. "the old lamp is out": one insertion (old). 1/4 = 0.25. Check (run with py -X utf8, after the chapter's wer function): print(wer("the lamp is out", "the lamp was out"), wer("the lamp is out", "the lamp out"), wer("the lamp is out", "the old lamp is out")) Output from the run: 0.25 0.25 0.25 Also run: "the lamp is out now please" scored 0.5 (two insertions in four reference words). All three differ from the reference by one edit, so they score the same, although in practice a change of meaning (is -> was) may matter more than a dropped word. WER counts words, not their importance; I did not look up whether the Speech recognition article comments on this.