Computational Linguistics 2, Exercise 1: three tokenizers

Sentence: "It's a well-known fact, isn't it?"

Code (run with py -X utf8):

import re
s = "It's a well-known fact, isn't it?"
print(s.split())
print(re.findall(r"\w+", s))
print(re.findall(r"\w+(?:[-']\w+)*", s))

Output from the run:

["It's", 'a', 'well-known', 'fact,', "isn't", 'it?']
['It', 's', 'a', 'well', 'known', 'fact', 'isn', 't', 'it']
["It's", 'a', 'well-known', 'fact', "isn't", 'it']

Which keeps each whole:
- It's: split on spaces, and the third tokenizer. The second breaks it into
  It + s.
- well-known: split on spaces, and the third. The second breaks it into
  well + known.
- isn't: split on spaces, and the third. The second breaks it into isn + t.

Only the third also removes the punctuation (fact, it?) without breaking
the contractions and the hyphenated word. Even it is only right for these
cases; the chapter showed it breaks "p.m.", so no tokenizer is correct
everywhere.