prize18-10  Exercise 1: A Symptom Reporter
==========================================

Plan: one small test per row of the field guide. Each test looks for a sign of that fault in the string and adds a finding. The tests are heuristics: a clean
string passes, but unusual text can fool them (a "?" between letters is only a hint, and genuine text can contain "Ã").

ex1.py:

  import re

  BACKSLASH = chr(92)

  def report(s):
      found = []
      if "\ufffd" in s:
          found.append("U+FFFD present: data was already lost")
      if re.search(r"[A-Za-z]\?[A-Za-z]", s):
          found.append("'?' inside a word: maybe lossy ASCII conversion (check)")
      if "\u00ef\u00bb\u00bf" in s:
          found.append("BOM bytes shown as text: UTF-8 BOM read as a single-byte encoding")
      if "\u00c3" in s or "\u00e2\u20ac" in s:
          found.append("UTF-8 bytes read as Windows-1252/Latin-1 (mojibake)")
      if re.search(r"&#\d+;|&#x[0-9a-fA-F]+;", s):
          found.append("HTML numeric entities in plain text")
      if re.search(re.escape(BACKSLASH) + "u[0-9a-fA-F]{4}", s):
          found.append("literal backslash-u escapes (text not decoded)")
      if "\x00" in s:
          found.append("NUL characters: 16-bit text read as 8-bit?")
      return found or ["nothing suspicious"]

  samples = [
      "caf\u00c3\u00a9",
      "caf\u00e2\u20ac\u2122s",
      "\u00ef\u00bb\u00bfname",
      "caf\ufffd\ufffd",
      "café",
      "caf" + BACKSLASH + "u00e9",
      "h\x00i\x00",
      "plain text",
  ]
  for s in samples:
      print(f"{ascii(s):30} {report(s)}")

Real output (Python 3.14):

  'caf\xc3\xa9'                  ['UTF-8 bytes read as Windows-1252/Latin-1 (mojibake)']
  'caf\xe2\u20ac\u2122s'         ['UTF-8 bytes read as Windows-1252/Latin-1 (mojibake)']
  '\xef\xbb\xbfname'             ['BOM bytes shown as text: UTF-8 BOM read as a single-byte encoding']
  'caf\ufffd\ufffd'              ['U+FFFD present: data was already lost']
  'café'                    ['HTML numeric entities in plain text']
  'caf\\u00e9'                   ['literal backslash-u escapes (text not decoded)']
  'h\x00i\x00'                   ['NUL characters: 16-bit text read as 8-bit?']
  'plain text'                   ['nothing suspicious']

Each sample triggers exactly its own finding. The BOM sample contains no "Ã", so it is reported only as a BOM (the markers are tested separately).
The Python source avoids typing a backslash-u in a message, because the interpreter would try to read it as an escape: the code builds it with chr(92).