prize18-10  Exercise 2: Layers and Lost Data
============================================

Predictions: for 0, 1, 2 and 3 layers of damage, peel should report 0, 1, 2 and 3 layers peeled and recover the original every time (no byte is lost, because the
tolerant decode keeps the undefined bytes). For the two irreversible strings it should change nothing: the question-mark string has no markers to remove, and the
replacement-character string does not encode back to valid UTF-8 (the original bytes are gone).

ex2.py (the toolkit from the chapter, then the experiment):

  MARKERS = ("\u00c3", "\u00c2", "\u00e2\u20ac", "\u00ef\u00bb\u00bf")

  def suspicion(s):
      return sum(s.count(m) for m in MARKERS)

  def lenient_decode(raw):
      out = []
      for b in raw:
          try:
              out.append(bytes([b]).decode("cp1252"))
          except UnicodeDecodeError:
              out.append(chr(b))
      return "".join(out)

  def lenient_encode(s):
      out = bytearray()
      for c in s:
          try:
              out += c.encode("cp1252")
          except UnicodeEncodeError:
              if ord(c) > 0xFF:
                  raise
              out.append(ord(c))
      return bytes(out)

  def undo_once(s):
      try:
          fixed = lenient_encode(s).decode("utf-8")
      except (UnicodeEncodeError, UnicodeDecodeError):
          return None
      return fixed if suspicion(fixed) < suspicion(s) else None

  def peel(s, max_layers=5):
      layers = 0
      while layers < max_layers:
          fixed = undo_once(s)
          if fixed is None:
              break
          s = fixed
          layers += 1
      return s, layers

  text = "caf\u00e9 \u20ac5 \u2014 ok"
  damaged = text
  for layers in range(4):
      fixed, n = peel(damaged)
      print(f"{layers} layers damaged: peeled {n}, recovered={fixed == text}")
      damaged = lenient_decode(damaged.encode("utf-8"))

  lost = text.encode("ascii", errors="replace").decode("ascii")
  print(lost, peel(lost))
  replaced = text.encode("utf-8").decode("ascii", errors="replace")
  print(replaced, peel(replaced))

Real output (Python 3.14):

  0 layers damaged: peeled 0, recovered=True
  1 layers damaged: peeled 1, recovered=True
  2 layers damaged: peeled 2, recovered=True
  3 layers damaged: peeled 3, recovered=True
  caf? ?5 ? ok ('caf? ?5 ? ok', 0)
  caf�� ���5 ��� ok ('caf�� ���5 ��� ok', 0)

The reversible cases are fully recovered, including three layers. The last two lines are unchanged: nothing in them can bring the lost characters back.
The only remedy is to go back to the original source and read it correctly.