prize18-4  Exercise 3: Sniffing a Byte Order Mark
==================================================

Order matters: the UTF-32 little-endian BOM is FF FE 00 00, which begins with the UTF-16 little-endian BOM FF FE. If UTF-16 were tested first,
a UTF-32 file would be mistaken for UTF-16. So test the longer signatures first. (The reverse risk remains: a UTF-16 LE file whose first character is
a NUL would look like UTF-32. A BOM is strong evidence, not proof.)

ex3.py:

  import codecs

  BOMS = [
      (codecs.BOM_UTF32_LE, "utf-32-le"),   # must come before UTF-16 LE: FF FE 00 00 starts with FF FE
      (codecs.BOM_UTF32_BE, "utf-32-be"),
      (codecs.BOM_UTF8, "utf-8"),
      (codecs.BOM_UTF16_LE, "utf-16-le"),
      (codecs.BOM_UTF16_BE, "utf-16-be"),
  ]

  def sniff(data):
      for bom, name in BOMS:
          if data.startswith(bom):
              return name, data[len(bom):]
      return None, data

  for enc in ["utf-8-sig", "utf-16", "utf-16-be", "utf-32"]:
      data = "Hi".encode(enc)
      name, rest = sniff(data)
      print(f"{enc:10} {data.hex(' '):28} -> {name}, {rest.decode(name) if name else '?'}")

  print(sniff(b"Hi"))
  print(sniff(b"\xff\xfe\x00\x00"))

Real output (Python 3.14):

  utf-8-sig  ef bb bf 48 69               -> utf-8, Hi
  utf-16     ff fe 48 00 69 00            -> utf-16-le, Hi
  utf-16-be  00 48 00 69                  -> None, ?
  utf-32     ff fe 00 00 48 00 00 00 69 00 00 00 -> utf-32-le, Hi
  (None, b'Hi')
  ('utf-32-le', b'')

Note the third row: "utf-16-be" writes no BOM, so nothing is found. No BOM does not mean no encoding; it only means this method cannot tell you.
The last line shows the ambiguity: four bytes FF FE 00 00 are reported as an empty UTF-32 file, though they could be a UTF-16 BOM plus a NUL.