chatbot1-11  Exercise 1: What Reaches the Log
==============================================

ex1.py (save in the project root):

  import io, logging
  from bot.observability import SECRET_PATTERNS, scrub, setup_logging
  import re

  stream = io.StringIO()
  logger = setup_logging(logging.StreamHandler(stream), name="ex1")

  lines = [
      "order ORD-482913 shipped to the customer",
      "retrying with token xoxb-1234567890-abcdefghijklmnop",
      "support ticket TCK-2026-0042 opened by ada@example.com",
      "user reported 'my password is hunter2'",
  ]
  for line in lines:
      logger.info(line)
  print("BEFORE adding a rule (what reached the log):")
  for l in stream.getvalue().splitlines():
      print("  ", l.split('"message": ')[1].split(', "correlation_id"')[0])

  # a rule for something specific to this business: order numbers, which identify a customer's purchase
  SECRET_PATTERNS.append((re.compile(r"\bORD-\d{6}\b"), "[order-id]"))
  stream.truncate(0); stream.seek(0)
  for line in lines:
      logger.info(line)
  print("AFTER adding ORD-###### as a pattern:")
  for l in stream.getvalue().splitlines():
      print("  ", l.split('"message": ')[1].split(', "correlation_id"')[0])
  print()
  print("Still not caught: the free-text password, and the ticket number. A pattern list only catches shapes you thought of.")
  print("The only reliable rule is to log fewer free-text fields in the first place.")

Run with:  py ex1.py

Output:

  BEFORE adding a rule (what reached the log):
     "order ORD-482913 shipped to the customer"
     "retrying with token [slack-token]"
     "support ticket TCK-[number] opened by [email]"
     "user reported 'my password is hunter2'"
  AFTER adding ORD-###### as a pattern:
     "order [order-id] shipped to the customer"
     "retrying with token [slack-token]"
     "support ticket TCK-[number] opened by [email]"
     "user reported 'my password is hunter2'"

  Still not caught: the free-text password, and the ticket number. A pattern list only catches shapes you thought of.
  The only reliable rule is to log fewer free-text fields in the first place.

Reading the result:
  - The Slack token and the email were caught by the built-in patterns.
  - ORD-482913 was NOT caught until a pattern was added for that shape; afterwards it became [order-id].
  - "TCK-2026-0042" turned into "TCK-[number]": the phone-number pattern over-matched the digits. That is harmless here, and shows that patterns are blunt.
  - "my password is hunter2" passed straight through. No pattern can recognise a password written in a sentence.
The real fix for the last case is not a better pattern: do not log free text from users at all. Log what happened (a flag, a count, a duration), not what was said.
In a real test suite, do not append to SECRET_PATTERNS globally: it is a module-level list, so a pattern added in one test would affect every later test. Copy the list first and restore it afterwards.

WHY THIS WORKS AS AN ANSWER
---------------------------
The ScrubFilter runs on every record and applies each (pattern, replacement) pair in order, then chapter 6's redact(). Adding your own rule is one list entry. The exercise shows both halves of the lesson: patterns are cheap to extend and reliably catch what you describe, and they cannot be a complete defence, because they only know shapes.