chatbot1-7  Exercise 3: A Full Reliability Stack
==================================================

ex3.py (save in the project root):

  import asyncio
  from bot.providers import FakeProvider, ServerError
  from bot.reliability import BreakerProvider, CircuitBreaker, FallbackProvider, RetryingProvider

  MSG = [{"role": "user", "content": "hi"}]

  class Clock:
      now = 0.0
      def __call__(self): return self.now

  class NoSleep:
      async def __call__(self, s): pass

  async def main():
      clock = Clock()
      # the primary fails on its 3rd to 8th calls and works otherwise
      primary = FakeProvider(["primary ok"] * 2 + [ServerError("down")] * 6 + ["primary ok"] * 20)
      primary.name = "primary"
      backup = FakeProvider(["backup ok"] * 20)
      backup.name = "backup"
      breaker = CircuitBreaker(failure_threshold=2, reset_after=20, clock=clock)
      stack = FallbackProvider([BreakerProvider(RetryingProvider(primary, max_attempts=2, sleep=NoSleep()), breaker),
                                RetryingProvider(backup, max_attempts=2, sleep=NoSleep())])
      print(f"{'request':>8} {'time':>5} {'breaker':<10} {'answered by':<26} primary calls")
      for n in range(1, 13):
          clock.now = n * 5
          before = breaker.state
          reply = await stack.complete(MSG)
          print(f"{n:>8} {int(clock.now):>4}s {before:<10} {reply.served_by:<26} {len(primary.calls)}")
  asyncio.run(main())

Run with:  py ex3.py

Output (simulated time, five seconds between requests):

   request  time breaker    answered by                primary calls
         1    5s closed     breaker(retry(primary))    1
         2   10s closed     breaker(retry(primary))    2
         3   15s closed     retry(backup)              4
         4   20s closed     retry(backup)              6
         5   25s open       retry(backup)              6
         6   30s open       retry(backup)              6
         7   35s open       retry(backup)              6
         8   40s half-open  retry(backup)              8
         9   45s open       retry(backup)              8
        10   50s open       retry(backup)              8
        11   55s open       retry(backup)              8
        12   60s half-open  breaker(retry(primary))    9

The rows where the primary was not tried:
  Requests 5, 6 and 7 (25 s, 30 s, 35 s): the breaker is open. It opened after requests 3 and 4 each failed (each cost two primary calls, because of the retry layer), and it stays open for 20 seconds from the last failure at 20 s. Calls fail instantly with CircuitOpen and the backup answers; the primary-call count stays at 6.
  Requests 9, 10 and 11 (45 s to 55 s): the trial at 40 s (request 8) used two more primary calls and failed (the script still had failures left), which reopened the breaker for another 20 seconds, so these requests skip the primary again.
  Request 12 (60 s): the breaker is half-open again; its trial call succeeds, so the primary answers and the breaker closes. The primary-call count rises only by one.

WHY THIS WORKS AS AN ANSWER
---------------------------
Each layer does one job. The retry layer absorbs a brief failure (but here the failures lasted longer than its two attempts). The breaker notices that failures continue and stops calling the primary, which saves every later user the wait. The fallback keeps the bot answering from the backup throughout. After the wait the breaker lets a single trial through, and the trial's result decides whether
traffic returns. The "answered by" column shows the stamp (served_by) that the fallback layer adds, which is how the meter and the logs know who to credit.