chatbot1-4  Exercise 2: Choosing a Budget
===========================================

ex2.py (save in the project root). Each budget is tried on a freshly built, identical ten-exchange history, so the runs do not affect each other:

  import asyncio
  from bot.context import message_tokens
  from bot.core import Bot
  from bot.providers import FakeProvider


  async def sent_for(budget: int) -> tuple[int, int]:
      """Build the same ten-exchange history from scratch, then ask one more question under `budget`."""
      fake = FakeProvider(["ok"] * 11)
      bot = Bot(fake, history_budget=100_000)
      for i in range(10):
          await bot.handle("s", f"question {i}")
      bot.history_budget = budget
      await bot.handle("s", "the next question")
      return len(fake.calls[-1]), len(bot.store.messages("s")) - 2   # messages sent; messages stored before this question


  async def main():
      print(f"{'budget':>7} {'sent':>5} {'dropped':>8}")
      for budget in (10_000, 100, 80, 60, 40, 20, 5):
          sent, before = await sent_for(budget)
          print(f"{budget:>7} {sent:>5} {before + 1 - sent:>8}")
      print()
      print("budgets at which the number of messages sent changes:")
      previous = None
      for budget in range(1, 70):
          sent, _ = await sent_for(budget)
          if sent != previous:
              print(f"  budget {budget:>2} -> {sent} messages")
              previous = sent
      print("estimated tokens: the new question =", message_tokens({"role": "user", "content": "the next question"}),
            "| one stored exchange =", message_tokens({"role": "user", "content": "question 3"}) + message_tokens({"role": "assistant", "content": "ok"}))

  asyncio.run(main())

Run with:  py ex2.py

Output:

   budget  sent  dropped
    10000    21        0
      100    15        6
       80    11       10
       60     9       12
       40     5       16
       20     1       20
        5     1       20

  budgets at which the number of messages sent changes:
    budget  1 -> 1 messages
    budget 21 -> 3 messages
    budget 33 -> 5 messages
    budget 45 -> 7 messages
    budget 57 -> 9 messages
    budget 69 -> 11 messages
  estimated tokens: the new question = 9 | one stored exchange = 12

The smallest budget that still sends an earlier exchange along with the new question is 21 (3 messages: the last exchange plus the question). From the printed sums: the new question costs 9 estimated tokens and one stored exchange 12, so 9 + 12 = 21, and
each further exchange needs 12 more (33, 45, 57, 69).

Choosing a budget for a model with a 200,000-token limit (this part is judgement): the budget is for history only, so start from the limit and subtract the system prompt, the facts and summary, and the largest reply you allow (max_tokens), then leave a
safety margin because the estimate can be badly low for some languages (see chapter 4's token table). You would usually choose a budget far below the limit, since every token you send costs money and time on every message; a long window is something to pay for only if
the bot needs it.

WHY THIS WORKS AS AN ANSWER
---------------------------
The window keeps the newest messages that fit and always keeps the final message, so as the budget shrinks the number sent falls from 21 (everything, including the new question) down to 1. The step list shows it falling in pairs, because the window never starts with an
assistant message, so a lone reply is dropped together with the question that went with it. The "dropped" column is the difference, so it rises as the number sent falls. Nothing is removed from the store; only the view sent to the model changes. The exact thresholds depend on message
lengths, so find yours by measuring with your own conversations rather than guessing. (My first version of this exercise reused one bot for all budgets, and each pass stored its own question, which shifted the numbers; building a fresh history per budget fixes that.)