chatbot1-6 Exercise 1: Fail Closed or Fail Open? =================================================== ex1.py (save in the project root): import asyncio from bot.core import Bot from bot.moderation import SafetyPolicy from bot.providers import FakeProvider, ProviderUnavailable class DownModerator: async def check(self, text): raise ProviderUnavailable("moderation service is down") async def main(): for mode in ("closed", "open"): fake = FakeProvider(["Here is my answer."]) bot = Bot(fake, moderator=DownModerator(), safety=SafetyPolicy(disclosure=None, on_moderation_error=mode)) reply = await bot.handle("s", "hello") print(f"on_moderation_error={mode!r:9} reply={reply!r:45} model calls={len(fake.calls)} stored={len(bot.store.messages('s'))} events={bot.events}") asyncio.run(main()) Run with: py ex1.py Output: on_moderation_error='closed' reply="I can't check that message right now. Please try again in a minute." model calls=0 stored=0 events=[('s', 'moderation_unavailable')] on_moderation_error='open' reply='Here is my answer.' model calls=1 stored=2 events=[('s', 'moderation_unavailable')] Which would I choose (this part is judgement, not a measured fact): (a) A bot answering library questions: open is defensible. The worst thing it can say is limited by its scope and persona, an outage is probably short, and refusing every message during one is a bigger cost than a few unmoderated library answers. Keep the output guard and record the events so someone reviews them. (b) A bot that talks to teenagers: closed. If the safety check cannot run, the bot should say so and stop. The harm of an unmoderated conversation with a young person is much larger than the inconvenience of an outage. (For a service for under-18s you would also want more than a moderator: age-appropriate design and a human escalation route, which this course does not build.) WHY THIS WORKS AS AN ANSWER --------------------------- With "closed" the model was never called, nothing was stored and the user got the fixed "can't check" reply; with "open" the same message was answered and stored. In both cases the event moderation_unavailable was recorded, so the failure is visible either way. The setting is a policy decision about what a safety failure should cost, which is why it is a field on SafetyPolicy and not hard-coded.