Iterators & Context Managers Deep Dive

Python Advanced — Iterators & Context Managers Deep Dive

Python Advanced

Course 3 · Chapter 2 · Iterators & Context Managers Deep Dive

🔄 Iterators & Context Managers Deep Dive

Course 2, Chapter 5 showed yield as an easy way to produce lazy sequences. This chapter goes underneath that: the iterator protocol every for loop actually relies on, how to implement it yourself in a class, and the same "deep dive" treatment for context managers — the real __enter__/__exit__ protocol behind with, plus contextlib's easier decorator-based shortcut.

🔁 The Iterator Protocol

Every for loop you've written since Course 1 is sugar for a lower-level protocol: calling iter() once, then next() repeatedly until StopIteration is raised:

numbers = [1, 2, 3]

# what "for n in numbers:" actually does, unrolled:
iterator = iter(numbers)
while True:
    try:
        n = next(iterator)
    except StopIteration:
        break
    print(n)
        

An object is iterable if it defines __iter__ (which iter() calls, returning an iterator). An object is an iterator if it defines __next__ (which next() calls, returning the next value or raising StopIteration when exhausted). A list is iterable but is not itself an iterator — iter(numbers) returns a separate list_iterator object that actually tracks position.

🏗️ Building a Custom Iterator Class

class Countdown:
    def __init__(self, start):
        self.current = start

    def __iter__(self):
        return self          # Countdown IS its own iterator

    def __next__(self):
        if self.current <= 0:
            raise StopIteration
        self.current -= 1
        return self.current + 1

for n in Countdown(3):
    print(n)
# 3 2 1
        

This is exactly what Course 2, Chapter 5's count_up_to generator function did automatically — a generator function is really just a much more convenient way to get an object with __iter__ and __next__ already implemented correctly, without writing the class yourself.

⚠ A Custom Iterator Has the Same Single-Use Limitation as a Generator

Because Countdown.__iter__ returns self, and self.current only ever counts down, a single Countdown instance is exhausted after one full loop through it — iterating the same instance again yields nothing, the exact same one-pass limitation Course 2, Chapter 5 covered for generators. Create a fresh instance (Countdown(3) again) for a second pass, the same fix that chapter recommended for generators.

🔒 Context Managers: enter and exit

class Timer:
    def __enter__(self):
        import time
        self.start = time.time()
        return self                # becomes the "as" variable

    def __exit__(self, exc_type, exc_value, traceback):
        import time
        print(f"Elapsed: {time.time() - self.start:.2f}s")
        return False              # False = don't suppress any exception — see the warning below

with Timer() as t:
    sum(range(10_000_000))
# Elapsed: 0.34s
        

This is the class-based version of Course 2, Chapter 4's with open(...) as file: — __enter__ runs at the start of the with block (its return value becomes the as variable), and __exit__ always runs at the end, even if an exception occurred inside the block.

⚠ __exit__ Returning True Silently Swallows the Exception

__exit__ receives details about any exception that occurred inside the with block (exc_type, exc_value, traceback — all None if nothing went wrong). If __exit__ returns a truthy value, Python treats the exception as handled and suppresses it entirely — the code after the with block keeps running as if nothing happened. This is almost never what you want; return False (or nothing, which is falsy) unless you deliberately intend to swallow specific exceptions.

🎁 contextlib.contextmanager

from contextlib import contextmanager
import time

@contextmanager
def timer():
    start = time.time()
    try:
        yield                     # the "with" block's code runs here
    finally:
        print(f"Elapsed: {time.time() - start:.2f}s")

with timer():
    sum(range(10_000_000))
# Elapsed: 0.34s — same result, far less code than the Timer class
        

@contextmanager turns a generator function into a context manager: code before yield becomes __enter__'s logic, code after yield (inside finally, guaranteeing it runs) becomes __exit__'s logic. For simple cases, this is noticeably less boilerplate than writing a full class with both dunder methods — the same trade-off decorators (Course 2, Chapter 6) generally offer over writing the underlying mechanism by hand.

Custom Iteration: Go vs Kotlin vs Python

Language	Approach
Go	No built-in iterator protocol until relatively recently (range-over-func) — historically, channels combined with goroutines were the idiomatic way to produce a lazy sequence.
Kotlin	An `Iterator<T>` interface with `hasNext()` and `next()` — structurally very close to Python's `__next__`/`StopIteration` pair, just with an explicit boolean check instead of an exception.
Python	`__iter__`/`__next__`, with `StopIteration` as the "no more values" signal instead of a boolean check.

Kotlin's explicit hasNext() check versus Python's exception-based StopIteration is a real philosophical difference worth noticing — Python leans on "ask forgiveness, not permission" here, the same style choice that shows up in its exception-handling culture generally (Course 2, Chapter 3).

iter / next

The protocol every for loop relies on; next() raises StopIteration when done.

Iterable vs iterator

Iterable has __iter__; an iterator additionally has __next__.

enter / exit

The class-based protocol behind with; __exit__ must return falsy to not suppress exceptions.

@contextmanager

Turns a yield-based generator function into a context manager, less boilerplate than a class.

💻 Coding Challenges

Challenge 1: An Evens-Only Iterator

Write a class EvensUpTo implementing __iter__ and __next__, that iterates only the even numbers from 0 up to (and including) a given limit. Use it in a for loop with a limit of 10.

Goal: Practice implementing the iterator protocol directly in a class, reusing this chapter's Countdown structure.

→ Solution

Challenge 2: A Class-Based Logging Context Manager

Write a class LogSection that takes a name in __init__, prints f"Entering {name}" in __enter__, and prints f"Exiting {name}" in __exit__. Use it in a with block around a couple of print statements.

Goal: Practice writing __enter__/__exit__ directly, reusing this chapter's Timer structure.

→ Solution

Challenge 3: The Same Logging Context Manager with contextlib

Rewrite Challenge 2's LogSection as a @contextmanager-decorated generator function called log_section(name) instead, producing the same "Entering .../Exiting ..." output.

Goal: Directly compare the class-based and @contextmanager-based approaches to the exact same problem.

→ Solution

🎯 What's Next

Next chapter: Concurrency — threading, multiprocessing, and the GIL explained.

Iterators & Context Managers Deep Dive

🔄 Iterators & Context Managers Deep Dive

🔁 The Iterator Protocol

🏗️ Building a Custom Iterator Class

🔒 Context Managers: __enter__ and __exit__

🎁 contextlib.contextmanager

Custom Iteration: Go vs Kotlin vs Python

__iter__ / __next__

Iterable vs iterator

__enter__ / __exit__

@contextmanager

💻 Coding Challenges

Challenge 1: An Evens-Only Iterator

Challenge 2: A Class-Based Logging Context Manager

Challenge 3: The Same Logging Context Manager with contextlib

🎯 What's Next

🔒 Context Managers: enter and exit

iter / next

enter / exit