Chunked reading and single-pass generators

The practical payoff is reading a huge file in fixed-size chunks rather than loading it whole:

def read_chunks(path, size=10_000):
    with open(path, encoding="utf-8") as f:
        while True:
            lines = [f.readline() for _ in range(size)]
            lines = [ln for ln in lines if ln]
            if not lines:
                return
            yield lines          # hand back one chunk, keep our place

pandas builds the same idea in: pd.read_csv("big.csv", chunksize=100_000) gives a generator of DataFrame chunks, so you can sum a column across a file far bigger than memory by adding up one chunk at a time.

There's a lighter-weight cousin of generators — a generator expression, written like a list comprehension but with parentheses, which streams its values instead of building a list:

squares = (x * x for x in range(1_000_000))   # nothing computed yet
total = sum(squares)                           # streamed, low memory

Laziness has a cost you must respect: a generator is single-pass. You can iterate it once and then it's exhausted; there's no indexing and no len(). Handing a generator to code that expects a reusable list is a classic source of quiet bugs where the data seems to "vanish" the second time you use it.
“Recursion: base case and walking trees”.*