Lazy Expressions and When a List Is Simpler

Chapter 3 introduced the generator expression as a comprehension without the brackets, and promised a reason later. This is later.

[question.points for question in quiz]     # builds a list
(question.points for question in quiz)     # builds a generator object

The word for what the second one does is lazy: creating the generator computes none of the point values. It produces each value only when asked, and it never needs to hold the full result at once.

Watching it be lazy

Nothing is produced until something asks. Compare a list comprehension, which runs the whole thing before it hands anything back:

That is the whole difference, and everything else follows from it.

When laziness matters

Stopping early. If you only need the first match, a list comprehension does all the work anyway:

It checked 3, checked 9, found it, and stopped. The list version would have checked all four.

Too much to hold. A million rows in a list is a million rows in memory. Passed lazily, one at a time, it is one row.

Chaining steps. Generators can feed generators, and no step builds a collection:

Each generator names one step without building another collection. The original scores list remains, and list(doubled) collects the final result.

Stop without taking one extra

Stopping after two results should also mean requesting exactly two input items. A for loop asks its iterator for the next item before it enters the loop body, so this check comes too late:

for item in items:
    if len(collected) == 2:
        break
    collected.append(item)

By the time the condition sees two collected items, the loop has already requested a third. Check immediately after accepting the item that reaches the boundary:

def first_two(items):
    taken = 0
    for item in items:
        yield item
        taken = taken + 1
        if taken == 2:
            return

For a requested count of zero or less, return before entering the loop at all. This is exact bounded consumption: the function stops both its results and its input requests at the promised boundary.

When a list is simpler, which is often

Laziness is not free, and the costs are real:

You can only walk it once. The bug from Lesson 1, now easy to write by accident.

It has no length. len() on a generator raises TypeError. So does indexing.

It is harder to debug. Printing a list shows you the values. Printing a generator shows you that it is a generator.

The work happens somewhere else. An exception raised inside a generator surfaces at the for loop consuming it, which can be a long way from the code that looks responsible.

A short, practical rule:

For ten questions, the memory saving is nothing and the readability of a list is worth more than the elegance of laziness.

Returning a generator from a function

If a function returns a generator, say so, because the caller now has a one-shot value with no length:

def passing_records(records, mark):
    """Yield each record reaching the mark.

    Returns a generator: walk it once, or wrap it in list() to keep it.
    """

That is the honest version of the trade. The default for a public method that returns a collection is a list, and the exception is when the collection is genuinely large or the caller genuinely stops early.