Generators in Python

In Python, a generator is a function that yields a sequence of results over time, rather than returning a single value when called.

Unlike traditional functions, which return once and terminate, generators can yield multiple values across multiple calls. This is achieved by using the yield keyword instead of return, allowing the generator to provide a value to the caller while preserving its state, thereby enabling the function to resume where it left off on the next call. 

How do they work?

Each time a generator yields, it emits a value to the calling code, then "freezes," maintaining its state until it's invoked again.

The generator function's local namespace is preserved for reuse in subsequent calls.

Thus, upon being invoked again, the generator picks up its execution precisely at the yield point where it had last left off, ensuring a seamless continuation of its operation.

Why use them? This feature makes generators especially useful for handling data streams or large sequences efficiently, as they consume less memory. Thus, generators offer an efficient way to implement iterators, enabling you to create iterable objects effectively.

Here's a practical example of a generator.

This code defines a generator named my_generator that yields three values: 1, 2, and 3.

def serie():
    yield 1
    yield 2
    yield 3

Invoking a generator function yields a generator object, which allows for the sequential production of elements one by one.

object=series()

Having acquired the generator object, individual elements can be accessed sequentially through the __next__() method.

print(object.__next__())

1

Each call retrieves the next element in the sequence.

print(object.__next__())

2

Upon reaching the sequence's end, a StopIteration exception is thrown, signaling the end of generation.

print(object.__next__())

3

Alternatively, the generator can be effortlessly iterated over with a loop for automatic progression through its elements.

When a for loop invokes the generator, the function runs its code block up to the first yield, produces the value, then pauses.

for value in serie():
    print(value)

With each iteration of the loop, the function continues from where it left off until it reaches the next yield or completes.

Overall, the generator is called three times, printing the next value in the sequence with each iteration.

1
2
3

Generator Expressions

Beyond generator functions, Python also supports generator expressions, a compact syntax for creating generators.

Generator expressions are akin to list comprehensions, but instead of constructing a list, they generate a series of items on demand.

Here's a practical example.

Generator expressions are enclosed in parentheses:

gen_expr = (x ** 2 for x in range(10))
for value in gen_expr:
    print(value)

This generator expression yields the squares of numbers from 0 to 9, one at a time.

0
1
4
9
16
25
36
49
64
81

To select only the even numbers from 0 to 9, you can add a condition within the expression.

gen_expr = (x ** 2 for x in range(10) if x%2==0)
for value in gen_expr:
    print(value)

0
4
16
36
64

When are Generators Useful?

Generators are memory-efficient because they generate values without needing to store the entire sequence in memory.

This makes them highly valuable for working with large data sequences, such as log files or large datasets, where storing all data in memory would be impractical or inefficient.

Moreover, generators implement lazy evaluation, meaning they produce values only when required, adding to their efficiency and practicality in data processing and handling.