Generators and yield

A source of confusion for a lot of people new to Python and for anyone who has not used them for a while is the yield keyword. As this must be the third time I’ve had to relearn generators I thought I’d make a few notes.

My way of visualizing a generator is a function that returns (or should that be generates?) an iterator. Having the yield keyword in a function is enough to turn it into a generator. Once you have called the generator to get the iterator you can use it as you would any other iterator. Jeff Knupp has a much fuller explanation on his blog so give it a read and then return.

For an example I created a Fibonacci number generator with the following code along with a few examples using an iterator.

def fibonacci(a = 0,b = 1,maxiter=-1):
    while True:
        yield a
        a,b = b,a+b
        if maxiter > 0:
           maxiter -= 1
           if maxiter < 1:
              return

print([f for f in fibonacci(maxiter=10)])
for f in fibonacci(3,5,20):
    print(f)

First a few notes on the generator itself. You can specify the starting numbers a and b (naturally defaulting to 0 and 1) and a maximum number of Fibonacci numbers (or iterations to perform) with maxiter when you call the generator. Without setting maxiter the iterator will continue indefinitely and could not be used for list comprehension (the first example) and the for loop would be an infinite loop.

The use of return in a generator ends the iteration and is equivalent to raising a StopIteration exception (see PEP 255).  Replace the return keyword with raise StopIteration if you want to prove it.

Lastly, if you are wondering about the line a,b=b,a+b it is just a compact (and I think elegant) way of writing:

temp = a + b
a = b
b = temp

Behind the scenes the loop is calling a next method to get the next value from the iterator. There is nothing to stop you manually calling the next method as shown below. Also the generator will create a new iterator each time it is called. Each iterator will encapsulate their own values for a, b and maxiter as shown below.

i = fibonacci()
j = fibonacci(13,21)
print("variable i is %s\nvariable j is " % (i,j))
print("First 3 from i: %d , %d , %d" % (i.next(),i.next(),i.next()))
print("First 3 from j: %d , %d , %d" % (j.next(),j.next(),j.next()))

Hopefully Jeff’s explanation and my example above goes some way to explaining how generators work.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s