A source of confusion for a lot of people new to Python and for anyone who has not used them for a while is the yield keyword. As this must be the third time I’ve had to relearn generators I thought I’d make a few notes.
My way of visualizing a generator is a function that returns (or should that be generates?) an iterator. Having the yield keyword in a function is enough to turn it into a generator. Once you have called the generator to get the iterator you can use it as you would any other iterator. Jeff Knupp has a much fuller explanation on his blog so give it a read and then return.
For an example I created a Fibonacci number generator with the following code along with a few examples using an iterator.
def fibonacci(a = 0,b = 1,maxiter=-1): while True: yield a a,b = b,a+b if maxiter > 0: maxiter -= 1 if maxiter < 1: return print([f for f in fibonacci(maxiter=10)]) for f in fibonacci(3,5,20): print(f)
First a few notes on the generator itself. You can specify the starting numbers a and b (naturally defaulting to 0 and 1) and a maximum number of Fibonacci numbers (or iterations to perform) with maxiter when you call the generator. Without setting maxiter the iterator will continue indefinitely and could not be used for list comprehension (the first example) and the for loop would be an infinite loop.
The use of return in a generator ends the iteration and is equivalent to raising a StopIteration exception (see PEP 255). Replace the return keyword with
raise StopIteration if you want to prove it.
Lastly, if you are wondering about the line a,b=b,a+b it is just a compact (and I think elegant) way of writing:
temp = a + b a = b b = temp
Behind the scenes the loop is calling a next method to get the next value from the iterator. There is nothing to stop you manually calling the next method as shown below. Also the generator will create a new iterator each time it is called. Each iterator will encapsulate their own values for a, b and maxiter as shown below.
i = fibonacci() j = fibonacci(13,21) print("variable i is %s\nvariable j is " % (i,j)) print("First 3 from i: %d , %d , %d" % (i.next(),i.next(),i.next())) print("First 3 from j: %d , %d , %d" % (j.next(),j.next(),j.next()))
Hopefully Jeff’s explanation and my example above goes some way to explaining how generators work.