Advanced Flow Control

In this chapter we’ll be looking at some more advanced, and possibly even obscure, flow-control techniques used in Python programs with which you should be familiar at the advanced level.

Specifically, we’ll cover:

• else clauses on loops
• else clauses on try blocks
• emulating switch statements
• dispatching on type

By understanding these unusual language features you’ll be able to understand more code that you may encounter. You’ll also be able to reduce the complexity of your own code by eliminating unnecessary conditionals.

else clauses on loops

You doubtless associate the else keyword with optional clauses complementary to the conditional clause introduced by the if statement. But did you know that else can also be used to associate optional code blocks with loops? That sounds pretty weird, and to be honest it is an unusual language feature. That’s why it’s only rarely seen in the wild and most definitely deserves some explanation.

while.. else

We’ll start out by saying that Guido van Rossum, inventor and benevolent dictator for life of Python, has admitted that he would not include this feature if he developed Python again. ²³ Back in our timeline though, the feature is there, and we need to understand it.

Apparently, the original motivation for using the else keyword this way in conjunction with while loops comes from Donald Knuth in early efforts to rid structured programming languages of goto. Although the choice of keyword is at first perplexing, it’s possible to rationalise the use of else in this way through comparison with the if...else construct:

In the if...else structure, the code in the if clause is executed if the condition evaluates to True when converted to bool — in other words, if the condition is ‘truthy’. On the other hand, if the condition is ‘falsey’, the code in the else clause is executed instead.

Now look at the while..else construct:

We can, perhaps, glimpse the logic behind choosing the else keyword. The else clause will be executed when, and only when, the condition evaluates to False. The condition may already be False when execution first reaches the while statement, so it may branch immediately into the else clause. Or there may be any number of cycles of the while -loop before the condition becomes False and execution transfers to the else block.

Fair enough, you say, but isn’t this equivalent to putting the else code after the loop, rather than in a special else block, like this? :

You would be right in this simple case. But if we place a break statement within the loop body it becomes possible to exit the loop without the loop conditional ever becoming false. In that case the execute_condition_is_false() call happens even though condition is notFalse:

To fix this, we could use a second test with an if statement to provide the desired behaviour:

The drawback with this approach is that the test is duplicated, which violates the Don’t Repeat Yourself — or DRY — guideline, which hampers maintainability.

The while..else construct in Python allows us to avoid this second redundant test:

Now, the else block only executes when the main loop condition evaluates to False. If we jump out of the loop another way, such as with the break statement, execution jumps over the else clause. There’s no doubt that, however you rationalise it, the choice of keyword here is confusing, and it would have been better all round if a nobreak keyword had been used to introduce this block. In lieu of such a keyword, we heartily recommend that, if you are tempted to use this obscure and little-used language feature, you include such a nobreak comment like this:

So much for the theory; is this any use in practice?

We must admit that neither of the authors of this book have used while.. else in practice. Almost every example we’ve seen could be implemented better by another, more easily understood, construct, which we’ll look at later. That said, let’s look at an example in evaluator.py:

This code evaluates simple ‘stack programs’. Such programs are specified as a stack of items where each item is either a callable function (for these we use any regular Python function) or an argument to that function. So to evaluate 5 + 2, we would set up the stack like this:

When the plus operator is evaluated, its result is pushed onto the stack. This allows us to perform more complex operations such as (5 + 2) * 3:

As the stack contains the expression in reverse Polish notation, the parentheses we needed in the infix version aren’t required. In reality, the stack will be a Python list. The operators will be callables from the Python standard library operators module, which provides named-function equivalents of every Python infix operator. Finally, when we use Python lists as stacks the top of the stack is at the end of the list, so to get everything in the right order we’ll need to reverse our list:

For added interest, our little stack language also supports comments as strings beginning with a hash symbol, just like Python. However, such comments are only allowed at the beginning of the program, which is at the top of the stack:

We’d like to run our little stack program by passing it to a function execute(), like this:

Let’s see what such a function might look like and how it can use the while..else construct to good effect. The first thing our execute() function needs to do is pop all the comment strings from the top of the stack and discard them. To help with this, we’ll define a simple predicate which identifies stack items as comments:

Notice that this function relies on an important Python feature called boolean short-circuiting. If item is not a string then the call to startswith() raises an AttributeError. However, when evaluating the boolean operators and and or Python will only evaluate the second operand if it is necessary for computing the result. When item is not as string (meaning the first operand evaluates to False) then the result of the boolean and must also be False; in this case there’s no need to evaluate the second operand.

Given this useful predicate, we’ll now use a while-loop to clear comment items from the top of the stack:

The conditional expression for the while statement is the program stack object itself. Remember that using a collection in a boolean context like this evaluates to True if the collection is non-empty or False if it is empty. Or put another way, empty collections are ‘falsey’. So this statement reads as “While there are items remaining in the program.”

The while-loop has an associated else clause where execution will jump if the while condition should ever evaluate to False. This happens when there are no more items remaining in the program. In this clause we print a warning that the program was found to be logically empty, then returning early from the execute() function.

Within the while block, we pop() an item from the stack — recall that regular Python lists have this method which removes and returns the last item from a list. We use logical negation of our is_comment() predicate to determine if the just-popped item is not a comment. If the loop has reached a non-comment item, we push it back onto the stack using a call to append(), which leaves the stack with the first non-comment item on top, and then break from the loop. Remember that the while-loop else clause is best thought of as the “no break” clause, so when we break from the loop execution skips the else block and proceeds with the first statement after.

This loop executes the else block in the case of search failure — in this example if we fail to locate the first non-comment item because there isn’t one. Search failure handling is probably the most widespread use of loop else clauses.

Now we know that all remaining items on the stack comprise the actual program. We’ll use another while-loop to evaluate it:

Before this loop we set up an empty list called pending. This will be used to accumulate arguments to functions in the stack, which we’ll look at shortly.

As before, the condition on the while-loop is the program stack itself, so this loop will complete, and control will be transferred to the while-loop else-clause, when the program stack is empty. This will happen when program execution is complete.

Within the while-loop we pop the top item from the stack and inspect it with the built-in callable() predicate to decide if it is a function. For clarity, we’ll look at the else clause first. That’s the else clause associated with the if, not the else clause associated with the while!

If the popped item is not callable, we append it to the pending list, and go around the loop again if the program is not yet empty.

If the item is callable, we try to call it, passing any pending arguments to the function using the star-args extended call syntax. Should the function call fail, we catch the exception, print an error message, and break from the while-loop. Remember this will bypass the loop else clause. Should the function call succeed, we assign the return value to result, push this value back onto the program stack, and clear the list of pending arguments.

When the program stack is empty the else block associated with the while-loop is entered. This prints “Program successful” followed by any contents of the pending list. This way a program can “return” a result by leaving non-callable values at the bottom of the stack; these will be swept up into the pending list and displayed at the end.

for-else loops

Now we understand the while..else construct we can look at the analogous for..else construct. The for..else construct may seem even more odd than while..else, given the absence of an explicit condition in the for statement, but you need to remember that the else clause is really the no-break clause. In the case of the for-loop, that’s exactly when it is called — when the loop is exited without breaking. This includes the case when the iterable series over which the loop is iterating is empty. ⁴

The typical pattern of use is like this: We use a for-loop to examine each item of an iterable series, and test each item. If the item matches, we break from the loop. If we fail to find a match the code in the else block is executed, which handles the ‘no match found’ case.

For example, here is a code fragment which ensures that a list of integers contains at least one integer divisible by a specified value. If the supplied list does not contain a multiple of the divisor, the divisor itself is appended to the list to establish the invariant:

We set up a list of numeric items and a divisor, which will be 12 in this case. Our for-loop iterates through the items, testing each in turn for divisibility by the divisor. If a multiple of the divisor is located, the variable found is set to the current item, and we break from the loop — skipping over the loop- else clause — and print the list of items. Should the for-loop complete without encountering a multiple of 12, the loop-else clause will be entered, which appends the divisor itself to the list, thereby ensuring that the list contains an item divisible by the divisor.

For-else clauses are more common than while-else clauses, although we must emphasise that neither are common, and both are widely misunderstood. So although we want you to understand them, we can’t really recommend using them unless you’re sure that everyone who needs to read your code is familiar with their use.

In a survey undertaken at PyCon 2011, a majority of those interviewed could not properly understand code which used loop-else clauses. Proceed with caution!

An alternative to loop else clauses

Having pointed out that loop else clauses are best avoided, it’s only fair that we provide you with an alternative technique, which we think is better for several reasons, beyond avoiding an obscure Python construct.

Almost any time you see a loop else clause you can refactor it by extracting the loop into a named function, and instead of break -ing from the loop, prefer to return directly from the function. The search failure part of the code, which was in the else clause, can then be dedented a level and placed after the loop body. Doing so, our new ensure_has_divisible() function would look like this:

which is simple enough to be understood by any Python programmer. We can use it, like this:

This is easier to understand, because it doesn’t use any obscure and advanced Python flow-control techniques. It’s easier to test because it is extracted into a standalone function. It’s reusable because it’s not mixed in with other code, and we can give it a useful and meaningful name, rather than having to put a comment in our code to explain the block.

The try..except..else construct

The third slightly oddball place we can use else blocks is as part of the try..except exception handling structure. In this case, the else clause is executed if no exception is raised:

Looking at this, you might wonder why we don’t call do_something_else() on the line after do_something(), like this:

The downside of this approach, is that we now have no way of telling in the except block whether it was do_something() or do_something_else() which raised the exception. The enlarged scope of the try block also obscures our intent with catching the exception; where are we expecting the exception to come from?

Although rarely seen in the wild, it is useful, particularly when you have a series of operations which may raise the same exception type, but where you only want to handle exceptions from the first such, operation, as commonly happens when working with files:

In this example, both opening the file and iterating over the file can raise an OSError, but we’re only interested in handling the exception from the call to open().

It’s possible to have both an else clause and a finally clause. The else block will only be executed if there was no exception, whereas the finally clause will always be executed.

Emulating switch

Most imperative programming languages include a switch or case statement which implements a multiway branch based on the value of an expression. Here’s an example for the C programming language, where different functions are called depending on the value of a menu_option variable. There’s also handing for the case of ‘no such option’:

Although switch can be emulated in Python by a chain of if..elif..else blocks, this can be tedious to write, and it’s error prone because the condition must be repeated multiple times.

An alternative in Python is to use a mapping of callables. Depending on what you want to achieve, these callables may be lambdas or named functions.

We’ll look at a simple adventure game you cannot win in kafka.py, which we’ll refactor from using if..elif..else to using dictionaries of callables. Along the way, we’ll also use try..else:

The game-loop uses two if..elif..else chains. The first prints information dependent on the players current position. Then, after accepting a command from the user, the second if..elif..else chain takes action based upon the command.

Let’s refactor this code to avoid those long if..elif..else chains, both of which feature repeated comparisons of the same variable against different values.

The first chain describes our current location. Fortunately in Python 3, although not in Python 2, print() is a function, and can therefore be used in an expression. We’ll leverage this to build a mapping of position to callables called locations:

We’ll look up a callable using our position in locations as a key, and call the resulting callable in a try block:

In fact, we don’t really intend to be catching KeyError from the callable, only from the dictionary lookup, so this also gives us opportunity to narrow the scope of the try block using the try..else construct we learned about earlier. Here’s the improved code:

We separate the lookup and the call into separate statements, and move the call into the else block.

Similarly, we can refactor the if..elif..else chain which handles user input into dictionary lookup for a callable. This time, though, we used named functions rather than lambdas to avoid the restriction that lambdas can only contain expressions and not statements. Here’s the branching construct:

Again we split the lookup of the command action from the call to the command action.

Here are five callables referred to in the dictionary values:

Notice that using this technique forces us into a more functional style of programming. Not only is our code broken down into many more functions, but the bodies of those functions can’t modify the state of the position variable. Instead, we pass in this value explicitly and return the new value. In the new version mutation of this variable only happens in one place, not five.

Although the new version is larger overall, we’d claim it’s much more maintainable. For example, if a new piece of game state, such as the players inventory, were to be added, all command actions would be required to accept and return this value. This makes it much harder to forget to update the state than it would be in chained if..elif..else blocks.

Let’s add a new “rabbit hole” location which, when the user unwittingly moves into it, leads back to the starting position of the game. To make such a change, we need to change all of our callables in the location mapping to accept and return a position and a liveness status. Although this may seem onerous, we think it’s a good thing. Anyone maintaining the code for a particular location can now see what state needs to be maintained. Here are the location functions:

The corresponding switch in the while-loop now looks like this:

We must also update the call to location_action() to pass the current state and receive the modified state.

Now let’s make the game a little more morbid by adding a deadly lava pit location which returns False for the alive status. Here’s the function for the lava pit location:

And we must remember to add this to the location dictionary:

We’ll also add an extra conditional block after we visit the location to deal with deadly situations:

Now when we die, we break out of the while loop which is the main game loop. This gives us an opportunity to use a while..else clause to handle non-lethal game loop exits, such as choosing to exit the game. Exits like this set the position variable to None, which is ‘falsey’:

Now when we quit deliberately, setting position to None and causing the while-loop to terminate, we see the message from the else block associated with the while-loop:

But when we die by falling into the lava alive gets set to False. This causes execution to break from the loop, but we don’t see the “You have chosen to leave” message as the else block is skipped:

Dispatching on Type

To “dispatch” on type means that the code which will be executed depends in some way on the type of an object or objects. Python dispatches on type whenever we call a method on an object; there may be several implementations of that method in different classes, and the one that is selected depends on the type of the self object.

Ordinarily, we can’t use this sort of polymorphism with regular functions. One solution is to resort to switch-emulation to route calls to the appropriate implementation by using type objects as dictionary keys. This is ungainly, and it’s tricky to make it respect inheritance relationships as well as exact type matches.

singledispatch

The singledispatch decorator, which we’ll introduce in this section, provides a more elegant solution to this problem.

Consider the following code which implements a simple inheritance hierarchy of shapes, specifically a circle, a parallelogram and a triangle, all of which inherit from a base class called Shape:

Each class has an initializer and a draw() method. The initializers store any geometric information peculiar to that type of shape. They pass any further arguments up to the Shape base class, which stores a flag indicating whether the shape is solid.

When we say:

in main(), the particular draw() that is invoked depends on whether shape is an instance of Circle, Parallelogram, or Triangle. On the receiving end, the object referred to by shape becomes referred to by the first formal parameter to the method, which as we know is conventionally called self. So we say the call is “dispatched” to the method, depending on the type of the first argument.

This is all very well and is the way much object-oriented software is constructed, but this can lead to poor class design because it violates the single responsibility principle. Drawing isn’t a behaviour inherent to shapes, still less drawing to a particular type of device. In other words, shape classes should be all about shape-ness, not about things you can do with shapes, such as drawing, serialising or clipping.

What we’d like to do is move the responsibilities which aren’t intrinsic to shapes out of the shape classes. In our case, our shapes don’t do anything else, so they become containers of data with no behaviour, like this:

With the drawing code removed there are several ways to implement the drawing responsibility outside of the classes. We could us a chain of if..elif..else tests using isinstance():

In this version of draw() we test shape using up to three calls to isinstance() against Circle, Parallelogram and Triangle. If the shape object doesn’t match any of those classes, we raise a TypeError. This is awkward to maintain and is rightly considered to be very poor programming style.

Another approach is to emulate a switch using dictionary look-up where the dictionary keys are types and the dictionary values are the functions which do the drawing:

Here we lookup a drawer function by obtaining the type of shape in a try block, translating the KeyError to a TypeError if the lookup fails. If we’re on the happy-path of no exceptions, we invoke the drawer with the shape in the else clause.

This looks better, but is actually much more fragile because we’re doing exact type comparisons when we do the key lookup. This means that a subclass of, say, Circle wouldn’t result in a call to draw_circle().

The solution to these problems arrived in Python 3.4 in the form of singledispatch, a decorator defined in the Python Standard Library functools module which performs dispatch on type. In earlier versions of Python, including Python 2, you can install the singledispatch package from the Python Package Index.

Functions which support multiple implementations dependent on the type of their arguments are called ‘generic functions’, and each version of the generic function is referred to as an ‘overload’ of the function. The act of providing another version of a generic function for different argument types is called overloading the function. These terms are common in statically typed languages such as C#, C++ or Java, but are rarely heard in the context of Python.

To use singledispatch we define a function decorated with the singledispatch decorator. Specifically, we define a particular version of the function which will be called if a more specific overload has not been provided. We’ll come to the type-specific overloads in a moment.

At the top of the file we need to import singledispatch:

Then lower down we implement the generic draw() function:

In this case, our generic function will raise a TypeError.

Recall that decorators wrap the function to which they are applied and bind the resulting wrapper to the name of the original function. So in this case, the wrapper returned by the decorator is bound to the name draw. The draw wrapper has an attribute called register which is also a decorator; register() can be used to provide extra versions of the original function which work on different types. This is function overloading.

Since our overloads will all be associated with the name of the original function, draw, it doesn’t matter what we call the overloads themselves. By convention we call them _, although this is by no means required. Here’s an overload for Circle, another for Parallelogram, and a third for Triangle:

By doing this we have cleanly separated concerns. Now drawing is dependent on shapes, but not shapes on drawing.

Our main function, which now looks like this, calls the global- scope generic draw function, and the singledispatch machinery will select the most specific overload if one exists, or fallback to the default implementation:

We could add other capabilities to Shape in a similar way, by defining other generic functions which behave polymorphically with respect to the shape types.

Using singledispatch with methods

You need to take care not to use the singledispatch decorator with methods. To see why, consider this attempt to implement a generic intersects() predicate method on the Circle class which can be used to determine whether a particular circle intersects instances of any of the three defined shapes:

At first sight, this looks like a reasonable approach, but there are a couple of problems here.

The first problem problem is that we can’t register the type of the class currently being defined with the intersects generic function, because we have not yet finished defining it.

The second problem is more fundamental: Recall that singledispatch dispatches based only on the type of the first argument:

When we’re calling our new method like this it’s easy to forget that my_parallelogram is actually the second argument to Circle.intersects. my_circle is the first argument, and it’s what gets bound to the self parameter. Because self will always be a Circle in this case our intersect() call will always dispatch to the first overload, irrespective of the type of the second argument.

This behaviour prevents the use of singledispatch with methods. All is not lost however. The solution is to move the generic function out of the class, and invoke it from a regular method which swaps the arguments. Let’s take a look:

We move the generic function intersects() out to the global scope and rename it to intersects_with_circle(). The replacement intersects() method of Circle, which accepts the formal arguments self and shape, now delegates to intersect_with_circle() with the actual arguments swapped to shape and self.

To complete this example, we would need to implement two other generic functions, intersects_with_parallelogram() and intersects_with_triangle(), although will leave that as an exercise.

Combining class-based polymorphism and overloading-based polymorphism in this way would give us a complete implementation of not just single-dispatch but double -dispatch, allowing us to do:

This way, the function is selected based on the types of both shapeandother_shape, without the shape classes themselves having any knowledge of each other, keeping coupling in the system manageable.

Summary

That just about wraps up this chapter on advanced flow control in Python 3. Let’s summarise what we’ve covered:

• We looked at else clauses on while-loops, drawing an analogy with the much more well-known association between if and else. We showed how the else block is executed only when the while-loop condition evaluates to False. If the loop is exited by another means, such as via break or return, the else clause is not executed. As such, else clauses on while-loops are only ever useful if the loop contains a break statement somewhere within it.
• The loop-else clause is a somewhat obscure and little-used construct. We strongly advise commenting the else keyword with a “nobreak” remark so it is clear under what conditions the block is executed.
• The related for..else clause works in an identical way: The else clause is effectively the “nobreak” clause, and is only useful if the loop contains a break statement. They are most useful with for-loops in searching. When an item is found while iterating we break from the loop — skipping the else clause. If no items are found and the loop completes ‘naturally’ without breaking ,the else clause is executed and code handling the “not found” condition can be implemented.
• Many uses of loop else clauses — particularly for searching — may be better handled by extracting the loop into its own function. From here execution is returned directly when an item is found and code after the loop can handle the “not found” case. This is less obscure, more modular, more reusable, and more testable than using a loop else clause within a longer function.
• Next we looked at the try..except..else construct. In this case, the else clause is executed only if the try block completed successfully without any exception being raised. This allows the extent of the try block to be narrowed, making it clearer from where we are expecting exceptions to be raised.
• Python lacks a “switch” or “case” construct to implement multi-branch control flow. We showed the alternatives, including chained if..elif..else blocks, and dictionaries of callables. The latter approach also forces you to be more explicit and consistent about what is required and produced by each branch, since you must pass arguments and return values rather than mutating local state in each branch.
• We showed how to implement generic functions which dispatch on type using the singledispatch decorator available from Python 3.4. This decorator can be applied only to module scope functions, not methods, but by implementing forwarding methods and argument swapping, we can delegate to generic functions from methods. This gives us a way of implementing double-dispatch calls.
• In passing, we saw that the Python logical operators use short-circuit evaluation. This means that the operators only evaluate as many operands as are required to find the result. This can be used to ‘protect’ expressions from run time situations in which they would not make sense.