Using Exception Handling for Control Flow (in Python)

For those that have a strong opinion about what constitutes good code, a topic ripe for contention is that of using exceptions for control flow. Indeed, the last time I had this debate with Akshay Shah, an avid disciple of the Go Programming Language (which has no exceptions), it ended with him pepper spraying me in the face.

I for one see exceptions as just another tool within programming that can be extremely powerful if used properly and fairly detrimental otherwise.

Example Use Case

Before talking about trade-offs and so forth, I just wanted to share a case where I found exceptions to be extremely useful and clean for control flow. For business logic in particular, the associated code from a technical standpoint is completely arbitrary. No outcome among possibles is necessarily better or worse without the overall context to the business operation the code is involved with.

Consider an example program we want to write what sounds simple enough: given some restaurant with a given state, find an appropriate seat for an incoming user. A user can sit inside, sit outside, wait in a priority queue, or leave the restaurant. Typically, a user will want to sit inside, and a seat is probably available, so we can seat the user. But maybe they want to sit outside, so we need to account for that case. But maybe there’s seating available outside, but not inside, and a user would prefer sitting indoors, so now we need to see if the user would rather sit outdoors or wait in a queue. Maybe there are no seats available, and there’s a certain wait time, and now we need to see if that wait time is satisfactory with the user. Or maybe there are seats available, but the available seating is held for a reservation, and we need to wait a certain amount of time before we can give the seat up. Now what if there’s seating available inside, but only for regular tables, but a user wants to sit in a booth. Or maybe there’s only space available at the bar. Or maybe the only available seating would require splitting a party up. Or maybe the only table available has vomit on it.

The number of cases can become fairly extreme, but the irony is that 95% of the time, the usual case will work just fine: a user wants to come sit in a restaurant and there’s seating available indoors. Done. But of course, with software, we need to account for every possible case if we want to make a polished product, and at the same time, we don’t the majority of our code polluted with handling edge cases (at least, this wouldn’t be how to create clean, intuitive code).

So this is some arbitrary code I made that tries to illustrate this point:

class Restaurant(object):

    def handle_incoming_user(self, incoming_user):
        if incoming_user.prefers_indoors and self.indoor_seats_available(incoming_user.party_size):
            if incoming_user.prefers_booth and [seat for seat in self.indoor_seats_available(incoming_user.party_size) if seat.is_booth]:
                self.move_user_to_booth(incoming_user)
            else:
                self.move_user_to_table(incoming_user)
        elif incoming_user.prefers_indoors and self.indoor_seats_available(incoming_user.party_size / 2):
            temp_table = self.indoor_seats_available(incoming_user.party_size / 2).pop(incoming_user.party_size / 2)
            remaining_people = incoming_user.party_size - (incoming_user.party_size / 2)
            if self.indoor_seats_available(remaining_people):
                self.move_user_to_table(incoming_user)
            else:
                self.push_table_to_available(temp_table)
        elif incoming_user.prefers_outdoors and self.outdoor_seats_available(incoming_user.party_size):
            self.move_user_to_outdoor_table(incoming_user)
        elif incoming_user.prefers_indoors and not self.indoor_seats_available(incoming_user.party_size):

            # determine if user is ok with sitting outside
            wait_for_seat_score = self.wait_for_seat_score(incoming_user.acceptable_wait_time, self.current_wait_time)
            if wait_for_seat_score > self.ACCEPTABLE_WAIT_SCORE:
                # user can not sit outside or leave the restaurant
                if incoming_user.amenable_to_sitting_outdoors:
                    self.move_user_to_outdoor_table(incoming_user)
                else:
                    # kick the user out of the restaurant via the bouncers
                    self.evict_user(incoming_user)
            else:
                # first determine if any of the non-available seats are just
                # because of reservations
                seats_for_reservations = [seat for seat in self.indoor_seats_available(incoming_user.party_size) if seat.is_reserved]
                if seats_for_reservations and seats_for_reservations[0].reserve_time + self.MAX_RESERVE_HOLDING_TIMEDELTA <= datetime.datetime.utcnow():
                    # give up the seat
                    seat_to_give_up = seats_for_reservations[0]
                    # FIXME: this causes a mutation to the class
                    self.cancel_reservation(seat_to_give_up.reserving_user)
                    self.move_user_to_table(incoming_user, specific_seat=seat_to_give_up)
                else:
                    self.add_user_to_priority_queue(incoming_user)

Perhaps you’ve seen code like this. It’s totally arbitrary and it’s basically trying to breathe life into a flow chart with branches of IF statements. The only way to make it somewhat readable is to put comments all over the place, and since you’re doing a whole bunch of branches, it’s kind of difficult to decompose the problem a little more (i.e. I could move one of the IF blocks to a function, but each block is already pretty short). If I did keep this style and just moved things to functions without exceptions, I need to figure out how to represent return types in the event of a failure, and if that’s done using None or a null reference, then the calling code will need to be cluttered with checks to see about success. And like I mentioned earlier, the primary block of code that 95% of cases will hit is probably just the first block.

I did make this code look intentionally bad…it’s as though I’m on an informmercial and I’m displaying the inferior product in black and white and illustrating how difficult and unwieldy the other product is. But, I have seen code like this.

Alternatively, I put together a sample of how I might write something to use exceptions for control flow:

class HeuristicFailedException(Exception):
    pass


class Restaurant(object):

    def handle_incoming_user(self, incoming_user):
        ordered_heuristics = (
            self._try_indoors_preference,
            self._try_outdoors_preference,
            self._try_split_tables,
            self._try_move_to_queue,
            self._try_move_outdoors,
            self._try_cancelling_reservation,
        )
        return self._execute_heuristics(ordered_heuristics, args=[incoming_user])

    def _execute_heuristics(self, ordered_heuristic_function_list, args=tuple()):
        for heuristic_func in ordered_heuristic_function_list:
            try:
                return heuristic_func(*args)
            except HeuristicFailedException:
                continue
        raise HeuristicFailedException("No viable heuristics to satisfy current state")

    def _try_indoors_preference(self, incoming_user):
        if not incoming_user.prefers_indoors:
            raise HeuristicFailedException("User does not want to sit inside")
        if not self.indoor_seats_available(incoming_user.party_size):
            raise HeuristicFailedException("No indoor seats available")

        booth_seats = [seat for seat in self.indoor_seats_available(incoming_user.party_size) if seat.is_booth]
        if booth_seats:
            return self.move_user_to_booth(incoming_user)
        return self.move_user_to_table(incoming_user)

    def _try_outdoors_preference(self, incoming_user):
        if not incoming_user.prefers_outdoors:
            raise HeuristicFailedException("User does not want to sit outside")
        if not self.outdoor_seats_available(incoming_user.party_size):
            raise HeuristicFailedException("No seats available outside")
        return self.move_user_to_outdoor_table(incoming_user)

    def _try_split_tables(self, incoming_user):
        if not incoming_user.prefers_indoors:
            raise HeuristicFailedException("User does not want to sit inside")

        temp_table = self.indoor_seats_available(incoming_user.party_size / 2).pop(incoming_user.party_size / 2)
        remaining_people = incoming_user.party_size - (incoming_user.party_size / 2)
        if not self.indoor_seats_available(remaining_people):
            self.push_table_to_available(temp_table)
            raise HeuristicFailedException("Not enough tables to split")
        return self.move_user_to_table(incoming_user)

    def _try_move_to_queue(self, incoming_user):
        wait_for_seat_score = self.wait_for_seat_score(incoming_user.acceptable_wait_time, self.current_wait_time)
        if wait_for_seat_score < self.ACCEPTABLE_WAIT_SCORE:
            self.add_user_to_priority_queue(incoming_user)
        raise HeuristicFailedException("Wait time is too long for the user")

    def _try_move_outdoors(self, incoming_user):
        if not incoming_user.amenable_to_sitting_outdoors:
            raise HeuristicFailedException("User is not amenable to sitting outside.")
        self.move_user_to_outdoor_table(incoming_user)

    def _try_cancelling_reservation(self, incoming_user):
        seats_for_reservation = [seat for seat in self.indoor_seats_available(incoming_user.party_size) if seat.is_reserved]
        if not seats_for_reservation:
            raise HeuristicFailedException("No open seats are waiting on a reservation.")
        if seats_for_reservation[0].reserve_time + self.MAX_RESERVE_HOLDING_TIMEDELTA <= datetime.datetime.utcnow():
            seat_to_give_up = seats_for_reservation[0]
            self.cancel_reservation(seat_to_give_up.reserving_user)
            return self.move_user_to_table(incoming_user, specific_seat=seat_to_give_up)
        raise HeuristicFailedException("All of the reservations are within time window to wait for reservee.")

On the surface, you might agree with me about some upsides:

  • To begin with, you can avoid using comments because the exception messages supply context (comments are arguably bad because they require context switching and are inherently not maintained as well as code).
  • There is a reasonable mechanism to decompose the problem into smaller methods
  • As the heuristics are traversed, it’s less and less likely for the case to be hit, so the primary use cases are more clearly de-coupled
  • You could go as deep as you wanted with multiple levels of abstraction without worrying about managing return types
  • Re-ordering the priority of pieces of business logic are fairly straightforward.
  • From my perspective, it’s easier to abstract away possible cases, trust that they’re handled, and move on to the current block (i.e. in a few methods, a point for raising an exception happen multiple times).
  • There is no use of None or NULL references, another arguable indication of code smell (no None means that type consistency is maintained)

Furthermore, the usual reasons argued against exceptions aren’t quite applicable here. The exception type isn’t creating additional dependencies outside of this class because it’s intentionally managed internally (I do raise to the outside caller if no heuristic works, but that could easily be modified). The principle of least astonishment is another argument against exceptions. It states that the result of performing some operation should be obvious, consistent, and predictable. Here, I argue that exceptions are predictable since they are deliberately used in harmony to decompose a business problem.

More Arguments Against from StackOverflow

I read some examples from StackOverflow on why NOT to use exception handling for control flow. From my perspective, it seems to me that the real arguments against it are against the poor use of it, and its use as a potentially powerful tool are dismissed.

As of this writing, the top-voted counter argument is because of the principle of least astonishment, which I addressed above. The answer elaborates that exceptions are particularly costly in compiled languages where catch blocks aren’t optimized as well. In a non-compiled language like Python, however, the opposite is true. If I need to retrieve millions of items from a large dictionary, and it’s a valid assumption that most keys are present, it’s more performant to try/except for KeyErrors than it is to check if the key is present every iteration.

The next answer essentially raises an example of a program that has rotted so terribly that hundreds of exceptions are fired and caught per second. Indeed, using exception handling as duct tape for various exceptions will create terrible maintenance problems, but this is expected when you’re not addressing the root problem. The answer elaborates that if exceptions are used for control flow, it also makes it harder to spot the exceptions are truly the result of a problem. This is also an easily addressable problem since you’re free to create and subclass exceptions at will. Exceptions for control flow can easily be done with your own, specific exception, while runtime errors can throw the regular TypeErrors and ValueErrors and so forth.

Finally, my own aversion to misused exceptions is when it’s packed together inside of a function that’s already designated for something else. Truly clean code has functions that only do one thing. Exception handling is a thing and so too should be isolated to its own function.

Conclusion

This might be one of my more controversial articles that is more likely to end up on the cover of Us Weekly. So leave your thoughts in the comments, but please, limit the hate speech.

  • Chris Okasaki

    There are two things going here. First, you are creating the basics of a “combinator library” for heuristics. Second, you are using exceptions as a way to manage the plumbing. My recommendation would be to take the first part even farther. You currently have a single combinator (_execute_heuristics), but you could easily create more combinators that would let you combine heuristics in different meaningful ways. The key is to structure it so you build bigger heuristics by combining smaller heuristics. You can see this kind of approach used in many libraries for parsing combinators. As you do that, you may or may not find that exceptions are still working to manage the plumbing, and possibly switch to things like continuations or monads.