December 12, 2007

...Learn TDD with Codemanship

More Places Where Bugs Can Hide

Hello there. I'm glad I've bumped into you, actually.

You see, I've thought of two other places that bugs can hide. And, as we all know, when we leave places that bugs can hide, they do rather tend to take advantage of our hospitality.

Before I tell you about these new Bug Hotels, let's refresh our memories about some of the other places that bugs are known to frequent:

* Complex blocks of code - all those big nested if... statements and twisty-turny methods are crammed full of the meaty badness that we know bugs like to eat

* Untested blocks of code - bugs are afraid of the light, so they'll make a beeline for anywhere our torches aren't shining

* Untested paths through the code - even when we've exercised every individual code statement, bugs can still hide in those peculiar execution sequences that our code allows, but we didn't think of

* Untested combinatons of logical conditions - once we've sanitised every line of executable code, and poured test disinfectant on every interesting path through the code, bugs can still retreat behind tricky combinations of A AND B, or A AND NOT B, (or NOT A AND B, and so on) that we didn't write tests for.

But none of this is news to you, of course. You and I, my friend, we laugh in the face of such bugs. Ha ha ha ha ha ha ha ha ha (ha ha ha ha). Because we know that we haven't left anywhere for bugs to hide in our code. No sir. (Or Madam.)

Well, I'm afraid you're about to get a visit from the Bad News Bear. There are still places that the hardiest of bugs could survive, even through that onslaught of test coverage.

Bugs can hide inside untested interactions. Sure, we tested the [colourful vernacular removed for reasons of taste and decency] out of class X, covering every line of code in every method, and every path through every method, and every logical combinaton of conditions in the code. But what we didn't test was that all of the methods on class X are used correctly by every client of class X. (Quick, sonny! Fetch Dad's gun! They're inside the building...)

Some bugs hiding yesterday

Yes, that's right. I'm talking about preconditions. Now things are getting complicated. How can we get good coverage on preconditions? The mind boggles. Well, mine does, usually.

Here's the beef; we'd have to test that everywhere a method is called, the precondition is never broken. A tall order, to say the least.

We could do it by inspection, searching for references to each method and then walking through the client code to make sure that there's no chance of the method being called when it shouldn't oughta be.

That sounds like fun, doesn't it? (Quick, sonny! Fetch Dad's pipe, we're in for a long evening...)

Another possible remedy might be to use some kind of mocking framework that can check constraints on all important interactions, perhaps using something like the Specification pattern. But just think of the code clutter that could create. Yuck! (Quick, sonny! Fetch Dad's shovel. Someone's been mocking in the garden again...)

Finally, we could go all old-fashioned and Victorian and opt for the steam-powered solution - namely assertions. (Quick, sonny! Fetch Father's hearing trumpet! They're talking about the old days...)

There may well be better solutions to this problem, of course. And if there are then please let me know so I can pretend it was my idea and take all the glory. (Well, hey - if it's good enough for [name of defamed author removed for legal reasons], then it's good enough for me.)

And as if that wasn't bad enough, now that we've established that your code is all bug-infested again, then even if we eliminate the precondition/interaction problem, we still have to contend with another place that bugs can hide: bugs can hide inside untested polymorphic interactions.

Yes, folks. That complexity doesn't go away just because we got rid of our switch... statements. We may have deferred our decision-making to the type system, but if the feed() method could be different on the Lion, Tiger and Spider subtypes of Animal, then we have to test that every one of those methods does what every client expects of an abstract feed() method, both pre and post-conditions.

When Barbara Liskov told us that you should be able to substitute an instance of one type of objects for an instance of any of its subtypes, what she really meant was that every subtype must pass all of the tests associated with that supertype, as well as any specific tests of its own.

This raises the possible need for some kind of polymorphic testing. Which I've been arguing for for several years. But the Agilistas poo-pooed my ideas, saying I was mad and impractical, and demanding to know how I got their home phone numbers in the first place.

But who's having the last laugh now?

Not me, that's for sure.

Sleep well.

* Yes, I know that there are plenty of other places that bugs can hide - like untested concurrency scenarios, for example.
Posted 2 weeks, 4 days ago on December 12, 2007