May 28, 2006

...Learn TDD with Codemanship

Bug-Free Part I - Building It Right

Everybody has their own views on the correct way to build software, and especially on how to improve the quality of the software we build. And I'm no exception!

Today we have access to a broad range of techinques for requirements analysis, specification, implementation and verification. But software quality seems to be as bad as ever. Arguably there should be no excuse. And, for one class of bugs, I quite agree.

I group quality into two areas:

    * Building the right thing
    * Building it right

The second part is where we should have no excuses. Once we know what the software is supposed to do (and what it's not supposed to do), we should be capable of writing code that does exactly that. Writing software that conforms to a well-defined specification is the easy part.

Here are 5 techniques for producing (almost) bug-free code that satisfies a well-defined specification:

1. Test-driven Development - if you do if for real. In TDD, the tests are the well-defined specification. How do you know if you're doing it right? Well, for starters you shouldn't write so much as a single character of production code unless you've got a failing test that needs it. If you've got 99% code coverage in your tests, then you're only 99% test-driven. And let's not forget, code coverage is just the beginning. I would suggest that 100% code coverage is your table stakes for bug-free software.

2. Inspections - whether it's through pair programming or with the whole team in a room with a laptop and projector (or both). The best way to do inspections is to step through a test scenario in the code one line at a time, making assertions about what should be true before and after each step. (Imagine every single line of code has pre and post-conditions). Pick key scenarios, or randomly select them, or mix and match. Again, coverage is key. Pair programming should mean that every single line of production code has been looked at by at least 2 people. Try not to get side-stepped into debating design or coding standards. Check the logic is correct first, then worry about whether to use a Factory Method or an Abstract Factory after you've delivered something that works.

3. Design By Contract - unit tests verify that methods satisfy their post-conditions. Who's verifying that clients satisfy the method's pre-conditions? Consider using DBC-style assertions to check pre-conditions for critical methods. This is especially important for methods that are widely reused.

4. "Most Wanted" Bug List - if you keep a record of your bugs, and their root causes, you may well discover patterns. We tend to introduce the same kinds of bugs over and over. For example, the null reference bug is extremely common. You may also discover that roughly 20% of bug types account for about 80% of the bug-fixing effort (according to the old 80/20 rule). Maintain a "most wanted" list of these pesky critters and make it everybody's business to know them and to know how to avoid them. Avoiding bugs can be easy if you know what kind of tests tend to show them up, and get into the habit of using those tests whenever you write a certain kind of bug-prone code. There are teams out there who rarely, if ever, introduce a null reference bug, for example. They've gotten into the habit of checking that the object isn't null before they try to call any of its methods. Think of it as being like getting into the habit of checking that you've got your keys before you leave the house... After a while, the most wanted bugs will become a distant memory and your attention can turn to bug types that are lower down the list.

5. Concurrency - raises the complexity stakes in your code by several orders of magnitude. They say the only code guaranteed to be bug-free is the code you didn't have to write. The same is true of concurrency. The best way to produce bug-free multi-threaded code is to avoid writing any. If you absolutely must, then here's a little tip for you: sequence diagrams. If you want to visualise multiple threads accessing methods on the same object at the same time, sequence diagrams help make concurrent logic clangers that little bit more obvious. Unit tests are, of course, next to useless when testing multiple threads because the order in whcih instructons being executed by each thread are interleaved cannot be guaranteed to be the same every time. A test may show up a bug one time, but not the next. Concurrent bugs tend to be difficult to reproduce, and this make them much harder to find and fix (and to avoid in the future.) IBM have created a unit testing tool that explores a whole range of interleaving scenarios called ConTest, but I haven't tried it myself yet, so I can only suggest you try it for yourself.

I've tried to stick to low-cost cheap and cheerful techniques for keeping bugs out of your code. To be honest, I've never used all 5 techniques on the same project, so I don't know what the end product would be if you went the whole hog and used all of these techniques rigorously. To be even more frank, I don't believe that anybody out there has used all of them on a project ever. Which is a little odd, because I can't think of any practical reasons why not. You don't need management buy-in to download JUnit and start vwriting unit tests. Or to walk through other people's code. Or to write assertions. Or to maintain a bug list. Or to explore concurrency scenarios with sequence diagrams or a tool like ConTest.

Hey, y'know what - you could be the first! Like I said: there's no excuses...

Bug Free Part II - Building The Right Thing
Posted 14 years, 7 months ago on May 28, 2006