March 25, 2011
"But TDD Doesn't Guarantee Defect-Free Code..."Sir Ian McKellan cannot play every possible part, therefore he's a crap actor.
Charity won't end starvation, therefore we shouldn't give.
Test-driven development will not catch all bugs, therefore we shouldn't bother with it.
And so on.
Writing software is a concert of related disciplines and techniques, and no single one of them takes care of all the business of producing working, useful, maintainable, performant, secure, reliable, robust, sociable, clean-shaven code.
TDD, for example, does not remove the need for testing. We flesh out our algorithms and OO designs using - ideally - a minimum list of examples that describes the overall behaviour. once we've reached the example that leads us to the general solution, all nicely refactored on the inside, then as far as TDD is concerned we are done.
If you think there may be scenarios for which you didn't have an example, but that won't require you to write new code, by all means write tests for them. This is not TDD. We just call it "testing". And it's fine.
For example, when calculating Fibonacci numbers, I can "name that tune" in just five examples: F(0), F(1), F(2), F(3) and F(5). But I might decide to add tests after the fact for F(19), or F(49), just to reassure myself that my test-driven solution really is correct.
Doing Fibonacci numbers as a kata in Java, I was merrilly using int as the return value of my function, since that worked just dandy up to F(5). An additional test for F(49) would have shown that Fibonacci numbers can be too big to store as an int, and therefore it needs to be a long for my function to work in all conceivable cases.
That my roadmap to a finished algorithm missed this edge case in no way devalues TDD, though. It's safe to say that test-driven code, done fairly rigorously, usually has low bug counts. There's evidence to support that from several studies. But nobody promised us zero defects from TDD. For high-integrity code, you need to go further.
But the fact that TDD doesn't produce zero-defect code is often used as an objection to doing TDD at all. Which I find very interesting, because in every instance where someone has thrown that objection my way, it's been the case that they're own code has much higher bug counts than we normally find with TDD. Practicing TDD almost certainly would improve the reliability of their code, and quite significantly.
So this is a dubious criticism, in my experience. Firstly because no one practice guarantees perfection, so why single out TDD, which has proven itself to add value in a wide range of situations? And secondly because, in most cases, teams that raise this objection are not doing anything else - or planning to do anything else - to make their code cleaner and more reliable.
That's my biggest bugbear. So you don't want to do TDD? Fine. Teams have produced amazingly reliable code without it, and there are definitely alternatives which can produce equal - if not better - results. How about Design By Contract? Or Formal Specification coupled with rigorous unit testing? Or Precise UML and MDA? Or guided inspections? Symbolic execution? Model checkers? Parameterised testing? Or just plain old hardcore testing - like the techniques Robert Binder talks about in his massive book "Testing Object Oriented Systems" (which I think every developer should have a copy of)? Or a combination of any or all of these techniques?
What irks me is that, often as not (in fact, always, in my own experience), when folk play the "TDD doesn't guarantee correctness" card as a reason not to do it, they're not using any other techniques to eliminate defects, and have no plans to, either.
I'm the first one - indeed, one of the few - who believes TDD is not enough where quality is concerned. And in previous blog posts going back over years, I've talked about complementary techniques for catching those especially elusive bugs, like DBC and guided inspections. Given that on average developers only write 10 lines of code a day, I think there's plenty of scope for going that extra mile, and plenty of good, hard commercial reasons for doing it.
So from now on, when you say to me "ah, but TDD doesn't guarantee defect-free code", don't be surprised when I ask to see your defect-free code. Well, it's only fair.
Posted 3 weeks, 3 days ago on March 25, 2011