January 30, 2015

...Learn TDD with Codemanship

It's True: TDD Isn't The Only Game In Town. So What *Are* You Doing Instead?

The artificially induced clickbait debate "Is TDD dead?" continues at developer events and in podcasts and blog posts and commemorative murals across the nations, and the same perfectly valid point gets raised every time: TDD isn't the only game in town

They're absolutely right. Before the late 1990's, when the discipline now called "Test-driven Development" was beginning to gain traction at conferences and on Teh Internets, some teams were still somehow managing to create reliable, maintainable software and doing it economically.

If they weren't doing TDD, then what were they doing?

The simplest alternative to TDD would be to write the tests after we've written the implementation. But hey, it's pretty much the same volume of tests we're writing. And, for sure, many TDD practitioners go on to write more tests after they've TDD'd a design, to get better assurance.

And when we watch teams who write the tests afterwards, we tend to find that the smart ones don't write them all at once. They iteratively flesh out the implementation, and write the tests for it, one or two scenarios (test cases) at a time. Does that sound at all familiar?

Some of us were using what they call "Formal Methods" (often confused with heavyweight methods like SSADM and the Unified Process, which aren't really the same thing.)

Formal Methods is the application of rigorous mathematical techniques to the design, development and testing of our software. The most common approach was formal specification - where teams would write a precise, mathematical and testable specification for their code and then write code specifically to satisfy that specification, and then follow that up with tests created from those specifications to check that the code actually works as required.

We had a range of formal specification languages, with exotic names like Z (and Object Z), VDM, OCL, CSP, RSVP, MMRPG and very probably NASA or some such.

Some of them looked and worked like maths. Z, for example, was founded on formal logic and set theory, and used many of the same symbols (since all programming is set theoretic.)

Programmers without maths or computer science backgrounds found mathematical notations a bit tricky, so people invented formal specification languages that looked and worked much more like the programming languages we were familiar with (e.g., the Object Constraint Language, which lets us write precise rules that apply to UML models.)

Contrary to what you may have heard, the (few) teams using formal specification back in the 1990's were not necessarily doing Big Design Up-Front, and were not necessarily using specialist tools either.

Much of the formal specification that happened was scribbled on whiteboards to adorn simple design models to make key behaviours unambiguous. From that teams might have written unit tests (that's how I learned to do it) for a particular feature, and they pretty much became the living specification. Labyrinthine Z or OCL specifications were not necessarily being kept and maintained.

It wasn't, therefore, such a giant leap for teams like the ones I worked on to say "Hey, let's just write the tests and get them passing", and from there to "Hey, let's just write a test, and get that passing".

But it's absolutely true that formal specification is still a thing that some teams still do - you'll find most of them these days alive and well in the Model-driven Development community (and they do create complete specifications and all the code is generated from those, so the specification is the code - so, yes, they are programmers, just in new languages.)

Watch Model-driven Developers work, and you'll see teams - well, the smarter ones - gradually fleshing out executable models one scenario at a time. Sound familiar?

So there's a bunch of folk out there who don't do TDD, but - by jingo! - it sure does look a lot like TDD!

Other developers used to embed their specifications inside the code, in the form of assertions, and then write tests suites (or drive tests in some other way) that would execute the code to see if any of the assertions failed.

So their tests had no assertions. It was sort of like unit testing, but turned inside out. Imagine doing TDD, and then refactoring a group of similar tests into a single test with a general assertion (e.g., instead of assert(balance == 100) and assert(balance == 200), it might be assert(balance = oldBalance + creditAmount).

Now go one step further, and move that assertion out of the test and into the code being tested (at the end of that code, because it's a post-condition). So you're left with the original test cases to drive the code, but all the questions are being asked in the code itself.

Most programming languages these days include a built-in assertion mechanism that allows us to do this. Many have build flags that allows us to turn assertion checking on or off (on if testing, off if deploying to live.)

When you watch teams working this way, they don't write all the assertions (and all the test automation code) at once. They tend to write just enough to implement a feature, or a single use case scenario, and flesh out the code (and the assertions in it) scenario by scenario. Sound familiar?

Of course, some teams don't use test automation at all. Some teams rely on inspections, for example. And inspections are a very powerful way to debug our code - more effective than any other technique we know of today.

But they hit a bit of a snag as development progresses, namely that inspecting all of the code that could be broken after a change, over and over again for every single change, is enormously time-consuming. And so, while it's great for discovering the test cases we missed, as a regression testing approach, it sucks ass Gagnam Style.

But, and let's be clear about this, these are the techniques that are - strictly speaking - not TDD, and that can (even if only initially, like in the case of inspections) produce reliable, maintainable software. if you're not doing these, then you're doing something very like these.

Unless, of course... you're not producing reliable, maintainable code. Or the code you are producing is so very, very simple that these techniques just aren't necessary. Or if the code you're creating simply doesn't matter and is going to be thrown away.

I've been a software developer for approximately 700 million years (give or take), so I know from my wide and varied experience that code that doesn't matter, code that's only for the very short-term, and code that isn't complicated, are very much the exceptions.

Code that gets used tends to stick around far longer than we planned. Even simple code usually turns out to be complicated enough to be broken. And if it doesn't matter, then why in hell are we doing it? Writing software is very, very expensive. If it's not worth doing well, then it's very probably - almost certainly - not worth doing.

So what is the choice that teams are alluding to when they say "TDD isn't the only game in town?" Do they mean they're using Formal Methods? Or perhaps using assertions in their code? Or do they rely on rigorous inspections to make sure they get it at least right the first time around?

Or are they, perhaps, doing none of these things, and the choice they're alluding to is the choice to create software that's not good enough and won't last?

I suspect I know the answer. But feel free to disagree.


My apprentice, Will Price, has emailed me with a very good question:

"Why don't we embed our assertions in our code and just have the tests exercise the code?

What benefit did we gain from moving them out into tests?

It seems to me that having assertions inside the code is much nicer than having them in tests because it acts as an additional form of documentation, right there when you're reading the code so you can understand what preconditions and postconditions a particular method has, I should imagine you could even automate collection of these to
put them into documentation etc so developers have a good idea of whether they can reuse a method (i.e. have they satisfied the preconditions, are the postconditions sufficient etc)"

My reply to him (copied and pasted):

That's a good question. I think the answer may lie in tradition. Generally, folks writing unit tests weren't using assertions in code, and folks using assertions in code weren't writing automated tests. (For example, many relied on manual testing, or random testing, or other ways of driving the code in test runs).

So, people coming from the unit testing tradition have ended up with assertions in their tests, and folk from the assertions tradition (e.g., Design By Contract) have ended up with no test suites to speak of. Model checking is an example of this: folks using model checkers would often embed assertions in code (or in comments in the code), and the tool would exercise the code using white-box test case generation.

This mismatch is arguably the biggest barrier at the moment to merging these approaches because the tools don't quite match up. I'm hoping this can be fixed.

Thinking a bit more about it, I also believe that asserting correct behaviour inside the code helps with inspections.

Posted 5 years, 6 months ago on January 30, 2015