June 10, 2012
Late Night Thoughts On "It Works For Me"Before I retire up the Apples & Pears to Bedfordshire, I just wanted to share some thoughts on an ongoing discussion I've been having in That Twitter with Dan North (@tastapod).
Now, I'm aware that I can be overly dismissive of people making claims that aren't supported by evidence, and I feel it's important to go beyond the limitations of 140 characters to try - and probably fail, as usual - to express what I'm really thinking about all this.
To cut a long story short, Dan's been writing and speaking a lot recently about a discovery he's made that involves writing software that is not - GASP! - test-driven. Indeed, there may be no automated tests at all. And he's finding that in the context he and his colleagues are working in, not TDD-ing is sometimes better and faster at delivering value, and, presumably, at sustaining that pace of innovation for business advantage.
Dan, if you're not aware, comes very highly recommended by programmers who also come very highly recommended. If programmer kudos was PageRank, and recommendations were web links, Dan's home page would be bbc.co.uk. So, at a personal level, I'm inclined to just shrug and say "fair enough, what he said".
But I've been at this game a while (and I've even won the odd round), and my two decades programming for shiny objects and sexual favours has taught me that our industry is rife with claims.
Some are out-and-out lies. The people making them know full well it's not true, and what they're saying is designed purely to appeal to the people who are holding the purse strings - a highly suggestible bunch at the best of times.
I think it's very doubtful that Dan doesn't believe what he's telling us, from what I've heard of him. But some very genuine people, with all the best intentions, also make claims that turn out not to be true. Software is a very complicated business, and mirages are not uncommon.
I know how prone I am to succumbing to that feeling of "productivity" I get when I cut corners. It's very seductive.
My Mum used to drive a Citroen 2CV (mint green with stripes on the roof - it looked like a boiled sweet on wheels), and I remember the sheer thrill of us coasting down hills, feeling like the car could take off at any moment. We must have been doing all of 45 miles per hour.
I've discovered, from my own experiments into quality-centred practices that, when I actually look back at what's been achieved objectively, what felt fast while I was doing it can turn out to be slower in real terms.
So, my issue is this: it's not that I think Dan's misleading us, or that he's necessarily misleading himself, either. What he's discovered may well be real, and may even be reproducible.
But, right now, he's that lone parent who didn't vaccinate their children and found that their children got better. Or that they think their children got better, and that it had something to with not vaccinating them. Maybe they did, and maybe it was.
However, before I start advising teams to not bother with the vaccinations - vaccinations whose efficacy is supported by a growing body of evidence in a wide range of situations (everything from embedded software in vending machines to labyrinthine distributed "enterprise" systems via the BBC iPlayer) - I need to see a similar body of evidence to persuade me that in some situations, skipping the jabs will be better for them.
I'd also like to understand why. I'm fairly convinced now of the causal mechanisms that link defect prevention to higher productivity, having seen so many wide studies published by the SEI, IBM, NASA and other august bodies. Taking steps to prevent issues saves more time later than it costs now. Simples.
I'm also aware of the limits of defect prevention on saving us time and money, and why those limits exist (e.g., in safety-critical software).
The same goes for the relationship between our ability to retest our software and systems quickly, frequently and cheaply. I'm not aware of any other way than by automating our tests, and I'm especially aware of the economic value of automated unit tests (or some automated equivalent - e.g., model checkers), having spent very little time in a debugger personally since about 2002.
It's not inconceivable that somewhere in the spectrum of quality vs. cost vs. test automation etc etc, there is an oasis that Dan's discovered of which we're all currently unaware. But if there is, then it's a tropical island in the middle of the Arctic ocean. It runs contrary to the picture that surrounds it - a picture that's still being corroborated as more and more data comes in, and for which no credible data currently exists to contradict it.
Right now, Dan's telling us he's been to this undiscovered island, and is describing it to us in vivid detail - thrilling tales of strange and exotic animals, wierd and wonderful plant life and azure-blue waters lapping at golden sands. But he's yet to give us the photos, videos, or any samples of unique flora and fauna that might convince me that he wasn't actually in Fiji (that's the danger of flying without instruments). Most important of all, he needs to give us the grid reference so we can all go and find this island for ourselves.
He tells me he's in the process of doing this now, so we can try his approach on our own projects and see what we think. This is very encouraging.
My hope is that we'll finally see this mysterious Lost World for ourselves and know that he was right.
Either that or we'll confirm that he is indeed lost in Fiji.
Posted 5 years, 9 months ago on June 10, 2012