April 4, 2015

...Learn TDD with Codemanship

Why Software Craftsmanship Needs To Stop Worrying About The Colour Of The Bike Shed & Focus On The Big Issues

Why do software development efforts run into trouble?

When we examine the factors that tend to make development harder, and that raise the risk of failure - that is, failure for development to achieve its goals - we find the same culprits again and again.

1. Lack of customer involvement and "management support"

2. Lack of a clear vision or goal(s)

3. Unrealistic expectations

4. Projects that are too big and complicated to succeed

5. Failure to get feedback & incorporate learning throughout development

6. Lack of care taken to deal with issues early and cheaply

7. Overstaffing of teams to make them go faster

8. Hiring under-skilled teams

9. Being driven by the technology rather than the problem

10. Too much emphasis on the process (and its artefacts) and not enough on outcomes (and the proof)

These risk factors often can be seen conspiring to create some almighty shitstorms.

My overriding experience as a freelance developer and tech lead, going from project to project as we do, was giant overstaffed projects being executed by developers who were largely hired on price rather than ability, encouraged to hack code out with minimal attention to quality, and with stratospheric ambitions being squeezed into tiny schedules with arbitrary release dates cooked up by absentee customers who never gave us a clear idea of what it actually was we were setting out to achieve in the first place, but the teams had some pretty strong ideas about what technologies we should use (usually dictated by what they would like to add to their CV).

That, it would appear, is the norm in software development.

The inverse of that: small projects staffed by small and highly skilled teams, with clear and testable business goals from a committed customer whose involvement is pretty much daily, iterating towards working (tested) solutions on timescales decided by real progress towards our goals, using whatever technology is best to get the job done.

Parkinson's Law of Triviality (the "bike shed effect", named after a nuclear reactor's design committee obsessing over a bike shed while ignoring major issues) predicts that teams will focus on the things that matter least, but they are most comfortable talking about. Hence, we can devote a lot of time to debating the deployment architecture while non of us dares ask "but where is our customer?" It's understandable; many developers do not feel empowered or have confidence in tackling such matters as lack of customer involvement, dysfunctional hiring policy or unrealistic deadlines. So we obsess about Lambda expressions and Maven instead. That's stuff we know; that's stuff we can control.

But, in the grand scheme of things, Lambda expressions and Maven don't amount to a hill of beans compared to the big stuff. A project can hobble along quite nicely using while loops and Ant scripts. But a project that's too big is almost certain to fail, crushed under its own weight.

We're mistaken, though, that we have no power to influence these factors. In actual fact, as the people who make the software happen - without whom, no software will happen - we have the ultimate power. We can say "no".

When the customer is absent, stop development until you've got the access you need.

When the goals aren't clear and realistic, stop development until you have at least one testable and achievable goal.

When the boss insists on hiring a developer who's not up to snuff, stop development until they're removed from the team.

If these risks can't be addressed, the project has likely failed anyway. You're doing everyone a favour by effectively canning it from the developer's side. It's within our power to do these things.

Other professions can and do do this. Some are legally obliged to, and can go to prison if they don't. I don't advocate legislating to make developers more professional. But if we continue to roll over on these issues, eventually someone somewhere will.

Now, the meeker and milder among us, who find themselves retching at the mere thought of taking a stand, will say "I would have a conversation with them first". And of course you should. But here's the thing, and we've all been there: there will be times when we raise our concerns and the boss says "do it anyway". So what then?

I believe that knowingly choosing failure is bad. It's wrong. It really shouldn't be allowed. But it is, and teams do it all the time. They still get paid, of course. And that's one of the reasons why long schedules with release dates months or years away are still popular. It's how we make a living from failure. The longer we can make it before the shit hits the proverbial fan, the easier it is to sustain a living effectively throwing shit at fans.

Now, if you're the only person on the team who thinks this way, then realistically you're choosing between staying and probably failing, and walking. People with mortgages and school fees and car payments do not like this option. Which is why I coined the term mortgage-driven development, specifically to describe an approach to software development that knowingly embraces failure to ensure continuity of income.

And that, too, is pretty much the norm in software development. You are very likely in a minority on your team if you think that these big risks must be addressed. You are outnumbered by mortgage-driven developers who will accept working without clear goals, but with unrealistic expectations, in inexperienced and probably overstaffed teams, with absentee customers, and all the other risk factors that sink projects - for as long they keep paying them.

Until that balance changes, choosing failure will continue to be the default approach.

I suppose, secretly, I hoped the software craftsmanship movement would tip that balance. I hoped that more teams would take a stand. Bu I've seen no evidence of this happening. Mortgage-driven Development is as ubiquitous as ever. Only now, it's more like "Test-driven Mortgage-driven Development (with Angular.js and Microservices)", building on that old tradition of focusing on the things that matter least.

That's not to say that craftsmanship - or rather, the narrow reinterpretation of craftsmanship that's dominated the last 8 years or so - isn't important. It's important enough that I choose to focus my business on it almost exclusively. But it's only one piece of the jigsaw - a vital piece - of the puzzle that should be software craftsmanship.

As you may know, I'm no fan of renaming things. Because the craftsmanship community has been "doing craftsmanship wrong" all these years does not mean that we need a new name for software craftsmanship to make it explicit that it should be about taking care over all aspects of getting software right, including the big issues we prefer not to tackle. I want us to keep the name, and the emphasis on the practical that's characterised craftsmanship in the UK and Europe in particular - it should still be about creating working software, and not just be a talking shop about "what it means to be a craftsman" (we've had enough navel-gazing to last us a while).

There's a tendency for movements in software development to tip us off-balance. We seem to lurch from one over-emphasis (e.g., project management) to another (e.g., code quality), and never seeming to achieve actual balance of all these forces. Unbalanced Forces is itself considered a software project anti-pattern. We've actually woven unbalanced forces into the fabric of how we progress as a profession, constantly rediscovering ideas we knew about decades ago but then forget as the next shiny new idea came along.

There just needs to be greater emphasis now on issues like hiring, project size, agreeing goals, and all the stuff that hopefully will lead to us writing Clean Code on projects that give a damn. We've seen hints of it; and some clever approaches to making it hands-on and practical, as it should be.

There. I've said it.

Posted 2 years, 8 months ago on April 4, 2015