November 21, 2015

...Learn TDD with Codemanship

Getting Software Teams Back On Track

A recent tweet about things teams do when the schedule is slipping that can actually make things worse seems to have resonated with a lot of people.

It naturally begs the question: what could we do to get us back on track?

Well, the first - and most painful lesson - is to accept that just maybe we can't get back on track. And therefore we'll have to rethink the plan. There are two ways we can remake a plan that we know can work:

1. Move the delivery date back to fit all the work in

2. Deliver less to meet the delivery date

It's not a conversation a lot of teams relish having with their customer, but there is some silver living re: #2

When I say "deliver less", I mean "less software". If we're clear in our minds that software and value aren't the same thing, it becomes possible to deliver most of the value a customer needs - to solve their key problem, essentially - without delivering all the features we planned to.

This is the software development equivalent of saying "The house won't be finished on time, but you will be able to move in on that date".

If you're doing Agile Software Development, this may already have happened. And if it hasn't, then maybe you're not really doing Agile Software Development.

This solution does require us to have a clear idea of what the real end goals of the software are. You have to shift the emphasis from delivering a shopping list of features to solving a set of problems. Then you can buy yourself flexibility on design. More choices can be a powerful thing when you need to solve a problem in a fixed amount of time.

When I've inherited team leader roles on compromised projects, the first thing I've done is stop everything and get the team to step back and ask "What are we trying to achieve here? Is there a simpler way?" So be prepared to ditch the design and to ditch the plan that goes with it. In my own experience of saving compromised projects, this is the key. Prepare for some full and frank conversations with your customer.

But what of the three things I tweeted about that we know can make late deliveries even later?

Hiring More Developers has long been known to make things worse.

The reasons why hiring more developers exacerbates the problem are explained beautifully in Fred Brook's famous book The Mythical Man-Month. I believe every software developer and every software development manager should read it.

Developers who were busy getting shit done suddenly become busy getting new team members up to speed. It's like promoting your most effective sales people to management roles; watch those sales plummet!

Add to that an exponential explosion in the number of lines of communication within the team required to keep everyone abreast of what everyone else is doing - which, sadly, in software is a necessity because it's all connected - and we are fighting a double productivity hit of losing your most productive developers to mentoring duties and the rapidly diminishing returns of bigger teams.

So, the advice is very simply DON'T DO IT. No matter how tempted you are. No matter how much pressure you're under from above. DON'T DO IT.

But what if the team's already too big? What if the reason you're not getting shit done is because the team overhead is holding you back? Here, you may have to make very difficult decisions, ranging in severity from moving people on to other projects - if they exist - or side-lining people who are getting in the way, to letting people go as a last resort.

If you are a bit top-heavy, or have a few too many trainees, it's possible to absorb the hit. If, on the other hand, you have 3 architects, 2 project managers, 6 tech leads, 4 business analysts, 30 developers and 10 testers... well, as far as I'm concerned a lot of those people are enjoying what is effectively a kind of in-work welfare. They're getting paid - and possibly paid very well - to not just add no value, but to positively suck value out of the team. You're paying them to slow you down. We all know it, but few dare say it.

In those cases when the team is very obviously too big, then you'll have very little choice but to simplify it if you want to deliver something of value when the customer needs it.

Alas, these kinds of decisions - to hire or to redistribute or to fire - are usually out of the hands of the likes of me and you, and managing upwards in these situations carries great risk; specifically the risk that the person going will be the one who blows the whistle. The default behaviour in these situations is to quite knowingly choose failure; perversely, for the good of the team.

Cutting Corners is a classic mistake. For about as long as we've known that hiring more developers will make things worse, we've also known that skimping on quality actually makes us go slower.

The reasons are simple, well-understood and backed up by a veritable Himalayas of hard evidence; bugs cost exponentially more time and money to fix the longer they go undetected. Less testing, less often, therefore, creates a downstream tsunami of necessary bug fixes that will take 10-100x times as long to fix than if we'd caught them sooner.

You can spot the teams that have been doing what we call "code-and-fix" development not just by the size of their bug list, but by the sheer amount of time and effort it takes them to "stabilise" the software to make it fit for release. I've watched teams go round and round in circles for months trying to make code good enough to ship.

This is why teams in the know, when the schedule's slipping, do more testing, more rigorously, and more often. Yes, it's more work now. But it's 10-100x less work later.

For sure, there are no silver bullets in software development. But when it comes to putting teams back on track, creating fast-running automated tests is the closest thing I've seen. It can completely transform the outlook.

Likewise moving to Specification By Example in our discussions about requirements with the customer can have a transformative effect. You'd be amazed how many misunderstandings you catch when you ask for concrete examples to clarify what they mean. A couple of hours spent clarifying a requirement with executable acceptance tests could save you weeks of unnecessary rework.

Same goes for the maintainability of the code. Less thought and care put into making our code easy to understand and easy to change (without breaking) can cost us dearly, and sooner than we think. I've found myself being hampered in the afternoon by code I wrote in the morning.

As in golf, the way to get a better score faster is to take more care over our shots.

This, too, is a tough sell both to team mates and to managers. The entirely debunked myth that better quality software takes longer to deliver still persists. And there may be new skills that have to be learned to make a defect prevention and "clean code" approach work. If the team's not already up-to-speed on the principles and practices, then - well, you're a bit buggered, really, because you're months (possibly years) away from conquering that learning curve.

If the team, on the whole, does have the necessary skills, then that would beg the question: why weren't you already doing these things? If the answer is "the boss won't let us", then here comes another difficult conversation, potentially.

My own experience, and the anecdotes I hear from people in that situation, is that you should never have that conversation. If you're having that conversation, all is very possibly already lost. It rarely goes the way you wish it to. By giving managers and customers a choice over quality, 99 times out of 100, they'll veto it in favour of "features". Even though, as we know, skimping on quality will deliver less features. But the perceived conflict between those two concerns beats most teams who make the mistake of offering it as a choice.

So, when it comes to meeting delivery dates, make quality non-negotiable. And the best way to do that is to never talk about it outside of the team. If you do TDD, for example, than that is the way you deliver working software. Just as surely as using a text editor is. When did you last offer the boss the choice of whether or not you should use a text editor?

And when things are slipping, you need to pull on those quality levers even harder. If you've been taking baby steps, take even babier steps. If you've been building and testing daily, build and test hourly. If you've been devoting an average 20 minutes to discussing a user story with the customer, devote an hour and agree executable tests, including all the edge cases you can think of. If you're averaging one unit test for 10 lines of code, aim to make your unit tests even more focused. If you're averaging one team code review in each iteration, do peer code reviews before every commit. And so on.

Finally, Working Long Hours - we know from many studies done over the last century - is not sustainable, hurts teams and projects, and is a false economy. Going as far back as Henry Ford, who discovered that when he reduced factory workers' hours from 50 to 40 a week, they got more done, we've known that hours worked != useful output.

Software development in particular is susceptible to burnout when we push teams into "crunch mode". It's an activity that requires focus and clarity, and focus and clarity are perishable goods that need to be replenished frequently. From my own experiments with teams, and with myself, I learned that the average developer will achieve roughly as much in 4 hours a day as she does in 8, and more than she would in 12 hours.

Some folk say "it's okay in the short term", but I've found no evidence anywhere that even this is true. Maybe an assembly line worker can work a few long shifts to meet a deadline, but that ain't software development.

We're also very susceptible to the illusion of productivity, and I think this explains where the "it helps in the short-term" myth comes from. Sure, we feel like we're getting shit done. But that may well be because we are blind to the downstream consequences of the mistakes we make along the way. Towards the end of an 8-hour day, I find - when I measure it objectively - that I'm making more mistakes. And so I've learned to quit while I'm ahead, and have found it beneficial to have teams do likewise. Learn to recognise when you're passing from a zone of "flow" into a zone of "liability".

And so, when I lead teams, I have a very low tolerance for longer hours as any kind of solution - even a short-term one.

And, yes, this too is a very tough sell to management. And to many developers still enraptured by the illusion of their own productivity. It takes real nerve - when a deadline is looming and everyone's frantically firefighting - to calmly get up from your desk at 5-6pm and go home. But it is, as far as I can tell, the best thing you can do for the project.

Similar - and closely related to the need to take more care - is the need to stay even more refreshed. Take more short breaks, have a proper relaxing (but not carb-heavy, because that'll make you sleepy) lunch. Be extra diligent about leaving at a reasonable time and having an even better work-life balance.

And then turn your focus on to what you actually do when you are at your desk instead of how long you spend at your desk.

One thing that really annoys me is how many organisations insist on long hours, but then merrily burn those hours on meaningless distractions like meetings and dysfunctional working environments like open-plan offices. I tend to find a strong correlation between the two. Wanna' get more done? Tackle the working environment, not the hours.

So, there you have it. To summarise; to improve your chances of meeting a delivery date:

1. Manage expectations of about what will be delivered and when. Simplify the product, simplify the plan. Focus on the end goal, not the shopping list.

2. Resist the temptation to hire more developers. If your team is already far too big, have that difficult conversation.

3. Turn the quality dial UP. Make quality non-negotiable.

4. Work FEWER hours. Take MORE breaks. Instead of focusing on hours, focus on what gets done in those hours, and remove obstacles to getting things done, like the working environment.

Of course, I say all these things safe in the knowledge that most teams will be too afraid to try them. It's easy on paper. But in practice, it takes a certain kind of team with a certain kind of leadership to make these things happen.

So, finally:


This doesn't necessarily mean someone bullish like me fighting the team's battles for them (though I have done it many times). But somehow, from somewhere within, the team needs to find this strength and self-belief and assert themselves on key issues like scope, deadlines, team size, working hours, office environments and so on. They've made you responsible for delivering: now assert the authority that has to accompany that responsibility if it's got to work.

Good luck!

Posted 3 weeks, 4 days ago on November 21, 2015