January 16, 2015

...Learn TDD with Codemanship

Can Restrictive Coding Standards Make Us More Productive?

So, I had this long chat with a client team yesterday about coding standards, and I learned that they had - several years previously - instituted a small set of very rigorously enforced standards. We discussed the effects of this, especially on their ability to get things done.

They've built a quality gate for their product that runs as part of their integration builds (though you can save yourself time by running it on your desktop before you try to commit).

To give you an example of the sort of thing this quality gate catches, and their policies for dealing with it, try this one for size...

Necessarily, parts of their code are multi-threaded and acting on shared data. Before we begin, let's just remind ourselves that this is something that all programming languages have to deal with. Functional programming uses sleight of hand to make it look like multiple threads aren't acting on shared data, but they do this typically by pushing mutable state out into some transactional mechanism where our persistent data is managed. And hence, at some point, we must deal with the potential consequences of concurrency.

So, where was I? Oh yeah...

Anyway, they find that unavoidably there must be some multithreading in their code. Their standard, however, is to do as little multithreading as possible.

So their quality gate catches commits that introduce new threads. Simple as that: check in code that spawns news threads, and the gate slams shut and traps it in "commit purgatory" to be judged by a higher power.

The "higher power", in this case, is the team. Multithreading has to be justified to an impromptu panel of your peers. If they can think of a way to live without it, it gets rejected and you have to redo your code without introducing new threads.

Why go to all this trouble?

Well, we've all seen what unnecessary complexity does to our code. It makes it harder to understand, harder to change, and more likely to be contain errors. Multithreading adds arguably the most pernicious kind of complexity, creating an explosion in the possible ways our code can go wrong.

So every new thread, when it's acting on shared data, piles on the risk, and piling on risk piles on cost. Multithreading comes at a high price.

The team I spoke to yesterday recognised this high price - referring to multithreaded logic as "premium code" (because you have to pay a premium to make it work reliably enough) - and took steps to limit its use to the absolute bare minimum.

In the past, I've encouraged a system of tariffs for introducing "premium code" into our software. For example, fill a small jam jar with tokens of some kind labelled "1 Thread", and every time you think you need to write code that spawns a new thread, you have to go and get a token form the jar. This is a simple way to strictly limit the amount of multithreaded code by limiting the number of available tokens.

This can also serve to remind developers that introducing multithreaded code is a big deal, and they shouldn't do it lightly or thoughtlessly.

Of course, if you hit your limit and you're faced with a problem where multithreading is unavoidable, that can force you to look at how existing multithreaded code could be removed. Maybe there's a better way of doing what we want that we didn't think of at the time.

In the case of multithreading - and remember that this is just an example (e.g., we could do something similar whenever someone wants to introduce a new dependency on an external library or framework) - it can also help enormously to know where in our code the multithreaded bits are. These are weak points in our code - likely to break - and we should make sure we have extra-string scaffolding (for example, better unit tests) around those parts of the code to compensate.

But the real thrust of our discussion was about the impact rigorously enforced coding standards can have on productivity. Standards are rules, and rules are constraints.

A constraint limits the number of allowable solutions to a problem. We might instinctively assume that limiting the solution space will have a detrimental effect on the time and effort required to solve a problem (for the same reason it's easier to a throw a 7 with two dice than a 2 or a 12 - more ways to achieve the same goal).

But the team didn't find this to be the case. If anything, to a point, they find the reverse is true: the more choices we have, the longer it seems to take us.

So we ruminated on that for a while, because it's an interesting problem. Here's what I think causes it:

PROGRAMMERS ARE PARALYSED BY TOO MANY CHOICES

There, I've said it.

Consider this: you are planning a romantic night in with your partner. You will cook a romantic dinner, with a bottle of wine and then settle down in front of the TV to watch a movie.

Scenario #1: You go to the shopping mall to buy ingredients for the meal, the wine and a DVD

Scenario #2: You make do with what's in your fridge, wine rack and DVD collection right now

I've done both, and I tend to find that I spend a lot of time browsing and umm-ing and ah-ing when faced with shelf after shelf of choices when I visit the mall. Too much choice overpowers my feeble mind, and I'm caught like a rabbit in the headlights.

Open up the fridge, though, and there might be ingredients for maybe 3-4 different dishes in it right now. And I have 2 bottles of wine in the rack, both Pinot Noir. (Okay, I do, however, have a truly massive DVD collection, which is why when I decide to have a quiet night in watching a movie, the first half hour can spent choosing which movie.)

Now, not all dishes are born equal, and not all wines are good, and not all movies are enjoyable.

But I generally don't buy food I don't like, or wine I won't drink (and that leaves a very wide field, admittedly), or movies I don't want to watch. So, although my choices are limited at home, they're limited to stuff that I like.

In a way, I have pre-chosen so that, when I'm hungry, or thirsty or in need of mindless entertainment, the limited options on offer are at least limited to stuff that will address those needs adequately.

The trick seems to be to allow just enough choice so that most things are possible. We restrict our solution space to the things we know we often need, just like I restrict my DVD collection to movies I know I'll want to watch again. I'll still want to buy more DVDs, but I don't go to the shop every time I want to watch a movie.

This is the effort we put into establishing standards in the first place (and it's an ongoing process, just like shopping.)

Over the long-term, at our leisure, we limit ourselves to avoid being overwhelmed by choice in the heat of battle, when decisions may need to be made quickly. But we limit ourselves to a good selection of choices - or should I say, a selection of good choices - that will work in most situations. Just as Einstein limited his wardrobe so he could get on with inventing gravity or whatever it was that he did.

Harking back to my crappy DVD library analogy - and I know this is something friends do, too, from what they tell me - I will not watch a particular movie I own on DVD for years, and then it'll be shown on TV, and I will sit there and watch it all the way through, adverts and all, and enjoy it.

This might also have the effect of compartmentalising trying out new solutions (new programming languages, new frameworks, new build tools and so on) - what we might call "R&D" (though all programming is R&D, really) - from solving problems using the selection of available solutions we land upon. This could be a double-edged sword. Making "trying out new stuff" a more explicit activity could have the undesired effect of creating an option for non-technical managers that wasn't there before. Like refactoring, it's probably wise to make this non of their business.

And, sure, from time to time we'll found ourselves facing a problem for which our toolkit is not prepared. And then we'll have to improvise like we always do. Then we can switch into R&D mode. In Extreme Programming, we call these "spikes".

But I can't help feeling that we waste far too much time getting bogged down in choices when we have a perfectly good solution we could apply straight away.

Oftentimes, we just need to pick a solution and then get on with it.

I look forward to your outrage.



Posted 3 years, 1 month ago on January 16, 2015