February 19, 2006

...Learn TDD with Codemanship

Sleep On It

Yet more scientific evidence that our unconscious minds are better at dealing with complexity than our conscious minds, this article in Nature describes an experiment in which people given complex decisions to make, who were then distracted, made better decisions than people given the time to think things through.

This ties in with work on quantum consciousness that proposes a non-computational element in our thought processes. Creativity, inspiration, genius, intuition - call it what you may, but the picture being painted in various strands of related research suggests there's something distinctly not 20 GOTO 10 about human intelligence (and, quite possibly, animal intelligence, too).

Since most decisions we have to make are complex (do I like this music or not? am I a Winter person? etc), we surely must have found a solution to non-computable problems, or else how did we get this far?

Interestingly, the complexity of decisions is also a major factor in the suggestability of the decision maker. The less obvious the solution, the more swayed people are by what other people think or do. Since choosing what hat to wear this spring is a complex decision, I may end up wearing whatever hat the bloke on the cover of Maxim is wearing. He looks like the kind of guy who knows his hats, after all...

Whether it's as unconscious quantum computers or just plain fashion victims, our decisions are still effectively unpredictable. My latest theory is that when something is effectively unpredictable, you treat it as a random phenomenon. It may have boundaries and attractors (strange or otherwise), but in any specific instance that's the best we can know in advance.

What this research suggests is that we should trust our unconscious instincts more when we're dealing with complex problems. It also suggests that screwing your eyes up tight and thinking about it really hard probably won't help - it might even make things worse.

I've always been a great believer that if you're stuck on a problem, your best bet is to study it for a bit - take in the information, swish it around in your head for a while - and then walk away and do something completely different. With a small pinch of luck, a solution tends to present itself at some point. You can't know in advance exactly when this solution will appear, and therefore it's absolutely imperative to acknowledge the uncertainty in any process of problem solving or innovation.

This is a good explanation as to why software projects (and many other creative endevours) tend to behave like dice-throwing games. It's because the appearance of solutions is effectively random, like throwing a dice. You might be able to throw a 6 in a single roll of the dice, but it could take 20 rolls of the dice. Theoretically, it could take an infinite number of rolls of the dice - but that's infinitely unlikely.

I'm not sure why this model of software development hasn't been proposed before. It makes perfect sense and appears to match the statistics we get from real projects. It could be a coincidence, but the more I explore, the less coincidental it seems to be. It certainly dovetails very nicely with other ideas that are emerging in disciplines like consciousness studies and complexity science.

Of course, our typical project environment is geared up for the keep-staring-at-the-screen-until-you-solve-the-problem school of innovation. To be sure, there are times when we need to be 100% focused on implementing a solution or studying a problem. But there are also times when we need to be distracted. I find I work best at home because when I need to focus I can take the phone off the hook and just get on with it without any interruptions. When I'm stuck I can pick up the guitar that's always plugged in right next to me, or watch some TV, or listen to a CD, or go for a walk in the local park. When I'm on a client's site I feel a little self-conscious walking away from my desk for too long. I know I shouldn't. I should be committed to delivering. Delivering isn't something you necessarily do for 8 uninterrupted hours between 9am and 6pm. Delivery in our job isn't like delivering a parcel, or delivering a baby. Delivery means solving problems. And solving problems requires some innovation. Innovation tends not to happen when we're trying to force it, quite possibly for exactly the same reasons that most people can't think of anything funny to say whenever someone tells them to "say something funny".
Posted 14 years, 11 months ago on February 19, 2006