February 10, 2011
Software Craftsmanship - We Need To Raise Our GameDoing business on a macro scale has only really become commercially viable since the Second World War. And we all know what's been the driving force behind many large-scale post-war business models - computers.
Many existing business models, such as retail and banking, became supercharged by the ability to process millions of transactions, track millions of items of stock and manage the accounts of millions of customers electronically. New models emerged that were made possible by computing - for example, supermarkets.
Meanwhile, computing in the science lab and engineering design office transformed our ability to figure stuff out quickly, and the pace of scientific discovery and technological innovation has accelerated as number-crunching power grows exponentially.
Computers, and the software they run, have been the defining factor - the shaping force - of the last 70 years. They have transformed every aspect of our lives - commerce, science, medicine, communication, education, government, construction, manufacturing, fashion, media, the arts, sport, architecture, transport, war. You name it, computers have changed it.
Much of the UK and world economy relies on software to function now. Your local supermarket would have to shut if its software systems went down for even half an hour. The food on the shelves would not arrive the next morning if the software that runs logistics failed. Electrical power would cease to flow from the sockets in your homes and offices if the grid systems failed. Your car would not start - or your brakes might not work - if the millions of lines of code in it failed. You would not be able to call the emergency services when the phone system crashed. Not that it would do you much good even if you could, because they would be unable to respond without their IT systems to co-ordinate them.
Your case would collapse if the law firm representing you had to go back to manual typewriters and working entirely on paper. There'd be nothing on TV if the digital broadcast systems failed. The final Harry Potter film would take 10 years to make and cost a billion dollars if they had to do all the effects using practical, optical and chemical techniques.
As a technological society, our reliance on software is absolute. If a solar flare wiped all our drives tonight, society would collapse tomorrow. Completely and utterly.
Of course, that's very unlikely. There are back-ups and disaster recovery plans and most key organisations that rely on computers will be back up and running fairly quickly. The damage would probably only run into a few billion pounds for those few hours with no systems working.
But there is something in our software that's already hurting society, and it's something we really need to wake up to as we become even more reliant on software.
I've personally witnessed companies of appreciable size (> £100 million annual turnover and > 1,000 employees and 1 million customers relying on them) being brought to their knees by their inability to roll out new or improved software systems.
Increasingly, new products and services cannot happen without new software to support them. And that's the case in most large businesses these days. A delay of a month on deploying new features and changes delays the rollout of the business innovations that require them by a month.
Put more simply, and we can call this Gorman's Law Of Software-Dependent Business Evolution if you like:
Software-dependent businesses can only evolve as fast as their ability to write and evolve their software allows them to
Since the bulk of UK GDP comes diretcly or indirectly (via smaller suppliers) from large businesses (turnover > £100 million), and since all large UK businesses are heavily reliant on their software, it stands to reason that the ability of the software teams in these businesses to move the systems forward will have a considerable impact on GDP as a whole.
If a supermarket or high street bank could reduce the average cycle time from identifying a new system requirement to having software deployed that satisfies that requirement by, say, 10%, I strongly suspect it would have a positive effect on the bottom lines of said businesses and would be visible in the performance of the economy as a whole.
And this is where we come in. Over the decades, we've learned about the factors that affect our responsiveness to new and changing requirements. We have built a fairly robust understanding of factors like maintainability of code, number of defects slipping into a release, the time and effort required to build, test and deploy systems and so on, and their relationship to cycle time.
A decade of "Agile" has led us to an understanding of what kind of practices and good habits might bring us to this Nirvana of continuous, sustained delivery and the ability to respond more immediately to the changing business landscape over the lfietime of a system.
I suspect that if even just 20% of software teams in the UK got to grips with and applied these practices effectively, our economy could be transformed by the competitive advantage it would bring.
Ask a business person what difference it could make to them if, when they ask IT to get the system to do something new, the answer was not "it'll be in the next release in 6 months time" but "it'll be in next week's release".
That was what Agile was always about. Being more responsive, and giving businesses a greater capacity to evolve. In that respect, we have failed because we didn't apply the technical discplines that continuous, sustained delivery requires. Most developers and teams don't know such practices and couldn't apply them even if they wanted to. Most IT organisations are blissfully unaware that this is even a problem and have no plans to address it. Nor does our government, or our education system.
Whether you choose to make it a lifestyle, or form a guild, or wear a cape and a big pointy hat, there's no getting away from the need for craftsmanship in software.
And if you care about your craft, or about the long-term health of your economy and your society, then there's no getting away from the need to do something about it. It's no longer enough to focus on yourself and what you do, because no matter how much care and pride you take in your work, you are in a very tiny minority today. There's nowhere near enough of us to even make a dent in the problems we collectively face.
According to a statistic I gleaned on Twitter (via @dr_black), out of 28,000 new teachers who qualified in 2008 in the UK, only 3 (T.H.R.E.E.!) had computing-related degrees. Intake into CS and SE courses is down, and in general, computing education is in decline. (And not just here in the UK, I should add.)
Computing departments are closing down because of lack of funding caused by lack of interest. This is entirely at odds with the exponential rise of computing within society, which shows no signs of abating any decade soon.
Lack of interest inevitably leads to a lowering of the bar to fill places on degree courses and jobs in industry. The downward pressure on skills and quality can be felt in the workplace. In order to get pressing projects started, employers widen the recruitment goalposts. Better to limp to the finish line with a so-so team than to disqualify ourselves, surely?
Except, it isn't the finish line. As soon as our cobbled-together team of kind-of-sort-of programmers gets one delivery out the door, the change requests (and bug reports) start flooding in demanding another release, and soon. The race has only just begun, and it's not a short sprint, it's a marathon. And our code's in no fit state to continue much further.
The apparant victory of beating that first deadline with a motley crew of low-skilled developers is Fool's Gold. We've won nothing for the long term. We've just delayed inevitable failure.
This current climate creates serious challenges for software development, and for craftsmanship (or whetever you want to call "taking care over the quality of your code") in particular. At a time when we need standards to rise, most of the forces acting on our profession are pushing them down.
What we need is competition to get into and stay in software development. Even if it's just in a reasonably-sized bubble of excellent programmers working for enlightened employers who demand and appreciate that excellence.
Right now, the main barrier to entry is not talent or ability, but inclination. If you want to be a professional computer programmer, you can be. And so few want to be.
That is at the root of many of our problems. And tackling it will probably require a two-pronged approach:
1. More kids need to choose software development as a career. This opens a whole can of worms, including issues surrounding the failure of computing education to address software development in a meaningful, practical way.
2. More practicioners working today need to want to do it better
In both cases, little will be achieved by us continuing to preach to the choir. Telling a bunch of people at a software development conference that they need to get better at software development is unlikely to be winning many converts. For the most part, anyone who attends a software development conference probably already wants to get better at software development. (Same goes for blog audiences, Twitter followings and so on.)
Like the Judges in 2000AD, perhaps the time has come for us to take the Book of the Law (whichever one you choose to follow) out into the Cursed Earth and look for new converts among the true unbelievers.
In particular, it is our business leaders, government and educators who need winning over, chiefly by educating them on what software development really is and what role it really plays in people's lives. The Digital Britain Report and subsequent Digital Economy Bill paint a picture of our leaders being hopelessly unaware of the fact that Britain's economy and infrastructure has been "digital" since the 1950s. Tech journalists, too, fail to grasp this, choosing instead to focus on the tip of the computing iceberg.
Digital TV, music, films, computer games, social media and e-government represents a tiny, tiny slice of the software driving out economy forward. We can almost discount them for their relative unimportance. and yet, for the government, their advisors and the tech media, they seem to be the dominant focus. And no mention at all of the crisis in software development and computing education looming.
To many MPs and teachers, using an Excel spreadsheet or knocking up an HTML page is "computer programming". The educational focus is very much on using software, rather than making software. But we still call it "computing", which is as misguided as suggesting someone who can take an aspirin understands "medicine".
Evidently, the crisis is just not on anybody's radar - at least, not on the radar of anybody whose opinion matters.
These people are ultimately holding all the cards. Businesses can demand higher standards from software developers, and withold payment when they're not met. Governments can allocate more resources to building a better software development capability and help build awareness among the general population about what it all means to them, and educators can begin to address the chronic shortfall in bright, enthusiastic kids entering the field.
Ultimately, we're wasting our time trying to win over the Keith Bank-Account's and John Q. Clockwatcher's in our profession. While the money continues to roll in, regardless of the quality of the work they do, it'll be business as usual for them.
Let's focus on the developers who want to change, and on the people who can influence our work from the outside and slowly but surely make it harder to make a living writing crap code.
What's need, and needed soon, is a concerted, focus and highly organised campaign among those of us who care to bring these issues to the forefront in our society, and to seek out and offer practical and effective support to those who choose to join us in our efforts.
We're going to have to raise our game. No politician, entrepreneur or teacher is going to read our blog posts or follow our Tweets. The effort and smarts required here are going to be an order of magnitude bigger than any software movement we've seen so far.
As 2012 is the centenery of Alan Turing's birth, it seems appropriate to target that year for a co-ordinated campaign of adverts, events, politicial lobbying and other activities designed to bring us nearer the day when code quality is as much in the layperson's mind as the reliability of the brakes in their car is today. (Those two things are not unrelated, of course!)
I'm going to kick things off by organising a panel discussion to coincide roughly with Software Craftsmanship 2011, either here in London or at the conference itself in the evening.
Audience participation will be very much encouraged, with folk putting questions to the panel and giving their views.
Keep your eyes peeled for details.
Posted 2 weeks, 2 days ago on February 10, 2011