November 30, 2006

...Learn TDD with Codemanship

Code Aging & The Grand Canyon Effect

I've been thinking some more about the way in which software ages. I quite often come across systems that are well past their prime. I think I probably recognise this by what I call the Grand Canyon effect.

When you visit the Grand Canyon, you see this truly, staggeringly enormous hole in the ground. The scale of it is eye-boggling. But when you look down you see this little trickle of water.

I jokingly say that the software product lifecycle goes something like this:

Day 1 - Hire 4 good programmers to write 80% of the code
Day 365 - Hire another 40 programmers to write the next 18% of the code
Day 3,650 - You've got another 400 programmers, and you're still waiting for that last 2%...

The inescapable reality of software development seems to be that young code grows rapidly - and relatively cheaply - but as the code ages, the pace of development slows and the costs shoot up until a $10 line of code can end up costing $1,000.

It doesn't really matter why this happens; suffice for us to know that it happens. Code is complex enough that we should think of it as essentially organic. And one thing we know about organisms is that they age. Early nights, plenty of carrot juice and a brisk walk every evening might slow down the onset of decrepitude, but will not stop it. There is not stopping entropy, it seems. The arrow of time has its own gravity and we cannot escape its pull. The odds are against us. (And that's more significant a statement than I originally thought.)

But some organisms age more rapidly than others. And there are usually factors that influence the aging process. The model of life as being something that "fights entropy" - that works to keep itself out of equilibrium from its surroundings - might be useful here. As the physicist Erwin Shrodinger puts it:

What is the characteristic feature of life? When is a piece of matter said to be alive? When it goes on "doing something," moving, exchanging material with its environment, and so forth, and that for a much longer period than we would expect an inanimate piece of matter to "keep going" under similar circumstances. When a system that is not alive is isolated or placed in a uniform environment, all motion usually comes to a standstill very soon as a result of various kinds of friction; differences of electric or chemical potential are equalized, substances which tend to form a chemical compound do so, temperature becomes uniform by heat conduction. After that the whole system fades away into a dead, inert lump of matter. A permanent state is reached, in which no observable events occur. The physicist calls this the state of thermodynamical equilibrium, or of "maximum entropy.

How does the living organism avoid decay? The obvious answer is: By eating, drinking, breathing, and (in the case of plants) assimilating. The technical term is metabolism. The Greek word means change or exchange. Exchange of what? Originally the underlying idea is, no doubt, exchange of material . . . That the exchange of material should be the essential thing is absurd. Any atom of nitrogen, oxygen, sulphur, etc., is as good as any other of its kind; what could be gained by exchanging them? For a while in the past our curiosity was silenced by being told that we feed upon energy . . . Needless to say, taken literally, this is just as absurd. For an adult organism the energy content is as stationary as the material content. Since, surely, any calorie is worth as much as any other calorie, one cannot see how a mere exchange could help.

What then is that precious something contained in our food which keeps us from death? That is easily answered . . . The device by which an organism maintains itself stationary at a fairly high level of orderliness (= fairly low level of entropy) really consists in continually sucking orderliness from its environment . . . In the case of higher animals we know the kind of orderliness they feed upon well enough, viz. the extremely well-ordered state of matter in more or less complicated organic compounds, which serve them as foodstuffs. After utilizing it they return it in a very much degraded form—not entirely degraded, however, for plants can still make use of it. (These, of course, have their most powerful supply of "negative entropy" in the sunlight.)


According to Schrodinger's model, some organisms must be better at maintaining their disequilibrium than others. The theory suggests that - far from closing systems to change - the better way to maintain orderliness might be to open them up to external orderliness.

If the flow of incoming orderliness slows down to a trickle, like the river that runs through the Grand Canyon, then we should expect system death to come sooner. The irony is, of course, that those early periods of high productivity on a software project are what get us into this sorry state in the first place. Just as everything we eat, drink and breath isn't necessarily good for us, not everything that goes into code is necessarily healthy either. The programming equivalent of saturated fats and free radicals can cause the sort of creeping nastiness that most developers live in fear of being asked to maintain.

As to what those factors really are, well - we can hazard some educated guesses. But I'd like to explore the underlying model some more, rather than instinctively proclaim that it's "waterfall development" or "nested If statements" that cause premature code aging.

I blogged a few months ago about visualising the evolution (and, one assumes, the aging) of software by digging back through successive versions of the code in a repository and constructing a sort of "time-lapse video" of it's entire lifecycle up to the present day. Code makes trees (parse trees, as it were), and so I suspect we may recognise the forms as somewhat organic right from the start. I also suspect we'll see something uncannily familiar in the dynamics of the code's lifecycle. Like fruit sprouting on the tree, ripening, falling and decaying, we may see some startling resemblances to natural organic processes.

That's just a guess, though. I could easily be completely and utterly wrong.
Posted 14 years, 1 month ago on November 30, 2006