March 24, 2016

...Learn TDD with Codemanship

The NPM Debacle: Programmers Don't Know Their Own Strength

This NPM debacle has reminded me of a couple of issues in software development that I think maybe we all need regularly reminding of.

Most importantly, they collectively broke the First Law of Software Development - thou shalt not break shit that was working.

The programmer in question broke it by withdrawing all his packages. The guardians of NPM broke it by making it possible to actually do that at all.

It's shocking, of course, that a single developer can so easily shit on the duvets of literally millions of code bases. And we can point and gasp and say "well I never!" and roll our eyes. But, hey, it's not like almost every open source software development team does this on a regular basis, is it?

It turns out that breaking code that was working, by changing APIs and removing stuff folk were using and arbitrarily renaming things and so on, is pretty common on many open source libraries and frameworks, as well as web APIs, that gazillions of lines of code have been built on.

One shudders to think just how much time and money we're wasting every year fixing stuff that was working just because of this. My guess would be in the many billions of dollars.

Here's the thing, when we write reusable code, presumably in the hope that people will reuse it, we take on a responsibility. The more people reuse our code, the greater that responsibility. Whether you realise it or not, if you write a function that left-pads a string and everybody uses it, then you have the weight of billions of dollars invested in code on your shoulders. It beggars belief that a temper tantrum can end with an entire development community coming into work the next day to find their software mysteriously broken. How many hundreds of thousands of hours did it take them to fix that collectively? How many millions, or tens of millions of dollars was wasted because of that temper tantrum?

This is something as a society we're going to have to get used to. The incalculable impact of a single person over just a few lines of code has no direct comparison in any other industry. No other kind of professional can wreak that kind of havoc and cause that amount of damage with so little time and effort. 50 years ago, it would be extremely difficult for, say a disgruntled car factory worker to break every single car they've sold. Now, a pissed-off software developer could potentially do it with a single Git commit. Look at how many major car recalls recently have been software-related.

We don't know our own strength. When society remodels itself on software foundations, and someone's little string formatting function ends up in a billion devices, our technological civilisation becomes a house of cards.

So, in summary on that issue: we need to get our shit together before legislators get it together for us. It's only a matter of time before a VIP non-programmer notices just how much these things can cost and how much damage they can do.

The second issue the NPM-string-padding hoohah raises is should programmers create big dependencies on 3rd party libraries just to left-pad strings? This is like buying a Mercedes because you need the cigarette lighter.

I recall meeting a team who had a need to sort of a list of strings in a programming language that didn't have sorting built in to the core libraries, and decided the best solution was to do it using SQL Server.

I see developers being a little too eager to add more and more external dependencies to their projects, often to solve little problems like sorting a list or padding a string. As we're seeing - on a galactic scale - dependencies can hurt us. There's a balance to be struck here between not unnecessarily reinventing the wheel when a solution exists, and strapping our code to some giant - poorly managed - framework or library when it would actually be simpler to roll our own.

Open source dependencies, in particular, tend to get added with less thought and oversight - probably because they're free and developers don't need to go through a procurement process to get them. We can add this to the growing list of things that developers can do with little thought, that can have a potentially huge impact. I see developers occasionally making $multi-million decisions, blissfully unaware that they're making $multi-million decisions.

A bit more thought is required, please.

Posted 3 weeks, 3 days ago on March 24, 2016