July 28, 2017

...Learn TDD with Codemanship

How To Backwards Compatibility (And Why For)

One of my biggest bugbears as a software developer is how casually many developers of tools, libraries, services and frameworks make changes that aren't backwards compatible.

It breaks my first rule of software development:

Thou shalt not break shit that was working

I acknowledge there are times - rare occasions - when it's unavoidable. But we seem to do it with careless abandon, creating untold hours of needless work that adds no value for client code developers.

I know from experience that backwards compatibility is not that hard to get right. You just have to choose to do it.

Here are 5 ideas that I've had success with that you might like to incorporate into your development practices.

1. Run The Old Tests Against The New API

This is a no brainer. Some of your automated tests will work at the API level. Keep those in their own folder or project, so you can run them by themselves. In your builds, as well as checking the code passes the new API tests, check they also pass the old API tests. It can be as simple as merging the API tests from the previous release with the latest code.

If the tests won't compile (e.g., because an API method has an added parameter), you've broken syntactic compatibility. FAIL. Do not pass GO. Do not collect £200.

If the tests compile, but when you run them some of them fail, you've broken semantic compatibility. FAIL. Go directly to jail.

2. Apply The Rules of Extending Classes in Design by Contract

Bertrand Meyer's Design by Contract has strict rules about how we can extend classes so that we don't break the original contracts with client code.

Pre-conditions to calling methods can be weakened. That is, that method will work in the same or a wider set of circumstances. Otherwise, clients may find themselves invoking the method in situations that are now forbidden.

For example, in a video library service, let's say that originally any member could borrow any video. If we add ratings to the system and require that members must be old enough to borrow a video, potentially a whole bunch of client code breaks when it tries to borrow a copy of The Exorcist for a 9-year-old member.

Post-conditions to using methods can be strengthened. That is, the client will get the same or more than they did before.

In our video library example, we could include a free pizza when members borrow 5 or more videos. They get the videos AND free pizza. But if we changed it so that we can substitute titles (let's say they thought they were borrowing all of the Harry Potter titles for a marathon fan weekend, and then we send them 8 Star Trek movies instead), we've broken the original contract - and ruined their Potter Party.

Whenever you change an API, ask yourself "are we breaking the original contract?"

3. Overload API Methods Instead of Changing The Originals

If we want to change the way borrowing videos works so that members can specify they want it for an extended rental (e.g., if usual rental period is 48 hours, but they want to take it on a holiday for a week), we could just add a new parameter for that to the original API method. But that will break all the clients.

Instead, overload the method for borrowing a video so that new clients - who are aware of this new feature - can take advantage, but existing clients will still work with the default rental period.

4. Minimise The Pain

On those occasions when we really can't see another way, we can remove a lot of the pain and extra work for client code developers.

First of all, give them plenty of warning. And don't make the change in a single step. Put your replacement API method in place first, and deprecate the one you're getting rid of several versions ahead of actually removing it. Steer people writing new client code away from the old method, but keep old client code working for as long as possible.

And then, when it's time for the old API method to go, seriously consider giving client code developers an automatic way of migrating their code. 9 times out of 10, a regular expression will do the job. Match the signature of the old API method and replace it with the new one, including default values for new parameters.

5. Offer Real Value in Migrating. Avoid Change For The Sake of Change

I could probably live with API updates that break my client code if there was a good enough pay-off for the extra work. What really annoys me is when there's no real pay-off. The API's developers, of course, will list a whole bunch of "benefits" of the new version. But they've usually confused benefits with features.

We really do have to get over this culture of "but there's gotta be a new version with new features!"

No there doesn't. Consider carefully whether the benefits of the changes you're making really will outweigh the benefits of leaving client code working.

ADDENDUM: Occurred to me this afternoon, after hearing from devs who have chosen not to upgrade their unit testing framework because of lack of backwards compatibility (that means they'd lose the use of a range of tools relying on the previous API)... It's also a very risky product strategy to put out a non-backwards-compatible release.

If you raise the prospect of a migration path and extra work for client developers, this opens the door to competing products. Better, I think, to just develop an all-new product with a clean slate.

Posted 3 years, 9 months ago on July 28, 2017