December 10, 2018
The Gaps Between The ToolsOn a training course I ran for a client last week, I issued my usual warning about immediately upgrading to the latest versions of dev frameworks. Wait for the ecosystem to catch up, I always say.
This, naturally, drew some skepticism from developers who like to keep bang up-to-date with their toolsets. On this occasion, I was vindicated the very next day when one participant realised they couldn't get Cucumber to work with JUnit 5.
I abandoned an attempt to be bang up-to-date using JUnit 5 for a training workshop in Sweden the previous month. The workshop was all about "Third-Generation Testing", which required integration between unit testing and a number of other tools for generating test cases, tracking critical path code coverage, and parallelising test execution. I couldn't get any of them to work with the new JUnit Jupiter model.
So I reverted back to safe old JUnit 4. And it all worked just spiffy.
No doubt, at some point, the tools ecosystem will catch up with JUnit 5. But we're not there today. So I'm sticking with JUnit 4. And NUnit 2.x. And .NET Framework 3.5. And the list goes on and on. Basically, take the latest version, subtract 1 from the major release number, and that's where you'll find me.
For sure, the newer versions have newer features, which may or may not prove useful to me. But I'm more concerned about the overall development workflow, and that's why compatibility and interoperability mean more to me than new features.
We're notoriously bad at building dev tools and frameworks that "play nice" with each other. Couple that with a gung ho attitude to backwards compatibility, and you end up with a very heterogenous landscape of tools that can barely co-exist within much wider development practices and processes. It's our very own Tower of Babel.
In other technical design disciplines, like electronic engineering, tool developers worked hard to make sure things work together. Simulation tools plug seamlessly into ECAD tools, which talk effortlessly with manufacturing tools and even accounting solutions to provide a relatively frictionless workflow from initial concept to finished product. The latest release of your ASIC design tool may have some spiffy new features, but if it won't work with that expensive simulator you invested in, then upgrading will just have to wait.
Given that many of us are engaged professionally in integrating software to provide our customers with end-to-end processes, it's surprising that we ourselves invest so little in getting our own house in order.
Looking at the average software build pipeline, it tends to be a Heath Robinson affair of clunky adaptors, fudges and workarounds to compensate for the fact that - for example - every test automation tool produces a different output for what is essentially the exact same information. And it boggles the mind why we need 1,001 different adaptors to run the tests in the first place; every combination of build tool + test framework imaginable.
If test automation tools all supported the same basic command interface, and produced their outputs in the same standard formats, we could focus on the task in hand instead of wasting time reinventing the same plumbing over and over again. JUnit 5 would already work with Cucumber. No need to wait for someone to patch them back together again.
And if you're a tool or framework developer protesting "But how will the tools evolve if nobody upgrades?", my advice is to stop breaking my workflows when you release new versions. They're more important than your point solution.
I vote we start focusing more on the gaps between the tools.
Posted 1 month ago on December 10, 2018