October 14, 2013

...Learn TDD with Codemanship

101 Uses For Polymorphic Testing (Okay... Three)

Quick thoughts about some useful applications of polymorphic testing that I've tried with some success over the years.

Polymorphic testing is very simply done by writing tests such that the object under test can be dynamically substituted (e.g., by overriding a factory or builder method that creates it), making it possible to - for example - test that a SettlementAccount obeys the rules of its Account base class. (see Abstract Test pattern)

Unit testing frameworks like JUnit and NUnit will allow us to do this by extending test fixtures. They detect for running all the tests in the base fixture as well as new tests added to the subclass fixture. Simple and neat (and very possibly unintended.)

But what would you use this approach for in the real world?

Here are 3 examples:

1. Testing that classes satisfy supertype contracts

If you've worked on software that allows 3rd party developers to write their own components or extensions (e.g., device drivers for an OS), you will likely have come up against instances where developers implement your required interfaces but not necessarily obeying the rules that implementing those interfaces demands. Whatever a device driver does, it must obey the rules of being a device driver. If it doesn't, the result can be instability in the OS.
Having worked on products like these, I've seen many instances when writing a suite of tests that any classes implementing our interfaces must pass has paid big dividends later.

2. Testing for backwards compatibility

It's the First Law of Software Development - Thou shalt not break shit that was working.

This includes other people's shit.

When we release software to be used by other software, we're publishing an API against which 3rd party developers will invest time and effort. Every time we change our API, we potentially cause the need for them to spend even more time and effort adapting their code to the change.

This makes backwards compatibility not so much a "nice to have" as a commercial imperative for any software company that wants other developers to build solutions using their products.

I don't know about you, but when something's commercially imperative, I like to take steps to ensure it's happening.

Polymorphic testing, with some extra jiggery pokery, can be used to test that new versions of our public interfaces pass the tests we wrote against older versions.

We're talking specifically here about tests that bind only to our API. Change the internal design as much as you like. It's therefore useful to keep API-level tests separate from ordinary unit tests.

This can get a little technology-specific in practice. Running NUnit tests compiled against older versions of .NET assemblies is not entirely straightforward, for example. You have to go round the houses a bit to get around .NET's component versioning system. It's much more straghtforward in dynamic scripted languages like Ruby, where interfaces are discovered at runtime. But in most OO languages in use today, it's do-able, and economically worth doing in these situations.

Of course, if we've been observing the Open-Closed Principle like good little boys and girls, it's much easier.

3. Dynamically controlling the scope of test runs

This last technique I've found especially powerful. The ability to run the same functional test, say, with a real database and with data access mocked out can be very useful.

By overriding the methods that create objects under test, we can wire its collaborators together in all manner of permutations, and our test code is none the wiser.

This is especially helpful for teams who have hundreds of slow-running end-to-end system or acceptance tests. These tests are often where the logic of their code is being verified, having relied entirely on testing at that higher level, or on mostly interaction testing at the unit level. Although lack of unit tests will hurt you in other ways, at least with polymorphic testing it's possible to run those system-level tests entirely in-memory so they run fast.

So you might have a suite of acceptance tests implemented as JUnit fixtures that connect to a database etc, and then a suite that extends those fixtures to substitute objects that are wired to a mock persistence layer.






Posted 4 years, 6 months ago on October 14, 2013