April 2, 2008
RE: The Future of TDDIn response to my last post pontificating on a future where not only do our IDE's perform background compilation and real-time syntax checking, but they also run our unit tests - or some other kind of behavioural checks - in the background too, and inform us in real-time if the code we're writing has broken anything, the very helpful Peter Zsoldos has sent me a link to a tool for Eclipse that does pretty much exactly that.
My own thoughts about this were that the computing power needed to run a suite of unit tests fast enough to give real-time - or as near as damn it - feedback is probably a good decade away. The continuous testing Eclipse gets around that by not running all of the tests. It sounds a little like voodoo to me, but that might just be my limited grasp of English getting in the way. The authors of the tool explain:
The test suite in use by a developer may contain a test that covers a large fraction of the program code, or takes a long time to run. It may prove difficult for a continuous testing tool to provide early feedback when that test regresses, that is, when a change to the program being tested causes the test to stop passing and begin failing. To improve this situation, we are proposing test factoring. Test factoring, given a large test, produces one or more smaller tests. Each of these smaller tests is unlikely to fail unless the large test fails, and likely to regress when the large test regresses due to a particular kind of program change.
Interestingly, in another response to my post, Danny Faught blogs:
It's conceivable that a large system could have five minutes worth of well-written unit tests. Jason hinted that he's thinking of running all of the systemís unit tests when he says that we could try to take a shortcut and only run the unit tests that are dependent on the code that changed. But that doesn't sound like unit testing to me, it sounds more like integration testing. I do have suites of subsystem tests and system tests that take more than a minute to run, but I don't call them unit tests. So we're probably using different definitions of what a unit test is. I know that many people use the term "unit test" very loosely, which tends to mask the fact that people often aren't really getting the benefits of good unit testing.
I think Danny may have his wires crossed, though. To get reasonable assurance against regressions, I would certainly want to retest dependent code, and not just run the tests for the class or method I'm working on. I also don't see that "unit test" means that it applies to only one method or one class. A "unit" could be a component. Or a collaboration or cluster of classes inside a component. But that's just my opinion. Regardless, regression test assurance would still be required on dependent code.
It's very encouraging to see that a prototype already exists, though. Hopefully someone out there's working on this problem right now. It would be a terrific shame if we dropped this particular ball, because we already have the techniques we need. Now we just need to wait for computing power to catch up.
Posted 13 years, 2 months ago on April 2, 2008