June 5, 2017
The Codemanship TDD "Driving Test" - Initial UpdateA question that gets asked increasingly frequently by folk who've been on a Codemanship TDD workshop is "Do we get a certificate?"
Now, I'm not a great believer in certification, especially when the certificates are essentially just for turning up. For example, a certificate that says you're an "agile developer", based on sitting an exam at the end of a 2-3 day training course, really doesn't say anything meaningful about your actual abilities.
Having said all that, I have pioneered programs in the past that did seem to be a decent indicator of TDD skills and habits. First of all, to know if a juggler can juggle, we've got to see them juggle.
A TDD exam is meaningless in most respects, except perhaps to show that someone understands why they're doing what they're doing. Someone may be in the habit of writing tests that only ask one question, but I see developers doing things all the time that they "read in a book" or "saw their team doing" and all they're really doing is parroting it.
Conversely, someone may understand that tests should ideally have only one reason to fail so that when they do fail, it's much easier to pinpoint the cause of the problem, but never put that into practice. I also see a lot of developers who can talk the talk but don't walk the walk.
So, the top item on my TDD certification wish-list would be that it has to demonstrate both practical ability and insight.
In this respect, the best analogy I can think of is a driving test; learner drivers have to demonstrate a practical grasp of the mechanics of safe driving as well as a theoretical grasp of motoring and the highway code. In a TDD "driving test", people would need to succeed at both a practical and a theoretical component.
The practical element would need to be challenging enough - but not too challenging - to get a real feel for whether they're good enough at TDD to scale it to non-trivial problems. FizzBuzz just won't vut it, in my experience. (Although you can weed out theose who obviously can't even do the basics in a few minutes.)
The Team Dojo I created for the Software Craftsmanship conference seems like a viable candidate. Except it would be tackled by you alone (which you may actually find easier!) In the original dojo, developers had to tackle requirements for a fictional social network for programmers. There were a handful of user stories, accompanied by some acceptance tests that the solution had to pass to score points.
In a TDD driving test, I might ask developers to tackle a similar scale of problem (roughly 4-8 hours for an individual to complete). There would be some automated acceptance tests that your solution would need to pass before you can complete the driving test.
Once you've committed your finished solution, a much more exhaustive suite of tests would then be run against it (you'd be asked to implement a specific API to enable this). I'm currently pondering and consulting on how many bugs I might allow. My instinct is to say that if any of these tests fail, you've failed your TDD driving test. A solution of maybe 1,000 lines of code should have no bugs in it if the goal is to achieve a defect density of < 0.1/KLOC. I am, of course, from the "code should be of high integrity" school of development. We'll see how that pans out after I trial the driving test.
So, we have two bars that your solution would have to clear so far: acceptance tests, and exhaustive testing.
Provided you successfully jump those hurdles, your code would then be inspected or analysed for key aspects of maintainability: readability, simplicity, and lack of duplication. (The other 3 goals of Simple Design, basically.)
As an indicator, I'd also measure your code coverage (probably using mutation testing). If you really did TDD it rigorously, I'd expect the level of test assurance to be very high. Again, a trial will help set a realistic quality bar for this, but I'm guessing it will be about 90%, depending on which mutation testing I use and which mutations are switched on/off.
Finally, I'd be interested in the "testability" of your design. That's usually a euphamism for whether or not dependencies betwreen your modules are easily swappable (by dependency injection). The problem would also be designed to require the use of some test doubles, and I'd check that they were used appropriately.
So, you'd have to pass the acceptance tests to complete the test. Then your solution would be exhaustively tested to see if any bugs slipped through. If no bugs are found, the code will be inspected for basic cleanliness. I may also check the execution time of the tests and set an upper limit for that.
First and foremost, TDD is about getting shit done - and getting it done right. Any certification that doesn't test this is not worth the paper it's printed on.
And last, but not least, someone - initially me, probably - will pair with you remotely for half an hour at some random time during the test to:
1. Confirm that it really is you who's doing it, and...
2. See if you apply good TDD habits, of which you'd have been given a list well in advance to help you practice. If you've been on a Codemanship TDD course, or seen lists of "good TDD habits" in conference talks and blog posts (most of which originated from Codemanship, BTW), then you'll already know what many of these habits are
During that half hour of pairing, your insights into TDD will also be randomly tested. Do you understand why you're running the test to see it fail first? Do you know the difference between a mock and stub and a dummy?
Naturally, people will complain that "this isn't how we do TDD", and that's fair comment. But you could argue the same thing in a real driving test: "that's not how I'm gonna drive."
The Codemanship TDD driving test would be aimed at people who've been on a Codemanship TDD workshop in the last 8 years and have learned to do TDD the Codemanship way. It would demonstrate not only that you attended the workshop, but that you understood it, and then went away and practiced until you could apply the ideas on something resembling a real-world problem.
Based on experience, I'd expect developers to need 4-6 months of regular practice at TDD after a training workshop before they'd be ready to take the driving test.
Still much thinking and work to be done. Will keep you posted.
Posted 16 hours, 35 minutes ago on June 5, 2017