January 11, 2008

...Learn TDD with Codemanship

Response To My Post About Crappy Code in Crap4J

Bob Evans of Agitar writes in response to my post about finding crappy methods in Crap4J:

Indeed, crap4j does not check a method's line, statement, or expression count. We have written quite a bit about what we are after. I do happen to agree that long methods can be a problem, but I argue that complex, untested methods (long or short) are worse. We are looking for the lowest bar that the code might have failed in the metric. The method absolutely has to be bad.

This metric is a first approximation for detecting crappy projects and we want the method to be undoubtedly crappy when the tool flags it as crappy. When the CRAP score is low, then there are many other facets of projects that need to be investigated on the way to assessing its quality -- other metrics can be used at that point.
The problem now is that no metric is used consistently across the industry. We think that's a shame, so we are trying to build the simple-to-understand, simple-to-act-on metric that will get developers thinking about code quality quantitatively. It's like a training wheel metric in a sense. Naturally, anyone who is already using metrics in a mature fashion would not see Crap4j as the end-all be-all of metrics.

I think it's worth highlighting the fact that Bob works at a company that sells a tool designed to help teams improve their test coverage, so I can't help wondering if there isn't a little bias here towards metrics that highlight lack of coverage. Which is fine, because Bob's absolutely correct in saying that complex methods with low test assurance are pretty much universally crappy. It is a problem, and you should address it.

But so are classes that are heavily depended upon but aren't abstract enough, and variable names that don't make any real sense. I could list a whole heap of things that I would suggest make code "crappy", and you'd have a tough time arguing against any of them.

Getting developers started with quantitative code analysis - metrics - is one of the things I do for a living, and I've always found the opposite. You need to try to establish balance right from the start. Focusing in on just one area of quality, no matter how valuable by itself, can lead to people ignoring other important areas.

I routinely recommend comparing code complexity with unit test coverage, and I think it's pretty common to measure things like path coverage against cyclomatic complexity - well, it is where I've been hanging my hat, anyway - so I don't question the metric at all. I just question relying on it by itself, even as a "training wheel" for developers new to metrics.
Posted 12 years, 11 months ago on January 11, 2008