October 24, 2006

...Learn TDD with Codemanship

Exploratory Testing - Not Just A Reason To Have Testers

I've had an email from Jonathan Kohl about my post on Exploratory Testing in which I (jokingly) suggest that it creates work for testers who might otherwise have not much to do on Agile teams. He makes some interesting points, and I thought I should post it here for a bit of balance. He writes:

I saw your post on Exploratory Testing. I'm part of a community of ET practitioners and trainers. Cem Kaner (http://www.kaner.com) defined the term in 1983 as "simultaneous test design, execution, and learning". All testers do some sort of ET at least some of the time, to do it well requires interaction and careful thought. I got into the ET community because I was interested in how valued testers did their best work. My own experience led me to the work of Cem Kaner and James Bach.

I've worked on several Agile projects over the past several years, and I truly wanted to see if they didn't need dedicated testers or not.
What I found was that traditional projects had a better handle on testing in some areas, and Agile projects had a better handle on testing in others. In general, neither traditionalists or Agilists tend to see testing in as broad a sense as I and others like Cem Kaner and James Bach do. With so-called "Agile Testing" I found a lot of gaps, particularly in developer and customer testing. There are still problems, just different ones.

Along with Elisabeth Hendrickson (http://www.qualitytree.com) I was a pioneer in introducing ET on Agile teams, but Agilists seem to want to try to automate away the testing problem. In practice, I haven't seen a lot of adoption. In some cases, teams report doing some ET as an end of iteration ritual, but I haven't seen this in the wild that much.

As you might imagine, I take issue that ET is a testing practice that testers are using to justify their existence on Agile teams. :) Some might, but I have found the threat that the testing role will go away to be a myth in most cases. Much like the myth of Object Oriented development solving all ills -- almost ten years ago I was told my role would go away because of the development team was using Java. You can imagine what a laughable farce that was.

Five years ago I was told the same thing because of Agile methods. I took it to heart, and worked hard to see if it was true or not.
Experience has shown me I have found that good software testers add value to any team they are a part of. Poor software testers are easily replaced. I did an interview on this topic a while ago:
http://www.qthreads.com/interviews/testing/agile_teams_miss_out_by_having_a_narrow_focus_on_testing.html

In fact, I have seen the opposite - some Agile teams are coming to me for help because they are having testing issues, and realize there are gaps. Others are dealing with gaps jumping back to the "QA" charlatans who were doing bad testing for years, and were a reason why some early XP projects cut them out. Now they repackage their awful record/playback tools for "automated acceptance testing", and do the useless pre-scripting of test cases that narrows testing focus and call it "story-test driven development".

In short, no one has the testing issue completely sorted out. Some teams can get by just fine without testers, others need them. Nothing new here.

Like you, I have a strong bias towards iterative lifecycles, evolutionary design and development, high levels of communication and collaboration, and strong customer involvement. Exploratory testing fits this kind of model of development very well.


Posted 14 years, 2 months ago on October 24, 2006