DevelopsenseLogo

The Test Case Is Not The Test

A test case is not a test.

A recipe is not cooking. An itinerary is not a trip. A score is not a musical performance, and a file of PowerPoint slides is not a conference talk.

All of the former things are artifacts; explicit representations. The latter things are human performances.

When the former things are used without tacit knowledge and skill, the performance is unlikely to go well. And with tacit knowledge and skill, the artifacts are not central, and may not be necessary at all.

The test case is not the test. The test is what you think and what you do. The test case may have a role, but you, the tester, are at the centre of your testing.

Further reading: http://www.satisfice.com/blog/archives/1346

Subscribe to the Newsletter

12 replies to “The Test Case Is Not The Test”

  1. Test case is a noun, test is a verb.

    Test case is the means, test is the objective.

    A test case is a recipe used by the tester(chef) to test(cook).

    Test case is the script; tester is the actor; test is an act.

    A tester can test without a test script.

    The most important of all three is the tester, as the test is as good as the tester.

    Reply

  6. It’s a sacrilege to say something against them or be placed in a different light. Somehow this emphasis on test cases is encouraged by tools also, for example TFS. It seems that some pseudo-managers are won over by the simplicity of this management, meaning testing by just counting test cases.

    Reply

  8. Absolutely right! … and nobody wrote in test case: “Now you should galumphing for 1 hour to find some bugs”

    Michael replies: Yes. It would be a good idea to do that (although it might be helpful to be clear on what “galumphing” means; so here’s a reference).

    Reply

  9. In my view, a Test Case is the unit of measurement of the effectiveness of a Test. Segregating a Test into Test Cases can help us determine the Test Coverage. A Test tends to be generic whereas a Test Case is specific.

    Michael replies: People might think that way, but I can’t agree.

    With repsect to your first sentence: if a test case is a unit of measurement, what is the scale for the unit? Is a test of six steps and three evaluations to be counted the same way as a test of 24 steps and one evaluation? Before you answer, notice that “step” and “evaluation” suffer from the same scaling problem: is moving the mouse and clicking the button one step or two? Is selecting a check box a step? If it is, is typing a single keystroke in a field a step? Is typing 16 keystrokes into a credit card field a single step? Does that step include pressing Tab to go to the next field, or is pressing Tab a new step? What if the test case is composed simply of a set of data values—inputs and outputs—with no steps specified at all? For something to be a unit of measurement, you need a model such that units are commensurable. That’s hard enough to do when you leave out the cognitive aspects of the test, but how are you going to claim a “unit” when you add in the tacit stuff&mash;that is, what we’re observing and learning that is not explicit; the stuff at the very heart of the test.

    That’s a problem with treating a test case as a unit of measurement at all; even as a unit of effort. But there’s a deeper problem: there’s no intrinsic link that I can see that makes a test case a unit of measure of effectiveness. That seems like a pretty wild leap.

    With respect to your second and third sentences: I’d say there are other, more specific ways to describe our testing and our coverage. I agree that describing our coverage is important, but I don’t think you’re talking about test cases here in the way that most people talk about them. In Rapid Software Testing, we think of a test condition as “something that could be examined in a test, or that could influence the outcome of a test”. It seems to me that a test case covers many conditions—and here again some are explicit and some are tacit; some anticipated, and some only recognized after the fact. To the degree that you’re talking about conditions, rather than cases, I would find it easier to agree with you.

    Reply

Leave a Comment