Questioning questioning questioning

Shrini Kulkarni reports on a conversation that he had with Rex Black. Shrini apparently offered a definition of testing, developed by James Bach, that we use in our Rapid Software Testing course: testing is questioning a product in order to evaluate it. Rex didn’t agree with this definition. “Questioning a lifeless thing like software is bizarre. I cannot question my dog,” said Rex.

Despite the fact that a statement is a lifeless thing, let’s go through the something-less-than-bizarre process of questioning it.

Is Rex’s dog lifeless? I don’t know. Maybe it is lifeless; maybe it’s dead. Maybe it’s a stuffed toy. Or perhaps Rex means that asking a question of something that isn’t sapient is bizarre.

Is it bizarre to talk to something that isn’t sapient? Maybe. But some people do talk to their dogs, and some people swear at their computers. So maybe Rex means “not helpful”, rather than bizarre. Maybe he means that it’s bizarre to talk to something that can’t hear you, or that won’t answer you back in English.

Is it bizarre to ask questions of something that can’t hear you? Maybe. But is it bizarre to ask a deaf person a question, using sign language?

Is it bizarre to ask a question of something that doesn’t have a tangible existence or that doesn’t answer back in a way that you can hear? Maybe. But is it bizarre to pray? Is it bizarre to question your beliefs?

Is it bizarre to ignore a common use of the verb “to question”? Well, now I think we might be on to something. The Concise Oxford English Dictionary says this:

Question: v. tr. 1 ask questions of; interrogate. 2 subject (a person) to examination 3 throw doubt up; raise objections to 4 seek information from the study of (phenomena, facts)

If you want to go hyper-literal, “questioning the product in order to evaluate it” is an example of the transitive meaning exactly expressed by 4 above. And 3, and 2, I suppose. But in the Rapid Software Testing course, James and I also suggest that 1 is feasible with software, in a sense. In oratory and in the course notes, we say that we ask questions of the product. “Of”, in this sense, can mean about; that is, we can ask questions about the product. That’s consistent with Oxford too:

Question: n. 1 A sentence worded or expressed as to seek information. 2 a doubt about or objection to a thing’s truth, credibility, advisability, etc. (allowed it without question; is there any question to its validity?). b the raising of such doubt, etc. 3 a matter to be discussed or decided or voted on. 4 a problem requiring an answer or a solution. 5 (foll. by of) a matter or concern depending on conditions (it’s a question of money).

But we can also ask the product questions directly, in a less than literal sense. In the course material, we say:

The “questions” consist of ordinary questions about the idea or design of the product, or else questions implicit in the various ways of configuring and operating the product. (Asking questions about the product without any intent to operate it is usually called review rather than testing.) The product “answers” by exhibiting behavior, which the tester observes and evaluates. To evaluate a product is to infer from its observed behavior how it will behave in the field, and to identify important problems in the product.

(The quotation marks appear in the original text; the emphasis I’ve added here.)

Rex and I have had a number of chats and, on at least one occasion in Canberra, a very pleasant dinner. We disagree on our some of our approaches to testing. That’s okay; reasonable people can disagree reasonably. He deals respectfully with these differences in his most recent book, and kindly acknowledges me and my work, for which I’m grateful.

One reason that bugs in software exist is that people have multiple interpretations of our world and of the words that we use to describe it to one another. In particular when we’re dealing with idea-stuff—things that don’t have a physical existence—alternative interpretations abound. We can use the same words to mean different things, or we can use different words to mean the same thing. For example, as we say in the course:

Cem Kaner prefers this definition (of testing): “Software testing is a process of empirical, technical investigation of the product under test conducted to provide stakeholders with quality-related information.” We mean no disagreement here; Kaner’s definition means the same thing as our definition in every material respect. However, we sometimes prefer more concise wording.

Successful testing depends on embracing expansive definitions, generating expansive models, considering open possibilities, reflecting on alternative assumptions—and then questioning what we think we know. That’s right: we can question assumptions, even though assumptions are lifeless things.

Here’s the thing: I don’t expect Rex to accept anyone’s definition of software testing; the choice is up to him. That’s cool. But I can’t understand how Rex, an apparently smart guy and the past president of the International Software Testing Qualifications Board, can see the world so narrowly, failing to recognize such a common usage of such a common word. What’s up with that?