For almost as long as I’ve been a tester, with occasional lapses into process enthusiasm, I’ve been questioning the value of test automation as a presumed good, especially when the automation is deployed against the highest levels of the application. Automation is a tool, and there is great value in tools. But with that value comes risk.
The Agile Manifesto, properly in my view, emphasizes individuals and interactions over processes and tools, and it emphasizes working software over comprehensive documentation. The Manifesto notes that “while there is value in the things on the right, we value things on the left more.”
McLuhan had some remarkable observations on tools, which he considered a subset of media. I wrote about the value of McLuhan thinking for testers here. McLuhan famously identified writing—in particular the phonetic alphabet—as a technology. In his Laws of Media, he points out that one of the effects of a medium is to extend or enhance or enable or accelerate or intensify some human capability in some way. Another effect occurs when the medium is stretched or “overheated” beyond its original or intended capacity; it reverses into the opposite of its enabling or extending effect.
I was introduced to McLuhan’s work largely through a CBC Radio program called Ideas. In 1988, David Cayley covered a conference on Orality and Literacy, co-sponsored by the McLuhan Program at the University of Toronto and the Toronto Semotics Circle (for a description of the program and of the conference, go here, and then search for “literacy”). The conference was set up to question the idea of literacy as the centre of education—not to reject it, but to question it.
The motivation behind the questioning was to understand better the role of literacy in education and in the world in general. Some scholars pointed out that literacy has to be seen in its human context, as an extension of oral discourse, because it is as listeners and speakers that we evolved, not as readers and writers. As David Pattanayak, the director of the Central Institute of Indian Languages at the University of Mysore put it, “What I am worried about is that there are 800 million illiterates in the world, and for those 800 million illiterates, there is nobody to speak. We are speaking as though literacy is responsible for everything—for family welfare, GNP increase, for modernization, for all kinds of things, but I don’t think that is correct…The whole question is that there are illiterates and there are literates, and we should be looking for interaction among the illiterates and the literates, rather than trying to prove the superiority of one over the other.” Why? Because literacy is one means to an end; it helps with many things, but it certainly does not guarantee the accomplishment of our human goals. As he later goes on to say, “Literacy without social concern is meaningless.”
To me, this has a direct connection to our business and to our fascination with written and/or automated tests. For many years, we’ve tried to improve testing by trying as hard as possible to remove the messy human bits from it. James Bach describes this in his essay in The Gift of Time. We’ll call it the Chapel Hill approach to software testing; papers on it are collected in Hetzel’s Program Test Methods (something much closer to being the first book on software testing than Myers’ The Art of Software Testing, by the way). The Chapel Hill approach to reducing testing’s problems with those unreliable humans, says Hetzel, is to lean hard on media, improving “written specification methods” and using “unambiguous testable specification languages”, rather than treating testing as an open-ended investigative process.
A bunch of us, led by the work of Jerry Weinberg, Cem Kaner, and James have been questioning the value of putting media at the centre of testing for qute some time now. Questioning the value of written artifacts isn’t exactly new; it goes back still farther than that. How far? How about the Greek philosophers—Plato, and Socrates?
There’s a version of Plato’s dialogue Phaedrus online. The part that most interests me starts with the paragraph that you can find by bringing up the link and using your browser’s search function to look for “Theuth”. Read that paragraph and a little further, and you come across this, when Socrates talks of written-down speeches:
You would imagine that they had intelligence, but if you want to know anything and put a question to one of them, the speaker always gives one unvarying answer. And when they have been once written down they are tumbled about anywhere among those who may or may not understand them, and know not to whom they should reply, to whom not…”
Wait, it gets better. I recommend reading this bit slowly, savouring it:
Until a man knows the truth of the several particulars of which he is writing or speaking, and is able to define them as they are, and having defined them again to divide them until they can be no longer divided, and until in like manner he is able to discern the nature of the soul, and discover the different modes of discourse which are adapted to different natures, and to arrange and dispose them in such a way that the simple form of speech may be addressed to the simpler nature, and the complex and composite to the more complex nature-until he has accomplished all this, he will be unable to handle arguments according to rules of art, as far as their nature allows them to be subjected to art, either for the purpose of teaching or persuading;-such is the view which is implied in the whole preceding argument.
My emphasis—context-driven thinking! Wait; it gets better yet. Really slowly, now:
But he who thinks that in the written word there is necessarily much which is not serious, and that neither poetry nor prose, spoken or written, is of any great value, if, like the compositions of the rhapsodes, they are only recited in order to be believed, and not with any view to criticism or instruction; and who thinks that even the best of writings are but a reminiscence of what we know, and that only in principles of justice and goodness and nobility taught and communicated orally for the sake of instruction and graven in the soul, which is the true way of writing, is there clearness and perfection and seriousness, and that such principles are a man’s own and his legitimate offspring;-being, in the first place, the word which he finds in his own bosom; secondly, the brethren and descendants and relations of his others;-and who cares for them and no others-this is the right sort of man; and you and I, Phaedrus, would pray that we may become like him. (My emphasis added all over the place, there.)
What happens when we apply all this to testing? To me, this says
- that conversation, rather than documentation, is central to the work that we do;
- that notions of correctness are pointless unless they’re based on value;
- that we need to study and practice testing, or anything else that we wish to understand;
- that we must be cautious and think critically;
- that the focus on nomenclature and unquestioned bodies of knowledge proffered by the ISTQB and other certifiers is foolish; and
- that we should aspire to the values that Socrates proposes.
To me it also says that if we want to be great testers, it would be a good idea to study philosophy, focusing (as James suggests)(James Bach, not William James, although he’d probably agree) on ontology (how we conceive of the world, the nature of being) and epistemology (how we know what we know).
And, yes, there is irony too. Here’s Plato, griping through Socrates about how dangerous writing is, and he’s written down this dialogue. What does that tell us? To me, it suggests that everything that we have to say about what we think and what we do is based, not on absolute principles of truth and certainty, but on heuristics and skepticism.
What does this say to you?
First of all, Michael, you’ve written a terrific essay–but, unfortunately, you’ve written it, so it’s unavailable to illiterates without help.
Second, notice the assumption in the quote, “What I am worried about is that there are 800 million illiterates in the world, and for those 800 million illiterates, there is nobody to speak…”
Being illiterate doesn’t mean you can’t speak. My grandmother was illiterate in at least three languages that I know of, but she could certainly speak all three of them.
So, to further your argument, consider that even people who are considered “literate” may not be very good at writing their thoughts, and some who can write are not very good at speaking their thoughts.
I don’t think we should rail against any form of communication–writing, speaking, drawing, gesturing, demonstrating, or what have you–but only against omitting a priori any form of communication that might be helpful to the sender and the receiver. (Perhaps because you believe there’s one and only one “right” way.)
We are, after all, in the communication business. I wish I could tell your readers that in person, then have them experience it. By blogging, I’m (for the moment) settling for less, but I’m going to get you next month in the Experiential Exercise Design Workshop.
Hi, Jerry,
Thank you so much for commenting!
unfortunately, you’ve written it, so it’s unavailable to illiterates without help.
As you well know, I’m all too happy to talk to people as well. Illiterates are unlikely to be spared.
Being illiterate doesn’t mean you can’t speak.
David Pattanayak sounds erudite enough to me that I can believe he intended the irony there.
I don’t think we should rail against any form of communication–writing, speaking, drawing, gesturing, demonstrating, or what have you–but only against omitting a priori any form of communication that might be helpful to the sender and the receiver.
Amen; there’s been way too much of the a priori omission stuff over the span of my career, and doubtless over yours. The point of the Orality and Literacy Conference was not to attack literacy, but rather to de-centre it. (Yes, it sets up another joke about literacy, but that that is the way we spell "centre" in these parts.) In the same program, Eric Havelock noted that in addition to reading and writing, we should also teach kids to speak—he said "we speak appallingly badly in this country"—and to listen, and to sing, and to dance. (Hmmm… considering the disaster that we sometimes make of teaching, perhaps we should focus on letting the kids learn, rather than trying to teach them; or on letting them unlearn the “rules” that we’ve tried to teach them.)
I’m going to get you next month in the Experiential Exercise Design Workshop.
And I, for one, cannot wait. Thanks again for the comments.
—Michael B.
To illustrate Jerry’s comment:
I had once a discussion with a friend about using the phone or email at work. He argued that the phone is much more efficient because you have direct response and you are able to respons on the response, etc, etc.
All I could reply was that it depends on the message. When there is a need for discussion, calling seems indeed the best solution. But when more people must be notified, e-mail may be the better means of communicating.
As you’ve pointed out lot’s of times, an important part of testing is acquiring information about the product. That information is worthless if it isn’t communicated clearly and timely to the right persons. In my vision tools can help to make sure the message is acquired and composed fast and communicated in a reliable, clear, and unambiguous manner. I think good tools are essential for this, can make a difference with your competitors. But you’re right: it is about the message. The message is always most important.
An interesting read – but where is it meant to lead me to?
It starts with automation, and tools, but neither of those words are mentioned after the third paragraph. Were those words used to attract attention or are they germane to the discussion that follows? I'm not clear.
The McLuhan background isn't referenced after paragraph 4 but doesn't seem to segue into paragraph 5. Why is it there?
Para 5 says not much. "literacy without social concern is meaningless" – this doesn't follow from the text in the blog. It might be substantiated by Pattanyak in his pitch which you don't reference. OK, let's take that as given. (But why bother?)
No – I don't see the relevance inferred in para 6. What exactly is the connection? I also don't understand the 'Chapel Hill Approach…'. Who references that work except the context-driven folk? CH is an age ago – nobody except a few testing archaeologists read that sutff? But again – let's move on…
Para 7. No I don't think you've made the case that test scripts = media (most people *still* test in an ad-hoc way, don't they?), so your next segue doesn't follow.
The next sections are very interesting – but not for the reasons you suggest. The quotes are really proposing that speeches made without written prompts (Powerpoint 100BC?) are superior to written texts used by less capable orators.
It's a 2500 year old rant. Brilliant orators and thinkers who didn't write things down were jealous of published writers. The paragraphs may simply be sour grapes targeted at more (finanacially) successful writers who subsequently have been forgotten perhaps?
Extracted from an Amazon reriew of Plato : Phaedrus: A Translation With Notes: "The second speech (by Socrates), while an impeccable model of correct rhetoric, and reaching the correct conclusion is also essentially flawed- for it makes no appeal to the deepest fundamental causes of things. Simply put, it lacks soul." I'm no expert on the classics, but it sounds like he was a dreamer – with no experience of the real world, and certainly not our world 2500 years after his death. (He committed suicide 'on principle'). I've ordered a copy of the book – I'll make a more considered response in time.
Whatever.
Is there a connection with the classical view and unscripted testing? Purrrlease. The classical view seems to imply that the written word has little value; that the only rewarding performance is a live performance (by the original 'orator'). I'm sorry, but this sounds like the puerile boasting of some 14 year old who defends a crappy pop group by saying – "they are better live".
The only Shakespeare worth seeing was a live perfomance? Er no, I'm sure. If that was the case all his works should have been trashed in 1616. In fact, if no one had written down Plato's soliloquy's – they would have been long forgotten. Nah – doesn't compute.
I'm not sure of the connection here. No I am sure – there is none.
Onwards.
I agree mostly with testing being an intellectual, cerebral activity. (Sapient is presumptious). Philosophy, ontology and epistemology might characterise some disucssions by consultant types – but most testers would benefit tremendously by using their critical faculties alone. Let's focus on that hey?
Thank you, Paul, for your comments.
It starts with automation, and tools, but neither of those words are mentioned after the third paragraph.
You might like to have a look at the beginning of the sixth paragraph. You also note later (although you disagree) that I am claiming that tools are media, of which more in a moment.
The McLuhan background isn't referenced after paragraph 4 but doesn't seem to segue into paragraph 5. Why is it there?…
"literacy without social concern is meaningless" – this doesn't follow from the text in the blog. It might be substantiated by Pattanyak in his pitch which you don't reference.
David Pattanayak, mentioned in paragraph 5, was one of the scholars at the conference referenced in paragraph 4. I'm sorry if this wasn't sufficiently clear.
I don't see the relevance inferred in para 6. What exactly is the connection? I also don't understand the 'Chapel Hill Approach…'. Who references that work except the context-driven folk? CH is an age ago – nobody except a few testing archaeologists read that sutff?
For me, part of understanding the present involves trying to understand the past. I believe that few read about Chapel Hill less because it's old and irrelevant, but more because it was so successful. Its message is tacitly repeated every day in the words and actions of those who see testing as a simple technical problem, and who believe that if only we did enough checking (mediated by test scripts and/or automated), we'd be fine.
The quotes are really proposing that speeches made without written prompts (Powerpoint 100BC?) are superior to written texts used by less capable orators…I've ordered a copy of the book – I'll make a more considered response in time.
That's an interesting interpretation. I'd be grateful if you could point out evidence for it. Perhaps you can do that after you've read the piece in full. Maybe the Amazon reviewer, citing Socrates, missed the part in which Socrates notes "Know then, fair youth, that the former discourse was the word of Phaedrus, the son of Vain Man,…" That is, Socrates was parodying what Phaedrus would have saidI. I'm no expert on the classics either, but I generally prefer reading them myself before making conclusions about them based on an Amazon reviewer's review, even if he is a top 100 reviewer. (By the way, the review to which you refer is here: http://www.amazon.com/Phaedrus-Oxford-Worlds-Classics-Plato/dp/0192802771)
I'm not sure of the connection here. No I am sure – there is none.
Spoken like a developer! There is no bug. Is there a possibility that there is a connection, but that I haven't expressed it sufficiently clearly for you? Or that I (and others) might see a connection that you don't, and that being reasonable people, we might disagree reasonably? Apparently not; there is no connection.
More in a moment…
I don't understand Paul Gerrard's objection to studying the history of testing.
I missed the part where he tells us why we're better off being uneducated about our craft.
It's kind of a back-handed compliment, I guess. Apparently he thinks only the context-driven crowd can be expected to know what they are talking about!
The classical view seems to imply that the written word has little value; that the only rewarding performance is a live performance (by the original 'orator').
Well, you're certainly entitled to draw whatever inferences you like. However, I hope that you'd consider a different interpretation, based on a difference between questioning and rejecting.
I'm not saying that there's no value in (say) the text of Shakespeare's plays; but I am saying that it's important to question what the text leaves in, what it leaves out, and what it might put in. That was what McLuhan was very concerned about. He saw that issue starting with the phonetic alphabet, through movable type, and on to electronic media. (Note that McLuhan takes a much broader view of "media" than you seem to here; for McLuhan, a medium was anything that effected a change—but I suppose you didn't read my article either.) Representations of something change the thing; they take our experience of the original thing from immediate to mediated. That's not a bad thing, necessarily, but the experience of one is definitely different from experience of the other.
Read the blog post above carefully and you'll note (well, I hope you'll note) that I'm suggesting a change in focus, questioning and de-centring the document, and not rejecting the idea of documents altogether.
Philosophy, ontology and epistemology might characterise some disucssions by consultant types – but most testers would benefit tremendously by using their critical faculties alone.
I trust you intended the first part of the sentence ironically. However, I think that the development of critical faculties depends on philosophy (the love of wisdom), ontology (how we view the world), and epistemology (how we know what we know). That may not be how the conversation begins, but I think those issues emerge for those who care about the work that they do.
Perhaps you disagree. Or to put it another way, perhaps you have a philosophical objection because our ontologies disagree based on our epistemologies. Or to put it yet another way, "whatever", "onward", or "oh, bollocks".
Cheers,
—Michael B.