In Rapid Software Testing namespace, critical thinking is thinking about thinking with the goal of avoiding being fooled. We have a quick mnemonic to trigger critical thing: WHeReAS. Ask: Who? Huh? Really? And? So?
For a long time, we had only the latter four items on the list. Evaluating an observation or a claim depends on making sure that we’ve understood the statement or idea we’re examining. That’s “Huh?” “Really?” is about the validity of the claims. “And?” prompts us to consider things that we might have overlooked. And “So?” is about consequences and whether we should take action.
A little while ago, one of our colleagues pointed out something that was missing from our little model: the social dimension. Avoiding errors in our thinking depends in the motives, expertise, and trustworthiness of the people who are making claims. That’s why we added “Who?” Who says?
Let’s consider an important asymmetry. We all have the experience of listening to experts and being confused, when we’re new to a domain and its language, and the experts know much more than we do. In such cases, it might be tempting to assume that when someone is saying something that we don’t understand, that person must be an expert. That doesn’t necessarily follow. Maybe that person isn’t an expert at all, but simply incoherent, and serving up a word salad that sounds dazzling and impressive to non-experts because it’s confusing.
That’s where “Huh?” comes in. What is actually being said here? If we don’t understand what we’re hearing, it behooves us to make sense of it; to ask if we’ve misheard; or to ask for explanations for unfamiliar technical terms. If the person is unable to articulate or clarify ideas in an understandable way, we would be wise to question whether that person is an expert.
Maybe that person is an expert in a specific domain. That’s fine, but it can lead to the halo effect, a cognitive bias in which we erroneously transfer that expertise to a completely different domain. In 2016, Geoff Hinton predicted that no radiologists would be needed within five years. It’s 2025 as I write, and not only are radiologists still necessary, there’s a global shortage of them. Hinton is unquestionably an expert in machine learning, but that expertise doesn’t automatically confer expertise in radiology. What he said about that field should never have been taken seriously.
Worse, maybe the person is both inexpert and incoherent. That’s why we need to ask “Really?”, “And?”, and “So?”
“Really?”, again, is about evidence and its quality. Maybe a claim is impressive — but is the claim true? What data is the claimant providing — what can we see or hear — to provide support for what they’re saying? Is that data relevant to the claims? Consistent with them? When the claimant refers to the work of experts for support, is the claimant really familiar with those people and their work? Or is the claimant simply name-dropping, trying to prompt another form of halo effect? Similarly, when the claimant dismisses the work of experts, is the dismissal based on actual study of that work?
“And?” is about more evidence — or counter-evidence. And what else? Whatever is being mentioned, other things might be worth mentioning. And also…? Whatever is true, other things might also be true. And over there? Whatever is going on, other things might be going on somewhere we’re not looking. And so far; may we haven’t yet seen things we could see. And what comes next? Whatever is true now might not be true for long.
“So?” is about conclusions, significance, and consequences. If we’ve assessed a statement or a claim as being true or false, does it matter? How should it change our thinking, or our actions? Maybe the issue at hand is a big deal; maybe it doesn’t matter. Maybe there are hidden, or downstream effects and consequences. Is it appropriate for the expert’s perspective to influence us?
Asking Huh? Really? And? and So? can also help us in another way, by replace cynicism (rejecting beliefs) of with more appropriate skepticism (rejecting certainty about beliefs). Sometime we are prone to reject a claim because of our biases against who’s making it. Maybe we simple don’t like the person making the claim, or we don’t like their tone. Maybe we don’t trust a claim because the person making it has a record of being untrustworthy. Yet at the moment, the claimant might be saying something true and worthy.
Ignoring “Who?”, or failing to go beyond it into the other questions, can lead us to mistaken conclusions about who’s an expert and who isn’t; who’s worthy of trust and who isn’t; what’s true and what isn’t; what we should or should not do. When people fail to think critically, we know where that can go. Look what happens in management sometimes. Look what happens in politics.
Please think carefully. Please think critically.
Book recommendations:
Daniel Kahneman, Thinking Fast and Slow. A terrific and thorough guide to how we think — and systematic errors in our thinking.
Harry Collins and Rob Evans, Rethinking Expertise. A fascinating examination of expertise — including the notion of interactional expertise. This is the capacity to speak and understand the practice language of a specific domain, learned through interaction with experts with contributory expertise — which is the capacity to do work in that domain.
Christoper Cerf and Victor Navasky, The Experts Speak: The Definitive Compendium of Authoritative Misinformation The authors call this “a collection of unadulterated, fully authenticated, false expertise”. And? They left out “often hilarious”.