There’s a significant mistake that people might make when using LLMs to summarize a requirements document, or to produce a test report. LLMs aren’t all that great at summarizing. That’s definintely a problem, and it would be a mistake to trust an LLM’s summary without reviewing the original document. The bigger mistake is in believing that the output, the artifact, is the important thing. We might choose to share a … Read more
A prototype of this post originally appeared on LinkedIn. Today I was using Microsoft Word, and for the first time I took a look at a feature that’s probably been there for a long while. Also today, there’s at least one more LinkedIn poll with an interview question — apparently aimed at testers — on a fairly trivial aspect of Java programming. Questions of that nature might reasonable if the … Read more
This is a lightly-edited version of a repost on LinkedIn. The original post contained a photo of a conference talk. The presenter was a dude in a Spiderman costume. (I’ve always wondered how many Spiderman costumes we’d see at meetings of doctors, or journalists, or theoretical physicists. But I digress.) The screen displayed a slide “Everyone cares about User Experience, but no one cares about Developer Experience.” Spiderman outfit notwithstanding, … Read more
This post and the ones that follow represent an expansion on a thread I started on LinkedIn. On September 30, 2023, Vipul Kocher — a fellow with whom I have been on friendly terms since I visited his company and his family for lunch in Delhi about 15 years ago — posted a kind of testing challenge on LinkedIn. I strongly encourage you to read the post. I’ll begin by … Read more
Yet another post plucked and adapted from the walled garden of LinkedIn “What the large language models are good at is saying what an answer should sound like, which is different from what an answer should be.” —Rodney Brooks, https://spectrum.ieee.org/gpt-4-calm-down Note for testers and their clients: the problem that Rodney Brooks identifies with large language models applies to lots of test procedures and test results as well. People often have … Read more
This post has been brewing for a while, but a LinkedIn conversation today reminded me to put it in the bottle and ship it. Testing is socially challenging. There’s a double meaning there. One meaning is that testing involves challenging the product and our beliefs about it, in a social context. The other meaning is that probing the product and people’s beliefs about it can sometimes be uncomfortable for everyone … Read more
Last night, my wife was out on an errand in our car. She parked it, entered the store, and came out again. She tried to start the car. It wouldn’t start. She called home to consult with me. We tried a couple of things over the phone. We considered a couple of possible problems. From what I could tell, the starter motor wasn’t engaging. Not exactly a surprise, because I … Read more
It’s common these days to hear people say that they don’t want to focus on finding bugs; they want to focus on preventing bugs. They want to focus on “building quality in”. Let’s face it: building quality in is a pretty great idea, and preventing bugs from reaching customers is a really good thing. On this, reasonable people agree. To prevent bugs from reaching customers, you’ll have to become a … Read more
An amusing video from a few years back has been making the rounds lately. Dad challenges the kids to write exact instructions to make a peanut butter and jelly sandwich, and Dad follows those instructions. The kids find the experience difficult and frustrating, because Dad interprets the “exact” instructions exactly—but differently from the way the kids intended. I’ll be here when you get back. Go ahead and watch it. Welcome … Read more
A friend that I’m just getting to know runs a training and skills development program for new testers. Today he said, “My students are now starting a project which includes test design, test techniques, and execution of testing. Do you have any input or advice for them?” Here’s my reply. Test design, test techniques, and execution of testing are all good things. I’d prefer performing tests to “test execution”. In … Read more