To the Developer: About Your Impending “Promotion”

Dear Developer… I’d like to warn you that you’re about to get “promoted” — and doubtless without a pay increase — to a kind of management job that I bet you don’t want. Hold on a second; pardon me. Let me introduce myself first. I’m a tester. As a tester, it’s my job to point out problems and risks that you might not have recognized. A brief, informal survey of … Read more

ChatGPT and Test Data

People are touting ChatGPT as a tool for generating test data. Let’s try something simple: MB: You are a powerful, diligent assistant to a professional software tester. Give me a table of 30 numbers. In the first column, provide the number. In the second column, provide the English spelling of the number. Sort the column in alphabetical order by the values in the second column. ChatGPT 3.5: Certainly! Here’s a … Read more

Bing Chat, the Evaluate Function, and the Wolfram Alpha Plugin

When you read or even scan this post, you’re likely to say something like “Holy hopscotch, that’s a long post.”  And you’ll be right. And you might be inclined to say “…and it’s boring.” And depending on your perspective, you’ll be right about that, too. It certainly has taken a significant amount of time to edit and to narrate. If you’re interested in risk associated with Large Language Models and … Read more

Out of Order

To testers (and to designers, developers, and managers, and others who might do testing), this is your periodic reminder to do things out of order. By “do things out of order”, I mean: perform sequences of actions that do not follow the designer or the developer’s imagination of how things should happen; how things normally happen; what the customers might want; or what’s easiest to process. Here’s an example from … Read more

Tacit Knowledge Transfer Makes the News

In the Guardian, October 22 2023, John Naughton writes an article on the complications associated with moving semiconductor manufacturing from Taiwan to the United States. The article and the problems that it describes are pretty interesting. At the centre of the product is the difficulty of transferring tacit knowledge about chip fabrication. It turns out that documentation on its own doesn’t work very well. What matters most is having people … Read more


In the Rapid Software Testing Managed class today, James Bach was talking about experiences with CTOs. It struck me that it might be fun to come up with an expansion of the abbreviation that would represent the wishful thinking that James was talking about; like “Chief Wishing Officer”, but with a word for “wish” that starts with T. Aha! I thought. Here’s a great application of ChatGPT — get it … Read more

When the Developers Are the Users

This is a lightly-edited version of a repost on LinkedIn. The original post contained a photo of a conference talk. The presenter was a dude in a Spiderman costume. (I’ve always wondered how many Spiderman costumes we’d see at meetings of doctors, or journalists, or theoretical physicists. But I digress.) The screen displayed a slide “Everyone cares about User Experience, but no one cares about Developer Experience.” Spiderman outfit notwithstanding, … Read more

A Reply to “Running a crowd-sourced experiment on using LLMs for testing” — Part 2: Analysis

Vipul Kocher is a fellow whom I have known for a long time. I think we met in North America in the mid 2000s. I know I visited his company in Noida, New Delhi about 15 years ago, and spoke with his testers for an hour or so. On that occasion, I also visited his family and had a memorable home-cooked meal, followed by a mad dash in a sport … Read more

A Reply to “Running a crowd-sourced experiment on using LLMs for testing”

This post and the ones that follow represent an expansion on a thread I started on LinkedIn. On September 30, 2023, Vipul Kocher — a fellow with whom I have been on friendly terms since I visited his company and his family for lunch in Delhi about 15 years ago — posted a kind of testing challenge on LinkedIn. I strongly encourage you to read the post. I’ll begin by … Read more

Reliably Unreliable

ChatGPT may produce inaccurate information about people, places, or facts. Testing work comes with a problem: the more we test, the more we learn. The more we learn, the more we recognize other things to learn. When we investigate a problem, there’s a non-zero probability that we’ll encounter other problems — which in turn leads to the discovery of more problems. In the Rapid Software Testing namespace, we’ve come … Read more