A prototype of this post originally appeared on LinkedIn. Today I was using Microsoft Word, and for the first time I took a look at a feature that’s probably been there for a long while. Also today, there’s at least one more LinkedIn poll with an interview question — apparently aimed at testers — on a fairly trivial aspect of Java programming. Questions of that nature might reasonable if the … Read more
This post is a (more detailed) response to a post on LinkedIn by my good friend and colleague Antti Niittyviita. My intention is not to chide or scold him. Instead, I want to shine light on a problem in the way many people talk carelessly about the craft of testing, and to express my ongoing dismay. It’s not a new problem, but it sure is a persistent one. Antti says… … Read more
One of the reasons that software development and testing are screwed up is because people often name things carelessly. Jerry Weinberg was fond of pointing out that “floating point” was the kind of math where the decimal point stayed in the same place, where in “fixed point”, the decimal point moves around. People talk about “serverless computing”, when they really mean “computing using someone else’s servers”. “No-code testing tools”… well, … Read more
December 11, 2023; ChatGPT 3.5 (chat.openai.com) I believe that it’s unwise and irresponsible to promote Large Lanaguage Models for any testing purpose unless we have tested and assessed how well the LLM does on that task. We should not simply accept the claims and fantasies of AI fanboys; nor should we accept the idea that because Everything Will Be Fine in the Future, we should start applying LLMs indiscriminately today. … Read more
Here’s the prompt: “Create five sentences composed of five words; then for each sentence, leave out the first four words, and print the result.” And here’s the output: (Results from ChatGPT 3.5 (chat.openai.com), December 7, 2023. A colleague reports better results from ChatGPT 4.0.) It seems to me that most humans would offer something more like …either with or without the periods. Note ChatGPT’s misinterpretation of the assignment. It seems … Read more
A few nights ago, a colleague noted that he was impressed by ChatGPT’s capacity to evaluate a sentence. He had offered a prompt something like “Is honesty the best virtue” without the question mark, and was surprised that ChatGPT could infer that he had intended a question. (In this post, I will use cLLMs to refer to chatbots based on LLMs.) I was less surprised, since cLLMs reply by design. … Read more
Dear Developer… I’d like to warn you that you’re about to get “promoted” — and doubtless without a pay increase — to a kind of management job that I bet you don’t want. Hold on a second; pardon me. Let me introduce myself first. I’m a tester. As a tester, it’s my job to point out problems and risks that you might not have recognized. A brief, informal survey of … Read more
People are touting ChatGPT as a tool for generating test data. Let’s try something simple: MB: You are a powerful, diligent assistant to a professional software tester. Give me a table of 30 numbers. In the first column, provide the number. In the second column, provide the English spelling of the number. Sort the column in alphabetical order by the values in the second column. ChatGPT 3.5: Certainly! Here’s a … Read more
When you read or even scan this post, you’re likely to say something like “Holy hopscotch, that’s a long post.” And you’ll be right. And you might be inclined to say “…and it’s boring.” And depending on your perspective, you’ll be right about that, too. It certainly has taken a significant amount of time to edit and to narrate. If you’re interested in risk associated with Large Language Models and … Read more
To testers (and to designers, developers, and managers, and others who might do testing), this is your periodic reminder to do things out of order. By “do things out of order”, I mean: perform sequences of actions that do not follow the designer or the developer’s imagination of how things should happen; how things normally happen; what the customers might want; or what’s easiest to process. Here’s an example from … Read more