This morning, I started a local backup. Moments later, I started an online backup. I was greeted with this dialog: Looks a little sparse. Unhelpful. But there is that “More details” drop-down to click on. Let’s do that. Ah. Well, that’s more information. But it’s confusing and unhelpful, but I suppose it holds the promise of something more helpful to come. I notice that there’s a URL, but that it’s … Read more
I’m home in Toronto for a day after several weeks of helping people learn to test software, and as far as I can see, the whole Web is screwed up. Here are some of the things that have happened in the last 48 hours or so. A fellow on Twitter told me about an interesting Skype bug: send the string “http://:” (no quotes), and Skype hangs. Fpr me, it did … Read more
There’s a flaw in the way most organizations classify the severity of a bug. Here’s an example from the Elementool Web site (as of 14 January, 2015); I’m sure you’ve seen something like it: Critical: The bug causes a failure of the complete software system, subsystem or a program within the system. High: The bug does not cause a failure, but causes the system to produce incorrect, incomplete, inconsistent results … Read more
The longer I stay in the testing business and reflect on the matter, the more I believe the concept of “defects” to be unclear and unhelpful. A program may have a coding error that is clearly inconsistent with the program’s specification, whereupon I might claim that I’ve found a defect. The other day, an automatic product update failed in the middle of the process, rendering the product unusable. Apparently a … Read more
Several years ago, I wrote an article for Better Software Magazine called Testing Without a Map. The article was about identifying and applying oracles, and it listed several dimensions of consistency by which we might find or describe problems in the product. The original list came from James Bach. Testers often say that they recognize a problem when the product doesn’t “meet expectations”. But that seems empty to me; a … Read more
On Monday, January 10, Ajay Balamurugadas tweeted, “When programmer has fixed a problem, he marks the prob as fixed. Programmer is often wrong. – #Testing computer software book Me: why?” I intended to challenge Ajay, but I made a mistake, and sent the message out to a general audience: “Challenge for you: think of at least ten reasons why the programmer might be wrong in marking a problem fixed. I’ll … Read more
John D. Cook recently issued a blog post, How many errors are left to find?, in which he introduces yet another silly quantitative model for estimating the number of bugs left in a program. The Lincoln Index, as Mr. Cook refers to it here, was used as a model for evaluating typographical errors, and was based on a method for estimating the population of a given species of animal. There … Read more
Dave Nicollette responds to my post on the Ellis Island bug. I appreciate his continuing the conversation that started in the comments to my post. Dave says, “In describing a ‘new’ category of software defect he calls Ellis Island bugs…”. I want to make it clear: there is nothing new about Ellis Island bugs, except the name. They’ve been with us forever, since before there were computers, even. He goes … Read more
A couple of years ago, I developed a version of a well-known reasoning exercise. It’s a simple exercise, and I implemented it as a really simple computer program. I described it to James Bach, and suggested that we put it in our Rapid Software Testing class. James was skeptical. He didn’t figure from my description that the exercise would be interesting enough. I put in a couple of little traps, … Read more
And now for the immodest part of the EuroSTAR 2009 Test Lab report: I won the Best Bug award, although it’s not clear to me which bug got the nod, since I reported several fairly major problems. I tested OpenEMR. For me, one candidate for the most serious problem would have been a consistent pattern of inconsistency in input handling and error checking. I observed over a dozen instances of … Read more