DevelopsenseLogo

Experience Reports

Experience Report: mabl Trainer and runner, and related features

Screen shot from Mattermost while testing mabl

Warning: this is a long post. Introduction This is an experience report of attempting to perform a session of sympathetic survey and sanity testing, done in September 2021, with follow-up work October 13-15, 2021. The product being tested is mabl. My self-assigned charter was to perform survey testing of mabl, based on a performing a basic task with the product. The task was to automate a simple set of steps, … Read more

Bug of the Day: AI Sees Bits, Not Things

An article that I was reading this morning was accompanied by a stock photo with an intriguing building in the background. I wanted to know where the building was, and what it was. I thought that maybe Chrome’s “Search Google for image” feature could help to locate an instance of the photo where the building was identified. That didn’t happen, but I got something else instead. Google Images provided me … Read more

Lessons Learned from a Little Bug

Almost 10 years ago, I wrote a series of blog posts on project estimation and black swans. And, almost 10 years after that, Chris NeJame reported an observation about the following passage towards the end of Part 4 of the series: As Jerry (Weinberg) has frequently pointed out, plenty of organizations fall victim to back luck, but much of the time, it’s not the bad luck that does them in; … Read more

Testers Don’t Prevent Problems

Testers don’t prevent errors, and errors aren’t necessarily waste. Testing, in and of itself, does not prevent bugs. Platform testing that reveals a compatibility bug provides a developer with information. That information prompts him to correct an error in the product, which prevents that already-existing error from reaching and bugging a customer. Stress testing that reveals a bug in a function provides a developer with information. That information helps her … Read more

As Expected

This morning, I started a local backup. Moments later, I started an online backup. I was greeted with this dialog: Looks a little sparse. Unhelpful. But there is that “More details” drop-down to click on. Let’s do that. Ah. Well, that’s more information. But it’s confusing and unhelpful, but I suppose it holds the promise of something more helpful to come. I notice that there’s a URL, but that it’s … Read more

A Bad Couple of Days

I’m home in Toronto for a day after several weeks of helping people learn to test software, and as far as I can see, the whole Web is screwed up. Here are some of the things that have happened in the last 48 hours or so. A fellow on Twitter told me about an interesting Skype bug: send the string “http://:” (no quotes), and Skype hangs. Fpr me, it did … Read more

Taking Severity Seriously

There’s a flaw in the way most organizations classify the severity of a bug. Here’s an example from the Elementool Web site (as of 14 January, 2015); I’m sure you’ve seen something like it: Critical: The bug causes a failure of the complete software system, subsystem or a program within the system. High: The bug does not cause a failure, but causes the system to produce incorrect, incomplete, inconsistent results … Read more

I’ve Had It With Defects

The longer I stay in the testing business and reflect on the matter, the more I believe the concept of “defects” to be unclear and unhelpful. A program may have a coding error that is clearly inconsistent with the program’s specification, whereupon I might claim that I’ve found a defect. The other day, an automatic product update failed in the middle of the process, rendering the product unusable. Apparently a … Read more

FEW HICCUPPS

Several years ago, I wrote an article for Better Software Magazine called Testing Without a Map. The article was about identifying and applying oracles, and it listed several dimensions of consistency by which we might find or describe problems in the product. The original list came from James Bach. Testers often say that they recognize a problem when the product doesn’t “meet expectations”. But that seems empty to me; a … Read more

When A Bug Isn’t Really Fixed

On Monday, January 10, Ajay Balamurugadas tweeted, “When programmer has fixed a problem, he marks the prob as fixed. Programmer is often wrong. – #Testing computer software book Me: why?” I intended to challenge Ajay, but I made a mistake, and sent the message out to a general audience: “Challenge for you: think of at least ten reasons why the programmer might be wrong in marking a problem fixed. I’ll … Read more