Testing is…

Every now and again, someone makes some statement about testing that I find highly questionable or indefensible, whereupon I might ask them what testing means to them. All too often, they’re at a loss to reply because they haven’t really thought deeply about the matter; or because they haven’t internalized what they’ve thought about; or because they’re unwilling to commit to any statement about testing. And then they say something … Read more

I’ve Had It With Defects

The longer I stay in the testing business and reflect on the matter, the more I believe the concept of “defects” to be unclear and unhelpful. A program may have a coding error that is clearly inconsistent with the program’s specification, whereupon I might claim that I’ve found a defect. The other day, an automatic product update failed in the middle of the process, rendering the product unusable. Apparently a … Read more

Very Short Blog Posts (14): “It works!”

“It works” is one of Jerry Weinberg‘s nominees for the most ambiguous sentence in the English language. To me, when people say “it works”, they really mean Some aspect of some feature or some function appeared to meet some requirement to some degree based on some theory and based on some observation that some agent made under some conditions once or maybe more. One of the most important tasks for … Read more

Very Short Blog Posts (13): When Will Testing Be Done?

When a decision maker asks “When will testing be done?”, in my experience, she really means is “When will I have enough information about the state of the product and the project, such that I can decide to release or deploy the product?” There are a couple of problems with the latter question. First, as Cem Kaner puts it, “testing is an empirical, technical investigation of the product, done on … Read more

Harry Collins and The Motive for Distinctions

“Computers and their software are two things. As collections of interacting cogs they must be ‘checked’ to make sure there are no missing teeth and the wheels spin together nicely. Machines are also ‘social prostheses’, fitting into social life where a human once fitted. It is a characteristic of medical prostheses, like replacement hearts, that they do not do exactly the same job as the thing they replace; the surrounding … Read more

Can You Hear The Alarm Bells?

Many people seem certain about what happened to cause the fiasco. Stories are starting to trickle out, and eventually they’ll be an ocean of them. To anyone familiar with software development, especially in large organizations, these stories include familiar elements of character and plot. From those, it’s easy to extrapolate and fill in the details based on imagination and experience. We all know what happened. Well, we don’t. In … Read more

Very Short Blog Posts (2): Confidence

It is not the job of testing to build confidence in the product. Confidence is a relationship between the product and some stakeholder. It is much more the job of testing to identify problems in the product—and in people’s perceptions of the product—that are based on or that would lead to unwarranted confidence.

What’s Comparable (Part 2)

In the previous post, Lynn McKee recognized that, with respect to the Comparable Product oracle heuristic, “comparable” can be have several expansive interpretations, and not just one narrow one. I’ll emphasize: “comparable product”, in the context of the FEW HICCUPPS oracle heuristics, can mean any software product, any attribute of a software product, or even attributes of non-software products that we could use as a basis for comparison. (Since “comparable … Read more

What’s Comparable (Part 1)

People interpret requirements and specifications in different ways, based on their models, and their past experiences, and their current context. When they hear or read something, many people tend to choose an interpretation that is familiar to them, which may close off their thinking about other possible interpretations. That’s not a big problem in simple, stable systems. It’s a bigger problem in software development. The problems we’re trying to solve … Read more

Heuristics for Understanding Heuristics

This conversation is fictitious, but it’s also representative of several chats that I’ve had with testers over the last few weeks. Tony, a tester friend, approached me recently, and told me that he was having trouble understanding heuristics and oracles. I have a heuristic approach for solving the problem of people not understanding a word: Give ’em a definition. So, I told him: A heuristic is a fallible method for … Read more