Lessons Learned in Finding Bugs

This story is put together from several parallel experiences over the last while that I’ve merged into one narrative. The pattern of experiences and epiphanies is the same, but as they used to say on TV, names and details have been changed to protect the innocent. I was recently involved in testing an online shopping application. In the app, there’s a feature that sends notifications via email. On the administration … Read more

Expected Results

Klára Jánová is a dedicated tester who studies and practices and advocates Rapid Software Testing. Recently, on LinkedIn, she said: I might EXPECT something to happen. But that doesn’t necessarily mean that I WANT IT/DESIRE for IT to happen. I even may want it to happen, but it not happening doesn’t have to automatically mean that there’s a problem. The point of this post: no more “expected results” in the … Read more

As Expected

This morning, I started a local backup. Moments later, I started an online backup. I was greeted with this dialog: Looks a little sparse. Unhelpful. But there is that “More details” drop-down to click on. Let’s do that. Ah. Well, that’s more information. But it’s confusing and unhelpful, but I suppose it holds the promise of something more helpful to come. I notice that there’s a URL, but that it’s … Read more

Oracles from the Inside Out, Part 8: Successful Stumbling

When we’re building a product, despite everyone’s good intentions, we’re never really clear about what we’re building until we try to build some of it, and then study what we’ve built. Even after that, we’re never sure, so to reduce risk, we must keep studying. For economy, let’s group the processes associated with that study—review, exploration, experimentation, modelling, checking, evaluating, among many others—and call them testing. Whether we’re testing running … Read more

Oracles from the Inside Out, Part 7: References as Checks

Over the last few blog posts, I’ve been focusing on oracles—means by which we could recognize a problem when we encounter it during testing. So far, I’ve talked about feelings and private mental models within internal, tacit experiences; consistency heuristics by which we can make inferences that help us to articulate why we think and feel that something is a problem; brief exchanges—tiny bursts of conferences—between people with collective tacit … Read more

Oracles from the Inside Out, Part 6: Oracles as Extensions of Testers

The previous post in this series was something of a diversion from the main thread, but we’ll get back on track in this long-ish post. To review: Marshall McLuhan famously said that “the medium is the message”. He used this snappy slogan to point out that media, tools, and technologies are not only about what they contain; they’re also about themselves, changing the scale, pace, or pattern of human affairs … Read more

Oracles from the Inside Out, Part 5: Oracles as References as Media

Try asking testers how they recognize problems. Many will respond that they compare the product to its specification, and when they see an inconsistency between the product and its specification, they report a bug. Others will talk about creating and running automated checks, using tools to compare output from the product to specific, pre-determined, expected results; when the product produces a result inconsistent with expectations, the check identifies a bug … Read more

Oracles from the Inside Out, Part 4: From Experience to Inference

In the previous post, I gave an example of the happy (and, alas, rare) circumstance in which the programmer and I share models and feelings, such that the programmer becomes aware of a problem I’ve found without my being explicit about it. That is, on this table, I can go directly from top-left, where I experience the problem, to conference in the bottom left, where awareness of that problem has … Read more

Oracles From the Inside Out, Part 3: From Experience Directly to Conference

Yesterday I described the moment of recognition of a problem, which tends to happen in the upper-left corner of this table: When I perceive a problem during testing, it’s my job to let people know about what I’ve found, and to arrive at a shared set of feelings and mental models about the problem, represented by the lower left quadrant. That is, I want to move from experience—my private, internal, … Read more