DevelopsenseLogo

Reporting

Very Short Blog Posts (35): Make Things Visible

I hear a lot from testers who discover problems late in development, and who get grief for bringing them up. On one level, the complaints are baseless, like holding an investigate journalist responsible for a corrupt government. On the other hand, there’s a way for testers to anticipate bad news and reduce the surprises. Try producing a product coverage outline and a risk list. A product coverage outline is an … Read more

Very Short Blog Posts (33): Insufficient Information and Insufficient Time

Here’s a question I get from testers quite a lot: “What do I do when the developers give me something to test with insufficient information and time to test it?” Here’s my quick answer: test it. Here’s my practical answer: test it with whatever time and information you have available. (Testing is evaluating a product by learning about it through exploration and experimentation.) When your time is up, provide a … Read more

How is the testing going?

Last week on Twitter, I posted this: “The testing is going well.” Does this mean the product is in good shape, or that we’re obtaining good coverage, or finding lots of bugs? “The testing is going badly.” The product is in good shape? Testing is blocked? We’re noting lots of bugs erroneously? — Michael Bolton (@michaelbolton) January 31, 2018 “The testing is going well.” Does this mean the product is … Read more

Deeper Testing (2): Automating the Testing

Here’s an easy-to-remember little substitution that you can perform when someone suggests “automating the testing”: “Automate the evaluation and learning and exploration and experimentation and modeling and studying of the specs and observation of the product and inference-drawing and questioning and risk assessment and prioritization and coverage analysis and pattern recognition and decision making and design of the test lab and preparation of the test lab and sensemaking and test … Read more

What Is A Tester?

A junior tester relates some of the issues she’s encountering in describing her work. To the people who thinks she “just breaks stuff all day”, here’s what I might reply: It’s not that I don’t just break stuff; I don’t break stuff at all. The stuff that I’ve given to test is what it is; if it’s broken, it was broken when I got it. If I break anything, consider … Read more

Very Short Blog Posts (24): You Are Not a Bureaucrat

Here’s a pattern I see fairly often at the end of bug reports: Expected: “Total” field should update and display correct result. Actual: “Total” field updates and displays incorrect result. Come on. When you write a report like that, can you blame people for thinking you’re a little slow? Or that you’re a bureaucrat, and that testing work is mindless paperwork and form-filling? Or perhaps that you’re being condescending? It … Read more

Taking Severity Seriously

There’s a flaw in the way most organizations classify the severity of a bug. Here’s an example from the Elementool Web site (as of 14 January, 2015); I’m sure you’ve seen something like it: Critical: The bug causes a failure of the complete software system, subsystem or a program within the system. High: The bug does not cause a failure, but causes the system to produce incorrect, incomplete, inconsistent results … Read more

Very Short Blog Posts (20): More About Testability

A few weeks ago, I posted a Very Short Blog Post on the bare-bones basics of testability. Today, I saw a very good post from Adam Knight talking about telling the testability story. Adam focused, as I did, on intrinsic testability—things in the product itself that it more testable. But testability isn’t just a product attribute. In Heuristics of Testability (material we developed in a session of Rapid Software Testing … Read more

Very Short Blog Posts (12): Scripted Testing Depends on Exploratory Testing

People commonly say that exploratory testing “is a luxury” that “we do after we’ve finished our scripted testing”. Yet there is no scripted procedure for developing a script well. To develop a script, we must explore requirements, specifications, or interfaces. This requires us to investigate the product and the information available to us; to interpret them and to seek ambiguity, incompleteness, and inconsistency; to model the scope of the test … Read more

Evaluating Testing: The Qualitative Way

Evaluating Testing: The Qualitative Way | Michael Bolton | STAREAST

For years, testers and managers alike have wrestled with the problem of evaluating software products and testing efforts, often using approaches derived from manufacturing, construction, and physical sciences. These approaches have been only partially successful because software products aren’t physical products. Rather, software is part of a complex web of relationships among programs, computing equipment, networks, and, most importantly, people.

Read more