To a naïve manager who doesn’t understand testing very well, the visible manifestation of testing is a tester, sitting in front of a computer, banging on keys to operate a product, comparing output with a predicted result, and marking “pass” or “fail” on a spreadsheet. Therefore, thinks the manager: one way to make testing dramatically more efficient and effective is to automate the testing. Substitute the human tester for a … Read more
On November 21, 2017, I delivered a webinar as part of the regular Technobility Webinar Series presented by my friend and colleague Peter de Jager. The webinar was called “How To Get What You Want from Testing (for Managers)”, and you can find it here. Alas, we had only a one-hour slot, and there were plenty of questions afterwards. Each of the questions I received is potentially worthy of a … Read more
Testers: when we speak of “manual testing”, we help to damage the craft. That’s a strong statement, but it comes from years of experience in observing people thinking and speaking carelessly about testing. Damage arises when some people who don’t specialize in testing (and even some who do) become confused by the idea that some testing is “manual” and some testing is “automated”. They don’t realize that software development and … Read more
Here’s an easy-to-remember little substitution that you can perform when someone suggests “automating the testing”: “Automate the evaluation and learning and exploration and experimentation and modeling and studying of the specs and observation of the product and inference-drawing and questioning and risk assessment and prioritization and coverage analysis and pattern recognition and decision making and design of the test lab and preparation of the test lab and sensemaking and test … Read more
This is an expansion of some recent tweets. Do automated tests (in the RST namespace, checks) measure the quality of your product, as people sometimes suggest? First, the check is automated; the test is not. You are performing a test, and you use a check—or many checks—inside the test. The machinery may press the buttons and return a bit, but that’s not the test. For it to be a test, … Read more
Note: This post refers to testing and checking in the Rapid Software Testing namespace. This post has received a few minor edits since it was first posted. For those disinclined to read Testing and Checking Refined, here are the definitions of testing and checking as defined by me and James Bach within the Rapid Software Testing namespace. Testing is the process of evaluating a product by learning about it through … Read more
“Program testing involves the execution of a program over sample test data followed by analysis of the output. Different kinds of test output can be generated. It may consist of final values of program output variables or of intermediate traces of selected variables. It may also consist of timing information, as in real time systems. “The use of testing requires the existence of an external mechanism which can be used … Read more
Just as no one has the right to tell you what language to speak at home, nobody outside of your project has the authority to tell you how to speak inside your project. Every project develops its own namespace, so to speak, and its own formal or informal criteria for naming things inside it. Rapid Software Testing is, among other things, a project in that sense. For years, James Bach … Read more
Every now and then, in some forum or another, someone says something like “75% of the testing done on an Agile project is done by automation”. Whatever else might be wrong with that statement, it’s a very strange way to describe a complex, cognitive process of learning about a product through experimentation, and seeking to find problems that threaten the value of the product, the project, or the business. Perhaps … Read more
When a decision maker asks “When will testing be done?”, in my experience, she really means is “When will I have enough information about the state of the product and the project, such that I can decide to release or deploy the product?” There are a couple of problems with the latter question. First, as Cem Kaner puts it, “testing is an empirical, technical investigation of the product, done on … Read more