Whether you’re working in an Agile environment or not, one of the tester’s most important tasks is to ask and advocate for things that make a product more testable. Where to start? Think about visibility—in its simplest form, log files—and controllability in the form of scriptable application programming interfaces (APIs).
Logs aren’t just for troubleshooting. Comprehensive log files can help to identify the data that was processed and the functions that were covered during testing. Logs can be parsed to gather statistics or processed with visualization tools to reveal interesting patterns of behaviour. Ask for consistent structure, precise time stamps, and configurable levels of logging.
A scriptable API affords the opportunity for testers to drive the program at high speed or high volume, in well-ordered, variable, or randomized sequences. A scripting interface can allow testers to observe the program’s data structures, query its internal states, or adjust its configuration quickly and easily. Use APIs and tools for more than functional checking; use them for sophisticated, automation-assisted exploration. As a bonus, an API can add to the value of your product by making its functions more accessible to your customers.
You can’t depend on getting log files and APIs without asking for them. So, starting with your current sprint, ask early and ask often.
Related posts:
- Heuristics of Testability, by James Bach. This is the current word on testability in the Rapid Testing world.
- Testability (a blog post I wrote on the subject in 2009)
Increasing “checkability” is crucial too .
While testing embedded systems ,software testers in my team consistently advocated (with success) with developers to implemented product feedback (real time logging output) which automated test code could consume and drive protocol level actions.
This was break through of sorts for us as we were able to exercise & control scenarios which we were not able to before & keep an eye on regression to the protocol layer .
It was a great complement to existing exploratory test capabilities in the team.
[…] Blog: Very Short Blog Posts (18): Ask for Testability – Michael Bolton – http://www.developsense.com/blog/2014/05/very-short-blog-posts-18-ask-for-testability/ […]
[…] Very Short Blog Posts (18): Ask for Testability Written by: Michael Bolton […]
One of the things I research is for tools that will make the product more ‘externally verifiable’ (a term I believe I picked up from Crispin/Gregory). For instance, we started using Tibco queues so I asked for and received a tool that let’s me open the messages on the queues. Now I can see what is being sent which allows me to monitor details – important to know that my variations in tests are working. In your example, Michael, I might also use it to load a message to see how it is processed when it contains various details.