This is a lightly-edited version of a repost on LinkedIn. The original post contained a photo of a conference talk. The presenter was a dude in a Spiderman costume. (I’ve always wondered how many Spiderman costumes we’d see at meetings of doctors, or journalists, or theoretical physicists. But I digress.)
The screen displayed a slide “Everyone cares about User Experience, but no one cares about Developer Experience.”
Spiderman outfit notwithstanding, this is a crucial point for anyone with the mission to test a product through an API.
For any product, it’s typically important to check functions that transform prescribed input into specified output. Such functions can be checked relatively easily. By all means, do that — in fact, for the developer of the functions of an endpoint, it’s a professionally responsible thing to do; part of well-disciplined software development.
Functional correctness isn’t the only quality criterion for any product, though. Moreover, most of the functions in a product don’t deliver value directly; they enable the value to be delivered. Responsiveness is important. Reliability under load or stress is important. Security is important. Compatibility with other systems is important. These things matter to the end users of products or services, and to people who support them.
When it comes to the programming part of Application Programming Interfaces, developers are the users. Ease of learning and ease of use are vital. Descriptive, up-to-date, accurate documentation, with clear and vivid examples, is vital. Reliable and comprehensible error messages are vital.
Don’t think so? How much time do programmers waste trying to sort out problems with other people’s APIs? How much time should they waste working with yours?
Testing an API responsibly means testing what it’s like to use it; to try developing a product or service with it; to discover problems that people outside of its own developers’ group might have with it.
That kind of testing requires getting programming experience with the API. That in turn takes a degree of established or evolving programming skill, and real experiments using naturalistic scenarios. It requires the tester to play the role of a developer outside the tester’s organization. It requires the tester to identify not only general bugs in the API, but programming-related bugs; anything that would cause loss, harm, waste of time, or diminished value to the programmer.
This is far more than having an inexperienced tester — or an algorithm — check that this week’s output matches last week’s output. Alas, many organizations leave things at that.
Further reading: Exploratory Testing on an API, and the posts that follow it in a four-part series.