Certification and Its Discontents I

EuroSTAR has been a wonderful conference for me. Alas, I’ve been unable to attend many of the track presentations because I’ve been, well, conferring. I’ve had some very engaging and interesting chats, and it’s been great to connect familiar names with their faces. I look forward to many more visits to Europe.

Today’s activities included a conversation, sometimes quite animated, with Stuart Reid. I’ve been surprised at the strength of my emotional reaction to the conversation. It has deepened my already subterranean feelings about the ISEB/ISTQB Foundation Level certification. I think this aspect of the certification business is particularly odious–tantamount to a tax on the vulnerable and the credulous.

If I understood Stuart correctly, I believe that his principal argument was that the ISEB/ISTQB Foundation Level certification demonstrates an engaged interest in the testing profession (uh, applying for, or holding, a testing job indicates an engaged interest in the testing profession, doesn’t it?) and thereby makes it easier for HR departments to filter applicants for testing jobs. (As usual, if I am mischaracterizing the argument, I welcome a correction, and I’ll publish it.) If the poor businesses don’t have the certification to separate the wheat from the chaff, how are they supposed to qualify the applicants when they receive hundreds of resumes?

Well… I suppose that the hiring staff could select applicants that send a resume and a covering letter that is clearly and neatly written, and

• that states and articulates an engaged interest in the testing profession, in the applicant’s own words; or
• that points to relevant life experience; or
• that points to relevant work experience; or
• that highlights relevant education; or
• that points to seemingly irrelevant education or work or life experience, demonstrating a connection that might not be immediately obvious to the hiring manager, yet still intriguing; or
• that points to self-study via books or online courses on testing; or
• that provides a sample one-page test report for an open-source software product; or
• that provides a link to the applicant’s work in the bug-reporting system for an online software product; or
• that answers a set of three or four questions about testing basics provided in the advertisement; or
• that provides direct statements of endorsement from a personal reference; or
• that provides acknowledgment of the requirement for certification along with an argument as to why the hiring manager should consider the candidate anyway, despite the absence of the certification; or
• that places the letters “ISEB” after the candidate’s name, with a footnote that says “I’m Still Employable, Buddy”;
• or that uses any number of attention-getting devices that companies have been using to qualify entry-level applicants forever.

This would take too much work and too much time, was one reply. Too much compared to what? If we’re going to hire someone for a responsible technical position with the company, possibly for years, isn’t it worth a couple of minutes per applicant to select a set that we want to invite to an interview–where the real qualification begins? I don’t see any reason why our profession should shore up an incompetent HR department with a specious certification scheme. If the hiring manager doesn’t know how to hire or how to manage, why provide them with any help at all? (Jeez, why doesn’t the hiring manager go get a management certification?)

In addition, the resume-filtering argument ignores the fact that companies frequently hire testers from within. Testers often come from customer support (I did), from internal support, from development, from administration, or from internship. They come pre-qualified. Testers might also come through personal links and references–“I know someone who’d be great for that job.”

Have a look at this page of frequently asked questions from the Australia/New Zealand Testing Qualifications Board. Ask some more questions, less frequently asked: why is the exam a multiple choice test of forty questions for which there is only one correct answer per question? Why is a passing grade is 62.5% (25 out of 40)? Why are approximately half of the questions at the simplest level, such that getting the simplest questions correct, plus chance levels of achievement on the not-simplest questions is a passing grade? Why does it takes three weeks to mark? Why (especially considering the fee) is there no mechanism for feedback? Where have the ISEB/ISTQB and its affiliated boards managed to spend USD200/AUD300/GBP200 times 30,000 testers worldwide?

Anna Docherty gave a very good, though admittedly preliminary, presentation on some survey research that she did herself on certification (not just the foundation level) and its strengths and weaknesses in the field. The questions that she raised were pointed and pertinent, and she acknowledged that many questions remained unasked or unanswered in her research. In that sense, it was just like a good test report, and Anna immediately qualified herself under my personal certification scheme. I’ll discuss some suggestions for further investigation in a later post.

What truly astonished me today, though, was the number, immediacy, and directness of the statements, even from strong ISEB certification advocates, that the Foundation Level certification is “garbage”, “stupid”, and “useless”–those were their words. Come on guys; walk the talk. Trash it. Or make it free. Then trash it.