DevelopsenseLogo

Testers: Get Out of the Quality Assurance Business

The other day on Twitter, Cory Foy tweeted a challenge: “Having a QA department is a sign of incompetency in your Development department. Discuss.”

Here’s what I think: I’m a tester, and it’s time for our craft to grow up. Whatever the organizational structure of our development shops, it’s time for us testers to get out of the Quality Assurance business.

In the fall of 2008, I was at the Amplifying Your Effectiveness conference (AYE), assisting Fiona Charles and Jerry Weinberg with a session called “Testing Lies”. Jerry was sitting at the front of the room, and as people were milling in and getting settled, I heard him in the middle of a chat with a couple of people sitting close to him. “You’re in quality assurance?” he asked. Yes, came the answer. “So, are you allowed to change the source code for the programs you test?” No, definitely not. “That’s interesting. Then how can you assure quality?”

A good question, and one that immediately reminded me of a conversation more than ten years earlier. In the fall of 1996, Cem Kaner presented his Black Box Software Testing course at Quarterdeck. I was a program manager at the time, but the head of Quality Assurance (that is, testing) had invited me to attend the class. As part of it, Cem led a discussion as to whether the testing group should really be called “Quality Assurance”.

Cem’s stance was that individuals—programmers and testers alike—could certainly assure the quality of their own work, but that testers could not assure the quality of the work of others, and shouldn’t try it. The quality assurance role in the company, Cem said, lay with the management and the CEO (the principal quality officer in the company), since it was they—and certainly not the testers—who had the authority to make decisions about quality.

Over the years he has continued to develop this idea, principally in versions of his presentations and papers on The Ongoing Revolution in Software Testing, but the concept suffuses all of his work. The role for us is not quality assurance; we don’t have control over the schedule, the budget, programmer staffing, product scope, the development model, customer relationships, and so forth. And that’s okay; that’s management work, and that’s management’s job.

When we’re doing our best testing work, we’re providing valuable, timely information about the actual state of the product and the project. We don’t own quality; we’re helping the people who are responsible for quality and the things that influence it. “Quality assistance; that’s what we do.”

More recently, in an interview with Roy Osherove, James Bach also notes that we testers are not gatekeepers of quality. We don’t have responsibility for quality any more than anyone else; everyone on the development team has that responsibility.

When he first became a test manager at Apple Computer way back when, James was energized by the idea that he was the quality gatekeeper. “I came to realize later that this was terribly insulting to everyone else on the team, because the subtle message to everyone else on the team is ‘You guys don’t really care, do you? You don’t care like I do. I’m a tester, that means I care.’ Except the developers are the people who create quality; they make the quality happen. Without the developers, nothing would be there; you’d have zero quality. So it’s quite insulting, and when you insult them like that, they don’t want to work with you.”

Last week, I attended the STAR East conference in Orlando. Many times, I was approached by testers and test managers who asked for my help. They wanted me to advise them on how they could get programmers to adopt “best practices”. They wanted to know how they could influence the managers to do a better job. They wanted to know how to impose one kind of process model or another on the development team. In one session, testers bemoaned the quality of the requirements that they were receiving from the business people (moreover, repeating a common mistake, the testers said “requirements” when they meant requirement documents). In response, one fellow declared that when he got back to work, the testers were going to take over the job of writing the requirements.

In answer to the request for advice, the most helpful reply I can give, by far, is this: these are not the businesses that skilled testers are in.

We are not the brains of the project. That is to say, we don’t control it. Our role is to provide ESP—not extra-sensory perception, but extra sensory perception. We’re extra eyes, ears, fingertips, noses, and taste buds for the programmers and the managers. We’re extensions of their senses. At our best, we’re like extremely sensitive and well-calibrated instruments—microscopes, telescopes, super-sensitive microphones, vernier calipers, mass spectrometers. Bomb-sniffing detectors. (The idea that we are the test instruments comes to me from Cem Kaner.) We help the programmers and the managers to see and hear and otherwise sense things that, in the limited time available to them, and in the mindset that they need to do their work, they might not be able to sense on their own.

It’s not our job to take over anything, like writing the requirements documents. It’s absolutely fine for us to do any kind of work that we’re asked to do. When we take on such work, though, it’s important to be aware and to remind others that our testing work is not getting done.

Listen: if you really want to improve the quality of the code and think that you can, become a programmer. I’ve done that. I can assure you that if you do it too, you’ll quickly find out how challenging and humbling a job it is to be a truly excellent programmer—because like all tools, the computer extends your incompetence as quickly and powerfully as it extends your competence.

If you want to manage the project, become a project manager. I’ve done that too, and I was pretty good at it. But try it, and you’ll soon discover that quality is value to some person or persons who matter, and that your own standards of quality are far less important than those who actually use the product and pay the bills. Become a project manager, and you’ll almost immediately realize that the decision to release a product is informed by technical issues but is ultimately a business decision, balancing the value of the product and bugs—threats to the value of the product—against the costs of not releasing it.

In neither of those two roles would I have found it helpful for someone who was neither a programmer nor a project manager to whine to me that I didn’t have respect for quality; worse still would have been instruction on how to do my job from people who had never done that kind of work. In both of those roles, what I wanted from testers was information.

As a programmer, I wanted to know about problems the testers had found in my code, how they had found them. I wanted their help in identifying risk, and the ways in which the product might not work. I appreciated hearing about the clues that they noticed, and the steps I could take to point me towards finding problems myself.

As a program manager, I wanted to know what information the testers had gathered about the product. I wanted to know how they had configured, operated, observed, and evaluated the product to get that information. If our product were suddenly working differently from the ways in which it had always worked, I wanted to know about that. I wanted to know about inconsistencies within the product. The testers were able to tell me how our product worked in comparison to other products on the market; how it was worse, how it was better. I wanted to know how the product stacked up against claims that we were making for it.

So: you want to have an influence on quality, and on the team. You do that by helping the people who build the product. Want to know how help the programmers in a positive way?

  • Tell the programmers, as James suggests in the interview, that your principal goal is to help them look good—and then start believing it. Your job is not to shame, or to blame, or to be evil. I don’t think we should even joke about it, because it isn’t funny.
  • You’re often the bearer of bad news. Recognize that, and deliver it with compassion and humility.
  • You might be wrong too.  Be skeptical about your own conclusions.
  • Focus on exploring, discovering, investigating, and learning about the product, rather than on confirming things that we already know about it.
  • Report what you’ve learned about the product in a way that identifies its value and threats to that value.
  • Try to understand how the product works on all of the levels you can comprehend, from the highest to the lowest.  Appreciate that it’s complex, so that when you have some harebrained idea about how simple it is to fix a problem, or to find all the bugs in the code, you can take the opportunity to pause and reflect.
  • Express sincere interest in what programmers do, and learn to code if that suits you. At least, learn a little something about how code works and what it does.
  • Don’t ever tell programmers how they should be doing their work. If you actually believe that that’s your role, try a reality check: How do you like it when they do that to you?

Want to know how to help the managers?

  • Provide them with the information they need to make informed decisions, and then let them make the decisions.
  • Remain fully aware that they’re making business decisions, not just technical ones.
  • Know that the product doesn’t necessarily have to hew to your standard of quality.
  • It’s not the development manager’s job, nor anyone else’s, to make you happy. It might be part of their job to help you to be more productive.  Help them understand how to do that.  Particularly draw attention to the fact that…
  • Issues that slow down testing are terribly important, because they allow bugs the opportunity to hide for longer and deeper. So report not only bugs in the product, but issues that slow down testing.
  • If you want to provide information to improve the development process, report on how you’re actually spending your time.
  • Note, as is so commonly the case, why testing is taking so long—how little time you’re spending on actually testing the product, and how much time you’re spending on bug investigation and reporting, setup, meetings, administrivia, and other interruptions to obtaining test coverage.
  • Focus on making these reports accurate (say to the nearest five or ten per cent) rather than precise, because most problems that we have in software development can be seen and solved with first-order measures rather than six-decimal analyses derived from models that are invalid anyway.
  • Show the managers that the majority of problems that you find aren’t exposed by constant, rote repetition of test cases, but by actions and observations that the test cases don’t cover—that is, by your sapient investigation of the product.
  • Help managers and programmers alike to recognize that tools can be used for more, far more, than programming a machine to pound keys.
  • Help everyone to understand that tools extend some kinds of testing and greatly limit others.
  • Help to keep people aware of the difference between testing and checking.  Help them to recognize the value of each, and that excellent checking requires a great deal of testing skill.
  • When you’re talking about your work, emphasise that your job is skilled exploration of the product, and help managers to realize that that’s how we really find problems.
  • Help the team to avoid the trap of thinking of software development as a linear process rather than an organic one.
  • Help managers and programmers to avoid confusing the “testing phase” with what it really is: the fixing phase.
  • When asked to “sign off” on the product, politely offer to report on the testing you’ve done, but leave approval to those whose approval really matters: the product owners.

Want to earn the respect of your team?

  • Be a service to the project, not an obstacle. You’re a provider of information, not a process enforcer.
  • Stop trying to “own” quality, and take as your default assumption that everyone else on the project is at least as concerned about quality as you are.
  • Recognize that your role is to think critically—to help to prevent programmers and managers from being fooled—and that that starts with not being fooled yourself.
  • Diversify your skills, your team, and your tactics. As Karl Weick says, “If you want to understand something complicated, you have to complicate yourself.”
  • As James Bach says, invent testing for yourself. That is, don’t stop at accepting what other people say (including me); field-test it against your own experience, knowledge, and skills. Check in with your community, and see what they’re doing.
  • If there’s something that you can learn that will help to sharpen your knowledge or your senses or your capacity to test, learn it.
  • Study the skills of testing, particularly your critical thinking skills, but also work on systems thinking, scientific thinking, the social life of information, human-computer interaction, data representation, programming, math, measurement…
  • Practice the skills that you need and that you’ve read about. Sharpen your skills by joining the Weekend Testing movement or other experiential testing groups.
  • Get a skilled mentor to help you. If you can’t find one locally, the Internet will provide. Ask for help!
  • Don’t bend to pressure to become a commodity. Certification means that you’re spending money to become indistinguishable from a hundred thousand other people. Make your own mark.
  • Be aware that process models are just that—models—and that all process models—particularly those involving human activities like software development—leave out volumes of detail on what’s really going on.
  • Focus on developing your individual skill set and mindset, so that you can be adaptable to any process model that comes along.
  • Share your copy of Lessons Learned in Software Testing and Perfect Software and Other Illusions About Testing.

Ultimately, if you’re a tester, get out of the quality assurance business. Be the best tester you can possibly be: a skilled investigator, rather than a programmer or project manager wannabe.

Note: every link above points to something that I’ve found to be valuable in developing the thinking and practice of testing. Some I wrote; most I didn’t.  Feast your mind!

Subscribe to the Newsletter

119 replies to “Testers: Get Out of the Quality Assurance Business”

  1. Hi Michael,

    Interestingly, on my last project as performance tester, the performance team was actually part of the development team. We were following a Agile methodology, and our test team attended the daily stand-ups along with all the developers.

    A subsequent post will suggest that we are all developers–programmers, testers, business analysts, DBAs, graphic artists, documentors, program managers, network support folk. Technical support people may also be developers, too.

    This was the first time I had experienced a project that had broken down the formal lines between the two teams. On good testing efforts it is often the case that the two teams work closely together even though there are formal boundaries.

    There was a large functional test team that was separate from the development team on the same project, and this is typical of most testing experiences. I am wondering if the size of the teams determines how the groups interact successfully…

    Social research (and The Mythical Man Month, among many other books) suggest that it does. Thank you for the comment!

    Reply

  2. Thank you for this!
    It is a very good and much more thorough extension to Kaner’s chapter about this in “The ongoing revolution” (which was such a useful tool for us in the “QA”-team at my company around 2003-2004).

    It’s wonderful to receive recognition for this post, but I’d like to join you in emphasizing how much of it has been said before, by Cem, by James, and by Jerry. I can think of no better outcome of this post than for more people to become familiar with their work. (Ambigiuity–at least two interpretations of “their”–is intentional.)

    A small addition to the list about influencing management:
    * Tailor the test strategy so that it meet the objectives of the project stakeholders (and management). This way you create a connection between their expectations and the approach of the test work (which they pay for).
    This then also connects to your point “Provide them with the information they need to make informed decisions, and then let them make the decisions.”

    That’s an excellent addition.

    Reply

  3. Michael. A comprehensive post!

    I have the occasional struggle at my shop about the tester role. But it’s something I see in plenty of places – part entrenchment, part ignorance, part “bad” experience bias etc, etc.

    I’m trying my own brand of test evangelism – here in a post tipping my hat at you, Jerry, but also influenced by Cem Kaner & James Bach.

    I think it’s something that all testers who reject the QA label need to push/advocate in their shops.

    Thanks for the comment, Simon, and thanks for the link to your blog post. What can I say but that I agree!

    Reply

  4. Great article with solid advice. Thanks! I especially like the tips section about communicating with both developers and project managers.

    Thanks for commenting, Chris. Glad you got value from it.

    Reply

  5. All boils down to what I’ve been saying for a few years now. If all you do is Testing then you are a Tester. We are NOT Quality Assurance if all of our work activities are testing centric. QA is so much more than testing alone.

    Yes; we usually call it management (or perhaps self-management).

    Holy crap… something else I’m in agreement with you on. This is getting scary.

    That’s the second time you’ve expressed surprise at agreeing with me. With what did you ever disagree before? Why didn’t we talk about it? As for it getting scary to agree with me… learn to live with it. Learn to love it. [grin/]

    All joking aside, excellent post. There are definitely some links I will follow up on. Even old dogs can learn some new tricks.

    Thanks for the comments. I’m glad you liked it.

    Reply

  6. Michael,

    A great article. I work as a tester in a company where testing was an afterthought, but I rode out the wave until now we are fairly respected and considered necessary by the development team. We’ve had ups and downs, but as I read over your article I notice that our successes have largely come by adherence to the principles you outline. Your post also gives me some great ideas about how to be proactive in the future as well.

    Thanks for the kind words. I hope the ideas continue to be helpful. (Actually, I hope for a day when they’re history.)

    However, we’re still known as “QA”. Do you think the “QA” label automatically brings with it incorrect assumptions from the other teams / management? Also, how do you recommend convincing the rest of the company that we’re the “Testing Department” and not the “QA Department?”

    I think words are powerful forces in setting our models and our expectations. I think the label is pretty toxic.

    I call myself a tester, and I’m perfectly comfortable with that. Knowing what little I know about your situation, I’d encourage you and your colleagues to say “We’d prefer to be called testers. If you’d prefer to call us something else, that’s okay, but we’d be interested to know why you’d want to call us anything other than what we’d like to be called.” But again, I don’t know enough to advise, and even I did, I’m not you. So my recommendation is to follow your own lights.

    Reply

  7. Good reflection of what I think on the Testing vs QA discussion.

    I feel that a lot of the confusion also emanates from the use of the words. I know that if my mother/wife/friend asks what I do I say “Quality Assurance” because that will give them a far better picture of what I do quicker than if I say “Tester” or “Test Analyst”.

    I’m fully aware that I am actually saying the wrong thing but the picture in their minds for Quality Assurance is that of a Tester and that of a Tester is …..well…. mostly… nothing. So you end up explaining and at the end they say “Well like you’re doing Quality Assurance?”. That’s the part where you say “OK” and give up.

    Give up? Is it really that exhausting to explain what you do? You know how do handle this. You deal patiently with programmers who don’t understand the problem you’re reporting, right? Next time, try this: “I’m a tester. I learn about the product, and I search for problems in it. Some people call that “quality assurance”, but that’s a mistake. The people who have authority assure quality, and I help them out.” If your family doesn’t have time for that explanation (or you don’t have time to provide it), there are probably more serious issues to address than what you do for work.

    The thing is that this is OK for mother/wife/friends but not for developers, project managers, BAs, sales people, CFOs, CIOs,… but many of them see it the same way.

    You know, since it’s your mission and your relationship to your client that’s at stake here, I might be inclined to invest another few well-chosen sentences.

    They loosely talk about QA and think Testing or think Testing and sub-surmise it contains QA. But because the terms are used so differently world wide (which in my opinion is just wrong use of the terms) there is a entrenched misunderstanding of the areas of competence.

    You can do something about that. We all can. As Jerry Weinberg says, a tester is someone who knows that things can be different.

    QA is in my opinion as far removed from Testing as Business Analysis is. Yes, both have areas where the sort-of overlap or border on each other, help each other out but they are de-facto separate.

    And I wish the term QA would completely vanish from the Testing industry. It would make things much clearer and simpler for everyone.

    Again, you can help. I want to encourage you to do that. The revolution starts with us. Here. Now.

    Reply

  8. Michael – thanks for making a tweet into a wonderful resource and statement of the testing craft. Your points hit exactly many of the thoughts that were in my head when I made that tweet, and I’m thrilled to see it here.

    Cory

    Thanks, Cory. This post has been building up for a while, but you dropped the key glove. 🙂

    Reply

  9. Thanks Michael – For yet another classic like Checking Vs Testing. This one link is going to serve as the most beautiful and excellent source of information for any software tester.Any tester can start his journey of learning here
    –Dhanasekar S

    Thank you, Dhanasekar. It’s particularly important to me that people in India—and their managers—get this message.

    Reply

  10. Wow…and Thank You!

    You’re welcome, and thank YOU.

    Having had the distinct pleasure of sharing thoughtful and insightful discussions with you, I am delighted to have this concise, cogent and neatly packaged “brain dump”. Your passion and conviction is readily apparent throughout.

    I have struggled in the past to articulate some of the points you have shared so clearly and concisely.

    So have I.

    I look forward to referencing this blog countless times in the future, much in the same way I have used “Testing vs Checking”.

    Thanks again to you, Jerry, James, Cem and all the testers who participate in the “testing revolution”. Count me in!

    Cheers,
    Lynn

    Thanks again. I hope people find the post helpful. I’m looking forward to seeing you all in Calgary in June!

    Reply

  11. This is very interesting and also very practical, and I’m glad to say I recognize most points you address. I would like to add that with both developers and managers you should always discuss issues with the _product_: never play it personal.

    About the name game: I work at a QA company where we call the testers quality analysts to show that we are more than test dummies.
    In my current (scrum) project we are called test analysts to signify that our primary target is testing but also play an active role in designing, development and relations with the users.

    Thank you for adding to the conversation. “Quality analyst” certainly seems more accurate to me than “quality assurance”. To show that you’re more than test dummies, though, my ideal strategy wouldn’t be to change the label. It would be to show that you’re not test dummies. I expect that you’re doing that, and that the actions speak louder than the words.

    Reply

  12. Hi Michael,

    Thanks for the well constructed article, it’s nice and refreshing to read something that’s not been rehashed a million times 🙂

    For me this one line sums up my whole approach to managing testers and testing: Be a service to the project, not an obstacle. You’re a provider of information, not a process enforcer.

    Thanks for the comment, Steve. I agree with me too. 🙂

    Reply

  13. I agree that my role within projects is better defined by the term ‘Tester’ or ‘Test analyst’ than by the term ‘Quality Assurance analyst’. But QA is the label used within the industry. If one is seeking work as a Tester, one’s resume needs to include the words Quality Assurance to meet the search criteria used by many / most companies. I refer to myself as a Tester, but if I want to be considered for relevant contracts, I have to keep QA on my resume…

    Thanks for the comment. I know what you mean. I think I still include “quality assurance” in the meta tags in my blog (too lazy to look at the moment), but I’m hoping that’s a temporary condition. Mind you, there’s nothing wrong with adding keywords that refer to things that you want to make disappear. I expect there are doctors that include “cancer” or “scabies” in their meta tags. There are scary implications for the fact that searches are mediated by search engines, and that people often search for things different than what they’re actually looking for.

    We might be able to help the process of conversion along by persuading hiring managers to seek testers, rather than QA analysts.

    We might also use Cem’s trick: “QA? Oh… I thought you meant that stood for quality assistance. That’s what I do.”

    Finally, I wonder about “have to”. We don’t have to do anything, do we? For example, if an employer insists on putting me in “quality assurance” rather than testing, that’s information that tells me something about whether I want to work there.

    Reply

  14. Thanks for the insight. I can make up whatever title for my position I want (small company) and have struggled for years to come up with something that covers my far ranging responsibilities without having ‘QA’ or ‘Engineer’ in it. More and more I find myself falling back on ‘Software Tester’.
    I think my favorite suggested title so far is one I learned from Scott Barber at a conference a couple of years ago, which is ‘Quality Forecaster’. I haven’t had the guts to make it my official title yet, though 🙂

    Hi, Allen. Thanks for the comment.

    Hmmm. I like “software tester”, obviously. Much as I have respect for Scott, though, “Quality Forecaster” worries me a little. Think of “Weather Forecaster” or “Financial Forecaster” (post-2008). I’d like to stick with what can be observed and described now. How about “Investigative Software Reporter”?

    On second thought: Nah. Software Tester says it all.

    Reply

  15. Great article — well put. So many participants in the development process have preconceived notions of what success means and its mostly selfish and hinders the larger goal. Thanks for breaking it down in a clear, concise and impactful way. Still, I wonder where QA does belong. Clearly development and product owners have a role but who (or what) has the wherewithal, energy and skill to make it practically happen? Does reality require a person or organization to own quality to improve it? Or can it truly be distributed across every member? If you want security, you often need a security expert. If you want scalability usually the architects own that.

    Michael replies: Thanks for the comment, and for the opportunity to emphasize this:

    There are at least couple of kinds of quality assurance, as I point out above (apparently not clearly enough). There’s assurance of the quality of your own work. That’s something that individual programmers, graphic artists, user interaction designers, documentors, localizers, and the like (as you suggest, including security experts and architects) put into the product directly. The people who support those people—testers, build and configuration management, internal support, customer support, human resources, adminstrative staff, marketers, salespeople, and so forth (sorry to the groups I’ve left out)—can assure the quality of their own work too. They contribute to the product indirectly, or if you like, they contribute to the greater system of the product. Then there’s the overall responsibility for the quality of the greater product. That kind of quality assurance lives with the management functions—product owners, development management, other line managers, and up the chain to the CEO. Those are also people who don’t put quality into the product directly, but make all the decisions that make a quality product possible.

    On top of all that, I suppose (as my colleague Markus Gaertner suggested in private correspondence) I’mg going to have to do a post on the idea of quality as a relationship. More work to do!

    Reply

  16. @Michael,
    One word for this blog post *Rocking*.

    Thanks,
    Santhosh Shivanand Tuppadn m

    Thanks, Santosh. I had my Indian colleagues very strongly in mind when I wrote the post, and I’d be delighted if it were circulated widely there.

    Reply

  17. Michael – thanks for making a tweet into a wonderful resource and statement of the testing craft.

    Thanks for the thanks. And you’re welcome.

    Reply

  18. Hi Michael,

    Excellent article. It perfectly describes everything i am advocating here in the Netherlands. A lot of people here are still focussing on QA. Overhere testers are still held responsible for the problems caused by a bug when occured in production. When a bug is found, it’s always “badly tested”. We still need to do some advocating here 🙂

    I do would like to comment on one line in your article:
    “We don’t have responsibility for quality any more than anyone else; everyone on the development team has that responsibility”

    Maybe it should be better to address the difference between “feeling responsible” and “being responsible”. In my opinion everyone in the development team should FEEL responsible for the quality of the product. They ARE responsible to use all of there professional and personal abilities to add quality to the product. So a developer or tester can be held responsible for doing a crappy job in programming or testing. But in the end only one person should be held accountible (and therefore be held responsible) for the quality of the product. To me, that person is the contractor. In (a traditional) project most likely the project manager. It’s his responsibility to assure the quality of the product. It’s his responsibility to motivate others to feel responsible for the quality of the product. A project members can be held accountably for not using all of his/her professional or personal capability as a programmer or a tester to acchieve the goals set by the contractor.

    Yes, I agree. Individuals have control over the quality of their own work (and should exercise that control), but ultimately ownership of the quality of the entire product lies with the product owner.

    Reply

  19. Thanks for writing, David.

    Posted on http://afterhourtesting.blogspot.com/ January 6, 2010

    The myth of Software Quality Assurance

    I have the opportunity to interview every so often, and I like finding out what the candidate knows about the work they do. For this reason I almost always ask the candidate to tell me the difference between Quality Assurance (QA) and Quality Control (QC). At first I was astounded at the responses. I felt as though, I had asked the candidate to list all the heads of states for Zimbabwe since its inception. Now these candidates are usually those with many years of experience, you know 10 and considered senior testers. I felt safe asking because, I thought if you are working in an industry for more than a couple of years, you should know something basic about your job; sadly I was wrong.

    I have a different perspective: I think that the difference between quality assurance and quality control (as roles) is irrelevant to software testing, for all the reasons that I talk about above. That’s not to say that the quality part is irrelevant, but that the assurance and control parts are, since we neither assure nor control quality.

    I have a good friend who tells me that I am anal about the difference between QA & QC and she is probably right. But in interviewing these candidates I have found that there are unlimited views on the differences. I have been told that QA – “is following scenarios assuring quality”; “measuring the amount of quality”; “that it is analysis or detailed or in-depth validation of software”; “the job is to find defects”.

    Those experiences should give you a clue as to the actual relevance of the distinction.

    For QC it goes something like this – “there’s no difference”; “where faults are found in acceptable ranges”; “it’s an evaluation against standard”; “that QC is more comprehensive than QA”; “it is the job of engineering, development, and design”; “that quality is the composition for a product line”; “that QC is the verification of the end product”; “it is the flow through the application without errors”.

    It is amazing that so many of us in the software testing business don’t know what we do. Are we QA or are we QC? Many testers junior or senior do not know.

    I know: I am neither.

    I believe it because the majority of companies we work for do not understand what we do as well as they should. Consequently testers are placed in a department titled QA, and the testers test the software. Go to any Software Quality Assurance (SQA) group and what is the main topic of their conversations it is testing, improving testing, automated testing, manual testing, etc.

    Yes, I have a point. My point is that QA is not testing. QA is the assurance of quality and QC is the control of quality. Let me explain, I believe, as many of you, that you cannot test quality into any product. Quality comes when there are processes which are followed and the process are reviewed, followed by some analysis on the process to correct any flaws found in them, and then the flawed process is modified to produce a better product. This is QA; it is a short but simple answer. QA has nothing to do with testing. On the other hand, testing is QC. QC is the validation of the product to a set of specifications, requirements, or standards. The validation is accomplished through the action of testing the product against the requirements. We all work for the QA department yet we do QC work and we perpetuate the myth by calling ourselves “QA”. I have been corrected many times by testers which state very forcefully that “we are QA”.

    I agree with you that they’re mistaken in that impression. Yet there’s more. If you look at your own experience, you will soon see that testing is much more than validation of the product against requirements. In the Rapid Software Testing course, James Bach and I suggest that to test a product means “trying it to learn sufficiently everything that matters about how it can work and how it might not work”. This is not merely about validation to requirements (which, by the way, are typically mistaken for requirements documents); it is about exploring, discovering, investigating, and learning. Conformance to requirements, both explicit and implicit, is important, to be sure. However, testing is also important in discovering requirements, in refining requirements, in questioning requirements. All through the project, we’re finding that what we thought we knew at the beginning is different—often radically different—from what we thought we knew and what we thought we understood. Excellent testing helps to reveal that. “Validation”, I think, doesn’t—or does it very weakly.

    My stand is that we who test any software product are actually in the business of QC – quality control; in that we focus on controlling the quality of the product, once it is delivered to us. We are not QA, for we assure nothing.

    I agree; and I feel the same way about control.

    We just validate the requirements have been met and report any deviation in the product from them.

    Again, I hope that you can look at your experience and see that you actually do much more than that. Also: beware of that word “just”.

    Reply

  20. Hey Michael,

    Hey, Herb… thanks for writing.

    I don’t know how you can so easily separate the people who are filtering, finding and giving the information from the people who are using the information.

    Sure, the President is ultimately responsible for taking action, but if the CIA gives him filtered information that there are weapons of mass destruction, shouldn’t they get a fair bit of flak as well? The President can’t be expected to know everything the CIA knows, that’s why he has a CIA.

    Yes. And that’s also exactly why the CIA shouldn’t set policy—because there’s a President who’s responsible for that. As I’ve said, it’s typically a bad thing when the eyes start running the brain instead of informing it.

    I agree with you that Testers/Test Managers shouldn’t be the *only* people that are responsible for product quality, but they are responsible for it. We (testers/Test managers) are the ones providing the information. We, with input, are deciding what features and functions to test and how much time and effort to spend on these features. This has a direct impact on product quality because the decision makers are already basing their decisions upon a filter created by testing, albiet with others’ input as well.

    That’s right. And this is the sense in which testers are responsibility for the quality of their own work, as are the programmers, the designers, the documentors, and so on. But let’s not confuse the quality of the product overall with the quality of the work that went into it. I can assure the quality of my own work, but I can’t assure the quality over work for which I am not respsonsible—the work of others, and the product overall.

    If you believe that we also shouldn’t be deciding what to test, or how much time or effort goes in, and that decision is also made by someone else, then I would agree with you.

    I believe that to a great degree we should be deciding what to test, and that the decision should be reached in collaboration with our clients and our stakeholders. Again, we’re in a service position. Our clients know some things, and we know others, and exchanging that knowledge is often illumiinating. Consider the restaurant analogy: as your waiter, I’m certainly going to bring you what you ask for, but I’m also going to work with you to determine that that’s what you really want, based on things that I know about that you might need to know. As a customer, I appreciate this collaboration. I hear about the Death by Chocolate dessert and order it, and yet I appreciate it when the waiter thinks to mention that there are hazelnuts in it; I’m allergic to them, and I sometimes forget to ask.

    In a testing question, one of the first things for a tester to ask is, “Is it okay if I ask questions?” If it’s not okay, you can proceed or not as you wish, but if you do proceed, it’s important to make the client aware of the associated risk. I get the sense that, in a recent Administration, the client wasn’t interested in having too many questions asked.

    Testers are the pre-users, they had better have an opinion of quality because users will, and it should show on what they advocate for, what they test, and how they can *help* to improve the quality of the product by influencing others.

    It’s my sincere hope that my information will be helpful in getting my client what s/he wants. I worry about my influence when it diverges from what my client wants. A belief that I’m responsible for the quality of the product in a way that usurps my client’s responsibility is the risky area here.

    Reply

  21. Michael,

    I had another thought on this after hearing some musings on why the label “Quality Assurance” even exists. If we look at the origins and understand the basis for QA and QC in manufacturing there is merit to the concept of assuring quality. The disconnect occurs where you transition the same philosophies from the mass production of “widgets” – weaponry, car parts, match sticks – too the development of software. Understanding how we arrived at the current state of despair is one level of consciousness, realizing we are able to move forward and define a brighter, better future for software testing is the next. We all need to remember the quote “A tester is somehow who knows things can be different.” by Jerry Weinberg. Let’s be mindful to push the boundaries and limitations around software testing, many of which we actually self impose.

    – Lynn

    Very nicely put; thanks.

    Reply

  22. […] My colleague and friend Michael Bolton has been cranking out the blog posts – most recently Testers: Get Out Of The Quality Assurance Business and When Testers Are Asked for a Ship/NoShip Decision. Both of these articles stand on their own, […]

    Reply

  23. This is a fantastic article and discussion, and I really appreciate the advice and discussion points.

    I’ve pondered the QA vs QC aspects since my starts in the Software Industry.

    I seem to be in the minority here, but I’ve felt a sense of pride in calling my work teams ‘Quality Assurance.’ In recent organizations where I’ve worked, there’s been a level of credence or prestige given to an organization that recognizes Quality as an institution and sees it as important enough to have a department who focuses solely on measuring and ‘assuring’ an appropriate level of quality in products we ship.

    My daughter is five years old. She was saying something with “air quotes”, using her fingers as she was speaking. I asked her what that meant. She said “That means ‘doesn’t mean'”. I notice that you put ‘assuring’ in quotes above. So I have a question for you: if my daughter is right, why are you saying something you don’t really mean?

    Now, I really don’t want to ask that in a nasty way. I really don’t. I am saying, though, if you know the notion is a fiction (as the quotes suggest), why try to perpetuate it?

    That said, I will also respectfully disagree with you, though: if the organization really believes in quality why should there be a separate group for it? It’s like having a separate Ethics Department, or a Virtue Division. I’d prefer to work in an organization where virtue, ethics, and quality were considered essential aspects of every department, and were incorporated in every person in the organization.

    I in no wise aspire to own product quality 100% for my current company, and all executives regularly communicate that every employee is responsible for recognizing and contributing top quality. But, as a QA Director, I push my team to be top experts in our industry, our client’s needs, and in our software. We promote and mentor quality, in a cooperative manner, with sales, analysts, developers, and customer support staff, to regularly assure quality at every step of our lifecycle, where we can have a positive influence (but not in a forceful or policing manner).

    Testing is often our first job responsibility, but in these ways, and by following many of the suggestions posed in the article above, I am of the opinion staff who primarily test software have opportunity to be a positive influence that does ‘assure’ quality — and there is great intrinsic value in having a company that has a group designated as ‘QA.’

    There are those quotes again. But hey: you and your company do get to choose your own road, and your own title. I hope that the rest of the company finds your contribution helpful. I don’t know your business, and I’m certainly not a policeman. But if my business is testing and my department name is incongruent with my role, the first blow that I strike for quality is to align my business and my (named) role.

    All the best!

    Reply

  24. Hello Michael,

    I’ve been following your blog for a while, and I want to thank you for putting this post together. There are resources here that I was unaware of and new ideas that I can use as I try to influence my org.

    – Federico

    Thanks, Federico. I love that you and Matt are doing the CodingQA podcasts. Wanna chat sometime? 🙂

    Reply

  25. Since the early days of my career I have known that the term ‘QA’ did not make any sense when used as synonymously with Testing. I was lucky enough to work at companies where they also understood this. I recently started a new job, and am in a situation where the Test groups is called the ‘QA’ group. I have done what I can thus far to attempt to begin dispelling the common misconceptions of what testing is and is not, but I do not like the fact that the department is called ‘QA’. Do you have any suggestions/insight into how I can get the name of the department changed?

    There are many mailing groups and processes in place where the ‘QA’ term is used, for example in JIRA, our Wiki, mailing lists, etc. so there is work associated with changing the name. I know why it is important to me for it to be changed, but I am not sure how I am going to sway the decision makers.

    They probably don’t care. If they did care, they would realize that they, not you, are the quality assurance department. Try putting up a big sign at your desk that says “Quality Assistance”. Get some t-shirts made up that say “Questions Answered”. Get a button maker to make “Tester” buttons, and hand them out to your colleagues. Drop a copy of my post onto some managers’ desks. Get a copy of “The Ongoing Revolution in Software Testing” passed around if you really want to get an energetic conversation started.

    Any advice would be appreciated,

    Thanks,
    Steve

    Questions are appreciated. Hope this helps.

    Reply

  26. Michael,

    an interesting exchange of tweets just now. Thought I’d come and comment on the article you pointed me to just now although the comments are as much about the tweets as the article.

    First off I’m a great believer in specialist skills even in agile teams. Yes you want to have a bunch of multi-skilled people and yes, no one person should be saddled with sole responsibility for any aspect of the system or the process. But the fact remains that some things are best done by people who have particular expertise in that area and their job is to help other people on the team appreciate, understand and respect that area. The kinds of things I’m talking about: ‘Architecture’, User Experience, Database Admin, Deployment, and Testing. Your point about everyone contributing to development and everyone being responsible for quality is spot on … but that doesn’t mean we, as a team, can’t look to certain people to take the lead in certain areas.

    I don’t think I ever suggested otherwise. If you see that, let me know and I’ll try to fix it.

    I’ve always balked at calling what I’d call ‘QA people’ testers. It feels a bit dismissive … just like calling developers ‘coders’.

    You mean like calling programmers “coders”, right? Where I come from, all the people you mention above are developers. Meanwhile, for those who want to be dismissive of testers, you could call them “script monkeys” or some such. But to me, “tester” captures it perfectly. Calling testers “quality assurance” is wrong, and it’s a form of title inflation. Now, if you want to inflate our titles, you could do that do. In that case, you could call us “epistemological evaluators” or “empirical skeptics”. But again, tester is fine with me.

    Ultimately people can be called whatever they want but Quality Assurance is actually what I’m looking for in people who specialise in testing. That isn’t to say that they are the guarantors of bug-free code, or that they are held accountable for issues with the system (the whole team takes responsibility for that), but rather that they assure in the British (aka ‘correct’ ;)) definition of the word: insurance against things that are bound to happen at some point. Life Assurance is one of the oldest financial products in London … it doesn’t guarantee life or contribute to the presence of life, it provides a safety net that does its best to guard against the consequences of death.

    You do see how testers don’t do that, right? Not even insurance companies guard against the consequence of death; they simply cough up money when it happens, for the beneficiary to do with it what he will. We don’t provide insurance. That is, we don’t offer “the equitable transfer of the risk of a loss, from one entity to another, in exchange for payment”. It’s not a metaphor that works for me. We provide information. If you want a metaphor, think “investigative reporter”, “crime scene investigator”. Perhaps you might think “minesweeper” (“sapper”, in the UK), except we don’t particularly suffer anything like the same degree of consequences when we make a mistake.

    Enough of the metaphors; what do I mean in terms of QA for software? First off, in a TDD process, this isn’t ‘just’ about finding bugs. When bugs are inevitably found its about helping the developers understand the gaps in their unit and functional tests. Sometimes QA is about code reviewing or pairing those tests – providing specialist skills and experience to help the developers write better tests (skilled developers are adept and making things work, skilled QA people are adept at showing how they don’t). QA is about representing the view of the user; sometimes explaining to the product owner and the devs that they might think the system is ‘good enough’ but actually it just isn’t. And its about the whole team saying to their users and customers that we care enough about what we do to be prepared to be told, by one of our own, that what we’ve worked hard on just isn’t good enough.

    It is, to me, presumptuous and insulting to other people on the project to suggest that testers have some special kinship with the user; that we speak for the user and that, implicity, others do not. And as a tester, I’m not an arbiter of “good enough”, because I don’t run the project. What I might do is to say “this problem appears to threaten the value of the product overall because there’s a weakness with respect to this quality criterion”. But after that, the programmers and the product owner can do with the information what they will.

    This is what I look for in a ‘QA person’ and anyone who said “oh, I don’t do any of that, I’m just a tester – I write scripts and I perform manual test cases” isn’t likely to be invited to be part of the team.

    That’s not what I say. I say that “I provide the team with information about how the product actually works, not how we wish or hope it works. I do that by modeling the test space, by identifying quality criteria and oracles, by identifying product elements and coverage, by choosing techniques (which may or may not be assisted by automation). I configure, operate, observe, and evaluate the product; I decide when I’ve obtained enough information to be useful, and I convey that information to you. That information enables you, Dear Programmers and Dear Product Owners, to make informed decisions about the product. As above, you have the authority to change the code if you’re a programmer. If you’re a product owner, you have the authority to determine the scope of the product, to set the schedule, determine the staffing, set contractual obligations with customers, respond to market conditions, allocate the budget; and you also have much more business information than I do. I encourage you to run the project, and in order to assist you, I’ll give you the best information that you don’t yet have. I’ll light the way, but you’re in charge. You set the course and steer the ship.” Or would you rather have me telling you what you should be doing? Wouldn’t you find that a triflle… annoying? Presumptuous? Insulting?

    Inevitably the first thing I read in your post was the line “Having a QA department is a sign of incompetency in your Development department.” Totally agree. But I’m not talking about departments here I’m talking about individuals. In a long-term, large-scale agile project I ran one of our first actions was to take a team of 5 testers out of the QA department and give them a first-class role in the team, each one assigned to a sub-team of 8-10 devs (it was a too-large-scale project when we inherited it). It took a while but one of them commented after some time: “I actually feel like I’m contributing to quality now rather than just telling the developers how shit the system is”. That’s what ‘QA’ means to me.

    We testers do contribute to quality in the sense that we provide information that informs it. Maybe the difference in your current team is that the programmers and product owners listen more carefully to that information. And maybe that happened because the testers realized that if you’re telling programmers how shit the system is, you’re inflicting your vision of quality on them instead of providing, dispassionately, the factual information that the programmers really need to do their jobs.

    Reply

  30. Hi Michael,

    Thank you for writing such a detailed and thought-provoking post.

    Several years ago I thought that QA and testing were synonymous terms but about four or five years back I suddenly realised that the tester’s job was to inform others to help them make that critical ship/don’t ship decision.

    What gave me this understanding was the discovery that although I felt uncomfortable about shipping the product, others were prepared to accept the risks because they had access to different business reports than me; reports that I have no need to access but that were critical to the decision.

    I learned then that generally speaking a decision to release or not is a complex business decision and should be left in the hands of people with the business and commercial knowledge to make that decision. In many organisations testers are unlikely to be in possession of that knowledge.

    Thank you again.

    Reply

  31. […] first time presenting this session since crafting this enlightening blog post by the same name,“Testers: Get Out of the Quality Assurance Business”. There has been a fair amount of conversation generated on Michael’s blog and Twitter since […]

    Reply

  32. Who started mixing up these entirely different organisational bodies?

    Well, you’re the historian. 🙂 (For those who don’t know, Joris has developed a list of milestones in testing; http://www.testingreferences.com/testinghistory.php)

    Software testers test the software product, QA tries to control the software development process. Product vs process, clear distinction, different goals, different mindset, end of discussion.

    With respect to testing, what does it mean to you to test the product? Many people and organizations are confused about this. For example, in some organizations, there are stages in developing a product, including a “development phase” and a “testing phase”. What are the programmers doing during the “development phase”? Developing the product. What are the testers doing? They’re testing. In the “testing phase”, what are the programmers doing? They’re fixing the product. What are the testers doing? Testing. So it seems to me, the two phases aren’t differentiated by testing; they’re differentiated by developing vs. fixing. This suggests to me that at some point, people in that organization have conflated the ideas of testing and fixing. In other places, testing is seen as a bureaucratic, clerical comp In order for people to clear the fog in their heads about testing, we’re going to have to talk about stuff like this. I don’t think we’re anywhere near the end of the discussion.

    With respect to “QA tries to control the software development process”… well, you can see the problems there, right? In a development organization, there’s already a role that’s responsible for managing the project. We usually call that “management”. Most programmers I’ve met don’t like being controlled at all, although they might tolerate being managed by their managers. When a third party (typically a tester, typically someone who has worked neither as a programmer nor as a manager) tries to control the project or tries to impose process, that person is either taking on or usurping a role that properly belongs to management. Such people—those who have neither the experience nor the authority to control the process—are usually tolerated and ignored at best. At worst, they’re considered “damage to be routed around” (I think Cem Kaner said that once, echoing someone’s description of censorship on the Internet), and routed around they will be. That’s pernicious if you’re also a tester, because it’s unlikely that you’ll want to do anything to disrupt the flow of information to you and around you.

    In any case, as long as testers are called “QA” (or somewhat worse, call themselves “QA”) elements of confusion will remain.

    Reply

  33. Excellent post, but there is a small defect there: The one link that I clicked, in order to read more, is broken – gave me a 404. 🙂

    Here’s the line with the broken link:

    * Don’t bend to pressure to become a commodity. Certification means that you’re spending money to become indistinguishable from a hundred thousand other people. Make your own mark.

    Thank you very much; fixed.

    Reply

  34. I was software tester for at least 3 years with only vaguely being aware of the term Quality Assurance as equivalent term for software testing. Similarly so, years ago I was only vaguely aware of the term developer which is today most commonly applied to people who do software programming.

    Yes, it is. And yet isn’t someone who is in the business of analyzing requirements and providing requirements documents to the team also a developer? Isn’t someone who develops designs and graphics for the look and feel of the program also a developer? By that measure, isn’t a tester a developer too?

    By the same token, isn’t someone who writes code in quality assurance? Someone who provides equipment and tools to the programmers, or who removes obstacles for the programmer—isn’t that person in quality assurance? Are the aforementioned graphics people in quality assurance?

    Testers are certainly not the only people in the development group who are doing activities that amount to quality assurance on some level. Indeed, since testers don’t have authority over the code, the project, or the project, testers don’t assure quality. Our role is to help the programmers and product owners—the people who can assure quality.

    By the same token, programmers aren’t the only people who are doing development work. Testers are developing knowledge and information about the product. While we’re not adding things to the product directly, we’re certainly adding things to the system of the product.

    So, irrespective of what else happens, it would be well (in my view) for testers and everyone else alike not to grab a title or a role with the subtle suggestion that no one else is performing that role.

    I think both terms “quality assurance” and “development” got into the play because “tester” and “programmer” felt inadequate.

    Perhaps. Inadequate to whom? For what purpose?

    Programmers are not just programming but developing solutions. At least in companies who cannot afford to have people who just irresponsibly write code. Testers are not just testing (this especially sounds offending if testing is translated as checking), but finding and disclosing valuable quality related information. At least in companies that cannot afford to have ceremonial testers.

    Since customers, stakeholders, CEOs (generally, the one who pay money) are interested mostly in quality solutions it’s not strange that those “new” terms gain popularity not only because those are cool terms. However, while we can agree that (at least some) developers are indeed developing, seems there is no agreement that testers are indeed assuring quality. Especially there are lot unwillingness to go with the term “assurance” as in this article.

    That unwillingness is, to me, quite reasonable because testers are not assuring quality. They are providing information to those who can. Again, it’s an issue of who has the authority to make decisions about the product and the project. We don’t have that authority, nor should we—no more than the suit salesman has the authority to tell you what you’ll be buying today.

    But, what is with “quality” itself? Seems that the understanding of the term “quality” went through great deal of changes. We certainly do not speak about Aristotelian attribute or property of a being. More likely term aims to “best quality” as some hypothetical goal. But, what is “best quality?” Is this determined by subjective feeling of CEO or objective fact? Who is to determine the objective fact?

    Why do you believe that a determination of objective fact is desirable when no one is obliged to consider it a fact? We don’t make decisions based on rationality alone; indeed, rationality itself is greatly determined by what we value. Aristotle himself talks about this with respect to the measurement problem in The Nicomachaean Ethics. There are measurements of one kind (“two pounds of meat”), but there are also measurements of “too much, too little, or just right”, he says. Measurements of the first kind can inform decisions of the second, but when we determine whether we’ve eaten too much, that’s a feeling, not a fact. In addition, that fact is laden with many dimensions of what we might value. On the face of it, half a scoop of ice cream doesn’t sound like too much, does it? Yet, half a scoop of ice cream might seem like too much when I’m on a diet or if I’m intolerant of lactose. Half a scoop of ice cream might seem too much to me when I take it all and my daughter doesn’t get any. Half a scoop of ice cream might be too much when it’s on my shirt, instead of in a bowl. Quality can’t be evaluated outside a context, and people are the most important factor in any context.

    Quality is value to some person. It is futile to talk about (or, for that matter, to even attempt to assure) quality without considering it a relationship between someone and something. For that reason, it is absolutely the CEO’s subjective feeling (informed by what (s)he considers to be matters of fact) that determines the decisions that (s)he makes and delegates.

    Thank you for continuing the conversation.

    Reply

  35. […] to James’ challenge on another post. So now I have a challenge for you: what do you think of this post versus yours (that is, the one upon which I’m […]

    Alas, now a broken link. —MB

    Reply

  40. […] Read this post on developsense.com. This entry was written by Brian, posted on 10 May 2010 at 7:00 am, filed under Curated, Software testing. Bookmark the permalink. Follow any comments here with the RSS feed for this post. Post a comment or leave a trackback: Trackback URL. « Do it or document it? The importance of being independent » […]

    Reply

  42. […] is to try and help close down those communication gaps by pointing them out to people. Is that quality assistance? Tagged as: criticism, James Bach, marketing, testing Leave a comment Comments (0) Trackbacks […]

    Reply

  43. […] 1. Software Testing is not synonymous with Quality Assurance […]

    Reply

  47. I really enjoyed reading this, it clarified a lot of what I was feeling but had trouble articulating.
    To take it a step further, at my company the use of “QA” as interchangeable with “testing” is so bad that sentences such as, “who will be QAing?” is used without thought.

    Michael replies: Yes. Treating language that way should be banned under the Geneva Conventions.

    Reading your article has motivated me to get this put right!

    Thanks!

    Reply

  52. I think Quality Assurance may be a good name to a team in some cases, at my company the testing team have different responsibilities besides testing software such as document analysis, questioning development processes, software-related legal issues delegated to software, software-related political issues, etc… What name would you think for a department like this one?

    I understand you could think that the main problem here is the responsibilities taken by the testers but let’s face it, if it is what it is, and you only can switch the name, which would it be?

    Michael replies: Let’s make it clear that I can’t—and I shouldn’t—enforce orthodoxy, so this is an opinion, not a statement of dogma or best practice.

    You could call the testing group “quality assistance”, or “quality awareness”, if you want to keep the QA abbreviation. Consider that programmers do all of the things that you mention above, at least to some degree. Programmers do more than programming, but we still call them “programmers”. So it seems to me that “testers” could do all that stuff, and you could still call them testers. Why not call them “testers”?

    The point of the post is not really about the label; it’s about the idea. Whatever else testers might do, they don’t assure quality. They do provide information to the people who DO assure quality.

    Reply

  54. From my personal experience what developers tend to appreciate most of all it’s the fact that even when you’re a QA you can talk with them at the same level about the code, I use to be a former programmer a really good one, but because of a bad experience in a previous job (lack of creative ways or innovation paths to develop new features due to manager restrictions) I decided to switch to QA and what I have found it’s that most of the teams ussually start the project giving me the evil eye “grrrr that qa guy I don’t like him…” but when they start seeing me on the code reviews exposing my personal opinions on ***what are the best ways to implement something from a QA perspective***, they see the tools that I program to automate the several applications, or even libraries for commercial or open source automation/performance solutions and last but most importante they see a change on how I report the bugs that I found, I can’t stand out this enough but every QA should know where in the code is the defect that they just found, so you can report in an accurate way to the programmers and minimize the resolving time a great deal, it’s not the same when you say “login is failing when I type blah blah blah” instead of saying “the XXXX field on this function/method/class is throwing THIS exception when I give this input/s”.
    Then you have their respect and they’ll treat you as an equal that is helping them to develop a better software, and ofcourse you’ll have a direct impact in the project schedule and quality being a member that any team will want on their line up.
    Ofcourse is a must to be polite and always treat the code that those developers have spent hours of creative thinking to create with the respect that deserves as stated on this article.
    I disagree with the idea that if you know how to code and have opinions about the source of your current project you should move to the dev team, I’m more of the opinion that the days of the black box testing should be done and buried QAs need to know the code that they’re testing and the techonolgies that are being used to create it in order to provide even more information to the managers that will make the call to deliver the product or not.

    Michael replies: I agree that it’s a good idea for some testers, at least, to understand technology at the code level. And it’s a good idea for testers to understand the technologies that they’re testing. I’ll go further: often testers develop knowledge about the technologies that are being developed. But I don’t agree that “the days of black box testing should be dead and buried”. I fine with not seeing the code of things that I’m testing. It affords a different perspective that can be especially valuable when everyone else is focused on the code. If you believe that every tester must know software on a coding level, you’d have to agree that every tester should also understand design, usability, security, performance, diagramming, systems thinking, critical thinking, all domain knowledge, and should have a complete grasp of how to write grammatically correct English sentences. Otherwise you’d you’d have to explain why those things aren’t important. As alternative, perhaps you’d acknowledge that a really good development team consists of a number of people with diverse talents and skills working together, and that it’s okay for a tester not be a coder when the tester brings other skills and perspectives to the table.

    Reply

  55. Wow and thank you for your time to create this thread I am finding it invalueable. At present I’m seeking a career change and am a zygote within the IT arena, if its ok to label it that. What you have stated applies to all industries in a transferable skills/idea’s set way. That little box might be green this side however the opposite side might be red, oh wait… your colour blind and that box is actually big, now its not even a box… wow how can I see what in it if its not a box? hmm
    I guess sometimes we get so focused on the problems we forget to focus on their solutions. I’m going to look at what developers do within this industry to see if my skills can help there, thanks again for the reading tools.

    Reply

  58. I like a lot of what you said. Especially your recommendations to testers. But I think you have your “assure” and “ensure” mixed up. Quality Assurance doesn’t own, let alone “create” quality, but it provides confidence to the consumer of the product or results of a process that it is of a particular measure of quality (e.g., no sev 1 defects, or no salmonela poisoning).

    Michael replies: One of these days I’ll have to do a long post on the confidence issue. For now, here’s a short one (along with some longer comments).

    As testers, we cannot report that there are no Severity 1 defects in the product. At best, we can report that we have found no Severity 1 defects in the product, but we cannot achieve complete coverage, we are not omniscient, and we are not perfect. As with salmonella, you can’t observe the product (patient) and find evidence of no salmonella; you can only fail to find evidence of salmonella.

    Bugs don’t necessarily present immediately. Some do, but others remain latent for some time. Salmonella doesn’t present symptoms immediately, but over several hours, and the incubation period may be as long as a day. Observing your lunchtime customers doesn’t tell you in time for dinner that you’ve got clean prep surfaces. Even by lunchtime today, you don’t know that everything was perfectly clean yesterday; only that your customers have not developed an infection—but perhaps that’s because their immune response is very robust, and they were not exposed to enough contamination for illness to develop.

    The incubation period for discovering a bug may be even longer—years. Software is a domain that lives in the world of black and grey swans. We cannot present reasons why our clients can be confident, but we can show (or fail to show) reasons why they could doubt their confidence. Excellent testing and test reporting depends on understanding the extent and limits of what we know and what we can’t know.

    Reply

  59. Title:
    Software Assurance Trustworthiness and Rigor
    http://youtu.be/N2lT84hEhkU

    Description:

    Trustworthiness requires a commitment to rigor in both software production and its verification. No soft skill, rigor has a hard edge. In an environment of neglect and unmet need, the introduction of rigor would be a game changer, one that requires both cultural change and engineering know how with pass/fail training certification.

    Software Assurance only has meaning in the context of trustworthiness, that is, worthy of being trusted to fulfill the critical requirements needed for a particular software component, system, or system of systems. Software Assurance demands two capabilities associated with trustworthiness, the capability to produce trustworthy software products and the capability to verify that software products are trustworthy. Each depends on engineering and technology rigorously applied.

    The kernel of the layered defense approach to Software Assurance is Build Security In and Structured Programming with its rigorous and provably correct use of zero and one predicate prime programs along with proper programs composed of multiple prime programs limited to single entry and single exit.

    The layered defense approach described here includes Defense in Depth, Build Security In, Risk Management Calculation, and Resilience processes, engineering, and tool automation dependent on an elevated state of software engineering rigor and precision. There is much room for improvement in the Software Assurance methods and practices that assure such rigor.

    Michael replies: Just so I’m clear on this… you’re saying something about rigor, right?

    If so, you might be interested in this: http://www.satisfice.com/presentations/rigor.pdf

    Reply

  60. I hear what you are saying, but let me try another approach. I buy stuff. Mostly electronic stuff and cars, and my wife buys everything else. I rely heavily on Consumer Reports. I like the testing that they do and the fact that they are independent of the folks who are trying to sell me stuff. They provide me with “quality assurance.” They do nothing to actually “ensure” the product has quaility, but the fact that they are looking at it in some detail, and measuring it against some benchmarks, e.g., reported defects by readers in a survey, provides me with more assurance that a product will be reliable–or more reliable than some competing product–than if I just pick the pretty one. What they do is testing (and data gathering). What they provide to me the consumer or stakeholder is QA. It is not a lot of other things. It is not a garnatee that there are no defects. That’s not what I said. That’s not QA anyway, and your thesis isn’t no defects, it’s that testing is not QA. Consumer Reports can’t tell me that my new Honda has no defects. But they can provide me with some assurance that If I buy the Honda, I’m likely to get less defects than if I buy the Yugo. This is what Testers do for the customer. Confidence is also not binary. So in fact when CR tells me about their extensive testing process, and why the recommend the Honda, they do in deed provide reasons why I can be confident. Which is why I just bought my wife a new Honda.

    Reply

  61. Mike is very interest article. However in my view you cover the article from the “Role” point of view. What about the QA as a skill? A tester must have QA skills that will hep him to build better Test plans , Test cases etc,, So I find it a little dangerous as novice users may misread that they do not have to worry about QA as a skill.

    Michael replies: To me, quality assurance is not a skill. Skills do enable you to assure the quality of your own work to some degree, though.

    Reply

  64. I’ve been doing “QA” for well over 20 years. If I did not test the software, there would be little quality in many cases. I have to urge programmers to make changes to the software, not only because of bugs, but because of usability issues. If I did not URGE programmers who often have massive egos about the quality of their work, to fix or change the way that something does or does not work, most of it would not get done. So, I am assuring quality even though I am not the initial creator of the software. I’ve found that most programmers have little idea about what end users experience and are often detached from that reality. Without quality assurance testing and analysis, there is little quality. Quality Assistance? …maybe, but not just a tester.

    Michael replies: I notice that you call yourself “Mr. Test”. I wonder what “Mr. Develop” and “Mr. Manage” might say about the idea that they are detached from reality, or about the idea that you and you alone are “assuring” quality. I’m also curious about what you think being “just” a tester means.

    Reply

  65. I moved from dev to qa about 4 years ago.
    My experience:
    QA people substitute the knowledge, the competence with the best practices etc.
    QA people think, they are working independently.
    They are not and they should not.
    QA people hate anyone who can actually do the job. I can read and write software. They sense it on the day one.
    I am well hated from the day one. Have been in this situation, at least 3 times.

    Michael replies: I’d like to offer this question, gently: Are you sure that differences in technical skill represent the only explanation for what sounds like a more complex social problem? The one and only explanation? “Hate” is a pretty strong word.

    Reply

  66. While I agree with a great many of your assertions I think the point is being missed. There is a role for Quality Assurance vs. Quality Control. Testing is part of Quality control and is the assessment of the quality of the software relative to the requirements. Quality Assurance is the monitoring of the processes that assure the likelihood that the software is produced correctly and may be maintained in the future via production of deliverables (business case, requirements docs, design, test plans/cases etc.). They are different disciplines and responsibilities.

    Michael replies: I think you meant to say, “To me, testing is part of quality control…” To me, that (and the sentences that follow) require us to think of testing in terms of confirmation of what we hope or believe to be true. To me, that’s a highly oversimplified view of testing. The whole quality-control/quality assurance model comes from relatively ancient and not-very-helpful manufacturing models that don’t fit very well with software development. How would you distinguish between “quality assurance” and “management” in your description?

    Too many organizations do not recognize the difference and lump the two roles together. Most managers couldn’t describe how the two areas differ either. I’ve yet to meet a CTO/CIO/CEO that had a clue.

    Maybe the inability of people to distinguish or describe the distinction between the two roles suggests that the distinction isn’t very helpful.

    Just my 2 cents.

    Reply

  67. Actually, in general, I find myself agreeing with most of what is presented here. With one exception:

    From my experience working in small teams, very small businesses, and startups, you’re not going to get very far or be able to offer much value, with the mindset that someone in authority needs to instruct you about what’s important, and inform you as to your specific responsibilities, in terms of product quality.

    Michael replies: I agree. But did I say anything like that, or is that something that you read into the piece?

    In my world, we work in service of a business. I don’t need instruction in what’s important, but I also don’t presume that I know everything that’s important to the business; nor do I presume that the business knows about important things I might be able to reveal. We all (including the business people) start with ideas about what’s important, and we learn as we go.

    When it comes to building a quality product, hierarchical relationship structures are regressive, unresponsive to change, vulnerable to more flexible shops, and mostly unwanted in a small shop, or startup environment. Nobody wants a mere “worker”, and the old model of pyramidal authority is going the way of the buckle-shoe.

    Perhaps. But I think you’re changing the subject here; I didn’t say anything about a “mere worker” or “pyramidal authority”. What I am saying is that testers should not usurp the roles of the programmer or the product owner, but support them.

    What IS wanted, is a colleague. Someone capable of negotiating, debating, discussing, and yes, personally investing in not just the “testing” or “quality”, but the user experience, the marketability, the engineering and design, and the financial success of the product.

    They want someone who *thinks* about, and *cares* about, the product. Because they do too. And they don’t want to spend their day working along side someone who only cares about “providing a service”.

    It’s surprising to me that you would consider providing a service and caring about the product as being inconsistent with each other. Thinking about and caring about the product is an important part of informing the service that I provide.

    This means, I spend my day doing more than just ferreting out and reporting on functional bugs in professionally written reports (though it is the majority of the work). I also talk to product and design about useability issues I encounter. I discuss with the CTO and sales folks ideas I have for encouraging up-sales in various interactions within the product. I work with developers to actually *fix* some of the bugs I find, but also to learn about their design and implementation decisions, and to discuss alternatives.

    Yes. Can you point out where I’m suggesting otherwise?

    Also, if I did any of these in a manner you describe above in your bullet-points (making developers cry, telling them I know their job better than they do, making unreasonable demands of management, treating people smugly), I would be fired in very short order. Nobody wants to work with an a$$hat.

    Yes. That was exactly the point I was making,

    But my experience has also been, that if I *don’t* engage beyond the level of a mere “service provider”, who is simply following the marching orders given to him by the superior who “actually owns quality”, then I’m also basically doing nothing more than the minimum required to fill the role, and that’s not at all what is expected. Eventually, that will get you fired, too.

    Nobody said anything about not engaging beyond the level of a mere service provider, or about “simply following the marching orders given to him by the superior”. That’s stuff you made up.

    Reply

  69. QA IS different from testing. Testing is more common in Waterfall organizations. It happens after-the-fact, after design has been made and is a separate stage in which core functionality is allegedly set to work/scoped.

    Michael replies: Interesting; that’s a view of testing that, in our community, is as dated as Waterfall itself. For instance, in Rapid Software Testing, we maintain that testing is “evaluating a product by learning about it through exploration and experimentation, which includes to some degree: questioning, study, modeling, observation, inference, etc.” Does this happen “after-the-fact, after design has been made, in a separate stage”? Not by our reckoning, since “‘experimentation’ implies interaction with a subject and observation of it as it is operating, but we are also referring to ‘thought experiments’ that involve purely hypothetical interaction. By referring to experimentation, we are not denying or rejecting other kinds of learning; we are merely trying to express that experimentation is a practice that characterizes testing. It also implies that testing is congruent with science.” (You can read about this at http://www.satisfice.com/blog/archives/856)

    Now, you may say, “But that’s redefining testing.” We would disagree with that, too.

    QA is more involved. The QA is more of an guide to the team – the subject matter expert who encourages the team to move towards Best Practices for Quality. What is different and separates a QA from a Tester will be involvement in the Team. Typically Testers are less involved – do not get involved in decisions early on and accept products as they come. QA is involved early in inception of these products and can guide the team toward goals of better Quality.

    Here’s at least one problem with that: what gives “the QA” his or her stature? What gives “the QA” his or her authority? Quality is multi-dimensional; “value to some person(s) who matter”. Without stature (presumably derived from expertise and experience in several domains), there’s no good reason to listen to someone who protests about some dimension of quality or another. And without authority, one could view “the QA” as a wise coach (at best, assuming stature) or as an inexperienced, irritating pest. With authority, “the QA”‘s role would be indistinguishable from management, as I suggest in the post.

    I agree with all comments on this post except that QA and Testing are the same thing. Great post, by the way.

    Thank you for the compliment.

    Reply

  72. TL;DR = 1) There’s no reason testers can’t make code commits, if it helps them become better testers. 2) The people with the most product knowledge/business risks should make the release decisions, regardless of job titles, so perhaps we have different experiences with what “management” is or is not.

    Michael replies: TL;DR 1) There’s no reason a goalie can’t try to score goals either, if it helps them becomes better goalies AND if they and their team are willing to leave the net unattended. 2) The people with business responsibility and authority should make business decisions, and releasing a product is a business decision, one informed only partly by technical considerations.

    1) On the quote of “quality assurance”:
    I take exception to the initial question asked in that post though (where the QA is asked if “they are in quality assurance” and are “allowed to change the source code”. I think the logic of both parties in the conversation are flawed. The asker is assuming that you can only “assure” quality if you have direct code/change access, which is not true, while the tester is saying “of course not” to being able to edit the code, which also does not make sense to me. There no reason a tester cannot be technical enough to help the actually team make code commits. That might be part of their process to become a better tester, and saying “of course not” comes across as avery close-minded on the tester’s part. That is a mind, content to live in a box.

    You can only assure quality if you have authority or control over the work. An investigative reporter can assure the quality of her own work, but she cannot assure the quality of the political process or the politicians she’s investigating. She can run for office, but at that point she is no longer in an investigative reporter’s role. When a tester is helping the team with code commits, the tester is not in the testing role at that moment. I’m not quite sure why this is so hard for people to understand (but I can’t assure the quality of their understanding). Perhaps the problem is that people confuse “task” and “role”, or that they misunderstand the notion of roles altogether. Here is an attempt to explain that.

    In some contexts, a tester who helps do code commits every now and then might be a integral part of their development process and of that tester’s own workflow to become more masterful at their skill-craft of testing. I do not believe in a hard division/lines drawn in the sand of ‘this is what a tester does’ and ‘this is what a dev does’. Do I think there are guard-rails that govern those roles? Sure, but there’s no way we can claim that context is key, and then say in all cases ‘a tester should never focus on this or that’.

    It is of the essence of being a tester to focus on testing, just as it is the essence of being a flight attendant not to focus on attending passengers on piloting, and just as it is of the essence of being a pilot to focus on piloting rather than attending passengers.

    2) On the subject of release ownership (this section is based on working in environments where product owners are part of the team, embedded, and have all the business knowledge/risk information to make release decisions):

    I also used to be empowered by believing QA was a ‘gatekeeper’ but they are not.

    Meaning that your “empowerment” was fake—right?

    The whole team owns the quality of the product, as the team is made up of other smart professionals too. This is why I believe the tester works in conjunction with the PO/team to determine if something is a show stopper or not. I think we’re in agreement on that based on your article, but the implementation of it perhaps not.

    I don’t understand that last clause, there.

    You quote about project management saying that a release is a business decision is true. Sometimes we intentionally release with know bugs because we have calculated the risks and deemed it acceptable. We might do a patch later or work those bugs into upcoming sprints, but we’re ‘OK’ to release now in the current state. I get that, that must be done.

    There’s a fine line there too though. If we agree that the whole team (scrum team: Scrum Master, Product Owner, Devs, Testers) are in charge of the quality, then to me it’s a logical extension of that thought to say the team gets to decide what is/isn’t ready for production. Remember, the team includes the product owner, who understands the business risks and justifications for or against releasing something with known defects. We keep saying that testers cannot assume they know about all the risks, which they don’t, but part of being a good tester is also becoming a subject matter expert in the product, so you can make much more informed testing decisions. I am thrilled when the tester, in a backlog grooming session, is able to educate the product owner on some business rules or development risks that they did not know, thus changing the scope of the release. This also means the product owner has some learning to do, but that collaboration between the two can foster that. I think it is key that testers be product experts as well as testing experts, as one fuels the other (again, this is within the context of website software development). This again ties into my lack of interest in drawing hard lines in the sand on what a Dev is, what a tester is, what a PO is. If we share the responsibility of quality on the team, then we also share the other things as well.

    Curious to hear your thoughts on this, and it is fine if we disagree, but if you look at forward-thinking Agile shops like Spotify, they foster team autonomy by giving them control over the release approval process. Management helps remove impediments, mentor, guide, etc, but I would disagree that a good management structure involves someone outside of the team saying yes/no on a release as a practice. That should be an outlier in my opinion and the final say should lie with a discussion between tester, dev and product owner who understands the business ramifications. The most successful models I have seen are when the product owner (who is embedded within the team) ultimately decides the go/no go.

    You do understand that “product owner” is a management position in the sense that I’m talking about, right? That the product owner is responsible for management decisions on the team?

    Note: I am cognizant that my understanding of these things is in constant evolution. As a learner, I am open to being convinced otherwise.

    Reply

  73. Thank you for the reply. Clarifications below.

    On gatekeeping: Yes, you could say my empowerment was “fake”. I think a more appropriate description of it would be ‘subconscious wishful thinking’ at the time, which I felt into because I wanted to combat the stigma of testers traditionally not being as valued as Devs.

    On Product Owners: I am speaking to my experience when I talk about Product Owners, and perhaps we misuse that title here, but they are embedded within the team and do have the final call on what is/isn’t going to production, but they themselves are not considered “management”. We are using different vernacular there, so this may just be a hangup on the semantic of the “management” title.

    Reply

  79. I’m a QA engineer, and I do not perform tests on my daily base work routine.
    Because I’m the only one in my team in the testing/QA area, I make my tests plans/scenarios/test cases etc, I also perform the audit for software, and I’m involved in the development process.
    To be honest, only 30% of my work is actually testing.
    I do not really know what area I’m located in.

    Michael replies: Well, it seems to me that you know what area you’re located in; you say yourself that you’re in the testing/QA area. In fact I disagree with some part of that, but it’s up to you, not me, to decide your roles. Reflecting on that issue, as you’re doing, is a fine start. Notice that I say “roles”, there; roles can be picked up and dropped when it suits you.

    Reply

  80. […] assessment and make the ultimate business decisions as it relates to the product (more about that here from Michael Bolton). This may involve the Product Owner on your team, or it may involve a set of stakeholders who are […]

    Reply

  81. Interesting and great post Michael. I’m head of QA at our company. My boss and I have had discussions about quality responsibility. He claims that I am responsible for our quality and I’m not trying to dodge or avoid taking responsibility but I surely have to question it when reading your post. As you write in the beginning; the testers (my staff) can’t change the code etc.

    Michael replies: Were I in the same room with you, I’d ask you and your boss what you think “responsibility” means. I think it would be a good idea for you to have that conversation without me, too.

    If responsibility doesn’t come with corresponding authority to make decisions, set policies, change things, it’s not really responsibility. Literally, you don’t have the authority to respond such that things change.

    I know you also wrote: Cem Kaner said “the quality assurance role in the company, lay with the management and the CEO (the principal quality officer in the company), since it was they—and certainly not the testers—who had the authority to make decisions about quality.”

    So the question for me is. If I’m not responsible for our quality. Then what am I responsible for as head of QA?

    Only you and your boss can decide that. It’s not reasonable for him to make you responsible for quality assurance. It might quite be reasonable for him to make you responsible for quality awareness. In this case, your job would be to

    • gather relevant information about the project, including an understanding of the product, the customers, the tasks or purposes that they’re attempting to fulfill while using the product;
    • sift and sort that information—and since some of that information will be missing, develop the information that’s not yet available;
    • test the product, exploring and experimenting with it, actively looking for problems that might threaten its value;
    • report on the state of the product and of the project;
    • tell a story about problems that you’ve observed in the product (bugs), and potential problems that you haven’t observed yet but that you can infer might be there (risks);
    • talk about your coverage—the testing you’ve done so far—how you have configured, operated, observed the product;
    • talk about what you haven’t covered—the testing you could do that you haven’t done yet (or will not do at all, unless your client objects);/li>
    • talk about the things in the product and the project that are making testing more difficult, harder or slower.

    If there’s a bug in the product, the information that testing reveals can help in getting the bug fixed. If there are problems in the project, that information can help to bring those problems to the attention of management—and perhaps help to change the conditions that help to breed bugs.

    Sorry for the overuse of the r-word 😉

    If you don’t have authority to change things, you don’t have responsibility for quality. It might, however, be that you get the blame.

    Reply

  83. […] since no one person can “assure” quality. If interested, you can read more on that in, “Testers, Get Out of the Quality Assurance Business” by Michael […]

    Reply

  86. […] I then moved into the paradigm of ‘It is my job to find defects and report on the status of those defects but I still know better on what should be fixed since I am the tester’ and while I explored, testing outside of the acceptance criteria, I was doing ad-hoc/random testing rather than truly structured exploratory testing. I was still missing the boat on the true nature of what it meant to do exploratory testing and I wasn’t even thinking yet on reporting out on what I did not test, only what I did. I was getting to a point where I realized there were more important product risks, but still somewhat ignoring them because I had not fully broken out of the paradigm of the tester being the final “gatekeeper” of the product/release, which we are not. (Read more on that in Michael Bolton’s blog “Testers: Get Out of the Quality Assurance Business“) […]

    Reply

  87. Hi Michael, I read with interest your topic of Quality Assurance.

    As a tester I pride myself on being Quality Assurance. Part of my job is also to perform analysis on testing pass rates, number of bugs, where they cluster in the code and who coded them etc in order to determine why developers are making mistakes and how things can be improved to stop them making less mistakes.

    My analysis and subsequent chats with developers has uncovered 3 recurring issues that cause developers to make mistakes:

    1- Poorly communicated requirements from BA’s
    2- New developers on the project not having context of what they are building
    3- Developer fatigue

    I then discuss with the whole software team to refine processes to lessen the chance of developers making mistakes.

    It is only a tester that has an overall view of the project in this way because bugs & test cases are the artifacts we deal with all day with granularity of bugs as to who, when, were, how many times. Developers do not have time or interest to make these correlations and neither to ba’s or project managers.

    So although you say QA does not have the ability to change the code directly my view is that it is usually external factors that cause the mistakes in the code in the first place and a mindful QA can manipulate these external factors through research and understanding to get the code into a better state.

    By broaching the subject of software quality with developers in this way I have gained their respect because they know I am looking out for them.

    Kind Regards

    This does not sound like assurance to me. They do sound like worthy things to do, powerful forms of advocacy and assistance. However, it is the developers who make the changes, and the managers who run the project. Unless they take action, the quality of the product doesn’t change, and the project environment doesn’t change.

    I think it’s important that testers speak precisely about the extents and limits of what they can do. Without authority, you can’t assure quality—but that’s perfectly okay, in my view. Assistance can be valuable, and it your case, it sounds like it is. Keep it up.

    P.S. Be careful about the validity of pass rates.

    Reply

  90. […] on several roles beyond testing. You could say that testers in this school are getting themselves into the quality assurance business. Definitely some food for thought here, and something I expect to […]

    Reply

  92. […] In truth, I used to feel that Quality Assurance title sounds heavier and more “important” and would rather be called that. I remember the first time I came across someone correcting another person who called him a Quality Assurance Engineer, I was shocked. In my mind, I was like, “why on earth will you prefer to be called a Software Tester. So unsophisticated. Ugh!” The guy gave some reasons for having a problem with the title. You can find them here […]

    Reply

  97. Hi Michael,

    We recently had a conversation on LinkedIn when you directed me to this blog post.
    I remember reading this blog years back but when I read it again, it helped me gather my thoughts around. I am currently reading the comments section and will then move to references.

    It feels like a quest to find a way to help management understand how testers cannot assure Quality. I am hopeful to find a way so that when I meet the management next, I’ll have a few ideas to help them see Testing bit differently. Challenge is, I understand and agree with it, but how do I make them understand it?

    I tried in the past, shared BBST, Lessons Learnt in Software Testing and other thought provoking things but in the end, still couldn’t induce a lasting change. I guess the only option is, to be persistent, what are your suggestions?

    Regards,
    Rahul Gupta

    Michael replies: What, specifically, do they not appear to understand?

    When you say that you have no control over the code, the fixes, what is to be fixed, the schedule, the budget, the staffing, the scope of the product, the relationship with the customers, do they agree?

    If they agree, what is their reply when you ask how you assure quality?

    You can’t make them understand. It seems to me, though, that you can ask questions that problem them to understand on their own. You can also remind managers that they have control and authority over all of the items in the list above. Quality assurance is their business.

    Reply

  106. Michael, one question for me. In some companies, testers need to do both testing and QA related activities. What’s your suggestion for them?

    Reply

    • My suggestion is that you do what you were hired to do.

      If you were hired to do testing, do testing. If you were hired to do build work do build work. If you were hired to do both, do both.

      But it’s important to be clear on what you’re doing, what’s enabling it, and what’s disrupting it. It’s important to remind your manager that when you’re doing building work, it’s not the same as testing work, even if the build work is very valuable.

      A smooth and fast and well-checked build may enable us to get to experiential testing quickly and easily; that’s a good thing. Preparing builds may even include may even include some evaluation of the product and discovery of problems, which is testing work. All that may be worth the cost of the time we spend on it, if we eventually do get to such experiential testing as is needed. (And occasionally, we might not need a lot of experiential testing.)

      Too often, from my perspective, teams put emphasis on checking the build — looking for problems that matter to the builders. This displaces time and resources for testing the product — looking for problems that matter to all kinds of people, like customers or regulators or tech support folk or the ops people.

      That’s only my perspective. I can’t give you an answer that’s right for you and your group. But you can. Just be clear that any kind of work we are doing tends to displace other kinds of work we could do. Declare that; and ask if everyone is aware of the risk and happy with things as they are.

      Reply

  107. Agreed!

    A Tester is not the only one assuring quality! Testers’ role is to provide an accurate report on the bugs. It is the responsibility of the developers to make sure quality is not compromised. Testers are always blamed for compromised quality. Testers can’t be responsible for the code, right?

    Reply

  109. This is a very informative and well-written post on software testing. I appreciate how you broke down the different types of testing and explained the importance of each one. Your tips for effective testing were also very helpful. As someone who is new to software testing, I found this post to be a great resource. Thank you for sharing your knowledge and expertise!

    Reply
  110. Pingback: QA vs UAT: Difference between QA and UAT Testing

  111. Your perspective on testers shifting from Quality Assurance to Quality Engineering is intriguing. The evolution from traditional testing roles to active collaboration in the entire development lifecycle aligns with industry trends.

    Reply

    • As one might expect from a comment sent from a Search Engine Optimization service, there’s a misrepresentation here: my post doesn’t talk about testers shifting to Quality Engineering.

      Reply

  112. Thought-provoking perspective on testers’ role in quality assurance! This article challenges traditional notions and offers valuable insights into evolving approaches to software testing. Well-argued and informative, it sparks important discussions about the broader responsibilities of testers. Thank you for sharing this insightful viewpoint!

    Reply
  115. Pingback: Testerské střípky – U Marků

Leave a Comment