It had been a long day, so a few of the fellows from the class agreed to meet a restaurant downtown. The main courses had been cleared off the table, some beer had been delivered, and we were waiting for dessert. Pedro (not his real name) was complaining, again, about how much time he had to spend doing administrivial tasks—meetings, filling out forms, time sheets, requisitions, and the like. “Everything takes so long. I want a pad of paper to take notes, I have to fill out a form for it. God help me if I run out of forms!”
“How much time do you spend on this kind of stuff each week?” I asked.
Pedro replied, “An hour a day. Maybe two, some days. Meetings…let’s say an hour and a half, on average.”
Wow, I thought—that’s a pretty good chunk of the week. I had an idea.
“Let’s visualize this, I said.” I took out my trusty Moleskine notebook. I prefer the version with the graph paper in it, for occasions just like this one. I outlined a grid, 20 squares across by two down.
“So you spend, on average, an hour and a half each day on compliance stuff. One-point-five times five, or 7.5 hours a week. Let’s make it eight. Put a C in eight squares.” He did that.
“Okay,” I said. “You were griping today about how much time you spend wrestling with your test environments.”
Pedro’s eyes lit up. “Yes!” he said. “That’s the big one. See, it’s mobile stuff. We have a server component and a handset component to what we do, and the server stuff is a real bear.”
“Tell me more.”
“It’s a big deal. We’ve got one environment that models the production system. The software we’re developing has been so buggy that we can’t tell whether a given problem is general, or specific to the handset, so we have another one that we set up to do targeted testing every time we add support for a new handset. That’s the one I work with. Trouble is, setting it up takes ages and it’s really finicky. I have to do everything really carefully. I’ve asked for time to do scripting to automate some of it, but they won’t give that to me, because they’re always in such a rush. So, I do it by hand. It’s buggy, and I make the odd mistake. Either way, when I find out it doesn’t work, I have to troubleshoot it. That means I have to get on instant messaging or the phone to the developers, and figure out what’s wrong; then I have to figure out where to roll back to. And usually that’s right from the start. It wastes hours. And it’s every day.”
“Okay. Show me that on our little table, here. Use an S to represent each hour your spend each day.”
Whereupon Pedro proceded to fill in squares. Ten of them. Ten more. And then, eight more.
“Really?!” I said. “28 hours a week divided by five days—that’s more than five hours a day. Seriously?”
“Totally,” said Pedro. “It’s most of the day, every day, honestly. Never mind the tedium. What’s really killing me is that I don’t feel like I’m getting any real testing work done.”
“No kidding. There’s no time for it. There are only four squares left in the week. Plus, something you said earlier today about tons of bugs that aren’t related to setting up?”
“Right. When it comes to the stuff that I’m actually being asked to test, there’s lots of bugs there too. So my ‘testing time’ isn’t really testing. It’s mostly taken up with trying to reproduce and document the bugs.”
“Yes. In session-based test management, that’s bug investigation and reporting—B-time. And it does interrupt test design and execution—T-time—which is what produces actual test coverage, learning about what’s actually going on in the product. So, how much B-time?” He filled in three of the squares with Bs.
“And T-time?”
He had room left to put in one lonely little T in the lower right corner.
“Wow,” I laughed. “One-fortieth of your whole week is spent in getting actual test coverage. The rest is all overhead. Have you told them how it affects you?”
“I’ve mentioned it,” he said.
“So look at this,” I suggested. “It’s even more clear when we use colour for emphasis.”
“Whoa. I never looked at it that way. And then,” he paused. “Then they ask me, ‘Why didn’t you find that bug?'”
“Well,” I said, “considering the illusion they’re probably working under, it’s not an unreasonable question.”
“What do you mean?” Pedro asked.
“What does it say on your business card?”
“‘Software Testing’.”
“And what does it say on the door of the the test lab?”
“‘Test Lab’,” said Pedro.
“And they call you…?”
“Pedro.”
“No,” I laughed. “They say you’re a… what?”
“Oh. A tester.”
“So since you’re a tester, and since the door on the test lab says ‘Test Lab’, and your business card says ‘Testing’, they figure that’s all you do. The illusion is what Jerry Weinberg calls the Lumping Problem. All of those different activities—administrative compliance, setup, bug investigation and reporting, and test design and execution—are lumped into a single idea for them.” And I drew it for him.
“That’s management’s illusion, there. Since, in their imagination, you’ve got forty hours of testing time in a week, it’s not unreasonable for them to wonder why you didn’t find that bug.”
“Hmmm. Right,” said Pedro.
“When in fact, what they’re getting from you is this.” And I drew it for him.
“For testing—actual interaction with the product, looking for problems, you’ve got one-fortieth of the time they think you’ve got. One lonely little T. Is that part of your test report?”
“Oy,” he said. “Maybe I should show them something like this.”
“Maybe you should,” I said.
A couple of nights later, I showed that page of my notebook to James Bach over Skype. “Wow,” he said. “That guy could be forty times more productive!”
“Forty?”
“Well, no, not really, of course. But suppose the programmers checked their work a little more carefully, or suppose the testers practiced writing more concise bug reports and sharpened their investigating skill. One of those two things could cut the bug investigation time by a third. That would give more time for testing, when they’re not being interrupted by other stuff. What if they cut the setup time by a half, and that administrivia by half?”
“Four, fourteen…” I said. “That would give eighteen more hours for testing and bug investigation, for a total of 22 hours. And even if they’re still doing two hours of bug investigation for every one hour of testing time… well, that’s seven times more productive, at least.”
“Seven times the test coverage if they get some of those issues worked out, then,” said James.
“Maybe de-lumping is the kind of thing lots of testers would want to do in their test reports,” I said.
How about you?
Simply wow. I guess I should be buying one of those Moleskine notebooks with graphs. Its not management’s fault (only)…its our job to make them realize how they can help us being more productive. good post Michael…yet again.
Thanks,
Rahul
Great post!
TBS – [any other test measurement style] 1-0
A real eye opener!
Tonight I’m preparing a presentation like this to my management.
The simplicity of it should appeal to the management.
Wow indeed! That’s a great way to convey workload using colours. This makes me think that there must be other areas in testing which could be communicated better using images, colours, photos, etc. I know of James’s testing dashboard which I’ve used in the past (it has smileys!) but nothing else springs to mind which is as effective.
I love it. This is fantastic! It’s simple and so effective. Would not be a bad idea to run this exercise with the team to find out where we all stand. Thanks Michael.
[…] work. The diagrams used help visualise the problem of inefficient or poor time usage. Nice reading! http://www.developsense.com/blog/2012/10/where-does-all-that-time-go/ Written by Jamie Posted in Everything else, Testing, Work Tagged with Jamie, […]
This is a great way to present the classic “T and BS” argument visually, and it makes quite a statement. In the past I’ve been using a pie chart, but I see a lot of appeal in this. Thank you Michael!
Thank you for this post, Michael. Very useful.
Yet something bugged me as I read this, to the point of distraction.
You’re sitting at a restaurant, you have an idea, so you pull out a moleskin and a black pen. I’m with you so far. I’ve been there. But then, there are four categories of time begging for illustration, and you were able to produce four different colors on the spot! Is this normal for you? How many categories deep could you have gone?
Michael replies: I wear cargo pants! I carry a big heavy backpack! I keep stuffing things in both of them! That night, I happened to have in my pocket a bunch of the Crayola markers that I distribute around the room at the beginning of a class. Certain colours of those markers work really well as highlighters; that’s what I used that night.
These days, under normal circumstances, the limit is four colours (plus black). I’ve been carting around a Moleskine notebook and a blue pen for ages, but a year or so back, I discovered PrismaColor 05 pens, which I pick up from Midoco here in Toronto. James Bach carries around a similar collection of pens, but he’s got this rubbery bandolier from Etsy that goes around the notebook and holds the pens in one nice, neat package. I have cargo pants, or a murse (man-purse) for when I go through airport security. So this is what my writing kit looks like.
A couple of things worth noting.
I have two blue pens. I prefer to write in blue, most of the time, so they run out faster, and if a pen blows up on an airplane, I don’t want to be without a blue one, at least. I also have green, red, black, and orange. For me, that’s a decent set of choices for emphasis and for mixing things up.
Both the notebook and the pens are a little expensive, yet they’re substantial and pleasurable to use. That draws me into using them, which for me is psychologically important. Cem Kaner said a similar thing (I think in Testing Computer Software) about getting and using a really good pen.
I write my name on the edge of the Moleskine’s pages, so that people can see at once who it belongs to. On the long edge, I write the starting and ending date of the book, so that I can look at a stack of them and pick out the one I’m interested in.
Notice that my Moleskine has a slab of red electrical tape on the spine. If you cart them around and use them enough, the spines can wear out. The upside is that your choice of patch makes the book distinctive, which can be important when you’re hanging out with a bunch of people who also have Moleskines. That happens at the CAST conference and at peer conferences and such.
Finally, the photos in this post were reconstructions, done on my kitchen table, for the purpose of telling the story. The original looks a tiny bit different, in that I didn’t draw 40 Ts, just one big one, and now that I look at it, there are 12 C’s instead of eight, and 24 S’s instead of 28. The point of the story remains the same, though.
[…] Where Does All That Time Go? (Developsense Blog) […]
Superb, Great post Micheal, Thank you!!
I don’t know whether to be inspired or depressed by this. I’m a bit of both I guess. The problem is so, so familiar. But yes, it is the job of testers, especially test managers, to speak out about such problems, however unwelcome the news is. This is a good way to do it in a clear and non-confrontational report. It’s certainly better than the tactic I tried at a progress meeting when I simply said “no progress to report this week because of all the time wasting crap” and then shut up leaving a horribly embarrassed silence.
Because our profession usually overlaps with several others, we often find ourself in a position where we do work which is not actual testing. I don’t think that’s necesserly a problem. For me that main question is: does what I do bring value? If the answer is yes, then it’s my duty to do it. I mean quality assurance means more than the act of finding bugs, right?
Michael replies: Yes, we often do work which is not actual testing, and that’s not necessarily a problem. It might become a problem if that work is confused (or “lumped”) with testing, so identifying that work and accounting for it is precisely the point of the post immediately above.
I’ve long held that testers should get out of the quality assurance business. One way of interpreting that is to say that testers are not in the business of assuring quality, but instead are in the business of supplying information to the people who put the quality in and who manage the project. Here’s another reasonable interpretation, in line with this post: it’s perfectly reasonable to do things other than testing—to generalize and to help the team in all kinds of ways— but when you’re doing something other than testing, by definition you’re not testing. If that’s so, account for your time accordingly.
This post completely ignores the fact that many developers/testers seldom tell the truth on how they spent their day.
An hour and a half may be as little as 15 minutes for the rest of us.
Michael replies: If you’re a manager, I would argue, it’s up to you to validate the reports that you receive. That is, it’s up to you to observe the work being done, and to be familiar enough with it that you can tell the difference between a truthful report and a false one. It’s up to you to identify discrepancies between what you observe and what is being reported, and to resolve those problems. It’s up to you to hire and retain people that you can trust, and to provide an environment in which they have both freedom and responsibility to perform and account accurately for the best work they can do. In other words, to manage.
Now, if you’re the kind of manager who feels you must blindly accept everything that your people report, or that has systemic distrust for your own people or for testers or programmers in general, I’m not sure I can even begin to help you.
A great post again. Above graph says it all. If tester does not get time to test, is like a bus driver who travels each day to a distant city, but hardly gets time to get down and explore the city and look around.
Fantastic was of visualising what we actually do. Might suggest this for our developers who I feel fall under the illusion from above that all they do all day is write code.
I’d say there’re responsibility issues here:
1. I’ll be hard on Pedro first: I mean – identification of bugs and problems are the fruit of a tester, never-the-less he is a TESTER. If there is a lack of fruits or expected results are not met, he should analyse his own work (and not just sit back and complain). On regular basis. There’s no way the product is bullet-proof even if the bug list is short due to lack of test executive time! We’re living in Emmental cheese .. 🙂
In case there are obstacles for him to do a better job – which is TEST EXECUTION – he should be able to identify them (or at least find a help to do so) and escalate – not by coincidence as having a beer with you, but having that kind of »beer debate« on purpose, the moment he feels stuck in analysis or work.
2. His manager or test lead: follow up of »Why didn’t you find that bug?« should be a brainstorming with Pedro – or the whole team. The very same kind one as he had it with you. Just setting a »mountain peak« question doesn’t solve the problem.. as you’ve written above – he is the one who takes care of QA.
Michael replies: I think you’re suggesting that every tester should have both the rhetorical and visualization skills to make all problems clear to any class of manager, in any context at any time. I agree that would be very nice. Trouble is, people like Pedro have a hard time with that because a) no one ever really taught them how to test; b) no one ever made them really accountable for their work; and c) their managers often do not have the questioning and management skills to elicit information about the status of the product, the status of the testing, and the quality of the testing (largely because they’re in the same kind of position as Pedro with respect to (a) and (b). So your comment can be basically summarized as “I’ll be hard on Pedro because people should do a better job”. I think it might be more important to help people learn how to do a better job, in which case being hard on them won’t be necessary.
p.s. post that rises a blood pressure – as always :). Likeable
Thank you.
[wohoo!! – blood pressure is up & running :)]
No, I don’t agree with a summary – was not my intention to say that – respect to Pedro and alike .. he did ask (eventually), didn’t he – he asked you. Point of the critics for him was the timing of those questions. Results count and he did not have them, whether other make us accountable or not is not the case – we should make ourself accountable first, so I’d assume he should start digging for the answers the minute he felt uncomfortable with the situation.
A personal part of ones job is also improving in it, isn’t it? Though it is true, I don’t have an insight what did Pedro already read, tried, asked etc. before he met you.. I might as well be unjust towards him.. in that case – please, forgive me, Pedro!
It is just that many times a lack of knowledge and competences are a sweet excuse for not growing up or brake a loop we are stuck in (for any kind of reason).. in those cases managers should be able to see and give a hand.. but even if they don’t (are not capable or willing) – there’s www with huge amount of informations. With time we’re able to distinguish between poor and good infos. If we’re eager to grow.
Michael replies: I’d gently suggest that you have a look at http://www.developsense.com/blog/2012/09/premises-of-rapid-software-testing-part-1/.
Never the less – who tough you? You did not get it all on the plate, served, right? I sure know for myself there was no plate on my table… I remember the times when I was hardly able to explain what was an issue or what I assumed to be one. Sometimes it was based on a feeling – imagine: »Hi, I have a feeling we’re on the wrong track.« »What do you mean by that – what’s the issue?« »Well, I don’t know, it’s just that I have a feeling we’re not doing it right. That we could do better.« (Btw. I actually did that once and a team brainstorming produced a valuable outputs)
But truth is that I believe there’s no such thing as a stupid question (we all learn our whole life) so I searched and questioned. Well, I still do – life is full of loops to brake and knowledge to be gained.
It is hard tough, if in a whole Co.’s chain (tester – test lead – test manager – …) there is no single person to identify that there’s a problem or what is a source of it .. but it’s not dead end either – they can hire you to consult them.
So, a summary would be: »Be brave to be Pedro who asks – ask as soon the doubt pops up, but be eager to become Michel.« 🙂
That takes awareness, time, experience, practice, and skill. “Should” is really easy to say from one end of the learning curve; impossible to conceive of from the other, in many circumstances. Be careful of saying what someone should do.
Hi,
Thanks for the post!
Michael replies: You’re welcome.
I did the same in one of my previous company. I was a test lead at that time. I had en excel file there we collected information about how much time we’d spent on looking for requirements, how much time we spent on environment problems, how much time we spent on testing etc. Not for a session (we did not have SBT) but during a day.
The problem I had and which I could not solve (and still do not have a solution for) was: testers found it difficult to follow up everything during a day due to many reasons.
I wish we had some kind of activity logging program helping us to collect this information, prompting us from time to time with questions such as “what are you doing now?”, “are you still doing this…?” etc.
May be you know such a software?
Regards,
//Liza
There are a few ways to address this, it seems to me.
One is to use The Pomodoro TechniqueTM (although… it’s trademarked? Really?! Well, I acknowledge that “The Pomodoro Technique is a trademark of its owner, etc.).
Set up an old-fashioned windup egg timer on your desk.
Set a chime to ring on the hour, every hour, and get people to scribble a quick note on what they’re up to at the time. There must be a zillion timers on the Web. Google “alarm clock” or “time tracker”.
It’s important that it not be too interruptive, such that it completely destroys flow. I’d offer the suggestion not to worry about precision, and instead be concerned with accuracy. That is, ask people to note things that represented more than some number of minutes of interruptions—15 minutes, 30 minutes, an hour— whatever you consider significant. You can wait until the end of the day for that. Now, if the people can’t tell you about big interruptions, perhaps more scrutiny is necessary.
[…] (or someones) should do a time analysis of writing out scripts something like Where Does All That Time Go?. I suspect though that a lot of automation would be cancelled as a result […]
Michael,
I remember you showing this to me over dinner in March. At first I thought it was something of Weinberg’s creation…and still think it is worthy of Jerry.
Michael replies: I’ll take that. High praise indeed; thank you.
Not had a chance to give it a spin yet, but have something coming up where the testers seem to be in a crunch between management expectation and development reality. I suspect this could help out, and I’ll let you know.
-Iain
Please do! Experience reports are always welcome.
Michael
This is a very good example on demonstrating how you spend your time. As a manager this is the type of information I would like presented to me. It would help me understand the challenges in a way that is actionable. This information can then be presented to people who can help us remove those barriers increasing time for actual testing. And the barriers might be within our own department as we might need to train better, mentor more, etc. Bernice
Michael replies: Yes. Testing is about raising awareness—not just about the product, but about the project, about the business, and about the testing itself. Awareness tends to lead to better-informed decisions. Thanks for the comment.
Depressing, but so true!!!
We tend to be overly focused on manual testing vs automation, but there is so much time spent on environment set up, trial and error, reporting, meetings, etc. that testing is… well… just the side job.
I am so glad I 1) saw this post, albeit a bit late and 2) graduated to the same notebook you use. I’ve been carrying several colors of pens (I prefer the Sharpie pens, but they look to be about the same as yours) and find the grid paper very useful for things like this.
I am going to start using this idea today with the hopes of having my team use it as well. This is a very simple means of getting a high level view of time… and it is so much better than making people fill out timesheets.
[…] a really nice dinner together with some of my closes test colleagues and reading his blog post (http://www.developsense.com/blog/2012/10/where-does-all-that-time-go/) I did the same thing as him, for lets call her […]
Hi Michael,
Thanks for the tip! After talking to you and reading this blog post I did a grid like you did. If you are interested you can read more about the result on: http://thesupertester.com/?p=428
Thanks again!
[…] by Jacqueline C. Vischer; ”Interrupt Mood” by Brian Tarbox or “Where does all that time go” by Michael […]
[…] There has been separate discussion on this topic and they are also worth reading. […]
Michael says: Usually I simply delete the blog spam, but this one was too good.
Excellent web page. Plenty of handy data in this article. We are giving it with a friends ans furthermore giving around delightful. And obviously, thank you in your perspire!
[…] If you’re a tester, what do you spend most of your time doing? Testing, right? Well, wrong. Michael Bolton notes that we may only be spending a tiny amount of our total working time actually doing testing. […]
[…] Where does all that time go? […]
[…] it visible and legible — that is, readable and understandable. (There are examples here: https://www.developsense.com/blog/2012/10/where-does-all-that-time-go/ and […]