“How did you miss that bug?”

Dan Ashby

I’ve recently been involved in a discussion regarding the blame culture that lives within the software development industry.
You hear the stories everywhere, where a bug is found on a live environment and people (usually management) automatically start finger-pointing at individuals to blame them for the bug, either for missing it during testing, or for missing it in the code review or unit testing process…

In this discussion, I was told that “it’s important to point out the person who is responsible for the defect, so that they can learn from their mistake”… I have a few problems with this statement.

Firstly, we are making the assumption that the buck stops with the initial person being blamed, and there is not further cause for the mistake happening. A hypothetical example could be if a bug was found on the live system. Management automatically turn to the tester who was testing the system to blame them for not finding the bug during testing… “How did you miss that bug!?”… but in reality, it could have been missed due to time pressures placed on the testing process, or by an extremely late release being performed to fix a “quick” issue that has ended up breaking something else.

Secondly, I don’t think it’s a situation where only 1 person should learn from it. There is an opportunity for the entire project team, or the entire company to learn from this issue. the “learning from it” shouldnt be solely focussed on the person who made the mistake, the information on the cause and how it occurred is useful knowledge for everyone to know about and learn from.

Thirdly, I think it can be counter-active blame someone and put them in a situation where they lose confidence and other people start to possibly distrust them or look at them in a different, demeaning way due to a mistake. This can ruin teamwork and moral for everyone on the project.
I believe that it is important to find the root cause for the occurrence of the problem, but that you do this and allow everyone on the project to learn from this problem without having any blame culture towards any individuals.

It is important to discover why and where the problem has occurred. This is so we can learn from problems and mistakes, and so that  we can put measures and processes into place to test and prevent them from occurring again. I also believe that we should learn these problems as a team though, and not be forced to learn them in a demoralising manner from the finger of blame being pointed in our direction.

One way that you can efficiently get to the root cause of an issue, without any finger-pointing, is to perform the “5 Why’s” method for root cause analysis. (Just one of many resources that supply further information on the “5 why’s” method can be found here – or you can google “5 why’s” for more resources on it). This is far more productive than pointing the finger at the first person that you think might be the cause of the problem.

In a blame culture, management might turn to the testers (who are generally the first people in the blame line of fire) to point the finger at and ask how this bug could have been missed. This not only causes any moral or confidence to be sapped from the individual(s) that the finger is being pointed to, but will also having an effect on the other project members who will automatically subconsciously start to look down at the person being blamed for the problem. This in turn has an effect on the entire teamwork vibe for the remainder of the project and all future projects…
But in reality, if we stick with the team spirit (with the entire team holding the responsibility for the issue), and we continue to work as a team to communicate in a civilised and respectable manner to get to the root of the problem, this is more efficient and will consequently allow us all to learn from the problem while keeping moral high.

What do you think? Have you ever had any experiences where yourself or a colleague have been blamed for an issue occurring? Or have you ever been in a situation where you have blamed someone for a problem occurring? How did you handle this situation and would you have done things differently in hindsight?

17 thoughts on ““How did you miss that bug?””

    1. Thanks for the links Phil!
      There are some really good points made in the discussions, asuch as all humans making mistakes and the effectiveness of lessons learned meetings.
      Also teh point made about programmers being the creators of the defects, and how we all think of bug free code in the light “what have i not tried, there must be a bug in here somewhere…”
      🙂

      Like

      Reply

  2. Couldn’t agree more with your points.

    I will make one comment though, and that is: as a tester, when you hear that question, before you react,ask yourself what kind of environment you’re in, and in what spirit it’s being asked?

    Most testers will encounter a blame culture at some point during their careers. It’s important to shed that defensiveness when you get into a healthier environment.

    Like

    Reply

    1. I agree. There are definitely positive ways that you can deal with this situation rather than having a stand off and getting emotional about it all.

      I was in a situation in my junior tester days, where a localisation bug was found on the live environment – for the Finnish language, an incorrect (apparently offensive) word was used on a main screen of the application. The blame got pointed at the test team for missing the bug during testing…
      I cannot speak a word of Finnish, neither could any of my testing colleagues so we would simply do a “check” to ensure the correct language was loaded and then to a sanity test to ensure the functionality of the system was operational in that language.
      But the first place that management looked to blame was the test team for missing the defect. Not the development team for introducing it in their code in the first place (although I dont think blaming the developers is productive either).
      As I tried to explain my own situation on why I didnt think the responsibility was solely with the test team, I started to think about the consequences of the blame. I was worried about how this might affect the team, me personally and with my other member of the project.

      In the end, management accepted that the team should share the responsibilitywe ended up having a lessons learned meeting as a whole project, and managed to put measures into place in the development processes AND the testing processes to ensure that localisation bugs such as this one would be covered in the future. Everyone shared responsibility for the bug and everyone learned from the bug collectively.

      Like

      Reply

  3. It’s easy to understand why testers should be blamed for any bugs that are in the shipped product: the testers were the last ones who had it? From that, it’s clear that the testers are the ones to blame for whatever problems are subsequently found in the product. So there’s an easy way to get around THAT: fire all the testers. Then the testers won’t have had the product last, and therefore there won’t be any problems in the product. (Yes: a cat could come up with a better line of logic.)

    As for Anna(testerab)’s comments: there’s something much more important than shedding defensiveness. If you find yourself in a blaming culture, make sure that the person responsible for establishing it is found and punished severely.

    One serious comment now, for Dan: be careful of the notion of “root cause analysis”, since it too can lead to a blaming culture. Unless people are taking an exceptionally simplistic view of things, there’s never one single cause for any problem. (You could say that the root cause of every bug is easy: the programmer who put the bug into the code of the program should never have been born.) Instead, consider what the NTSB.

    I would love to sit around a table at a project meeting where, instead of fingers being pointed and blame being distributed, each person offers “Here’s what I could have done to find that problem earlier, and to contribute to getting it fixed.

    Like

    Reply

    1. Thanks for the link Michael. Its really interesting how the NTSB work it so that multiple problems can be fixed, rather than just the root cause of the incident. The NTSB investigator’s statement implies that they already have a list of issues that they’d like fixed so they use the report to their advantage to get this list of items fixed. It’s just a shame that that’s the lengths that they need to go to to get the fixes they desire – it takes a major incident before they are listened to.

      You make a great point about root cause analysis leading to blame… I was thinking about that as I was writing it in the blog. I guess its all down to the communication skills and the attitude of management as to whether they try to come to a sensible conclusion about the root cause and put multiple measues into place in an understanding, compasionate manner, or whether they take the angry approach and start to dish out some sort of “punishment”.

      Like

      Reply

      1. Let me be more clear here: again, just above, you mention “/the/ root cause”. Neither the NTSB nor I believes in /the/ root cause. There’s never only one.

        Like

      2. Thats a good point… And a bit of an eye opener actually.
        This basically renders root cause analysis methods such as “the 5 why’s” defunct!

        Essentially the single answers to each of the “why” questions will only lead you down a single path to a single root cause, so all other paths (or root causes), that relate to answering the questions differently, are essentially ignored… It seems the “5 why’s” method would only be useful if you exhaust all possible answers to each time the question is asked.

        But is this not going to be the same ending as you would get with finger pointing and blaming? Where management /think/ that they have gotten to the bottom of the problem (singular) wherever the buck stops, and want to dish out some punishment?

        Like

      3. Somewhat annoying that I can no longer reply at the bottom of the thread. Sigh.

        This basically renders root cause analysis methods such as “the 5 why’s” defunct!

        Only if you limit yourself to asking five whys down. You could ask five whys across: “What else was at the top of a contributing chain of causes?” That would end up as 25 Whys, I suppose.

        But in fact, there’s prior art here. Jerry Weinberg’s Rule of Three (a special case of the Rule of At Least Three): “If you haven’t thought of at least three plausible explanations for something, you probably haven’t thought of it enough.”

        You can allocate blame using any method; in fact, you can blame someone without any analysis at all. But I suspect that to those who invented them, The Five Whys (and certainly the Rule of Three) were not intended to find someone to blame, even though you could choose to use them that way. Each step down the chain of five gives five possible points at which a problem could have been addressed by some person, and those might be different people all the way down.

        If someone is eager to find and assign blame, you could also consider another of Jerry’s rules: The loss of X dollars is always the responsible of a manager whose financial authority exceeds X dollars.

        Like

  5. Pingback: Give up Giving up | Brummie Tester
  6. Pingback: Five Blogs – 10 January 2013 « 5blogs

  7. “But in reality, if we stick with the team spirit (with the entire team holding the responsibility for the issue), and we continue to work as a team to communicate in a civilised and respectable manner to get to the root of the problem, this is more efficient and will consequently allow us all to learn from the problem while keeping moral high.”

    I think that you’re on the money here. Most work places are really high pressure places to be assigning blame in, people’s jobs, advancement, raises, or whatever could hang on the line. I think in most cases, you’re going to find that assigning blame will push people apart, cause bitterness a nd poison the development environment. Workplaces need a safe space to assess how to improve products, without attacking individual people, and as you’ve intimated, usually the blame can’t rest with one person.

    Collaboration and open communication are amazingly valuable, and trust is very hard to build up once it’s been broken. I think that a lot of work places would do well to value those above blame-seeking. Knowing who did something wrong is inevitably less important than knowing how to do better in the future.

    Like

    Reply

    1. It’s a good idea to value that stuff. The trouble is that some people, for whatever reason, don’t.

      That’s why I think it’s very important for testers to remember two things:

      1) Don’t lay blame yourself. This seems surprisingly hard for some testers. Often some time spent as a program manager or as a programmer cures the problem.

      2) When blamed, respond congruently. See Jerry Weinberg’s work (especially the latter two books in the Quality Software Management series) for more on that.

      Like

      Reply

  8. Thanks for post and sharing valuable information.

    I am the lone tester in the dept where i am working..
    The Product under development/Before testing / after unplanned release without any testing — Managers come and see the application and obviously they find some thing,which i have not tested at all.

    But they always points at me – “Srinivas – Tester where are you? you didnt test this”..

    Usually i am very calm and silent and i speak honest and straight forward in nature.

    I feel so tense and gets out of my control and my words are always at that moment they feel like a shoot,
    They doesnt feel how a person (tester) responsible for that .. even if he hasnt tested at all?

    and tell me in reply i should accept the blame… “Why i should?”

    How to resolve these attacks from management ?

    Like

    Reply

  10. Two more links for you (both from me – sorry).

    The One that got away – http://www.stickyminds.com/sitewide.asp?ObjectId=16670&Function=DETAILBROWSE&ObjectType=COL&sqry=%2AZ%28SM%29%2AJ%28MIXED%29%2AR%28relevance%29%2AK%28simplesite%29%2AF%28alan+page%29%2A&sidx=1&sopp=10&sitewide.asp?sid=1&sqry=%2AZ%28SM%29%2AJ%28MIXED%29%2AR%28relevance%29%2AK%28simplesite%29%2AF%28alan+page%29%2A&sidx=1&sopp=10

    Bug Prevention in Five Minutes
    http://blogs.msdn.com/b/alanpa/archive/2007/05/29/bug-prevention-in-five-minutes.aspx

    Like

    Reply

Please leave a comment!