Learn from the “Inattentional Blindness” in your testing!

I’ve been researching a lot about “Inattentional Blindness” lately, and have been relating it to the world of software testing…

Inattentional Blindness is a phenomenon where, if you are focussing intensely on something that is mentally demanding, you fail to notice other things happen around the thing that you are focussing on.

There is a great demonstration called “The Monkey Business Illusion” from Christopher Chabris and Dan Simons that can be seen here (it’s the second video down) – Watch this before reading on, if you havent seen it already!!

It’s astonishing to think that when you are fixated on something that demands focus and a certain amount of brain power, that you can automatically subconsciously miss glaring things that happen right in front of your eyes. Even although it seems that you are giving 100% of your attention to the task at hand, your mental workload is solely focussed on that sole task, so turns off at all other areas around that task.

Thinking about this from a testing perspective, how many times have you been browsing through the software you’re testing and you are focussing on looking at one specific thing? Perhaps one specific function, or maybe you’re specifically looking for one type of bug?

The video proves how easy it is to miss something… Be it the Gorilla strolling through the basketball players beating its chest, or the fact that the colour of the curtain in the background changes from red to yellow, or even that a member of the black team walks off the stage. (If you didn’t notice these in the “Monkey Business Illusion” video, watch it again – it’s all true).

What I have noticed though, is that when you become aware of the changes, it is difficult NOT to spot them when watching the video again. In fact, if you watch ANY video that asks you to count the number of basketball passes for a team in a certain colour, you automatically subconsciously begin to look out for a gorilla walking through the set. This is because you are now aware of that scenario, and its impossible to be fooled twice.

But does this mean that when we are testing software, we are automatically on the look out for bugs and scenarios that we have found (or we have been fooled by)  previously, simply because we are aware of their possible existence?

I would say YES…
Take a typical search screen on a typical web app for example. If you are testing a search screen, and you have previously tested search screens before, you might automatically think about various areas such as: pagination for the results, the responsiveness of the results being returned when searching for the maximum amount of results possible, trying wildcards symbols (e.g. using “a%” to search for anything beginning with the letter A), entering search criteria in every field at once, etc… You’ll be looking at these areas because you have experienced them before.

When you’re concentrating on the search screen’s functionality, are you thinking about the screen’s branding? Or are you looking for any spelling or grammar issues? How about any W3C accessibility requirements? It’s impossible for your attentional capacity to handle all of these areas to test all at the one time (therefore it’s important to use the effects of expertise wisely during your test planning phases!!).

In a sense, it’s our experiences in these situations that help form our oracles that enable us to identify bugs in the software we are testing. This is the exact same as the gorilla video. Discovering that a gorilla walks through alerts our oracles so that we can identify it again when we see it again.

But still, there is a possibility that we might miss other various changing or obscure attributes due to only focussing on the one main thing to learn from and branch our oracles to… Take this second video for example. It’s also created by Christopher Chabris and Dan Simons – again, watch before reading on!

Did you see the change occurring in the first instance of watching the video? It’s very similar to the change in colour of the curtain in the first video, where the change happens gradually over time. This is a certain type of Inattentional Blindness, known as “Change Blindness“. So when we initially were watching the first video, and we discovered the truth about the gorilla walking through the basketball players, that became our main focus of learning not to be tricked by that again. The other changes (curtain changing colour and member of the black team walking off) came secondary when we were subconsciously learning not to be fooled again.

So to finish up, I’ll leave it with a 3rd video… This is a “whodunnit?” video that was used in a “Look out for cyclists!” advert. Can you spot the 21 changes?

8 thoughts on “Learn from the “Inattentional Blindness” in your testing!

  1. I’m quite a follower of psychological phenomena and the nature of experiencing vs objective reality. A good primer for inattention blindness, thanks for the post!

    I found it mentioned in a self-help lecture recently in that we get more of what we focus on. He told the room to look for every object in the room which was red then they were to close their eyes and shout out the name of every blue object they saw. Silence. He made the point that we find more of what we focus on, and we attribute meaning to things that don’t fit the pattern just so we can find more of it (confirmation bias thrown in for free). So in testing we’ll find more of what we’re looking for, and sometimes we’ll attribute meaning to something just to make it fit that pattern when it doesn’t. A healthy dose of scientific scepticism needed all around!

    Like

  2. Nice post. I found it interesting to note just how hard it was to correctly count the number of passes the players made in the “The Monkey Business Illusion” when I was watching for the curtain or the Gorilla. I wonder how many bugs get missed in a feature being tested just because the tester became aware of other issues around the page?

    It certainly seems to re-enforce my feeling that you should test in phases, first a sanity test to give everything a quick once over (it’s pretty embarrassing to discover a glaring bug after 2 days of testing just because it appeared at the bottom of the screen..), then a detailed test of the new feature and finally a detailed regression of the system or of the system that surrounds the new feature. Hopefully this would allow you to correctly count the passes AND see the Gorilla 🙂

    Like

    1. Hi Amy! Great angle! It’s really difficult to do both tasks (counting and looking out for the gorilla) at the same time, even although you know about them both. Do you think this might be where “Testing Tours” might be useful? Where you are potentially testing the system from each various angle?

      Like

  3. Why is there not a tool that doesn’t track this? That would be cool, no? You can get heatmaps for websites, mostly used for testing websites and where users click, it would be nice if this could be expanded to other uses. Switch it on, leave it running, do some testing then view which areas (visually) you have been focused on.

    Like

    1. Brilliant idea Rosie!
      I’ve got a list of test tools that I want to create, but just need to find the time…
      It would be useful during a UAT or Beta testing phase too, to see the most used and least used areas that the real users actually focus on.
      I’ll defo add this idea to the “tools to create” list too! 🙂

      Like

Please leave a comment!