6 areas to consider when thinking about testing estimations…

Setting estimates can be a difficult task. I’ve seen many mistakes made in the past from people missing out essential testing tasks in their estimations, to setting a testing estimation based on a percentage of the development estimation set for the project, to people completely guessing at a number of days/weeks/months that they believe the testing will take.

An inaccurate estimation can definitely cause problems on the project, where not enough time gets allocated to allow for adequate testing to be performed leading to either a poor quality product being released or a postponement in the product delivery date. Both of which scenarios create an unhappy customer…

It’s important not to set the testing estimate based on a percentage of the development estimate because the can be scenarios where a new function might only take half a day to develop, but it might affect every other functional area of the system that the new function is being integrated with, meaning that testing will take a lot longer than half a day, yet if you set a percentage of that half day, you might have supplied an estimate of an hour…

Testing estimations should be set on their own accord, by an experienced tester with adequate knowledge on what it takes to test the product.

So, where to start in setting an estimation?

When  asked to supply an estimation of testing time for a new project, you need to think about the bigger picture and take the following aspects into consideration:

1) The list of requirements (the same list of requirements that the developers would use to set their estimates): It’s important to communicate with the developers and stake holders to build up knowledge of the scope and expectations of the system. Without knowing what the scope is, then it will be difficult to know what testing effort is required.

2) Mind maps: Draw up some mind maps! This allows us to think of more scenarios and end to end processes that we might have missed just looking at the spec.

3) Think of the different types of testing that is required for the project: If Load Testing is required, then this will add considerable time to perform, which should be factored into your estimations. For Regression Testing, communicate with developers to discuss release cycles – Find out how many releases stages are anticipated. Towards the end of the project, regression testing will take more time than at the beginning due to the system growing with each iteration of releases. For UAT phases or Beta testing phases, it could be required that you aid the customer with their UAT phase or a Beta test to help and coach the users with their testing. It’s important to think about all the different testing methods that are required to be performed.

4) Think about the 3 aspects of the time around your testing; “Planning and preparing”, “performing” and “bug management and investigation”.

  • “Planning and Performing” is aimed at the amount of time that you will need to learn about the system, create any test documentation (test plans if required, mind maps to delve deeper, report documents, etc), and prepare for testing (set up test environments, obtain and set up relevent tools, and creating/setting up test data needed to actually perform the testing).
  • “Performing” is aimed at the amount of time to actually test the system, testing the requirements, performing end to end tests, integration and regression testing, and of course an element of retesting should be taken into account.
  • “Bug Management and Investigation” is aimed at the time that it takes to actually investigate found defects further (sometimes in collaboration with the developers), appropriately log the defects found and then chase up on any defects.

We need to think about all of these aspects when estimating time. I’ve seen people supply estimates without taking into account the  time it takes to plan your testing, or set up test data, or think about the time it takes to log bugs or prepare bug reports for the customer.

5) Use experience: think back to previous products and any problems with estimations that you have experienced, then apply the knowledge gained from those lessons learned with the estimations you are making.

6) Add some contingency! It’s important to add time for any unanticipated problems and for additional bug retesting that might be required.

Thinking of these aspects and taking them into consideration is a great start in being able to set an appropriate estimation for testing time for new products.

3 thoughts on “6 areas to consider when thinking about testing estimations…

  1. Good pointers although when it comes down to it, a estimate is a guess. And guesses are frequently wrong.

    Estimates seem pointless to me so I don’t give them as ultimately you are working to deadlines.

    So I prefer to spend my time differently and I’d rather prioritise and test what we can until we don’t have time to test anymore (for whatever the reason is).

    Alan has a good post on estimates: http://blog.eviltester.com/2011/06/how-can-i-estimate-my-testing.html

    Like

    1. Thanks for the comment Tony!
      And thanks for the link to Alan’s blog post too. I hadn’t read that one before.

      Good point that estimates are guesses! But I think there is a difference between a complete guess and an educated guess. When offering estimates, hopefully there’d be an educated guess based on past experiences of testing similar systems and from the same developer previously, where you know how long it took the last time, so this project could hopefully be the same.

      I know that it’s impossible to see into the future and issues arise that could totally throw off all the estimates supplied, but in this situation, it might be possible (hopefully) to update or modify the estimates that were supplied, taking into account the new issue…

      I’ve definitely been on projects where it is a set deadline too, and havent bothered to supply an estimate at all for these types of projects, but my blog was stemmed by a project that I was working on where I was told: “there is no deadline set yet as we want to take into account how much time you need, so how long do think you’ll need to test this new functionality?”. Needless to say I was shocked (and inspired at the same time)! 😀

      Like

Leave a reply to Tony Bruce Cancel reply