I’ve been reading a topic on the LinkedIn groups recently about a bug that was found but was rejected by a developer as “Could not reproduce”. The person who initiated the discussion had said that he went to re-test the defect, but found that it was un-reproducible and he was looking for advice.
As always, there were very many different responses from “the Developer fixing it, but marking it as un-reproducible for stat purposes”, to “Who cares? It’s fixed isn’t it?!!”.
I’m sure that most testers have experienced this situation and I’m also sure that some of us would have simply closed the defect accordingly, after all, it’s not reproducible by yourself again, so it must be fixed… right?
There are many reasons that the bug may be dormant and therefore un-reproducible, but still exists and relates to a separate entity that has not been thought of.
Take “timing” for example. It could be the case that the tester found the bug initially at 6pm, when a daily maintenance log was running in the background which affected the system and caused the bug. If the developer tries to investigate the bug the following morning, then it obviously wouldnt occur if the maintenance log isn’t running in the background. So the bug would still exist, but would only occur at 6pm during.
Or “data” could be the cause. if the tester has used random data that has triggered the defect, but doesn’t record the data that was used in the bug report, then the developer might have used completely different data which doesn’t reproduce the bug. Or angle where data could be the problem is if the bug was found on legacy or migrated cases on the system and the developer tries to reproduce on newly created cases on the dev site…
So what should we do next?
Well, this would really depend on the severity of defect… If the bug was a minor defect that was of low risk to the customer, then I would be tempted to close it after some discussions with the developer and PM.
On the other hand, if it was a major or critical issue (i.e. an ugly error screen with a big red X is thrown), then firstly I would update the bug log accordingly to detail that the bug appears to be sporadic – but I wouldn’t close it yet!
I would then plan to collaborate with the developer to discuss all of the variables that could be the cause of the bug and then try to emulate the scenarios as close as possible to when the bug was found initially. Investigation could also be done by the developer to determine if anything had been changed at all elsewhere on the test site by someone else, to see if that could be the reason for the defect not appearing anymore.
If the bug is still not reproducible after further investigation, then I would discuss the option of closing the defect. But if it turns out that the bug is reproducible under a certain scenario, then great! We can now return the defect back to the developer with the details of the conditions that the bug occurs in.
How would you handle this situation? Would you do anything differently?