Is This Bug Worth Reporting?



One of the primary pursuits of software testers is to verify if the application under test conforms to system specifications.  In this pursuit, testers create bug reports whenever test cases fail.  This is how the professional life or journey of a typical software tester starts.  At some instance in this journey a realization strikes.  It is a voice from within and it says, “The quality of bugs matters more than the quantity.”  This is when a software tester understands the true meaning behind the purpose of testing. This post is about a real life story to illustrate this phenomenon.

Anita, a software test engineer joined our project team several years ago.  That was her first project on her first job.  She held an undergraduate degree from a well-known college.   She was very fast in finding failures.  She would find failures against test cases and raise bug reports.  She topped in doing this.  She was happy with this progress. However her interaction and relationship and rapport with developers became unhealthy.  Every defect she found involved not only additional research and debugging but also a great deal of conversation and negotiation. Sometimes more than five out of ten defects reported by her would end up in categories such as ‘Not a Bug’ or ‘Duplicate’. Eventually developers started seeing no value in her bug reports.  She was demotivated.   When we analyzed this situation we understood that she was not collaborative enough and did not think through before creating bug reports. Somehow she was passionate about maximizing the number of defects but not paying attention to the quality of defects.  This episode turned out to be an unpleasant experience for her as well as the developers. One of the senior members in our team coached her and she was willing to learn and improve. It took her several weeks. She was getting better.

One day she was extremely happy because a bug she reported was fixed on the same day.  The developer who fixed it was impressed too.   I appreciated her and asked, “This is awesome. How did this happen?”  She retorted, “Thank you! I know, I did my homework before reporting this bug.  I discussed it with our developer, understood the context, did additional testing by considering related scenarios and created a better report. This approach helped us reduce debugging time.”  I smiled at her with a sign of satisfaction.

Gradually she become a successful professional and started contributing to large projects.

Several years have passed.  Nowadays, I am not playing the role of a full time developer or tester or project manager. I am a specialist. I work with organizations and teams. I write, speak, coach, consult and do several other things.

Couple of months ago I was using an online application that facilitates conference management and includes features such as attendee registration, speaker submission and so on.  This is a new system developed by an enthusiast who is a hardcore techie. Sometimes he works with one or two developers supporting him. Otherwise, he does it alone by adding features, resolving issues and making it better.  As a member of program committee of an upcoming conference, I use this system daily.  All of us in the program committee are aware of the known issues and one among them is a pagination issue – we had to refresh pages couple of times to get the right number of records.

The other day, I observed incorrect number of submissions under a specific category in spite of multiple attempts to refresh a page. I was reluctant to relate it to the pagination issue. I thought it could be due to some issue with the browser cache and went on to complete my tasks of the day before I investigate it further.  That issue did not leave my mind.   I did not react either. I remembered Anita’s happiness when she did it right and got that bug fixed efficiently. I asked myself, “Is this a bug or issue worth reporting?”  The answer was not affirmative.  I wanted to get back to the system and explore it further so that I can help the developer with my bug report.

Late in the evening, I went back to the system. Even though I was able to reproduce that defect I started examining the corresponding behavior under various scenarios and compared it with similar features.  To me, the ability to reproduce is a necessary but not a sufficient condition to report a bug and this bug was not worth reporting yet. I wanted to narrow it down and isolate it so that I can write an informative bug report. 

After multiple attempts, I started getting some clarity. I isolated it to a specific scenario across screens. It appeared to be a programming error or configuration error. I reported it with all my findings.  And the bug was fixed within an hour!

Looking back, I find that these are the small incidents worth sharing as they leave behind some takeaways. Test case failures are most probably the leading indicator of defects.  However, a test case failure need not be defect itself.  It is the responsibility of a software tester to do additional probing or research to isolate the failure in order to write a meaningful bug report.  Next time when you come across a failure, ask yourself, “Is it a bug worth reporting?”  Also, attempt to analyze it further and write a great bug report.  When you do this you will enjoy your work and become a valuable contributor in your team.