Public Beta Testing vs. Professional QA

To the optimist, the glass is half full.  To the pessimist, the glass is half empty.  To the professional tester, the glass is twice as big as it needs to be…

…and that is an example of the difference between public beta testing and professional quality assurance services.  Public beta testers start as people who are likely to use your application and may on the surface sound like a good resource for quality assurance.  However, they are not expert quality assurance engineers and as a result, their feedback is likely to fall short of the mark.

Talented and experienced testers are trained to “think outside the box” and provide as much information as possible to the developers so that the defect may be reproduced in the development environment and fixed.  The information typically includes:

• a short description of the defect
• a long description of the defect
• specific data for the platform upon which the defect was discovered
• a workaround if possible
• and most importantly the detailed steps to recreate the defect

Armed with this data, developers can easily zero in on the lines of code giving them trouble.  Whereas public beta testers can sometimes provide useful information into the functionality and/or usability of an application, the bug details are generally inadequate often causing developers more headaches than solutions.

Finally, professional quality assurance always comes with clean, referenced systems used for testing.  The lab environment is as important to the testing effort as the testers themselves.  Computers must be free from third-party applications that may conflict with the software under test.  Unfortunately, public beta testers always test on systems loaded with any number of applications.  Those applications not only cause potential conflicts, they themselves may possess defects of their own.

When faced with the need to test your software products, the prudent decision is to choose a resource for professional quality assurance services and leverage the talents of trained testers.