I worked with a developer once who was troubleshooting a defect I found, and after trying to duplicate it his response was “it works fine on my machine.” He was technically correct (it did work fine on his machine), and he wasn’t trying to avoid solving the problem, he was merely commenting that the code was functioning properly on his system. After working on it for some time, we found that at one point in the past he had an older version of the software on his machine which included a library file that was missing from the new version. Hence, when he ran the program it functioned properly, yet on a new install it would fail.
This shows the importance of ensuring that your testing systems are not compromised. It also shows that you should be testing from two standpoints: as the “existing customer” and as the “new customer”. As the “existing customer”, you should be testing on a system that had a previous version of the software installed. This includes upgrade testing as well as testing on systems where the software was previously installed but has been removed. This ensures that the current customer base can successfully use the new version without issue.
As the “new customer”, you should be performing clean install testing using a fresh machine or image where your product has never before been installed. This will catch any issues like the one I noted above. Also, this machine or image should not have any development tools installed, since they can give a “false pass” when testing. For example, if the development tools are installed and they include a library that your product depends on, the product may run correctly on that machine since the library exists. On a machine without the development tools it would fail.
Upgrade testing vs. clean install testing will ensure that both existing customers and new customers are able to use your product without issue, and eliminate the “it works on my machine” scenario.
Share on Technorati . del.icio.us . Digg . Reddit . Slashdot . Facebook . StumbleUponRelated posts:
