We have worked with lots of test systems and had many organizations consult with us about their performance and load testing efforts. Over the years we have encountered many common mistakes. These mistakes often greatly compromise performance and load testing effectiveness. In this post I will discuss some of the more common mistakes that are made during performance and load testing and how to avoid them.
1. Testing on a system that does not resemble the production system: The system used for load testing needs to mimic the production system as closely as possible. If your production system, for example, has 8 GB RAM but your test system only has 2 GB of RAM, it is highly likely that for a given number of virtual users the response times will differ. In addition, the operating system and any installed software on test system should be configured in the same way. This will lower the likelihood that the test system and the production system will behave differently.
2. No page validation in scripts: Load testing tools will report errors when an HTTP request produces a response with an unacceptable response code .e.g. a 404 (page not found), a 500 (server error) etc. However you may have still have errors that will produce perfectly valid HTTP response codes. For example, you may have a web page where you submit an invalid login name or password and the resulting page asks you to attempt to login once again. You might consider this situation an error but there is no way for the load testing tool to know this (since the page will probably produce a valid HTTP response code of 200). You need to specifically direct the load testing tool to invalidate such a web page response. You can do this by using a text checkpoint where you can look for specific text on the page and use that condition to validate the page.
3. Not starting load testing early enough: Performance bottlenecks are best identified as early as possible. If you wait until the system has completed development, you might find performance issues that require a architectural redesign. This would take time and would be likely to result in missed deadlines.
4. Test Cases do not represent real scenarios: This might seem obvious but I have seen many systems that underwent load testing but suffered significant performance problems and downtime after they were released. The most common reason is that the test scenarios that were used for load testing did not fully represent actual real world scenarios. The closer your test cases are to real world scenarios, the greater the likelihood that your production release will not encounter too many issues and will be successful.
There are several other mistakes that are made but the ones listed above tend to be the most common and appear to have the greatest impact on the success of the project.
