When we conduct load tests, it is extremely useful not just to know the performance metrics of the web application under test, but also to monitor the performance metrics for any associated components such as operating systems, database servers, web and application servers, networks etc. When we see poor performance on a load test, that performance is often correlated with the performance metrics of an associated component. For example, poor performance may be the result of a poorly optimized database or it could be the result of overloaded operating system that is running on a machine where other competing processes are running.
Let’s take a concrete real world example. Let’s suppose you are testing a web application. The web application runs on a Windows machine. It is built using the J2EE stack and runs on the JBoss Application Server. It connects to an Oracle database. You have created several scripts to test your web application. The first script loads a set of web pages some of which are dynamically generated by the J2EE servlets on the web application server. The second script loads a set of web pages dynamically generated by servlets that use the Oracle database to read data. When you start testing the first script you notice poor performance. However at the same time you notice both the Windows and the JBoss application server’s performance graphs look suspect. You are tracking many of the Windows performance counters one of which is the Memory\Available Mbytes counter. This counter measures the physical memory available for running processes. If this counter is say less than 5% of the total physical memory, this implies that you have insufficient physical memory. This results in increased paging and drastically affects performance. You find that this number is only 2-3% of your total physical memory. It becomes clear that the web application server does not have sufficient RAM. You add some additional RAM and the performance of the first script drastically improves and meets your acceptance criteria. The second script continues to fare poorly. A closer look at the mem.value and disk.value in your Oracle performance monitor reveal that the Oracle sort memory is not big enough. If the sort memory is insufficient, sorting will use the disk and query times will be affected. Your database administrator tunes the Oracle sort memory and voila! Your performance metrics now improve significantly and now meet the acceptance criteria.
You can see that the performance monitoring of associated components can often reveal the root cause of performance problems. This is extremely helpful in troubleshooting performance issues encountered during load testing. Troubleshooting performance problems without understanding the performance characteristics of associated components is like searching for a needle in a haystack.
