Case study: Load testing of a website for patients training

Our client is an innovative medical equipment and service company based in USA. Due to NDA restrictions we cannot explicitly mention the client name. The company created a new website designed for patients requiring regular medical procedures. It is a learning website and informational resource where people can train their skills in applying medical devices for personal use.

Test goals

Before releasing the website to the customers the company decided to test its capability of handling up to 5,000 simultaneous users providing industry standard level of service in terms of response time and error rate. The company sought to fully outsource the testing task to an independent team.

Technical features

  • The website content included a number of multimedia presentations with large-scale media data.
  • Due to the implementation technologies the application server provided essentially different data representation depending on the client browser.
  • Typical user session duration was about 30 minutes, which was longer than the regular case estimation.
  • In terms of the number of HTTP requests an average session was also longer than usual with over 1,000 hits per one site visitor.

Solution

Basing on the previously accumulated statistics regarding the behavior of online users the following browsers were selected for the emulation: Internet Explorer, Chrome and Firefox. The design of the test sessions and adjustment of the scenario parameters took one week. This included extensive communication with the client to make sure we were building a true realistic emulation.

Testing process and results

During the period of 3 months after the initial test implementation we performed 5 iterations of performance testing. Each iteration included a series of capacity, stress and endurance tests. The system had to be adjusted between iterations to fix the performance and functional problems revealed in the tests. This explains a relatively long period required to complete the project.

The first iteration of capacity and stress tests had shown that the site performance was below the target level. The site appeared to be capable of handling up to 2,000 concurrent user sessions without performance degradation. Under such load the error rate was almost zero and the response time did not change from the baseline numbers. However starting from that point all the way up to the 5,000 users mark we saw the proportional growth of response time with the factor of 20 being the final result achieved at the peak. Error rate had also reached an unacceptable level of 5%.

We identified most problematic transactions and created a report that helped the customer development team fix the issues and prepare the system for the next round of testing. Each of the subsequent three testing iterations revealed less severe, but still significant performance problems taking place under more specific conditions. The initial test was altered to implement each condition and outline the corresponding bottleneck in the report.

The final iteration was a long endurance test running for 24 hours with constant load. As all the performance issues had been fixed on earlier stages, this time the result was fully successful.

After analyzing the numbers obtained in all test iterations we provided additional recommendations to the client on further optional improvements to the system which could serve the resource usage optimization purposes.

Conclusion

The client appreciated the value of our load testing tools and services. After a short while the same client decided to utilize our services again for a completely different project which included testing of a CRM solution based on Salesforce integrated with Microsoft Dynamics.