Case study: Testing of iOS application with WAPT Pro

Noveo is an international software development company with offices in Europe and Russia. It provides full range of cost-effective IT services to organizations requiring deep technical expertise for their projects. Noveo's services include IT audit and consulting, software development, independent testing, product maintenance and migration of data and applications.

Background

Noveo participated in a startup business providing online ticketing integrated with additional services offered to the cinema customers. The end users of the web application were supposed to browse for information on movies and related news, discuss topics of interest using online chat and finally purchase tickets to their local cinemas.

Test goals

QA team at Noveo had performed functional testing on the application and faced a problem of checking the application performance under load. As the required performance parameters had been established in terms of maximum transaction time on each step, the goal was to check that the criteria would be met when the service is provided to 2,000 users concurrently.

Technical features

  • The client side was a mobile application working on iPhone or iPad.
  • The client-server communication protocol was implemented as REST API service.
  • WebSocket protocol was used for communication upon availability.
  • The server side utilized MySQL database as a storage. Its performance parameters were to be measured separately during the test.

Solution

Noveo had evaluated several load testing tools and chose WAPT Pro because of its ease of use, flexibility and performance. Another significant factor was a possibility to closely cooperate with SoftLogica's team during all test phases from the implementation of realistic user sessions to the interpretation of the obtained test results.

Testing process and results

Load testing was performed on the production server just before the release. The QA team had only two days for all testing jobs while the server was available before going live.

The very first load test had shown that even with a rather small growing number of concurrent connections the application demonstrated rapid performance degradation. It was imperative to find the cause of the problem, which could be a bandwidth limitation, server performance issue, or something related to one of the network components.

A fast hardware upgrade on the application server had not resolved the issue. The performance remained on the same level suggesting that the roots of the problem were different. Bandwidth and network components were also checked for performance separately and were unlikely to make significant contribution to the issue.

To further research the problem, the QA team enabled monitoring of key performance parameters on all backend servers. WAPT Pro could do this by means of SNMP and WMI interfaces for system parameters (such as CPU and RAM usage) and ODBC connection for database server counters (such as the number of slow requests, number of connections to the database, etc.). As a result, a number of issues related to server configuration had been revealed. In particular, we identified certain restrictions in the SQL server settings, such as the maximum number of connections available for the application, which prevented the system from scaling up.

After fixing those issues the testing had shown significant improvement in performance, however it still remained below satisfactory. We decided to look deeper into the user sessions and check each transaction separately. As a result a slow request was found in the session. Server response time for that particular request had been growing throughout the ramp-up test phase with degradation factor significantly exciding that of other similar transactions. Development team identified this as an implementation issue and was able to fix the problem shortly.

Few more tests had been conducted to optimize the system configuration, as it was required to check how the adjustments influenced overall performance. Consequently, the efficiency of database caching had been improved and communication with application server had been further optimized by redeploying two virtual servers on the same physical machine.

The final round of testing confirmed that after all fixes and configuration adjustments the system became capable of standing the load of up to 2,000 concurrent user sessions providing required level of service.

What Noveo says about WAPT Pro

WAPT Pro is an easy-to-use and yet a very powerful tool for load and performance testing. It is perfectly suitable for mobile applications and implementation of really complex client-server communication schemes. Most similar tools on the market provide such functionality for much higher price. Finally, the responsive team backing the product helped us do things faster and with less effort. In just few days we managed to carry out the whole realistic testing on our production server, found serious system bottlenecks and prepared the system for going live.