A recent study of our clients has revealed an interesting phenomenon: the number of universities, school districts, training centers and all other types of commercial and non-profit educational organizations is disproportionally high among the users of our load testing tools.
We released the latest versions of our load testing tools few months ago. All users who have had an opportunity to update their product installations already know that two major features in that release were the support of HTTP/2 protocol and the ability to execute concurrent requests in each user session. In fact, each of those features is a complement to the other one, because the development of HTTP/2 was inspired by the idea of concurrency in the first place.
Yet, it is true that not every web application has switched to the new version of the protocol by now. So, what if your test project does not require any of these major features? Will you get any benefits after upgrading to WAPT Pro 5.0 from the previous version of the tool?
When you plan a load test, one of the first things you need to know about the backend configuration is if it includes a load balancer. This is important because most load balancers distribute new user sessions by the client IP address. Some of them allow changing the distribution method. However this may be achievable only by a significant configuration change. You will hardly persuade the site admins to make such changes in the production environment.
What is a bit embarrassing in this respect is that even if you select the “distribute to the least loaded” option in LB settings, this may still mean that it will use this rule only for new IPs. After the initial connect, each IP is remembered for a certain time (from 1 hour to few days) and all new connections from the same IP will be directed to the same web server.
This creates a very big problem for load testing. If you use WAPT Pro, or any other efficient load testing tool, by default all your virtual users will share the same IP address and will be directed to the same web server behind the LB. As a result, you will not be able to test the whole system correctly.
As you probably know, the percent of secure HTTPS web sites is growing every day. Moreover, even if you do not care about security, the latest news from Google suggests that you will have to move to HTTPS in any case, because otherwise you will see your site dropping down in search results.
Depending on the implementation of your site, switching it to secure connection may take from few minutes to few weeks. You may face functional problems, broken links and thigs like that. Imagine that after you have done everything required, you finally see your site working under the perfect green line of the browser address bar. The question is: what have happened with the site performance, and should you load test it again?
After several years of work with our customers I decided to summarize my experience by gathering the most common user confusions resulted in questions to our technical support service. I think that most of these issues were caused by wrong ideas or interpretations people may have had. In other words, in each specific case the problem was caused not by the lack of information, but rather by some sort of incorrect assumptions.
Of course, nothing like that would be possible if everyone spent couple hours reading the user guide before trying to use the product. But for some reason people rarely do it this way.
When you load test a web application and gradually increase the number of virtual users during the test, you usually expect to see gradually degrading performance parameters such as response time. The number of pages (sessions or hits) per second will reach maximum in some moment and will probably stay the same for the rest of the test despite the growing load. This is simply because the site cannot serve more requests per second, so it has to postpone the new coming ones.
In reality you will probably see something like that, but only for a limited time. When the load goes even higher, you can get another picture that may seem surprising at first glance. It is illustrated by the following three charts. Black line in all charts represents the number of virtual users.
Let’s suppose that you have already resolved all problems related to unexpected errors in your test. I mean that your site may produce some errors on heavy load, but the emulation of the user sessions is performed correctly and the error rate is acceptable. If this is not the case, or you have any doubts, you may want to read my earlier posts here and here.
If the errors are not an issue, your next step is to look at the most evident indicator of the application performance – its response time. This term initially refers to a time required for your application to respond to a user action. This looks very simple, but you should keep in mind the following things, if you want to analyze the test results correctly.
If you see any errors after executing a load test, first of all you should check if this is a test design problem. I wrote about the most typical problems of that kind here.
Now let’s suppose that the test was designed correctly and any errors we see in the report are related to the load you created in the test. In other words, I would like to talk about errors that appear because of performance problems, not anything else. If you are not sure why an error takes place, try running the test with same profiles and a smaller number of users, and see if the same problem appears.
First of all I should note that there are different types of errors.
In the last days of 2013 we released a cloud version of our load testing solution. It is based on WAPT Pro, all extension modules and x64 Load Engines, so it is the most feature-reach and powerful combination of all the components we have.
This on-demand load testing solution is offered through the Amazon EC2 Marketplace. This means that with few clicks you can get the full functionality of WAPT Pro running on a virtual system instance in the Amazon Cloud. To do this you need to have an AWS account, however it is also very easy to create one. We provide step by step usage instructions right from our web site.
Some experienced users already know that all our products can be used in any cloud or virtual environment. So, technically this release is not a big step forward. With new cloud version it is easier to start and configure the product, but this is not the actual benefit. What really makes the change is the pricing for the new solution, which is now based on the hour rates.
When you start load testing a web site (especially if this is the first load test in your life) you may see a lot of errors in the report. Possibly you will even have to stop the test before it completes, because at some point in time it becomes clear that something goes wrong. If the problem is not so obvious, it is still recommended to check the report for the errors related to each virtual user profile (i.e. to each different type of virtual users) before looking at any performance data.
Usually you do not create a high load in the very first test of a web site, so if you see any errors in the report, most probably they appear because of the test design problems. In other words, the emulation of the real user activity is performed incorrectly. Your web application may produce errors and even refuse connections because it receives incorrect data from your load testing tool. Why this may happen?
