After several years of work with our customers I decided to summarize my experience by gathering the most common user confusions resulted in questions to our technical support service. I think that most of these issues were caused by wrong ideas or interpretations people may have had. In other words, in each specific case the problem was caused not by the lack of information, but rather by some sort of incorrect assumptions.

Of course, nothing like that would be possible if everyone spent couple hours reading the user guide before trying to use the product. But for some reason people rarely do it this way.

1. Confusion between virtual users and user sessions.

A very common mistake for novices is to think that each virtual user executes exactly one session. This is not so. Otherwise it would be hard to control the test volume during the test. To keep it on the same level you would need to add a new user right after some old one finishes its session. Instead of that, the same user just starts a new session, which is completely independent from the previous one. This is essentially the same thing, but in different terms.

So, in fact, each virtual user executes sessions one by one. As soon as it completes a session, it starts the next one, and so on until the test is finished. Thus, each virtual user emulates several successive real users and can execute corresponding number of sessions during the test. At any moment the number of active concurrent sessions is equal to the number of virtual users. In case of ramp-up load, the more users you add, the greater load will be created against the web site.

Some people confuse the number of concurrent users with the total number of user sessions in the test. Sometimes after seeing in logs several sessions for the same user, they ask how to make each user execute exactly one session. While this is possible to implement with the “Stop user” operator, this is rarely required in practice. You only need to understand that all successive sessions of the same user are totally independent from each other. Replacing one user with another is the same as starting a new session independent from the previous one.

2. Emulation of different browsers.

A very typical user question is: which browser is used in WAPT for the emulation of virtual users? This question probably arises from the necessity to test an application from a specific browser.
WAPT does not use any browser when running the test. That would be very inefficient, because browser object consumes significant system resources. It would be hardly possible to run thousands browsers on a system.

So, WAPT performs all the emulation by itself. To emulate a user session it sends the recorded sequence of HTTP requests with only some modification of the session-specific values.
The site determines the client browser by the “User-Agent” header. In general, you can change that header in a recorded profile. However in some cases web sites provide different content for different browsers. So, if you want to emulate a specific browser, use it for recording.

3. Requirements in terms of number of transactions per second.

A very typical question is: how can I make WAPT execute exactly 100 transactions per second? Why I cannot set this directly?

The answer is: because the number of virtual users required for this depends on the performance of your web site. WAPT emulates the behavior of real users, not some separate transactions. What if the site is not capable of serving more than 50 transactions per second? How can we emulate 100 ones in such case?

In fact, the number of transactions (sessions, pages or hits) per second may be a good indicator of the site performance. However it is not appropriate to set it as a requirement for the test specification.

4. Understanding of session execution speed.

Real users do not click links and buttons one by one. They make pauses to read page content, think, type something, etc. Load testing tools emulate those pauses by inserting delays between HTTP requests. The duration of each pause is called “think time” and can be adjusted in the properties of the corresponding request.

Some WAPT users do not take the above into account or do not understand the importance of those settings. If you increase or decrease think time, you can change the resulting test volume dramatically without changing the number of virtual users.

Initially think times are set on recording. That is why if you start recording a session, then decide to have a tea break, then get back and record the rest of the session, somewhere in the middle of your profile you will have a 15 minutes pause, which will be reproduced during the test.

Note also that when you verify a profile, think times are ignored. That is why verification may work significantly faster.

5. Test recording issues.

Sometimes it is not properly understood that a session can be correctly replayed only if it has been recorded correctly. To do this, you should always start the process from entering the landing page URL to the address bar. If you already have a page open in a browser, you should not start with clicking a link on that page.

It is also important to delete browser cache and cookie files before recording. WAPT can do this automatically if you leave the corresponding options checked in the “Recording Options” dialog.

If you do not follow the above rules, the recorded session will be valid only for the environment existing on your system before you started recording. WAPT will not be able to fully reproduce the corresponding state during the test and the session will be executed incorrectly.

In general, it is possible to append new content to an existing profile. It is even possible to move and duplicate requests inside a profile. However these features are really not for beginners. This means that you should use them only if you understand very well what you are doing. Otherwise you can easily break the session integrity.

6. Ignoring of logical errors in user sessions.

I can only guess how many people test their web applications with incorrectly working tests. They just record a user session, specify the load parameters and click the “Run” button. If they get some values for response time, they think that this is ok.

In fact, in such a manner you can test only very simple web sites with mostly static content. Many web applications will not let you use the same user account to login in multiple concurrent sessions. If the session changes somehow the internal state of the system, you cannot repeat it again and again with the same initially recorded parameters. Session-specific values should be replaced with variables before the recorded session can be replayed correctly.

The problem is that in some cases web applications do not report errors through HTTP codes. That is why you may see only successful sessions in the report even if the emulation is totally incorrect. To understand that something goes wrong, you need to check logs. Unfortunately many users do not care about that.

7. Wrong approaches to parameterization.

After the above mentioned problems are fully understood, people usually start thinking on how to avoid them. In the best case scenario they spend couple hours reading user guides or watching demo clips showing how to deal with session-specific values. Unfortunately many users feel that they are too busy to read anything, so they start inventing their own methods.

A very typical situation is when people duplicate the same profile to change some values in the created copy. So, instead of a single profile with parameterized values they create hundreds copies and manually replace values in every copy. They stop this only when they see that WAPT cannot handle so huge amount of test sources.

Another typical mistake is to try using predefined lists for session-specific values created by the server. In such cases you need to extract values from the server responses rather them take them one by one from the predefined files.

8. Misinterpretation of the 304 and 401 HTTP codes.

After verifying a test and seeing the 304 code in some responses people often start thinking that something went wrong. This is not so. All 3XX codes mean that the server redirects the request to a new URL provided in the response. In such case the client sends another request to that new URL. So, if you see the 3XX code, this only means that the corresponding page required sending more than one HTTP request to complete. There may be several redirects for the same page and the last result can be either success (200 OK) or error (4XX or 5XX).

A similar situation is the 401 code. Formally, this is an error, but it is always returned in case the site requires authentication. So, it is normal if you see it. All you need to do is to specify correct credentials in the profile properties. You will still see that message (or even two such messages) in the log for that page, but it will be followed by a response with another code and actual page content in the response.

Overall, I would recommend taking a look at the HTTP response codes reference before working on web testing. This may take some 10 minutes.

9. Handling of different authentication schemes.

There are several authentication schemes used in web applications. When you access your site from a browser, a popup window may appear asking from credentials. This means that the site requires authentication. In such case you need to enter the credentials in the profile properties.

However note the following.

• In some cases browser may use your system account credentials and will not show that dialog on recording. Still you need to enter credentials in the profile properties, if you see the 401 code on verification.
• User name may include domain name. This is often forgotten.
• If you enter user name and password on the login page (not in the standard security prompt window), most probably the authentication is performed by sending the credentials in the parameters of the login request. This is an application-specific authentication and in such case you do not need to specify anything in the profile properties. You can parameterize the values of the user name and password directly in the properties of the corresponding request as shown here.
• Some sophisticated sites may use encryption to avoid sending passwords in the explicit form. The only way to deal with such cases is to implement the same encryption in JavaScript. This only refers to the application-specific authentication.

10. Tests containing file uploads.

I do not know why I get so many questions on how to create a test containing file uploads. For some reason people think that file upload is a special feature. In fact, it is done with a usual POST request that contains a number of parameters in its body. One of those parameters is the file itself. In WAPT you simply specify the file to be used for this purpose. You will be prompted for this after recording the test.

Of course, you can parameterize the file, so that different ones would be used for different sessions. This is done by using variables in the file name specification. Note that in case of WAPT Pro the provided file name and path should be available from all load agents.

11. Testing through a proxy server.

If you are using WAPT inside a corporate environment, there is a high probability that your access to external network resources is limited. HTTP connections may be allowed only through a proxy server. This usually does not cause any problems when you record a user session. However to run a test from such environment you should either specify your proxy server in the load agent settings or ask your network administrator to allow direct connection.

The problem is that in most cases people prefer the first approach (testing through the proxy), which is very bad. This may work only if you are sure that the performance of your proxy server is far beyond the performance of the web application you are testing. By running test through a proxy you actually add one more component to the system. If that component becomes a bottleneck, your test results will be incorrect.

12. Test results interpretation mistakes.

Finally, people make many mistakes when interpreting test results. However this theme is too wide to discuss here. Again, the problem is usually that a certain value in the report is treated incorrectly. Report contains more than 20 tables with values. Each table is described in the help file that comes with the product. I would very much recommend reading the corresponding description before making any conclusions. If you are in doubts, you can always run couple small tests with different parameters and predictable results to check how each value is calculated. You can enable logging to confirm your understanding. If you see anything strange, just send your results and questions to our technical support and we will help you: support@loadtestingtool.com.