New release: WAPT 8.1 and WAPT Pro 3.1

Posted on July 10, 2013 by Ivan Lisitsyn

We are proud to announce the release of new versions of our products: WAPT 8.1 and WAPT Pro 3.1. This upgrade is free for all registered users of WAPT 8.0 and WAPT Pro 3.0 correspondingly. So, if you already have a license, you can download new versions and update your installation.

Note that load agents and x64 Load Engines should be also updated for use with the latest version of WAPT Pro.

Even though formally this is a minor upgrade, new versions have a number of features that really make the difference. Now I would like to mention couple technical details about the features that I treat as the most important.

Recording process. First of all, new version can determine which proxy server is used by your system browser and access your web site through that proxy when you record a test. If you do not have proxy server or if direct connection to any web site is available on your network, this feature is completely useless for you. However if you are trying to record a test inside a corporate network and your system administrator likes sophisticated features and he is deeply concerned of everything related to security, possibly your system uses “Automatic proxy configuration” and direct connection to any web site is forbidden. This is where you can check that:

Proxy configuration

In such case you will hardly record anything with any previous version of WAPT, but you have a very good chance with the new version.

If you use Firefox for recording, you will see that new version now uses a special Firefox plugin. It lets you change the proxy for your browser on the fly. Actually, the same is done for Internet Explorer and Google Chrome now. This means that you do not need to restart the browser to start or stop recording.

Custom performance counters. If you use performance counters to measure the performance of servers running your web application, you will see a dramatic change with this version. Until now you could only measure system CPU, network, RAM and disk consumption during the test. Now you can get ANY counter value available through WMI or SNMP interface, which means that you can monitor any parameters provided by server software, such as web and application servers.

For each counter you can now specify a JavaScript code for its calculation. Inside that code you can insert calls to functions that retrieve WMI/SNMP values. So, the resulting counter can be calculated basing on several different values.

Parameterization. If you are an experienced user and if you often need to work with profiles requiring parameterization of session-specific data, with new version you will do this work several times faster, because it provides extended “Find and Replace” functionality.

Now you can search an entire profile for a value and see all occurrences in one list. You can click on any item to get to it and edit, or you can mark several items and make replacements.
For example, you see that some parameter value looks like an ID, which is session-unique. You can just select that line in the list of request parameters, right click on it and choose “Find Value…” from the popup menu. The following dialog will appear.

Find and Replace

You can easily find the original place where that value was initially received from the server. In that place you can create a variable and read the value to it. At the same time you will see all the places where that value is used. You can mark them all and replace by the created variable.

Rendezvous points. We have finally implemented that feature! Now we have a special operator that pauses the execution of the current session and waits for other virtual users. When a specified number of users reach the same point, the execution of all sessions continues.

To tell the truth, I do not understand why this feature is so popular. In my view, load tests with rendezvous points can hardly emulate anything real. This is just a good way to stress a web site and break it. In real life users do not wait each other to click a link together. The only reasonable usage of that feature is the emulation of a sale at an online shop. When it begins, all users start to click the “Order” button immediately at the same time.

Anyway, we respect requests of our customers: you asked about that feature, you got it.

Authentication schemes. Well, it is actually hard to name this a new feature. To tell the truth, we just fixed several important problems related to Kerberos and cookie-based authentication. So, if your web site uses one and you could not record or execute tests before, everything should work fine now.

The full list of new features is provided here.

Posted in WAPT usage | Tagged , , , | Leave a comment

WAPT 8.1 and WAPT Pro 3.1 beta versions

Posted on June 5, 2013 by Ivan Lisitsyn

It has been a while since we last updated our products, and now we plan to do this very soon. This time we decided to provide beta versions to our users couple weeks before the official release, as there are several highly requested new features. So, I am happy to let you know that new versions are available for download.

WAPT 8.1: http://www.loadtestingtool.com/forum/builds/WAPT8.1.zip

WAPT Pro 3.1: http://www.loadtestingtool.com/forum/builds/WAPTPro3.1.zip

x64 Load Engine for Pro 3.1: http://www.loadtestingtool.com/forum/builds/WAPTProx64LoadEngine.zip

These versions are fully functional and hopefully not too raw to try. However if you find a bug, you are welcome to report it as usual. We will do our best to provide a fix as soon as possible.

New versions are compatible with the current official builds of all extension modules.

This upgrade will be free for all registered users of the current versions (WAPT 8.0 and WAPT Pro 3.0). So, if you already have a license, you can use your serial number to register the new installation.

Below is a list of new features with few comments on each one. Stars mark features available only in the Pro version.

Custom performance counters and support for secure SNMP*

You can now add your own performance counters for any values available through WMI and SNMP interfaces. You can monitor the performance of application servers and other system components. The latest secure SNMP v3 is now supported.

Full support for Kerberos, SSO and cookie-enabled authentication

You can test web servers requiring Kerberos, SSO and cookie-enabled authentication types. Test credentials can be provided in CSV files.

Rendezvous points

You can now create rendezvous points inside the emulated user sessions. This is done with help of a special operator that pauses the execution of the current session and waits for other virtual users. When a specified number of users reach the same point, the execution of all sessions continues.

Extended find and replace

When working on the test parameterization you often need to find all places inside a virtual user profile where a certain value is used. Now you can find all such places at once and easily navigate to different occurrences. You can also make massive replacements by simply checking the items that need to be changed.

Support for automatic proxy configuration

Automatic configuration of browser proxy settings is often used inside corporate networks. In such cases the name of the proxy server is not directly specified in the browser. Now it can be determined and used for test recording in all cases.

A number of optimizations*

  • Memory usage by load agents is reduced, so you can create more virtual users with the same hardware.
  • Communication between the workplace and load agents is performed more efficiently. This increases timing accuracy and reduces bandwidth requirements for load agents used remotely.
  • Any updates you make to the list of load agents are performed faster, so you can start the test immediately after configuring your agents.

A number of smaller features

  • Ability to measure time required for a specific task within a user session;
  • Errors produced by JavaScript operators are reported in a separate table;*
  • Support for non-ASCII data files used to read values from;
  • Ability to use variables inside function parameters;
  • Logging of variable values;
  • Automatic parameterization of JSON values (in the Module for JSON format);
  • AMF messages with object references are fully supported (in the Module for Adobe Flash testing);
  • Option to group requests on recording;
  • Extended SSL error reporting.
Posted in WAPT usage | Tagged , , | Leave a comment

Extreme online shopping with WAPT Pro

Posted on March 20, 2013 by Ivan Lisitsyn

Recently we have discovered that our load testing solution can be applied in a quite unusual way. One of our users has shared his “success story” with us. It perfectly shows how widely you can apply the product if you know its features very well and have a good fantasy.

The story is actually about Cyber Monday online shopping. You know, some web sites run very special actions to attract customers. They sell limited number of items at exceptionally low prices. In theory you can make a very good deal if you participate. In practice this is almost impossible, because each new item is ordered by someone immediately after it becomes available. Here is where WAPT Pro appeared to be a great automation solution.

The rest of the story is provided as is. (Personally I do not think that such things break any laws, but the original author asked me to keep his name confidential.)

In the late November GaGa Deals (gaga.aliexpress.com) launched a Cyber Monday sale. I was interested in purchasing some electronic devices, because the real discount on some items was about 90%, which seemed to be quite a good deal for me.

Each lot had a countdown to the moment when the “Buy Now” button became available. So, if you wanted to buy that lot, you should have been the first to click it.

After few manual attempts I realized that the required reaction time was less than one second. Moreover, it depended on the synchronization of your local system time with the server. So, it became quite obvious for me that the game was useless and I needed to apply a more serious solution. Fortunately I had it at hand.

Samsung

WAPT Pro is a load testing tool that we normally use to test the performance of web applications. It can emulate real web site users with virtual ones by executing a predefined sequence of actions. So, you can record a user session with help of a browser and then replay it with multiple users. In this specific case I did not need many concurrent users. My goal was to execute one session very fast, which was also possible in the product.

I recorded a typical session that I was going to emulate. For that purpose I selected a less attractive lot that allowed me to click the “Buy Now” button manually without haste. I verified the test and confirmed that most session-specific data was parameterized automatically by WAPT. I would only need to replace the lot ID to apply the recorded virtual user profile.

WAPT Pro window

I launched the “test” with the next “great deal” item. Well, that first attempt was not successful at all, because I discovered that my virtual user tried to click the “Buy Now” button sooner than necessary. The server discovered that and returned an error. Fortunately, a simple solution was shortly found: I added a loop in the profile and the button click was executing now until it returned a valid result. It was a conditional cycle, so I have to validate the response with a simple JavaScript function. Fortunately, the Pro version of WAPT supported that.

As a result, my virtual user was able to get to the order page. Here I came across another problem that seemed to be severe at first glance. The page contained a captcha. In other words, it was protected against robots, but WAPT was a robot after all…

I knew about few ways to get round the problem. For example, you can grab the captcha link and send it to one of the automatic recognition services. They return the sequence of letters and numbers that they read from the picture. (Don’t ask me how and why those services exist). Since I only needed to read a single image, I could simply view it myself (after taking from WAPT) and enter the data back to a file used by WAPT. However this solution seemed to be a bit slow to me, so I invented a better one, but it is a secret. If you decide to reproduce the whole process, you can use one of the ways mentioned above.

Now I had a virtual user profile for the whole ordering procedure and was ready to try with the next attracting lot. It was Samsung Galaxy S3 for as low as $99.99. I launched the process 30 seconds before the target time and when it did the thing, I started to browse logs with a great excitement. I was disappointed and surprised again. It did not manage to take the lot because it was too slow! According to the WAPT log files, it only spent 600 milliseconds to click the button, but that was too late. My competitors were a bit faster. Who would ever believe that they were humans?..

Delay in log

Ok, I started to research the problem again. I tried pinging the server and got 250 milliseconds response time. It was way too long for this kind of tricks. I run traceroute to the server and discovered that it was hosted somewhere in California.

The solution was quite straightforward. WAPT Pro can use load agents located remotely. This means that the whole communication to the target server is done from a remote location. If it is very close to the server, it will work very fast independently from the connection speed to the workplace component from which you manage the test execution.

Thanks to Amazon, it is very easy these days to get a load agent in California. I created a virtual system instance in the Amazon cloud at the West Coast location. It took just a minute to download WAPT Pro and setup a load agent to the newly launched system image. Now everything was ready. I mean really ready this time.

Fortunately right at the moment a new very tasty lot was about to fire. It was Nikon D7000 camera still for $99.99. I clicked the “Run test” button in WAPT and was waiting with bated breath for the outcome… And it finally happened! This time I had the Nikon D7000 for less than $100 in my cart!

Success

I paid the order and after some 20 days the lot was delivered to me without any problems. It was real Nikon D7000 and I am still happy to make great shots with it. Now preparing for the next year sales…

Posted in General, Testing practice, WAPT usage | Tagged , , , , | Leave a comment

Cloud load testing with WAPT Pro

Posted on March 12, 2013 by Ivan Lisitsyn

Clouds are very popular these days, because they provide a convenient virtual environment for a number of tasks from web hosting to scientific computing. In a cloud you can get a fresh ready to use system with almost any predefined parameters in a matter of minutes without need to purchase hardware and setup the OS. Besides that, clouds usually provide very fast and wide Internet connection. That is why it is usually a very good idea to use clouds for load testing of a web site. This is especially useful if the target site is hosted in the same cloud.

Since WAPT does not require any unusual hardware, it can be easily setup on any virtual system. There is no special license for this type of use. If you have a general WAPT or WAPT Pro license valid for one system installation, you can install the product on a virtual system as well. The only additional requirement is that to observe the license agreement in such case you can run only one instance of the system at a time.

Actually with cloud testing you can go even further. Since WAPT Pro can run distributed tests with multiple load agents and manage the process remotely, you can setup the agents in a cloud (or in many different ones) and manage the test from your local workplace running on your desktop.

The connection between the workplace component of WAPT Pro and load agents is established over TCP/IP protocol. Each agent is waiting for a connection from workplace on a specified port. By default the port number 9475 is used, but you can change it in the Load Agent Manager, if this one is occupied by another process. So, in order to run the remote agents in the cloud environment, you only need to open that port for external connections.

Let’s see how this can be done with a popular Amazon cloud. First, we need to create a Security Group for which we allow TCP and UDP connections to the ports in a range. We use 0-10000 range, which means any port from 0 to 10000. If you want, you can restrict this to 9475 only.

Security Group

Now we create a system instance where we will install the agent. We use the previously created Security Group for this.

Creating Instance

Finally, we add the agent to the list of agents in the WAPT Pro workplace. To do this we should simply specify the IP address of the instance created at Amazon.

Agent Settings

Everything is ready now for running the test. When you launch it, the workplace will send all the test data to the agent(s) and will start receiving performance statistics. The target web site will be loaded by the agent(s) running in the cloud.

Posted in WAPT usage | Tagged , , | Leave a comment

Module for GWT Testing

Posted on February 21, 2013 by Ivan Lisitsyn

We have recently released a new extension module for WAPT and WAPT Pro products. As its name suggests, it is targeted at handling web applications developed with help of GWT (Google Web Toolkit).

Till now we have had 4 extension modules. Two of them are optional (ASP.net and JSON), meaning that you can use them to facilitate testing (in many cases this means a significant improvement), but still there are no applications for which these modules are mandatory.

Two other modules (Flash and Silverlight) are used to convert binary data to XML and back again. So, if you have an application developed in Flash or Silverlight, you will not be able to load test it properly without the corresponding module.

The new GWT module stays somewhere in the middle in this classification. It converts to XML complex data structures contained inside requests and responses, but in this case the data is originally represented in text format, so in general it is possible to parameterize everything without such conversion. However, in complex cases this will require a lot of efforts without the module and just few steps with help of it.

The client part of a GWT application is often implemented in such a way that it can send remote procedure calls (RPC) to the server part. Such calls are passed inside special HTTP requests containing serialized data structures that are used as function arguments. The server replies with the function return value which can also be a complex data structure. This reply is provided in the corresponding HTTP response.

The Module for GWT Testing lets you import the jar/war files of your GWT project containing application .class files and .gwt.rpc files containing serialization policy in order to enable conversion of serialized data to XML.

GWT files

So, after installing and configuring the module on top of WAPT or WAPT Pro, you will see the XML documents inside the bodies of all GWT requests and responses.

Here is an example showing the content of a GWT-request before applying the module.

Before conversion

The same request body is converted by the module to the following XML.

After conversion

The same way you can convert any response and extract any values from it with help of the standard XPath() function. The extracted values can be used in subsequent requests for parameterization. On the test execution the module will convert the parameterized XML back to GWT structures expected by your server to ensure the correct emulation.

Good news for those customers who use our products regularly to load test different web applications. If you have already purchased the Module Pack (containing all 4 previously existing modules), we provide the GWT module license to you free of charge.

You may also notice that even though the regular price for the new module is $350, its addition has not increased the price of the Module Pack. So, you can now save much more by ordering all modules at once.

Posted in WAPT usage | Tagged , , | 5 Comments

Stop feeding fat cats

Posted on January 15, 2013 by Ivan Lisitsyn

I believe that when you see something completely wrong and cannot correct it, you should at least share your view with other people. This is the only way to change anything in this world.

This article is about load testing, but if you read it to the end, you will see that it is about the whole IT industry, about the whole economic reality and about our life. I will use an example which I know very well to demonstrate something very common, yet not very well understood.

A short technical introduction. Imagine that you are about to release a new web site. Will it work well when accessed by multiple concurrent users? This question can be answered if you perform a load testing. Obviously, this task cannot be accomplished manually by your staff members or beta testers. There are special software tools that perform load testing by emulating real users with virtual ones behaving same way.

Note that such testing not an optional activity. It is absolutely necessary for each new web site that includes any dynamic content. Now imagine how many new web sites are released every day and you will get some estimation of the overall demand in such tools. Shortly speaking, the market is really huge. I listed below few “typical” vendors together with their products. Let’s call this “X-list”:

Dollar catHewlett-Packard: LoadRunner
Borland: SilkPerformer
SmartBear: LoadComplete
Neotys: NeoLoad

Probably you have heard about such companies as HP and Borland. The other two are less well-known, but we will pay some attention to them first. Guess how much do they charge for a permanent license? They have different ones, but let’s consider a version that can create 2500 concurrent virtual users. This does not look like a really big number. The answer is: about $40,000. If you don’t believe this, you can check their web sites, which are quite easy to find.

A fat catThis is not the biggest surprise yet. The really surprising fact is that those guys really think that they provide “affordable testing solution”, because they usually compare their prices with HP and Borland. You cannot check HP prices so easily. If you search the web, you will only find some kind of legends, because such secret things cannot be revealed to ordinary people. Some insiders whisper that their companies have spent $100,000 and even more for a load testing solution. Moreover, they continue paying for support about $20,000 per year. I wonder what kind of support they mean… are all drinks included?

WAPT catIs everything really so bad? Is it true that you cannot perform such a common task as load testing for less than $40k? Of course, you can. There are solutions identical in functionality, but priced in a quite different range. A good example is a product called WAPT, which stays for “Web Application Performance Testing” or something like that. Its pricing starts from $700. Even if you need a full featured version with all add-ons and optional 64 bit load generator included, you will only spend about $3000. Such version can create 10,000 virtual users and even more, so in fact it outperforms all the X-list products mentioned above.

Now the question is: how is this possible? It is hard to believe that so similar things can be sold for so different prices. Maybe WAPT misses some important functionality? Well, since the product is still different, possibly you will find that some features are implemented in a different way. Some owners of NeoLoad licenses may even object that their product produces more beautiful graphs in report. However these graphs contain all the same data, so have they overpaid 10 times only for this? How about efficiency? If you compare WAPT and Neoload running on the same hardware, you will find that WAPT significantly outperforms its competitor. This means that in addition to $40k for a Neoload license, you will need to purchase expensive hardware to run it, whereas you can use WAPT even on a regular developer workstation.

Another catOk, is there any other reasons to choose Neoload or TestComplete? (Note that I do not mention LoadRunner and SilkPerformer, because I consider the choice of these tools a complete nonsense due to their pricing policy). When people are paying for software these days, they expect to receive something more than a license key. They want regular updates, efficient technical support, bug fixes and additional services. One would think that this is the point where we can find some difference between WAPT and its counterparts.

However it appears that these observations do not justify spending $40k as well, because WAPT is perfectly supported by its vendor, SoftLogica Inc. By the way, the support is free, which means that you don’t have to pay monthly fee. As for the quality of support, you can easily check it yourself by sending a letter to support@loadtestingtool.com. You can ask any questions on the provided products and services.

So, it appears that when choosing a load testing tool you can read and compare product features, try demo versions, contact vendors to discuss product applicability and even get extra service if you need help with your specific web site. So, you have everything required to make proper choice. So, why do people overpay for load testing tools (and software products in general)?

The problem is that guys from X-list have huge marketing and sales budgets, which is not quite surprising taking in account their prices. Using such resources they managed to create a wrong impression that their tools are quite different, and it is simply impossible to find an alternative solution for a reasonable price.

This is a new type of virtual monopoly: it exists just because most people believe that it exists. Once you search the web for a load testing tool, most probably you will come to one of the overpriced products, because they can easily pay several dollars for each click to their web site. You contact them, or just try to download demo of their product. After that they will call you every day persuading you that their tool is the only one and of course you cannot find anything less expensive.

Wake up! If you purchase overpriced products, you invest in their marketing, which will make these products even more expensive. These guys will perfectly load test your budget, before you can even try load testing your web site.

One more catI know that for some people it is a matter of principle. They never purchase “low-end” goods. But what is “low-end” in this case, if the quality is the same? Software product is not a luxury. If you want to look quite respectable and you have enough money, you can wear Rolex and drive Lamborghini. This will demonstrate your style and prosperity. However, a load testing tool is a completely practical thing. It is used by technical people inside a company and its make is not visible to your business partners and competitors. What you are going to demonstrate by choosing a tool that is 10 times more expensive than another one with the same functionality? People who know the technical side of the problem will simply think that you are spending money for nothing.

My main idea is that only the lack of information together with the marketing and sales tricks of the X-listed companies urges people to make wrong decisions and purchase their products while there are more affordable alternatives on the market. These companies cannot win the competition by providing better tools and services, so they are trying to apply approaches that actually work against customers.

My message to business owners: Do not trust sales persons when they say that you will pay extra money for quality. Extra money is always spent on sales and marketing. These activities of your vendor are completely useless for you. So, you can easily save your money by purchasing a different product from a company that concentrates on real work.

My message to managers: I understand that it is always a safer decision to purchase something from a company with a well-known name, provided that you do not have a severe budget limitation. However, you have a chance to cut expenses by 90% with a proper tool selection. This will be appreciated not only by your directors. All technical specialists will respect you more if they see how you make decisions basing on good technical understanding of the subject. They will also see that you are not throwing money for nothing.

My message to technical specialists: Please advise your managers. Remember that every company (even a prosperous one) is limited in expenses. If your company avoids buying overpriced products, it will have more resources for increasing your own salary, especially if you are the person who recommends the right decisions.

JMeter catWhat I am writing is not just a kinky attempt to promote WAPT. You know, there are absolutely free load testing tools. And these tools provide essentially the same functionality as WAPT and all products mentioned above. Sometimes I hear from people something like: “Ugh, WAPT is too expensive, I cannot afford it, I will better use JMeter, because it is free”. I can tell you frankly, I understand these people much more than I understand people buying SilkPerformer or LoadComplete. I strongly believe that poverty is not a sin. If you want to spend couple weeks trying to get to work a free product; if you do not need any technical support; if you are ready to fix bugs in the product yourself, then you do not need to spend anything and you are welcome to use JMeter, because it is completely free.

Please, do not think that I hate rich people and prosperous companies. I am not against anybody. I am for the progress of the mankind in general. I am sure that there are many people whose input to this progress is really huge and it is great if they are compensated for that. However I really don’t understand what progress we have from crowds of sales persons who do not produce any useful things, but invent imaginary benefits of the goods they sell in order to multiply the price and pocket the difference. I believe that wise people can cooperate without any “help” from these jugglers. I know that this sounds like an extreme and this philosophy is absolutely incompatible with today’s business practices and overall economic reality. If you like that reality, probably you will never share my ideas. In this case I am quite surprised that you have read that article to the end. But if you are not sure that what you see around you is perfect, please think about it. I really want to change this world to better, and as the first step in that direction I ask you to please stop feeding fat cats!

Stop it
P.S. All trademarks are the property of their absolutely respected owners.

P.P.S. Any comments are highly appreciated, especially if you have any objections to my point of view.

Posted in General | Tagged , , , , | 5 Comments

WAPT Pro 3.0 released!

Posted on October 3, 2012 by Ivan Lisitsyn

I wish I could make this post half a year ago. This is a really long awaited release, but it has finally happened. So, let’s see what’s new. Here is the list of added features. Sorry for so short descriptions. I will probably add more comments on some of these features in the next post.

Highly configurable test load

You can specify all load parameters separately for each virtual user profile. For example, you can join constant, growing and periodic load in the same test. You can also choose a specific load agent for each profile, if you do not want to use the automatic allocation.

Integration with your development environment

You can launch test execution using command line with extended options and specify a number of options for the output of your test results. You can organize the results of successive test executions into a storage containing reports, logs and CSV files marked with dates and test labels.

Improved test debug and verification capabilities

When you design tests you often need to parameterize the profiles to make them produce consistent and correct copies of the initially recorded session. To do this most efficiently you can now use the enhanced log viewer that will let you compare any part of the request or server response with the originally recorded data, check variable values, get extended timing information, etc.

Scheduled test execution

You can specify the date and time you would like to execute the specified test scenario, as well as repeat its execution regularly using the integrated scheduler.

Support for streaming HTTP requests

Some web applications, such as video players or stock quote viewers, constantly receive streaming data in the background even while the client application stays inactive or performs other activities. You can now create highly realistic tests for such web sites.

Emulation of multiple browser connections for a single session

Browsers usually keep several open HTTP connections within the same user session. Additional connections are used to load page elements and send AJAX requests initiated by the open page. This reduces total page download time and improves the overall web site performance from the user perspective. This browser behavior is now fully emulated in WAPT.

Parameterization of file upload

If your test includes one or more file upload requests, you can make different virtual users use different files. You now have two options for this: specify a folder to take files from, or parameterize text inside a single file. In the latter case WAPT will change the file content by inserting the values of variables inside it.

Posted in General, WAPT usage | Tagged , | 2 Comments

Choosing the best software for your business

Posted on March 15, 2012 by Ivan Lisitsyn

When you spend $1 for a new mobile game, you don’t care about the price and you want the game right now. There is no need to check competitive products, so, the immediate decision looks reasonable. The situation is different when you choose a software product for your business. In this case you cannot rely on your first impression. You probably have certain expectations regarding the product and you want to make sure that these expectations are met.

  1. The tool you choose should be reliable. You should be confident that it will never stop working suddenly on a changed data input or in other new conditions.

  2. Should meet your business goals now and (at least) in the near future. Note that it is not always possible to fully estimate this in a reasonable time using a demo version or a manual.

  3. You probably plan to use the product for some time, so you expect it to be regularly updated to follow the changing requirements of your business.

  4. It should be easy to use and easy to get familiar with. Probably your company will hire new people from time to time. They should be able to start using the product as quickly as possible.

  5. The vendor should provide good technical support for the product.

In my opinion the last requirement is definitely the most important. Moreover, you can forget about all the rest and concentrate on checking the vendor support service. If it is really good, the remaining doubts will be resolved during the process. If it is bad, don’t even estimate other possible benefits: companies that do not care about their customers cannot create good products.

In fact, I would not recommend purchasing any business tools without contacting the vendor first. I do not mean asking questions like “how to buy?” or “which version of your software do you recommend?”. Such questions can be answered by sales persons who have limited technical background. What you really need to check is if the vendor can resolve your technical problems quickly and efficiently.

Why is this so important? You may think that if the product is well documented and of a good quality, you will never need to contact the support service. However, let’s be realistic: bugs can be found in any product, even in a very expensive one from a respected brand. When you stumble on such problem, you may have to wait until the vendor provides a fix before you can move on with your project.

Bugs are not the only reason to contact technical support. Until you become very familiar with the product, you need to spend your time for learning. Product guides, help files, video presentations and online trainings may be very helpful in this process. However it will be much more efficient, if you ask vendor questions on problems you experience in your specific environment with your specific business goals.

Again, let’s be realistic: you can hardly expect that the vendor will do all the work for you free of charge and will tell you step by step how to implement your specific task with the product. However if for any question related to your specific project you get an answer like “we provide technical support only for registered users” or “this is a paid service”, you should probably consider a different vendor. If they are not interested in providing support in advance to make a sale to you, they will hardly be more interested after receiving your money.

So, the main idea regarding the support service is that you should be able to reach technical people easily and receive valuable feedback fast enough. It is also a benefit if the vendor can provide additional consulting service. If you are interested in completing a one-time project, you may consider outsourcing the whole work to them. At least you can ask for a quote and use the answer to estimate the vendor.

Let me give you one more advice on communication with tech support. Don’t try to call them by phone and expect that they will tell you which buttons you should click right now to make the whole thing work. They may be high professionals, but they are not wizards. So, if you are asking about something not trivial, it will be much better if you ask your questions in an email and attach to it files that can illustrate the problem: screen shots, data files you work with, etc.

If you do not receive an answer within 24 hours, or get anything telling about paid support, just choose another vendor. There are different companies. Some of them are interested in attracting new customers; some are interested in regularly collecting money from existing ones, charging them for every single move. In other words, there are growing companies, rich and prosperous ones, declining ones, and… dying ones. It is very important to estimate where on this scale is the place for your potential vendor. The only reliable way to do this is to contact them.

Looks like I forgot to mention something. All the above recommendations are useful only if you can get a demo version of the product and try using it before you actually make the purchase. That demo version should be fully functional except for some quantitative limitations like fixed trial period or maximum data file size. Otherwise you will not be able to ask technical questions, which will break the whole concept.

So, my recommendation is: never deal with vendors that do not provide functional demo versions of their products. They do not want you to get to the technical things until you make a purchase. This means that on that stage your will be able to communicate only with sales people on their end. They will paint a bright picture for you in any case.

Each company is like a nut. Its marketing and sales departments are like a shell. A purchase is a process of cracking that shell. When you do this, you get to the core and see if it is tasty or rotten. The trick is to feel a bit of taste without cracking the shell.

It may sound strange, but the best way to check how a company treats its clients is to find a bug in the product and report it to the support team. It is interesting what happens after that. Most people do not believe that it is possible to make the vendor provide a fixed version of the product for you in a reasonable time. This is true in most cases. Probably you will get an answer saying “we are working on the problem, blah, blah, blah…”. However if the company really cares about its customers, they will inform you about the real progress and will provide a fix or at least a workaround in a day or two. If you find such a vendor, this will be a very positive sign.

Let’s get back to our initial list of expectations. In fact, each item in the list means money that you save or earn using the product, because we are talking about a business solution, not an entertainment or a leisure product. So, when you estimate how a specific product meets each expectation, you can compare its price with its real value for you.

Pricing policies in today’s world is something that deserves a special discussion. It looks like everyone is trying to sell impressions, brands, innovations, arrogance or anything else that has nothing to do with real value. Shortly speaking in many cases the prices are ridiculous. For software products they always start from zero. I mean that you can find a free solution in almost any case. At the same time don’t be surprised if you immediately find a very expensive tool doing essentially the same things. Moreover, there are people buying those tools. You can think that it is strange to pay money for something you can get for free, but… You know, some companies adopt very interesting business practices: they simply forbid to use free tools inside the company. This approach looks a bit obsolete these days, but possibly its followers just do not believe that a free product can meet their expectations.

I would say that free products have made an impressive progress in recent years: many of them have grown from ugly little monsters to really convenient tools with friendly user interface and rich functionality. In many cases they outperform their paid competitors. There is only one problem: they are not supported by vendors. If you face a problem, you can only get help from so called “community”, which looks like asking for charity.

On the other hand, if you find a tool that costs 10 times more than you were going to spend, don’t give up. This does not mean that they all cost that much. Try to search more and you will find a reasonably priced solution from a vendor that has smaller marketing budgets and less known brand. The best choice cannot be the most expensive one, but it is rarely a completely free solution.

Posted in General | Tagged , , , | Leave a comment

Testing Silverlight Applications

Posted on February 29, 2012 by Ivan Lisitsyn

If you are going to load test a Silverlight web application with WAPT, first of all you need to install the corresponding extension module that you can download here.

This is a very easy task; you should simply download the file and run it. The module enables support for MSBIN1 format used by Silverlight applications to pass data between the client and server parts. This format is binary. It can represent any complex data structure containing objects of various types, which can, in turn, contain other objects and so on until we come to some elementary objects like text strings or numbers.

The Module for Silverlight Testing converts MSBIN1 messages to XML documents. The benefit is that such documents are easily readable inside WAPT. Moreover, you can extract any data from them and replace constant initially recorded values with session-specific data. In other words, you can parameterize values inside requests containing MSBIN1 content. When WAPT executes tests, it converts XML back to MSBIN1 and the server part of the application receives the data in its native way.

Now I am going to show how this is done on a live example. Unfortunately I do not have my own Silverlight web application, so I have to use a public web site for this purpose. I very much hope that the site owners will excuse me, because I am not going to do anything bad and possibly even make some promotion for them. Here is the web site that I am going to use: www.semyou.com.

If you want to follow my steps, you will need to register a free account on that site and get to the application store. Here we see several applications that are supposed to work “in the cloud”. This means that they provide a web interface and store data on the server. (As we can see, these days the word “cloud” is used to name things existed years before any clouds were invented.) Anyway, we will experiment with the application called “semNote”. It lets you create and edit simple text notes.

A Silverlight application

In our very first test I will simply log in, create a new note with some name and some text in it, and log out. I can do this by clicking the “New” button in the application, entering the text and clicking the “Save” button.

Let’s record such test with WAPT. First of all I should mention one important feature of Silverlight: when it is executed inside an embedded browser object, it blocks keyboard input. This is probably done for some security reasons. Looks strange in any case, but… that’s Microsoft.

What this means for us is that if we want to record a test of a Silverlight application, we should select to use the external browser. So, when I click the “Rec” button in WAPT, I choose that option in the “Recording Properties” dialog box.

Recording with external browser

Now I go to the web site, enter my email and password, run the “semNote” application and create a simple note titled “A note”. I click the “Save” button and close the browser.

Let’s see what we have in WAPT after the recording is finished. The very last request that was recorded after I clicked the “Save” button contains MSBIN1 content converted to XML. We can easily find a node containing the title that I used when created my note. I can edit any nodes here and even insert dynamic values to do any required parameterization. To show this I will add session number to the title. So, the request body will look like this:

MSBIN1 content

To illustrate the result I will run that modified profile with 3 virtual users, so that each user would produce exactly one session. So, my test volume settings are specified as follows.

3 virtual users load

As a result I get the following content inside the “semNote” application.

Result

Each of my virtual users created one note named with the session number. That is what I wanted for now. I am going to show how to make profiles using a more sophisticated parameterization in the future posts.

To be continued…

Posted in WAPT usage | Tagged , , , | Leave a comment