Testing Methods and Terminology
There are three main terms for testing by simultaneously acting multiple
users (real or simulated): load, performance and stress
testing.
All performance and stress testings require workload definition as a part
of the test. There may be no load, minimal load, normal load, above normal
load or extreme load. So we will use the term "load testing" as a general
category for all types of testing by simultaneously acting multiple users.
The definition for load testing is:
"Any type of testing where realistic (or hyper-realistic) workloads are characterized, simulated and submitted to
the system under the test."
Extreme loads are used in stress testing - to find the breaking
point and bottlenecks of the tested system.
Normal loads are used in performance testing - to ensure
the acceptable level of performance characteristics like response time
or request processing time under the estimated load.
Minimal loads are often used in benchmark testing - to estimate user's experience.
Anyway, load, not performance, is the shared characteristic across all
these types of testing.
In stress testing you should try to break the application by an extreme
load and expose the bugs that are likely to appear under the stress, such
as data corruption, buffer overflows, poor handling of resource depletion,
deadlocks, race conditions etc. The fact that performance metrics such
as response time can be measured during the stress testing is practically
irrelevant to the purpose of stress testing. So stress testing and performance
testing have totally different goals, which make it clear that stress
testing is not a species of performance testing.
Another reason for putting performance and stress under the load testing
is another type of testing known as background testing. In background
testing you use workloads (usually normal) to exercise the system under
the test while you run functional and/or regression tests against the
system. The goal of background testing is to test the functionality in
more realistic conditions, i.e. with a realistic background workload,
like the application will have in real use.
But some sources treat Performance testing as a general category instead
of Load testing. This approach is oriented towards test results (measured
timings) while the using of Load testing as a category is concentrated
on the test nature. We do not follow this approach, but it is rather widespread,
so it should be mentioned.
For example, Rational Unified Process defines these terms as follows:
Performance testing is a class of tests implemented and executed to characterize and evaluate the
performance related characteristics of the target-of-test such as the timing profiles, execution flow, response
times, and operational reliability and limits.
Included within this class are :
Load testing - Verifies the acceptability of the target-of-test's
performance behaviour under the varying operational conditions (such as
the number of users, number of transactions, etc) while the configuration
remains constant.
Stress testing - Verifies the acceptability of the target-of-test's
performance behaviour when abnormal or extreme conditions are encountered,
such as diminished resources or extremely high number of users.
Briefly:
Performance testing is the overall process,
Load testing checks if the system will
support the expected conditions,
Stress testing tries to
break the system.