Types of software testing: The System Test

After completing the tests on the individual units and verifying their correct integration, the whole software system is tested.
Performing tests on the entire system means affirming the quality of the final product to a certain extent. The features tested at this level can be different and numerous, and often depend on the type of application implemented.

These tests are usually based on the features expressed in the application specifications and requirements and may involve different software and hardware configurations.
Although the approach used is mostly black box testing, therefore oriented towards functional tests, other structural features are also checked, such as safety, usability and maintainability.
Safety tests, for example, are a crucial aspect of system testing, and necessarily depend on the requirements defined for the application being implemented, which must be able to guarantee features such as: authenticity, access control, data secrecy, data integrity and non-repudiation.

Security testing can be very demanding, complicated and expensive for applications that require a high degree of security, as to protect the software from attacks, it must be taken into account that these are generally caused by intelligent adversaries who exploit all the possible system weaknesses .
Careful testing of security mechanisms, developed to prevent, discover or restore the system after an attack, is always accompanied by risk analyzes, which can help identify and classify potential security problems and their impact.
While risk analyzes are approached with a white box testing approach, through a deep knowledge of the software architecture and a search and classification of its internal weaknesses, the testing of security mechanisms, instead, is usually performed with functional tests black box type.

Another classic system test, in addition to those related to safety, is the so-called “stress test”, which aims to destabilize the system by putting it in a state of stress, therefore with an overload or a subtraction of resources.
This type of testing aims to guarantee the qualitative aspect known as “recoverability”, or the ability of a system to react to unexpected errors.
After crashing the system it is necessary to carefully observe its reaction, measuring its tolerance towards sudden errors, verifying whether the shutdown occurs in a controlled way or immediately enters a lock state, checking if the restart resumes from the last been corrected and if the data was saved or lost.
Some examples of stress tests for a web application can be: shutting down and randomly restarting network switch and router ports that connect servers, setting the database offline and running resources-consuming processes (CPU, memory, disk and network) on the servers.

The requirements of an application also include the efficiency of the software product, typically controlled in system tests.
The purpose of this type of test is precisely to go and verify that those that are commonly defined as performance requirements are met, which must therefore be verified before the application is released.
This type of test is also often used for the comparison in terms of efficiency of different versions of the same application.
Usually in this type of test the main variable that is considered is the execution time.
In fact, for various types of operations the maximum execution times are fixed (that is, “baselines” are defined) and the software product does not exceed these time limits.
The maximum execution times considered may vary depending on the type of hardware and the software functions analyzed.

It is clear that if this test were performed for a web application installed on the latest generation servers with multiprocessor technology, the expected results would be very different from those of the same application on an obsolete server.
The same argument can be made for the computational complexity of the various operations examined, which will substantially determine these time limits.
These tests can be developed both from a functional and a structural point of view, ie by inspecting and controlling the system from the inside, considering the execution times as the sum of the times required for the individual operations.
This is explained by the collection and analysis of measures localized in different portions of code, which facilitate the determination of a slowdown in the application.

One or more performance tests can also be integrated in regression tests, to verify that changes to the software product have not introduced slowdowns.
Another type of system test is the “load test”, which provides for the constant increase of the load on the system through automatic tools.
For a web application, for example, the load is the number of concurrent HTTP users / connections.
Other examples of this type of test can be: the creation of a large number of mailboxes in a mail server, the writing of a very long document for a word processor or the printing of a heavy job for a printer.
This type of test is very useful to prevent defects related to memory management or buffer size.