Software testing: Selection of test techniques

Software testing

The most natural and traditional way to verify a product is to try it in a number of representative situations, ensuring that it behaves as expected. In general, it is impossible to test a software in all its possible operating conditions; therefore it is necessary to find some test cases that provide with sufficient evidence that the product will have acceptable behavior even in situations where it has not been tested.

The test is a critical activity in software engineering and should be performed as systematically as possible, clearly defining the results you expect and how you want to achieve them. In practice, however, the test is performed in a non-systematic manner and without applying predetermined criteria.

The test can be used to demonstrate the presence of malfunctions, not to prove their absence (Dijkstra’s thesis, 1972).
The test should therefore help to locate errors and not only to detect their presence. Before releasing the final software, engineers must test both functional and non-functional requirements, possibly using more than one test technique.

In this regard we recall some important definitions:

• test case: a test case is a set of inputs, execution conditions and a past / not passed criterion;
• test case specification: the test case specification is a specification of a requirement that must be met by one or more test cases;
• test suite: a test suite is a set of test cases

Problem of the selection of test techniques

One of the main problems concerning the software test is how to get an appropriate set of test cases useful for testing a software.
This set should ensure maximum effectiveness with as few test cases as possible; on the other hand there are numerous test techniques available, many are not used and only a few are widely used.

Some testers have very little information on the available test techniques, their usefulness, how suitable they are for the project they are working on and how to base their decision on which test techniques to use.
Very useful is to provide a catalog containing sufficient information to select the best techniques suitable for a given project. This ensures that the decisions the testers make are based on objective knowledge of the techniques rather than on perceptions, assumptions and hypotheses.

Another question concerns the difference between these techniques. We can find distinctions in the literature regarding the mechanical aspects and the contextual aspects of the techniques. However, there is still an open question which is the most suitable test technique in a given phase for a given project.
The truth is that, although there are some studies that compare the techniques, there are no studies that analyze the conditions of applicability of a technique or that, for each technique, allow to assess which are the most relevant parameters or conditions for its applicability.

There are two main reasons why developers and / or testers do not make good choices regarding the test technique:

• the information available on the techniques is normally distributed on different sources of information (books, articles and even people). This means that the developers have no general idea of ​​how many techniques are available and all the information of interest for each technique;
• they do not have access to information related to each test technique unless they have used it before. The developers also tend not to share the knowledge they have acquired with others; this means losing the possibility of knowing the experiences of others.

There are numerous techniques that are not universally applicable, so it makes sense to talk about a need for selection.
It is generally accepted that the selection process foresees a comparison between the characteristics of the techniques whose applicability conditions are closer to the conditions of the project on which one is working.

Therefore, the problem of selecting the best test technique can be divided into two sub-problems:

• the correct identification of the characteristics relevant to the selection;
• knowledge concerning all the techniques on which the selection is based.

It is reasonable to expect that the actions taken by a tester, who intends to carry out a systematic and balanced selection of test techniques, are:

1. list of techniques from which the selection must be made;
2. evaluate the characteristics relevant to the listed techniques;
3. evaluate the characteristics of the project in question;
4. compare the results of points 2 and 3 and decide which technique and / or techniques are most suitable for the project you are working on.

The problem that one tries to overcome is the following: the tester must be able to carry out the selection of the test techniques based not only on his direct or indirect knowledge.