Software testing: How to choose and select the test tool
Choice of instrument
In the software testing process, there are a variety of different issues that a test manager must consider when choosing test tools.
The most common choice has historically been to purchase an instrument from a supplier, which, in some cases, may be the only viable choice. However there are other viable options, such as open-source tools and custom tools.
Regardless of the type of instrument, a test manager must be careful to calculate the “total cost of ownership” of the instrument for the entire expected duration, performing a cost-benefit analysis. This topic is discussed later in the section on return on investment (ROI, Return on Investment).
Open source tools
For software testing, open-source tools are available for almost every aspect of the testing process, from test case management to defect tracking, to test case automation, just to name a few. An important difference of open-source tools is that, while the instrument itself does not generally have a high initial purchase cost, formal support may not be provided for the instrument itself. Many open-source tools, however, have people willing to provide non-traditional or informal support to users.
Furthermore, many open-source tools were originally created to solve a specific problem or address a single issue, so the tool cannot normally provide all the features of a similar market tool. For this reason, a careful analysis of the real needs of the test group must be performed before selecting an open-source tool.
One of the advantages of using open-source tools is that they can generally be modified or extended by their users. If the test organization has the basic skills, the tool can be modified to work with other tools or extended to fit the needs of the test team. More tools can be combined to solve problems that market tools cannot address. Obviously, the more tools used and the more changes are made, the greater the complexity and the management overhead. A test manager must ensure that the team does not start using open-source tools just for the sake of using them, but, as with other tools, the effort must always be aimed at achieving a positive ROI.
A test manager must understand the “licensing” scheme of the selected tool. Many open-source tools come with a variant of the General Public License, which specifies that the distribution of the software must be subject to the same conditions as the software originally obtained. Should the test team make changes to the tool to better support their test, these changes should be made available to all external users of the tool under license. A test manager should verify the legal implications of redistributing the software for your organization.
Companies that develop security or mission-critical software, or that are subject to regulatory compliance, may have problems using open-source tools. While many open-source tools are of very high quality, the accuracy of a given open-source tool has probably never been certified. Market instruments are often certified for their accuracy and suitability for a particular task. Although the quality of an open-source tool may already be good, certification, if required, is the responsibility of the group that uses it, and can create additional overhead.
The testing organization can see that it has a specific need for which no open-source or market tool is available. The reasons may be a proprietary hardware platform, a customized environment or a process that has been modified in an unusual way. In these cases, if the core competency exists in the team, the test manager can consider developing a custom tool.
The advantages of developing and adopting a customized tool are that the tool can exactly meet the needs of the team and that it can operate efficiently in the required context. The tool can be designed to interface with other tools in use and to generate data in the exact format required by the team. Furthermore, the tool can have uses in the organization beyond the project in question. However, before planning to export the instrument to other projects, it is important to review the purpose, scope, advantages and possible disadvantages in advance.
A test manager when he thinks of creating custom tools, must also consider possible negative issues. A custom tool often depends on the person who creates it. Therefore, customized tools must be properly documented so that they can be maintained by others. If this is not done, the instruments can become orphans and fall into disuse when the creator of the instrument leaves the project. Over time, custom tools can have their scope extended beyond their initial intent, which can cause quality problems in the instrument, leading to false positive defect reports or inaccurate data creation. The test manager must keep in mind that a customized tool is nothing more than an additional software product, and as such, subject to the same type of development issues. Custom tools must be designed and tested to make sure they work as intended.
Selection and selection process
The test tools are a long-term investment, which probably extends over several iterations of a single project and numerous other projects. A test manager must consider a tool to be acquired from different points of view:
For the business, a positive ROI is needed. In order to obtain a high return on investment, the organization must ensure that those tools that must interoperate, which can include both test tools and non-test instruments, can effectively work together. In some cases, in order to achieve interoperability, it is necessary to improve the processes and the connectivity of use of the instrument, and this can take some time.
For the project, the tool must be effective (for example: avoid errors in the execution of manual tests, such as typing errors during data entry). The tool can take a significant amount of time to start getting a positive ROI. In many cases, positive ROI can occur in the second release or during maintenance, rather than in the initial project, where automation was implemented. The test manager should consider the total life cycle of the instrument.
For the user of the tool, it must support project personnel to perform their task more efficiently and more effectively. Therefore the learning curve of the methods of use must be considered, otherwise the introduction of the instrument can only cause stress and disaffection. The test tools, when adopted, require training and tutoring by users.
In order to ensure that all points of view are taken into account, it is important to create a roadmap for the adoption of the test tool.