Software testing: How to estimate test activities

Estimate of the test

The problem of estimates is well known, ie no estimate, however accurate, can be considered sufficiently accurate. However, estimates are necessary for planning activities. Estimating methods and techniques can help you make accurate assessments. The estimates, therefore, heavily influence the plans and their lack of precision is often attributed to the failure to achieve the objectives of the project. For this, and not only, we cannot avoid making accurate estimates.

In other words, the estimate, as a management activity, is the creation of an indicative target of costs and completion dates in relation to the activities involved in a particular project. The best estimates:

• They represent the collective wisdom of professional experts and have the support of the participants involved
• They provide specific and detailed lists of costs, resources, activities and people involved
• They present, for each estimated activity, the most likely cost, effort and duration

In software engineering, it has long been known that the estimation is of high complexity, both at the level of techniques and policies, even if several optimal estimation practices in project management have been created and disseminated. The estimation of the testing is therefore the application of these best practices to the testing activities of a project or maintenance.

The estimate should cover all the activities involved in the testing process. On the other hand, the estimation of costs, efforts and, above all, the duration of the tests is often the one of greatest interest to the Management, since the execution of the tests is typically on the critical path of the project. However, test run estimates tend to be difficult to produce and still unreliable when the overall quality of the software is low or unknown. On the contrary, familiarity and experience with the system can probably positively influence the quality of the estimate. A common practice is to estimate the number of test cases needed during execution, but it works only if it can be assumed that the number of defects in the software to be tested is low. The hypotheses made during the estimation must always be documented as part of the estimate.

The testing estimate should take into consideration all the factors that can influence the cost, effort and duration of the test activities. These factors certainly include the following elements:

• quality level of the system requested
• system size to be tested
• historical data on previous test projects, which can be corrected with sector data or reference data from other organizations
• process factors, including test strategy, development or maintenance lifecycle, process maturity, and project estimation accuracy
• material factors, including automation of tests with related tools, environment and test data, development environments, project documentation (eg requirements specifications, etc.), and reusable test work products
• personal factors, including managers and technical leaders, commitment and expectations of senior management, experience and attitudes of the project team, its stability and inter-relations, support to the test and debug environment, availability of qualified consultants and knowledge of the domain
• process complexity, technology, organization, number of stakeholders in testing, number of test teams, especially when teams are geographically separated
• significant training, learning and driving needs
• assimilation or development of new tools, technologies, processes, techniques, customized hardware and software or number of testware
• need for a high degree of detailed testing specifications, particularly in the presence of an unfamiliar documentation standard
• complexity of the arrival times of the components, in particular for integration tests
• lack of test data (for example, data whose validity decays over time)

The quality of the software provided is an important factor, which Test managers should consider in their estimation. For example, if the developers have adopted “best practices” such as automated unit testing and continuous integration, at least 50% of the defects will be removed before the code is delivered to the test team.

Having said that, the estimate can be made either bottom-up or top-down. The following techniques can be used in testing estimates:

• intuition, conjectures and past experience
• project work breakdown structure (WBS)
• team estimation sessions
• business standards and regulations
• percentages of the overall project effort or staffing levels (for example, tester-developer ratio)
  organizational history and metrics, including models that estimate the number of defects, the
• number of test cycles, the number of test cases, the average effort of each test, and the number of regression cycles involved
• sector averages and predictive models, such as function points, lines of code, estimated development effort, or other design parameters.

In most cases, the estimates, once drafted, must be delivered to the management, together with their justification. Often, negotiation approaches follow, which translate into a re-elaboration of the estimate. Ideally, the final estimate of testing represents the best possible balance between organizational and design objectives, in the areas of quality, timing, budget and product characteristics.