Testing automation: Features, advantages and difficulties of automatic testing

In this article on automatic testing we will analyze the features, advantages and difficulties of using automatic testing in the end-to-end testing of web applications. Automatic testing is still not widespread among companies for various reasons but the main reason is certainly due to the fact that the effort of designing, writing and maintaining an automatic test suite is much more expensive than the fee required for manual tests.

Unlike manual tests, the development of automatic tests, in fact, is easily comparable to that of a software. Before writing an automatic test it is necessary to know the internal structure of the software to be tested, its critical points and many implementation details. In order to avoid the creation of illegible and difficult-to-maintain tests it is necessary to comment on the complex passages and the functions used; standards must be maintained and guidelines such as the use of red tests must be followed. Finally it is necessary that the test is speaking, that makes the tester understand, in case of error, where it is found and what could be. For these reasons, automatic test developers must have much higher skills than those needed to design and perform manual tests.
Like software programming, even automatic testing is therefore a difficult art to master.

Benefits and difficulties

The main advantages of using automatic tests are:

Possibility to test many aspects of a software in a short time
Reusability of tests on different applications
Possibility to launch tests in parallel to evaluate load capacities or real time collaboration
Result security (assuming that the test is well written)
Low percentage of false positives or negatives (as long as the test is solid)

The main reason why you want to build an automatic test suite is surely because you want to run tests repeatedly after each new release. If a test has to be performed only once, the effort to automate it could exceed the benefits offered.
From a purely theoretical point of view, the introduction of automatic tests in a software life cycle is a mandatory step for a company because, in the long term, it greatly simplifies all testing; however, clashing with the economic reality one realizes that the construction of a suite of automatic tests is a job, above all initially expensive and complicated. To write solid automatic tests, you need great knowledge in the field of programming, in addition to those concerning testing; a company could, therefore, find it difficult to renounce qualified personnel investable in areas such as design or development to assign it to automated tests.
Investing in automated tests means facilitating software maintenance, releasing more secure and reliable updates, greatly reducing the likelihood that the customer will find a bug in the product and reducing the time invested in the testing process in the long term. Even if, on paper, it may seem a superfluous and easily avoidable expense, the investment in automatic tests constitutes a fundamental step for a company that wants to produce quality software, updated and safe over time.

Time and money

One of the main reasons for automating a test is long-term time savings. Many tests are launched several times after some time. Whenever a piece of code is changed in the software, the test should be repeated. Performing a manual test each time is certainly more time-consuming than writing an automatic test and programming its launch.
Moreover, if the application is present on different browsers or operating systems, the amount of tests to be launched at each modification increases considerably. Once created, an automatic test can instead be launched countless times without any additional cost.
The performance of an automatic test is certainly faster than that of a manual test, which further reduces the testing time after each update, allowing the company to save time and money.

Precision

Even the most attentive tester can make mistakes during the monotonous execution of manual testing. The automatic tests perform exactly the same steps every time they are launched, without forgetting to record in detail the results obtained. Testers, who no longer have to perform manual tests, have more time to devote to writing new automated tests or implementing more complex features.

Test coverage

Automated testing can greatly increase the depth and scope of tests, helping to increase software quality. Long tests, often avoided in manual testing, can be developed and launched more easily through automated tests. They can also be launched on different browsers or devices, with different configurations. The automatic tests can enter the application and look at the contents of the memory, the data, the contents of the files and the internal status of the software, determining if the software is behaving properly. They can also execute hundreds of different and complex test cases simultaneously, providing an impossible code coverage for manual tests.

Contemporary users

Even the largest Quality Assurance team could not run a controlled web application test with thousands of users. Automated tests can easily simulate tens, hundreds or thousands of virtual users that interact simultaneously with a network, software or web application.
Despite the usefulness and power of automated tests, these fail (yet) to fully satisfy all application testing needs. In many cases, in fact, only manual tests can be considered consistent and safe. One of these cases is that of graphical interfaces and everything related to screen display. Since the automatic tests work directly on the application source code, in some situations they fail to notice errors due solely to the display. Typical problems of this kind can occur due to the incompatibility between different browsers or operating systems and are defects that are not easily found by analyzing only the source code.