Before being released, any software product goes through a long path of checks and improvements until it meets the expectations of business owners, developers and, eventually, users. We strongly believe that proper quality assurance is essential for any project, whether it is a one-page website, a simple mobile application or a complex enterprise product. What is proper QA then, manual or automated? In brief: manual always, automated often. Let’s elaborate!
Manual testing is when a meticulous QA engineer carefully checks every single detail of the design and logic, clicks all the buttons and links, and how it all works on different devices and in different systems. Then the person writes a beautiful clear report to explain to the developers what has gone wrong.
Automated testing is when a technically savvy QA or software engineer writes a script that tests the desired areas in the project automatically. There are different approaches in automated testing. There is E2E testing when the entire system or interface is tested. And there are unit tests, which are programs that check the operation of a small part of the code. Developers regularly update sites and applications, add features, refactor code and make changes, and then check how everything works with unit tests.
The most important thing in manual testing is the human. A QA engineer carefully examines the product, and pays attention to literally everything, even to what they didn’t seem to be asked for. At least a good QA, like our Zhanna.
Algorithms perform better and faster than humans when it comes to repeatable and predictable scenarios, as well as when it comes to checking the load on the system. But automation tools do not have a creative mind, they have no intuition and curiosity, they only check what they are programmed for and never think: “What will happen if the user does this or that?”.
Zhanna Shulha, QA at Kultprosvet
The human eyes and brain are indispensable in user experience testing. Only a human understands another human and can see what the algorithm does not see: a violation of the logic in the interface, its conveniences and inconveniences, etc.
On the other hand, humans get tired and make mistakes, and work slower than algorithms, therefore, the advantages and disadvantages of manual testing are all about humans.
Which projects are suitable for manual testing?
All of them. Not a single project can do without an attentive QA engineer.
For smaller projects, such as a simple mobile app or website, only manual testing is needed. The thing is, automation tools are expensive. Their setup and maintenance also require significant costs. When a company is working on a complex long-term project, that investment pays off over time. But with small applications, things are different.
In large projects with cool automated scripts, you still need a person to write and check these scripts, and also to check what algorithms are simply not capable of. The algorithm, for example, cannot check how your VR tennis application responds to hand movement.
Small changes in the product are also better to check manually because it will take a lot of time and money to write tests.
Algorithms work dozens of times faster than humans, and they make fewer mistakes. However, their capabilities are limited. No matter how perfect modern automation tools are, they cannot evaluate user experience. And no computer program can do the job of evaluating how an application responds to gesture controls as well and quickly as a manual tester can.
The initial investment is usually much higher than manual testing, but this investment will certainly pay off in the long run.
When to choose automated testing?
Autotests are better for repetitive tasks, they save time and money on large projects, and provide more complete test coverage.
In large and long-term projects, automated tests usually cover all critical scenarios that affect the success and profitability of the project. For example, adding a product to the cart and checking out in an online store.
Covering any scenarios with autotests, up to the whole project, is possible, but not always necessary. However, code updates are often covered by unit tests. Testing the entire system after each update is rather tedious and inefficient, and unit tests inspect small pieces of code.
Basically, there are two approaches in automated testing, these are E2E and unit tests. End-to-end tests scan the system as a whole, unit tests check separate parts of code in an artificial environment (think of testing a steering wheel in carton boxes with no wheels attached). Unit tests are fast, but since each component is tested in an artificial environment, they need to be improved, therefore I kind of often make a hybrid of unit tests and E2E tests.
Stas Volnyansky, Team lead at KPS
Performance tests such as load testing, stress testing, spike testing, etc. should definitely be tested by an automated tool.
Each project needs manual testing, and some but not all need automatic testing. Small projects and startups need at least one manual tester, preferably more, depending on the team size and pace of work. Large long-term projects often require a team of manual testers and a bunch of automated tests.