Selecting the Right Testing Approach – Manual vs. Automation Testing
- December 5, 2013
Software development teams are always looking for an edge to develop software features more quickly while ensuring a higher level of software quality. In order to achieve these goals, quality assurance teams have to decide the testing methods and approaches while preparing test strategy. QA teams can opt for manual testing, automated testing or a hybrid approach. Testing approach selection largely depends on certain factors like project duration, complexity, underlying technologies, budget and resources availability etc.
To select the best possible testing approach, apart from understanding project specific requirements, we also need to understand the difference between manual and automated testing as well as the benefits and drawbacks of using these approaches in different project scenarios. In this post, we’ll compare different factors that will help the testers in deciding the best testing approach.
Project Duration & Frequency of Test Runs:
If software is relatively small and has a shorter period of development, then it means that we will be running lesser number of test cases with less frequency. So, we will opt for Manual testing.
Project Duration & Frequency of Test Runs:
If the application is expected to be in development for longer period of time, then it means that a lot of regression testing will be involved. In such cases it is better to use automated testing – as it will not only reduce long term costs but will also reduce time to run the regression tests.
Allows ad-hoc & random Testing:
Manual testing is a much better option if you want to do ad-hoc and random testing. It is particularly helpful in such scenarios where not much documentation is available and you want to get the test results quickly.
Facilitates Systematic & Data-driven Testing:
Automation testing is more suitable in scenarios where you want to find bugs by running systematic test cases. Particularly when we have to do data-driven or keyword-driven testing, it is best to go for automated testing approach .
Not Suitable for Compatibility Testing:
Same test manually need to be run separately on different machines and browsers which is time consuming, tedious and tiresome job.
Quick and Economical Compatibility Testing:
Normally automated scripts are written in such a way that they can be run on different browsers, machines and configurations. So, automated testing helps us by saving our testing time and efforts while doing compatibility testing.
Short Term Costs are Reduced:
Manual testing does not involve script writing in initial stages which reduces the initial time and money investment.
Long Term Costs are Reduced:
Automated testing involves higher initial investment, but once the scripts are written they can be run for countless times, reducing the long term costs.
Not Suitable for Load Testing or Low Level Interface Testing:
It is difficult to do low level interface regression testing or load testing manually.
Not Suitable for Usability, UI, Accessibility and Localization Testing:
Automated test scripts cannot be used to test visual references like font colors etc. User experience testing is also not possible via test scripts.
Almost Everything can be Tested Manually:
Human testers are capable of testing anything that is testable. All that matters is the knowledge and skill set of the resources.
Everything cannot be Automated:
There could be plenty of test scenarios and application areas which are either impossible or are very costly to be automated. For example, testing flash based application areas is not easy and effective.
Manual Testing is good for Finding New and Un-expected Defects:
It is because it involves thinking and learning. One can find bugs by using the application in ways it was not intended to be used.
Automation Testing Generally does not Find New and Un-expected Defects:
It is merely the sequence of actions that are scripted, which can only find the expected bugs.
|Testing Application Developed on New Technologies & platforms: Automated tools for new technologies and platforms usually take years to develop and mature and that is when manual testing fills the gap and fulfills our testing needs.|
Testing Mobile Applications Across Different Devices:
There are hundreds of mobile devices available in market today . It is not possible to manually test the application on all of these devices and that is where automated testing helps us to quickly test the apps on different devices using services like AppThwack or CloudMonkey.
Manual Tests provide Limited Visibility:
If developers, team leads, managers want to verify and see the test results by themselves they have to repeat all test steps. Additional effort is required every time.
Automated Tests provide Global Visibility:
Developers, team leads, managers can run the tests with single click and see the results by themselves without any additional effort.
In light of the above comparison, it can be concluded that both manual and automated testing have their strengths and weaknesses. None of these two testing approaches can replace each other. It is better to adopt the automated testing approach for those areas of application, whose requirements will not change frequently and where a lot of regression testing will be involved. On the other hand it will be better to use the manual testing approach, when the project is short termed, involves less regression and when we want to do the usability, accessibility and GUI testing which is not possible via automated test scripts. So, we can conclude that final approach towards testing any application can vary from project to project depending on all these factors. It is up to us to identify and analyze all these factors, and decide which areas should be automated and which are better to be tested manually.