Automation Feasibility Analysis

Back to the above example, the project team decided to develop a custom tool which can meet project requirements. Suppose they have been given 100 test cases to automate and they estimated 5 days to develop a tool which can automate all of those test cases.

Here is the result of their work

As in above scenario, the issue is that the test tool cannot automate all the test cases of test specification. It means that not all application features can be thoroughly tested using the test tool.

If the functionality of application under test changes frequently or is too complicated, it is difficult to create test automation for all the application features, because every tool has its own limitations.

If you don’t want to be in such situation, before selecting the test tool, you must analyze the test cases and decide which test cases should be automated and which test cases should not. This is the Automation Feasibility Analysis activity.

Automation Feasibility Analysis is the very significant contributor in testing. In this analysis, you need to check if the application under test is qualified for automated test.

Some factors you need to consider:

To know more about these factors, you can use a checklist which is included in this article

Tool selection process

To select the most suitable testing tool for the project, the Test Manager should follow the below tools selection process

Step 1) Identify the requirement for tools

How can you select a testing tool if you do not know what you are looking for?

You to precisely identify your test tool requirements. All the requirement must be documented and reviewed by project teams and the management board.

Consider the following example:

Step 2) Evaluate the tools and vendors

After baselining the requirement of the tool, the Test Manager should

• Analyze the commercial and open source tools that are available in the market, based on the project requirement.
• Create a tool shortlist which best meets your criteria
• One factor you should consider is vendors. You should consider the vendor’s reputation, after sale support, tool update frequency, etc. while taking your decision.
• Evaluate the quality of the tool by taking the trial usage & launching a pilot. Many vendors often make trial versions of their software available for download

Step 3) Estimate cost and benefit

To ensure the test tool is beneficial for business, the Test Manager have to balance the following factors:

A cost-benefit analysis should be performed before acquiring or building a tool

Example: After spending considerable time to investigate testing tools, the project team found the perfect testing tool for the project Guru99 Bank website. The evaluation results concluded that this tool could

• Double the current productivity of test execution
• Reduce the management effort by 30%

However, after discussing with the software vendor, you found that the cost of this tool is too high compare to the value and benefit that it can bring to the teamwork.

In such a case, the balance between cost & benefit of the tool may affect  the final decision.

Step 4) Make the final decision

To make the final decision, the Test Manager must have:

• Have a strong awareness of the tool. It means you must understand which is the strong points and the weak points of  the tool
• Balance cost and benefit.

Even with hours spent reading software manual and vendor information, you may still need to try the tool in your actual working environment before buying the license.