Our documentation site has been updated to AIO Tests Knowledge Base

Automation Coverage Report

Owned by Noopur Varshney [AIO Tests]
Last updated: Feb 17, 2023
3 min read

The Automation Coverage Report can be used by managers to seek details about the automated cases against the manual cases. They can measure and compare the savings in effort, due to automation testing. The report allows to specify cases for which automation coverage can be seen across multiple parameters.

The different case selection criteria for generating the report are - Cases, Sets, Cycles & Requirements.

Generating Automation Coverage Report

Click on the right chevron, next to the report name to get the input screen for the report. There are 4 ways to specify for which cases the report should run. With each cases selection criteria, there is an option to specify a parameter - “Distribution By”, using which the automation coverage will be shown.

  • Cases – Users can specify a list of cases by using filters or keep it empty to select all cases of the project The default distribution is shown by ‘Automation Owner’. But users can choose any of the available options

     

     

  • Sets – Users can specify a list of sets by using filters. Report will be generated for unique cases belonging to specified sets. The default distribution is shown by ‘Sets’. But users can choose any of the available options

     

     

  • Cycles – Users can specify a list of cycles by using filters. Report will be generated for unique cases belonging to specified cycles. The default distribution is shown by ‘Cycle’. But users can choose any of the available options

     

     

  • Requirements – Users can specify a list of requirements by specifying Jira issue IDs, by using JQL or by using Jira’s saved filters. Report will be generated for cases linked to the specified requirements. The default distribution is shown by ‘Requirement’. But users can choose any of the available options

     

Click on “Generate” after specifying cases criteria and the distribution parameter. The report will be generated as shown below.

 

 

Understanding the Report Output

The three sections of the report are Overview, Distribution and Coverage details.

Overview

The Overview section shows the

  • Total Cases -> total number of cases as per the selection criteria (excluding the draft cases of other users)

  • Automated -> Number of automated cases (based on Automation Status = Automated)

  • Non-Automated -> number of cases yet to be automated (based on Automation Status != Automated)

  • Total Effort -> Sum of estimated effort of all specified cases

  • Effort Savings -> Sum of estimated effort of automated cases

  • Manual Effort -> Sum of estimated effort of non-automated cases

If the user does not have access to some cases, then all fields will only reflect accessible cases data.

 

Distribution

The distribution pie chart shows the cases in different Automation statuses - Automated, In Progress, to be Automated, Manual and Not Assigned. On hovering upon each slice of pie, the status count can be seen. 

 

Coverage Details

This section shows a quick summary of the automated cases with respect to the distribution parameter given in the input criteria.
The grid columns are divided into two sub-categories - Automation Status and Effort.