Executing a Dataset Based Case in a Cycle
- Niharika [AIO Tests]
The Executions of dataset based cases in a Cycle works quite similar to normal cases, with a run comprising of one execution per dataset.
This article describes in detail how users can execute dataset based cases and record their results.
Dataset based case display in Cycle
Dataset based cases exist along with non-dataset based cases in the cycle screen.
To narrow down cases having datasets, the datasets filter can be used as below:
Understanding the Case Row
The details shown in the grid at the case level remain the same :
case keys and title
a pencil icon to edit case directly from cycle (on hover over of a case row). Once the case is updated, user can choose if he wants to update the existing run or create a new run
assignee for execution and a pencil icon to change the assignee (on hover over of a case row)
number of runs associated with each case
an “Add run” link for executing additional runs of a case. Users can choose this option when they want to execute the same case in the same cycle again. Users get the option to use the latest version as well as to copy results and values from the previous run
execution status of the case
number of defects associated with the latest run of the case
an icon to remove the case from the Cycle - removal of case would remove all dataset based executions and all runs
Executing a Case
To execute a case, users will open the case using the carat icon (>) on the left. As for normal (non dataset based) cases, Users will see ‘Run 1’ of the case created by the system for the first execution of the case.
However, the run itself has no steps in case of dataset based cases, since steps belong to the dataset executions.
Each new run of a dataset based case will have multiple executions created based on the number of datasets defined in the case.
Understanding the Run Row
The run row shows the details of the run as well as the count of dataset executions under that run, including the status and metadata of the run. The latest run of the case cannot be deleted.
The run number [latest runs are at the top]
In curly braces, the count indicates the number of existing dataset executions in the run
‘More Details’ icon - shows the estimated effort of the case, created/updated run date and time and executed by information of the latest dataset execution
Reset Run - resets all the status, comments, attachments, stepwise results and defects of all dataset executions as well as the encapsulating run
Execution Mode - Either manual or automated and is set only at run level and not at dataset level
Effort - Actual effort logged against the run. If effort is added at dataset level, then the sum of all dataset executions actual effort becomes the run’s actual effort. It can be overridden at run level by clicking on the pencil icon next to the effort field
Timer - Timer can be started at encapsulating run level to track all executions as whole.
Status selector and display - Status of a run can be selected at run level. It is also automatically calculated if dataset execution status changes. Refer to Status Change rules.
Defects - Defects at run level show the total of all unique defects at run level as well as dataset level
Comments/Attachments - Comments and Attachments which are common across all datasets can be captured in this section.
Pre-conditions, if defined, is displayed just below the Run row
Custom Fields for Run, if defined, are displayed below Pre-conditions or below Run row (if Pre-conditions are not defined for the case). Users can edit Custom fields using the pencil icon
Understanding the Dataset Row
Each dataset execution can have it’s own status, effort, comments, defects and attachments. A dataset can also be deleted from a particular run. However, the last dataset of a run cannot be deleted.
In case the last dataset needs to be removed, then it would mean a case or run is no longer required and so, either a case can be removed or the run can be deleted.
The dataset row number - this maps to the row number of the case. In case user deletes dataset 2, then 1 and 3 datasets would be displayed.
‘More Details’ icon - shows the execution and creation metadata of the dataset execution
Reset Dataset - resets all the status, comments, attachments, stepwise results and defects of the selected dataset run - recalculates the effort and status of the run
Delete Dataset - deletes the single dataset - recalculates the effort and status of the run
Effort - Actual effort logged against the dataset. All efforts are summed up to calculate the total effort of the run.
Timer - Timer can be started at dataset level to capture execution time of a particular dataset
Status selector and display - Status of a dataset can be selected at dataset level. It impacts the overall status of the run. Refer to Status Change rules.
Defects - Defects at dataset level can be captured. Step level defects are also summarized in the defects view.
Comments/Attachments - Comments and Attachments specific to selected dataset
Datasets Steps
On expanding the dataset row, the steps are displayed with the dataset values being replaced in the steps. The datasets values are highlighted in the steps to better represent the data.
Users can add actual results for each step using the “Add actual result” link. Each step row also has
Execution status icon so that users can mark the status of each step (changing step status impacts the status of the dataset execution, the encapsulating run as well as the overall case as per the percolation rules
Defects icon to log defects (how to log defects is discussed in detail below)
Comments icon to tag folks and post relevant comments
Attachments icon to upload evidence of execution
Marking Execution Status and Status Percolation
Execution Status options (Not Run, In Progress, Blocked, Passed, Failed and any customized statuses) are shown on the click of the status icon from Case, Run, Dataset and Step on the cycle details page allowing user to set the desired status at whatever granularity the user chooses.
Status Percolation Rules
From Datasets to Runs
Rule | Dataset Run 1 | Dataset Run 2 | Run status |
|---|---|---|---|
If all datasets are passed, then run status is passed | Passed | Passed | Passed |
If any dataset is failed, then run status is failed | Failed | Passed | Failed |
If any dataset is blocked, then run status is Blocked [Failed rule supersedes Blocked] | Blocked | Passed or any other status | Blocked |
| Blocked | Failed | Failed |
If no dataset is in Blocked or Failed and one of the datasets in In Progress | In Progress | Passed | In Progress |
If any dataset is in Progress and others are Not Run, run status is updated to In Progress | In Progress | Not Run | In Progress |
If all datasets are in Not Run status, then run status is updated to Not Run | Not Run | Not Run | Not Run |
From Steps to Dataset Executions
The rules of Steps to Dataset Executions remain similar to normal runs, but instead of percolating up to the run, the status percolates up to the dataset. Once the dataset status is updated, it updates the main run status too.
Capturing Evidence and Effort
Users can capture the rest of the data like effort, comments, defects and attachments similar to how it is done for the rest of the runs.
Actual Effort
Effort logged against each dataset execution is summed up at the run level. User’s can choose to override the overall effort by changing the time from run row.
Similarly, instead of manual execution, timers can also be triggered at dataset level.
Defects, Comments, Attachments
Capturing defects, comments and attachments remains similar to normal runs.
Defects at run level show the defects raised at dataset level
Defects at step level of datasets will be shown at dataset level as well
Adding New Runs
Users can click on Add New Run to create new runs. If datasets have been deleted in the previous run, it doesn’t impact new runs being created.
As an example, if user has deleted dataset 2 in run 1, on clicking Add Run, the new run will have all datasets based on the test version being used.
If copy run results is selected, wherever the dataset matches fully, the results are copied. In the above example, Run 1 had dataset 2 deleted.
On adding a new run, all 3 datasets are created and results are copied for existing datasets whose data values map exactly.