Cucumber
AIO Tests provides integration with Cucumber tests by allowing users to import their Cucumber-based tests/scenarios into AIO Tests, either by mapping to existing cases or by creating new ones from the Cucumber results JSON file. Since AIO Tests rely only on the JSON report generated by Cucumber, the tests can be written in any of the supporting languages like Ruby, Perl, Java, Javascript, etc.
Having the automated results reporting in AIO Tests can help in having a holistic view of the coverage of requirements (unit to manual to functional and end-to-end automated tests).
This guide discusses how to import your existing Cucumber scenarios as AIO Tests and how to capture their execution results in AIO.
In this documentation, you’ll understand:
- 1 Cucumber
- 2 Status Mapping Cucumber → AIO Tests
- 3 Mapping Cucumber Tests to AIO Tests
- 3.1 Scenario 1: I have existing automation scenarios, but do not have any cases corresponding to them in AIO Tests. I want to create cases from my existing automation tests.
- 3.2 Scenario 2: I have created my scenarios in AIO Tests. Now I am automating those. How do I define a mapping?
- 3.3 Scenario 3: I have mapped scenarios in my feature files to AIO Tests. My steps have been updated. Do I need to update them in AIO Tests as well?
- 4 Mapping a Case to a Folder
- 5 Scenario Outline
- 6 Screenshots for Failed Cases
- 7 Importing Results
Cucumber
Cucumber is an open-source framework facilitating Behavior Driven Development (BDD), which helps create a formal understanding of system behavior among all stakeholders, from the business to the technical teams.
Cucumber is driven by simple domain-specific language (DSL) called Gherkin, which uses natural-language constructs like Given-When-Then
to describe a system’s behavior and its expected outcomes.
Sample Cucumber Scenario
Gherkin uses a set of keywords to formalize a behavior. Most lines start with one of the keywords.
Cucumber tests can be structured as Features. A feature can look at encapsulating a particular functionality of a system.
Features are made of Scenarios, which are essentially different user flows of the functionality being tested.
Scenarios are made of Steps. Each step starts with Given
, When
, Then
or And
.
Feature: Capture the number of trees planted and show relation to global warming
@P0
Scenario: Trees effect on global warming
Given There are no trees numbered in a given region
When Each user plants a tree and updates
Then The temperature is updated in the temperature log
When Temperature is captured over a few days
Then The graph should show the relation between the number of trees and temperature
Generating a JSON Report
Cucumber supports generating JSON reports by default, amongst other formats like HTML, JUnit etc.
For e.g., to generate a JSON report in Java, either in JUnit or TestNG, use the annotation @CucumberOptions
Cucumber Sample JSON Report
The JSON report generated by Cucumber gives a detailed step-by-step report as follows :
Status Mapping Cucumber → AIO Tests
Cucumber JSON | Description | AIO Tests Mapping | Run Status |
---|---|---|---|
Passed | Passed step | Passed | Passed, if and only if, all steps are passed |
Failed | Failed step | Failed | Failed, if even a single step is failed |
Skipped | Step was not executed due to failure in a previous step | Not Run | In Progress, if one step is Not Run and other steps are passed. Failed , if any step is failed and others are Not Run |
Undefined | Step does not have a corresponding implemented step definition | Not Run | In Progress, if one step is Not Run and other steps are passed. Failed , if any step is failed and others are Not Run |
Mapping Cucumber Tests to AIO Tests
Scenario 1: I have existing automation scenarios, but do not have any cases corresponding to them in AIO Tests. I want to create cases from my existing automation tests.
Since these tests do not exist in AIO Tests yet, you can simply Import the Cucumber JSON report, either through the Import Results REST API or through the Cycles tab in AIO Tests.
Make sure to check the option to create new cases if no mapping is found option.
On import, cases would be created with the following mapping from the JSON report