/
Cucumber

Cucumber

AIO Tests provides integration with Cucumber tests by allowing users to import their Cucumber-based tests/scenarios into AIO Tests, either by mapping to existing cases or by creating new ones from the Cucumber results JSON file. Since AIO Tests rely only on the JSON report generated by Cucumber, the tests can be written in any of the supporting languages like Ruby, Perl, Java, Javascript, etc.

Having the automated results reporting in AIO Tests can help in having a holistic view of the coverage of requirements (unit to manual to functional and end-to-end automated tests).

This guide discusses how to import your existing Cucumber scenarios as AIO Tests and how to capture their execution results in AIO.

In this documentation, you’ll understand:

Cucumber

Cucumber is an open-source framework facilitating Behavior Driven Development (BDD), which helps create a formal understanding of system behavior among all stakeholders, from the business to the technical teams.

Cucumber is driven by simple domain-specific language (DSL) called Gherkin, which uses natural-language constructs like Given-When-Then to describe a system’s behavior and its expected outcomes.

Sample Cucumber Scenario

Gherkin uses a set of keywords to formalize a behavior. Most lines start with one of the keywords.

Cucumber tests can be structured as Features. A feature can look at encapsulating a particular functionality of a system.

Features are made of Scenarios, which are essentially different user flows of the functionality being tested.

Scenarios are made of Steps. Each step starts with GivenWhenThen or And.

Feature: Capture the number of trees planted and show relation to global warming @P0 Scenario: Trees effect on global warming Given There are no trees numbered in a given region When Each user plants a tree and updates Then The temperature is updated in the temperature log When Temperature is captured over a few days Then The graph should show the relation between the number of trees and temperature

Generating a JSON Report

Cucumber supports generating JSON reports by default, amongst other formats like HTML, JUnit etc.

For e.g., to generate a JSON report in Java, either in JUnit or TestNG, use the annotation @CucumberOptions

import io.cucumber.testng.AbstractTestNGCucumberTests; import io.cucumber.testng.CucumberOptions; @CucumberOptions( features = "src/test/resources/features", glue = {"com.aiotests.samples.stepdefs"}, plugin = { "pretty", "json:target/cucumber-reports/Cucumber.json", "junit:target/cucumber-reports/Cucumber.xml"}) public class MainRunner extends AbstractTestNGCucumberTests { }
@RunWith(Cucumber.class) @CucumberOptions( features = "src/test/resources/functionalTests", glue= {"stepDefinitions"}, plugin = { "pretty" }, monochrome = true ) public class JUnitRunner { }

Cucumber Sample JSON Report

The JSON report generated by Cucumber gives a detailed step-by-step report as follows :

Status Mapping Cucumber → AIO Tests

Cucumber JSON

Description

AIO Tests Mapping

Run Status

Cucumber JSON

Description

AIO Tests Mapping

Run Status

Passed

Passed step

Passed

Passed, if and only if, all steps are passed

Failed

Failed step

Failed

Failed, if even a single step is failed

Skipped

Step was not executed due to failure in a previous step

Not Run

In Progress, if one step is Not Run and other steps are passed.

Failed , if any step is failed and others are Not Run

Undefined

Step does not have a corresponding implemented step definition

Not Run

In Progress, if one step is Not Run and other steps are passed.

Failed , if any step is failed and others are Not Run

Mapping Cucumber Tests to AIO Tests

Scenario 1: I have existing automation scenarios, but do not have any cases corresponding to them in AIO Tests. I want to create cases from my existing automation tests.

Since these tests do not exist in AIO Tests yet, you can simply Import the Cucumber JSON report, either through the Import Results REST API or through the Cycles tab in AIO Tests.

Make sure to check the option to create new cases if no mapping is found option.

On import, cases would be created with the following mapping from the JSON report