Cypress framework is a JavaScript-based end-to-end testing framework built on top of Mocha, which allows you to write end-to-end, integration as well as unit tests. Now, AIO Tests supports direct reporting from Cypress, with its cypress-aiotests-reporter npm package.

In this documentation, you’ll understand:

1 What Does the AIO Tests Native Cypress Reporter Do?
2 How to Get Started?
3 How to Report Results?
- 3.1 Mapping Automated Cypress Tests to AIO Tests
- 3.2 Mapping Automated Cypress Cucumber Tests to AIO Tests
- 3.3 Setup & Configuration
  - 3.3.1 Configure
    - 3.3.1.1 Cloud
    - 3.3.1.2 Server
    - 3.3.1.3 Cypress 10 and above
  - 3.3.2 Configurable Values
    - 3.3.2.1 Create New Cycle options
    - 3.3.2.2 Parallel builds
4 Logging
5 Using multiple reporters?
6 Queries/Suggestions?

What Does the AIO Tests Native Cypress Reporter Do?

With AIO Test’s Cypress reporter, AIO Tests simplifies the reporting of results from the automated Cypress tests to AIO Tests for Jira. By hooking into Cypress events, the AIO Tests Reporter reports results in the after:spec event, after every spec run finishes. The reporter can:

report results against AIO Tests cases.
create a new cycle for the executions or reuse existing cycles, based on the configuration done in cyress.json.
upload attachments for failed executions.
Retries can either be reported as new runs or used to update the existing run.

The reporter can work only with the cypress run and not with cypress open command, since Cypress app doesn’t provide the results information in the after:spec event.

How to Get Started?

AIO Tests Cypress reporter comes as an npm package and can be installed via the standard install command.

npm install cypress-aiotests-reporter

How to Report Results?

AIO Tests reporter reports results after every spec file. Once the tests in a spec file are finished, the reporter searches for mappings in the tests and uploads mapped test case results to AIO Tests.

The below section covers how to map Cypress automated cases to AIO Tests, how to setup the AIO Tests Reporter and the possible configuration values.

Mapping Automated Cypress Tests to AIO Tests

The AIO Tests Case key can be added to the ` describe ` and ` it ` function descriptions. <br> If there are multiple case keys in a single description, then the result of one test will be updated to multiple cases.

The case key can appear anywhere in the description.

Mapping Single Case

describe('example to-do app', () => {
  beforeEach(() => {    ..  })

  it('displays two todo items by default (NVTES-TC-72)', () => {
    cy.get('.todo-list li').should('have.length', 2)
    cy.get('.todo-list li').first().should('have.text', 'Pay electric bill')
    cy.get('.todo-list li').last().should('have.text', 'Walk the dog')
  })

Mapping to Multiple Cases


describe('example to-do app', () => {
  beforeEach(() => {    ..  })

  it('NVTES-TC-72, NVTES-TC-73 : displays two todo items by default', () => {
    cy.get('.todo-list li').should('have.length', 2)
    cy.get('.todo-list li').first().should('have.text', 'Pay electric bill')
    cy.get('.todo-list li').last().should('have.text', 'Walk the dog')
  })

Mapping Automated Cypress Cucumber Tests to AIO Tests

The AIO Tests Case key can be added to the scenario name. If there are multiple case keys in a single scenario, then the result of one scenario will be updated to multiple AIO Tests cases.

The case key can appear anywhere in the scenario name.

Mapping Single Case- The scenario is mapped to NVTES-TC-1003 in the below case

Feature: Settings Tests
  Scenario: NVTES-TC-1003 visiting the frontpage
    When I visit Settings page
    Then I see all configurable options

Mapping to Multiple Cases

Feature: Settings Tests
  Scenario: NVTES-TC-1003, NVTES-TC-1007 Validate settings and API keys
    When I visit Settings page
    Then I see all configurable options
    When I click on API Keys
    Then I see option to generate an API key

Scenario Outline with Examples

Feature: Settings Tests
  Scenario Outline: {TCId} Validate links work
    When I visit Settings page
    Then I see all configurable options
    When I click on {PageToTest}
    Then Profile {PageToTest} page opens with options
  Examples:
  |TCId     |PageToTest     |
  |NV-TC-11 |API Token      |
  |NV-TC-12 |Email Settings  |

Setup & Configuration

Versions before Cypress 10 Register the plugin to the Cypress plugins file as below:

// cypress/plugins/index.js
const { registerAIOTestsPlugin } = require('cypress-aiotests-reporter/src')

module.exports = (on, config) => {
  registerAIOTestsPlugin(on,config);
}

Cypress 10 and beyond In Cypress 10, the pluginsFile option was removed. This option was replaced with the new setupNodeEvents(). So, the plugin registration has to happen as below in the cypress.config.js

// cypress.config.js
const { defineConfig } = require("cypress");
const { registerAIOTestsPlugin } = require('cypress-aiotests-reporter/src')

module.exports = defineConfig({
  e2e: {
    setupNodeEvents(on, config) {
      registerAIOTestsPlugin(on,config);
      // implement node event listeners here
    },
  },
 }

Configure

The AIO Tests Reporter config needs to be set in the env property of cypress.json. Or it can be programmatically modified in your plugins/index.js.

Depending on the Jira hosting, the authentication information needs to be provided as below.

Cloud

For Jira Cloud (eg. https://yourco.atlassian.net/..), the "cloud" property needs to be set in the env.aioTests config for authentication.

Local runs : The API Key generated from AIO Tests, needs to be set as "apiKey" value.
CI/CD: For CI runs, you can set the AIO_API_KEY environment variable to pass it externally as a SECRET.

// cypress.json
{
  "env": {
    "aioTests": {
      "enableReporting": true,
      "cloud": {
        "apiKey": "<your API KEY>"
      },
      "jiraProjectId": "SCRUM",
      "cycleDetails": {
        "createNewCycle": true, //possible values "true","false","CREATE_IF_ABSENT", true, false
        "cycleName": "Cypress first run from plugin",
        "cycleKey": "NVTES-CY-2",
        "folder": ["Cloud","Smoke Test Nightly"],
        "tasks": ["SCRUM-1","SCRUM-2],
        "customFields": [
          {
            "name": "Reviewed? [Boolean CF]",
            "value": "Yes",
          },
          {
            "name": "Set (Single Select CF)", 
            "value": {"value":"P1"}
          },
          {
            "name": "Teams (Multi Select CF)", 
            "value": [{"value":"TeamAlpha"},{"value":"Zeta"}]
          },
          {
            "name": "NumberCF", "value": 0
          },
          {
            "name": "TextValue CF", "value": "This can be a long note",
          },
          {
            "name": "Reviewed Date", "value": "2024-08-29T04:38:36.437Z",
          },
          {
            "name": "SME [User CF]", "value": "<accountid of user>"
          }
        ],
      },
       "runDetails": {
        "customFieldsToUpdate": [
          {
            "operationType": "ADD_TO_EXISTING",
            "name": "StepOwner",
            "value": [{"value": "Val2"},{"value": "Val1"}]
          },
          {
            "name": "Env", "value": {"value":"UAT"}
          }
        ]
      },
      "addNewRun": true,
      "addAttachmentToFailedCases": true,
      "createNewRunForRetries": true,
      "addTestBodyToComments": true,
      "parallelBuild":{ //optional
        "masterBuild": true,
        "waitForSeconds": 10 //defaults to 2 seconds
      }
    }
  }
}

Server

For Jira Hosted or DataCenter versions, the "hosted" property needs to be set in the env.aioTests for authentication.
The "jiraUrl" needs to be specified with the base URL of the hosted Jira instance.

Authentication is supported either by providing a Jira username and password or by using the Jira PAT. More information can be found on Server Authentication here.

Local Runs: For local runs, either "jiraUsername" + "jiraPassword" can be set or one can simply set the "jiraPAT" value.
CI/CD: For CI runs, you can set the JIRA_USERNAME and JIRA_PASSWORD or JIRA_PATenvironment variable to pass it externally as a SECRET.

// cypress.json
{
  "env": {
    "aioTests": {
      "enableReporting": true,
      "hosted" : {
        "jiraUrl": "https://jira.yourco.com",
        "jiraPAT": "PAT from Jira Tokens | JIRA_PAT as environment variable",
        "jiraUsername": "Jira Username. If PAT is specified, then username is not required",
        "jiraPassword": "Jira password, required if authentication is through username/password"
      },
      "jiraProjectId": "SERV",
      "cycleDetails": {
        "createNewCycle": true, //possible values "true","false","CREATE_IF_ABSENT", true, false
        "cycleName": "Cypress first run from plugin",
        "cycleKey": "NVTES-CY-2",
        "folder": ["Cloud","Smoke Test Nightly"],
        "tasks": ["SCRUM-1","SCRUM-2],
        "customFields": [
          {
            "name": "Reviewed? [Boolean CF]",
            "value": "Yes",
          },
          {
            "name": "Set (Single Select CF)", 
            "value": {"value":"P1"}
          },
          {
            "name": "Teams (Multi Select CF)", 
            "value": [{"value":"TeamAlpha"},{"value":"Zeta"}]
          },
          {
            "name": "NumberCF", "value": 0
          },
          {
            "name": "TextValue CF", "value": "This can be a long note",
          },
          {
            "name": "Reviewed Date", "value": "2024-08-29T04:38:36.437Z",
          },
          {
            "name": "SME [User CF]", "value": "<accountid of user>"
          }
        ],
      },
       "runDetails": {
        "customFieldsToUpdate": [
          {
            "operationType": "ADD_TO_EXISTING",
            "name": "StepOwner",
            "value": [{"value": "Val2"},{"value": "Val1"}]
          },
          {
            "name": "Env", "value": {"value":"UAT"}
          }
        ]
      },
      "addNewRun": true,
      "addAttachmentToFailedCases": false,
      "createNewRunForRetries": false,
      "addTestBodyToComments": true,
      "parallelBuild":{ //optional
        "masterBuild": true,
        "waitForSeconds": 10 //defaults to 2 seconds
      }
    }
  }
}

Cypress 10 and above

For Cypress 10, the settings have moved to the cypress.config.js. Set the above aioTests configuration in the cypress.config.js as below:

module.exports = defineConfig({
  env: {
    "aioTests": {
      "enableReporting": true,
      "cloud": { //Replace with server authentication if using Jira Server
        "apiKey": "apikey for cloud"
      },
      "jiraProjectId": "NVTES",
      "cycleDetails": {
        "createNewCycle": true, //possible values "true","false","CREATE_IF_ABSENT", true, false
        "cycleName": "Cypress first run from plugin",
        "cycleKey": "NVTES-CY-2",
        "folder": ["Cloud","Smoke Test Nightly"],
        "tasks": ["SCRUM-1","SCRUM-2],
        "customFields": [
          {
            "name": "Reviewed? [Boolean CF]",
            "value": "Yes",
          },
          {
            "name": "Set (Single Select CF)", 
            "value": {"value":"P1"}
          },
          {
            "name": "Teams (Multi Select CF)", 
            "value": [{"value":"TeamAlpha"},{"value":"Zeta"}]
          },
          {
            "name": "NumberCF", "value": 0
          },
          {
            "name": "TextValue CF", "value": "This can be a long note",
          },
          {
            "name": "Reviewed Date", "value": "2024-08-29T04:38:36.437Z",
          },
          {
            "name": "SME [User CF]", "value": "<accountid of user>"
          }
        ],
      },
       "runDetails": {
        "customFieldsToUpdate": [
          {
            "operationType": "ADD_TO_EXISTING",
            "name": "StepOwner",
            "value": [{"value": "Val2"},{"value": "Val1"}]
          },
          {
            "name": "Env", "value": {"value":"UAT"}
          }
        ]
      },
      "addNewRun": true,
      "addAttachmentToFailedCases": true,
      "createNewRunForRetries": true,
      "addTestBodyToComments": true,
      "parallelBuild":{ //optional
        "masterBuild": true,
        "waitForSeconds": 10 //defaults to 2 seconds
      }
    },
  },
  e2e: {...
  }}

Configurable Values

Value	Description

Value	Description
enableReporting	Set to true to make the current run update results to AIO Tests. Default false.
jiraProjectId	Jira Project key to update results to
cycleDetails.createNewCycle	Options: [true, false, "CREATE_IF_ABSENT"]. Set to true to create a new cycle for run being reported.
cycleDetails.cycleName	Works if createNewCycle is true, sets the cycle name of cycle getting created
cycleDetails.cycleKey	AIO Tests cycle key that should be updated. Used if createNewCycle is false
cycleDetails.folder	Folder hierarchy, where first item in array is parent folder and so on eg.["Parent","Child"]
cycleDetails.tasks	List of Jira Issue Keys to attach as Tasks to created cycle, impacts only when creating new cycle
addNewRun	Create a new run or update an existing run in the cycle
addAttachmentToFailedCases	Set to true to attach screenshots, if available, for failed cases
createNewRunForRetries	Set to true if each retry should create a new run
addTestBodyToComments	Set to true test script body should be added as a comment in a failed case
runDetails.customFieldsToUpdate	List of run level custom fields. Options in example above.
customFieldsToUpdate.operationType	Options: ADD_TO_EXISTING, REPLACE_EXISTING, DELETE_EXISTING
debugMode	Default false. Set to true to increase verbosity of logs while debugging an issue
parallelBuild.masterBuild	Optional. Default true. See below for details on parallelBuild
parallelBuild.waitForSeconds	Optional. Default 2 seconds. See below for details on parallelBuild

Create New Cycle options

createNewCycle = true or "true", uses the cycleName and cycleDetails value to generate new cycle
createNewCycle = false or "false", uses the cycleKey value to find an existing cycle and updates the cycle. If cycle is not found, an error is thrown
createNewCycle = "CREATE_IF_ABSENT" uses the cycleName to search for an existing cycle with an exact match. If cycle is found, then the cycle is updated. If it is not found, then a new cycle is created using cycleName and cycleDetails

Parallel builds

If multiple builds are being triggered in parallel, the parallelBuild setting can be used to specify the masterBuild.
One of the parallel builds can be configured to set masterBuild as true. This build should have createNewCycle either set as true or CREATE_IF_ABSENT.
The other builds running in parallel can have masterBuild set to false, which would imply, they would wait for the masterBuild to run the cycle creation code, waiting for waitForSeconds (defaults to 2 seconds), before trying to find the cycle.

If multiple builds are not being run in parallel, the parallelBuild value can be ignored.

Logging

AIO Tests Reporter logs can be seen in the run logs below for successful updates.

Errors received while updating will appear in a similar way.

Using multiple reporters?

Cypress has a known issue - when multiple plugins hook into the same events like 'before:run' or 'after:spec', Cypress cannot invoke the multiple event handlers and can only fire one of the plugins. So one can only have one listener for a specific event. Cypress Issue which tracks the limitation https://github.com/cypress-io/cypress/issues/5240

If there is a need to use multiple reporters, please follow the below workaround:

Install the cypress-plugin-init plugin

npm install cypress-plugin-init

Invoke the plugins as below - Below example shows two plugins being used [ AIO Tests Plugin and Cypress Mochaawesome Reporter

const { registerAIOTestsPlugin } = require('cypress-aiotests-reporter/src')
const  mochaReporter  = require('cypress-mochawesome-reporter/plugin');
const { initPlugins } = require('cypress-plugin-init')
module.exports = defineConfig({
  reporter: 'cypress-mochawesome-reporter',
  e2e: {
    setupNodeEvents(on, config) {
      initPlugins(on, [registerAIOTestsPlugin, mochaReporter], config);
      },
    },
  env: {
    "aioTests": {
    ......

Queries/Suggestions?

For any queries, suggestions or issues, please feel free to reach out @ mailto:help@aiotests.com.

For further queries and suggestions, feel free to reach out to our customer support via help@aiotests.com.

AIO Tests Knowledge Base

Cypress AIO Tests Reporter