WebdriverIO + Mocha

AIO Tests supports importing WebdriverIO (WDIO) results through its support for JUnit reports or via the hooks available in WebdriverIO.

WebdriverIO is an automation framework which leverages the power of the Webdriver protocol and is built to automate web and mobile applications. It is an open-source project that is run with open governance and owned by a non-profit entity called OpenJS Foundation. The WDIO runner currently supports Mocha, Jasmine, and Cucumber frameworks and is capable of generating a host of reports including a JUnit report.

There are two ways to integrate AIO Tests with Webdriver IO either via the JUnit report or via AIO REST API calls in afterTest hooks of Webdriver IO. This document provides an overview on :

  • Generating the Junit report from WebdriverIO tests and uploading it to AIO Tests.

  • Using AIO Tests REST APIs to report results and much more, using the WDIO framework hooks.

The demo example is based on the Getting Started project of Webdriver IO.

In this documentation, you’ll understand:

Required WDIO Setup

  1. npm init wdio ./path/to/new/project. On the prompt to select framework, select Mocha.

  2. For reporting results:

    1. For JUnit Report: npm install @wdio/junit-reporter --save-dev

    2. For Reporting Results via AIO Tests API: npm install axios (This can be replaced with any library to make API calls).

Running Your Tests

The sample test generated by Webdriver IO is as follows. One more test has been added to demo an error case.

const LoginPage = require('../pageobjects/login.page'); const SecurePage = require('../pageobjects/secure.page'); describe('My Login application', () => { it('should login with valid credentials', async () => { await LoginPage.open(); await LoginPage.login('tomsmith', 'SuperSecretPassword!'); await expect(SecurePage.flashAlert).toBeExisting(); await expect(SecurePage.flashAlert).toHaveTextContaining( 'You logged into a secure area!'); }); it('should login with invalid credentials : ', async () => { await LoginPage.open(); await LoginPage.login('tomsmith', 'SuperSecretPassword1'); await expect(SecurePage.flashAlert).toBeExisting(); await expect(SecurePage.flashAlert).toHaveTextContaining( 'You logged into a secure area!'); }); });

To trigger the WDIO tests, use:

npx wdio run wdio.conf.js

Or

npm run wdio

Reporting Results via JUnit File

Setting up WDIO JUnit Reporter

The WDIO JUnit reporter requires a configuration in the wdio.conf.js which is the configuration file for Webdriver IO. JUnit reporter along with any other reporter can be setup as follows :

framework: 'mocha', // Test reporter for stdout. // The only one supported by default is 'dot' // see also: https://webdriver.io/docs/dot-reporter reporters: ['spec',['junit', { outputDir: './',outputFileFormat: function(options) { return `wdio-junit-results.xml` }}]],

Running the tests generates the following Junit XML:

<?xml version="1.0" encoding="UTF-8"?> <testsuites> <testsuite name="My Login application" timestamp="2022-07-19T18:15:27" time="16.996" tests="2" failures="1" errors="1" skipped="0"> <properties> <property name="specId" value="0"/> <property name="suiteName" value="My Login application"/> <property name="capabilities" value="chrome.103_0_5060_114.macosx"/> <property name="file" value="./test/specs/example.e2e.js"/> </properties> <testcase classname="chrome.103_0_5060_114.macosx.My_Login_application" name="should login with valid credentials" time="4.504"> <system-out><![CDATA[ COMMAND: POST /session/99dc559d65c3435b3794bd7a9f4bce78/url - {"url":"https://the-internet.herokuapp.com/login"} RESULT: {"url":"https://the-internet.herokuapp.com/login"} COMMAND: POST /session/99dc559d65c3435b3794bd7a9f4bce78/element - {"using":"css selector","value":"#username"} RESULT: {"using":"css selector","value":"#username"} COMMAND: POST /session/99dc559d65c3435b3794bd7a9f4bce78/element/83a67eae-4c8d-47ed-8ecb-7f4b954431ba/clear - {} RESULT: {} ...ctor","value":"#flash"} COMMAND: POST /session/99dc559d65c3435b3794bd7a9f4bce78/element - {"using":"css selector","value":"#flash"} RESULT: {"using":"css selector","value":"#flash"} COMMAND: GET /session/99dc559d65c3435b3794bd7a9f4bce78/element/70db11bc-46c7-4998-a5f8-8741a67f971e/text - {} RESULT: {} ]]></system-out> </testcase> <testcase classname="chrome.103_0_5060_114.macosx.My_Login_application" name="should login with invalid credentials" time="12.491"> <failure/> <error message="Expect $(`#flash`) to have text containing&#xA;&#xA;- Expected - 1&#xA;+ Received + 2&#xA;&#xA;- You logged into a secure area!&#xA;+ Your password is invalid!&#xA;+ ×"/> <system-out><![CDATA[ COMMAND: POST /session/99dc559d65c3435b3794bd7a9f4bce78/url - {"url":"https://the-internet.herokuapp.com/login"} RESULT: {"url":"https://the-internet.herokuapp.com/login"} COMMAND: POST /session/99dc559d65c3435b3794bd7a9f4bce78/element - {"using":"css selector","value":"#username"} RESULT: {"using":"css selector","value":"#username"} COMMAND: POST /session/99dc559d65c3435b3794bd7a9f4bce78/element/d7a12216-7f4e-4b6d-980a-537ed9d801d2/clear - {} RESULT: {} COMMAND: POST /session/99dc559d65c3435b3794bd7a9f4bce78/element/d7a12216-7f4e-4b6d-980a-537ed9d801d2/value - {"text":"tomsmith"} RESULT: {"text":"tomsmith"} ... COMMAND: GET /session/99dc559d65c3435b3794bd7a9f4bce78/element/46471dfb-3bfb-4320-8f7e-006bd9fffeb1/text - {} RESULT: {} COMMAND: GET /session/99dc559d65c3435b3794bd7a9f4bce78/element/46471dfb-3bfb-4320-8f7e-006bd9fffeb1/text - {} RESULT: {} COMMAND: DELETE /session/99dc559d65c3435b3794bd7a9f4bce78 - {} RESULT: {} ]]></system-out> <system-err><![CDATA[ Error: Expect $(`#flash`) to have text containing [32m- Expected - 1[39m [31m+ Received + 2[39m [32m- You[7m logged into a secure area![27m[39m [31m+ You[7mr password is invalid![27m[39m [31m+ ×[39m at Context.<anonymous> (/Users/varshneyn/aiotcms/automation-demo-repos/webdriverio-mocha/test/specs/example.e2e.js:18:45) at processTicksAndRejections (node:internal/process/task_queues:96:5) ]]></system-err> </testcase> </testsuite> </testsuites>

Uploading Results to AIO Tests

Post execution of a suite, the TEST-<xxx>.xml file can be uploaded either via

Please follow the above links to continue to import results using either of the options.

Uploading the above file for the first time will

  1. Create new cases in the system. The new case is created with
    - Title as the name value from <testcase> tag of the JUnit report
    - Automation key as classname.name from the JUnit report.
    - Status as Published
    - Automation status as Automated
    - Automation owner as a user uploading the results

  2. Add the newly created case to the cycle being uploaded.

  3. Mark the details of the run

    1. Execution mode is set to Automated

    2. The duration of A run is set to the Actual Effort

    3. The status of a run is set based on the status mapping table below

    4. Failures and errors are reported as Run Level comments

If the same file is uploaded again, the cases will be identified using the automation key (classname.name )and would be updated, instead of creating new cases.

 

There is no way to map case keys to WDIO cases for the JUnit report since the WDIO JUnit reporter strips off all non-alphanumeric characters while generating the JUnit report.

e.g. If the test name isdescribe('My Login application SCRUM-TC-11'" the Junit report strips of the hyphens and reports it as
<testcase classname="chrome.103_0_5060_114.macosx.My_Login_application" name="should login with valid credentials SCRUM TC 11" time="14.112">

Status Mapping JUnit → AIO Tests

JUnit XML

Description

AIO Tests Mapping

JUnit XML

Description

AIO Tests Mapping

No tag inside <testcase> means Passed

Passed case

Passed

</skipped>

Skipped case either by @Ignore or others

Not Run

</failure>

Indicates that the test failed. A failure is a test which the code has explicitly failed by using the mechanisms for that purpose. e.g., via an assertEquals.

Failed

</error>

Indicates that the test errored. An errored test is one that had an unanticipated problem. e.g., an unchecked throwable;

Failed

Reporting Results via WebdriverIO Mocha Hooks and AIO Tests REST APIs

AIO Tests provides a rich set of APIs for Execution Management, using which users can not only report execution status, but also add effort, actual results, comments, defects and attachments to runs as well as steps.
AIO Tests also provides APIs to create cycles and to add cases to cycles for execution planning.

The basic sample below will show how Mocha Hooks in WebdriverIO can leverage the AIO Tests REST APIs to report results. In the wdio.conf.js, the afterTest method can be used to make an AIO API call.

Mocha aftestTest Method

The afterTest call is defined as below: [ref https://webdriver.io/docs/configurationfile ]


Establish a Convention for AIO Tests Case Keys

For the purpose of the example, we have established a convention to map cases - the AIO Tests case key is the prefix to the case title e.g. it('NVPROJ-TC-11: should login with valid credentials' contains NVPROJ-TC-11, which is the AIO Tests Case key.

Any convention can be established and the code consuming it can cater to the convention. In our case, we are using startsWith to identify the case key.


Reporting Result via API

In the example below, the postResults method uses Axios to make an HTTP call.

  1. It uses the test title to identify the case key [ based on the convention we established]

  2. Create a POST request

    1. URL: For cloud the URL host would be https://tcms.aiojiraapps.com/aio-tcms/api/v1. For the Jira server, it would be the native Jira server hostname.

    2. Authorization: Please refer to Rest API Authentication to understand how to authorize users. The authentication information goes in the headers: {'Authorization': '<Auth based on Jira Cloud/Jira Server>'},

    3. POST Body: The body consists of data from the test object provided by Webdriver IO. If the case has failed, the error is posted as comments.

    4. If required, the basic example can be extended to upload attachments against the case using the upload attachment API.

Running the above in the afterTest method will result in 2 cases being added to cycle NVPROJ-CY-48.

  1. NVPROJ-TC-11: should login with valid credentials - NVPROJ-TC-11 will be added to the cycle with status as passed and duration.

  2. NVPROJ-TC-12: should login with invalid credentials - NVPROJ-TC-12 will be added to the cycle with the status as Failed and comments would hold the error messages.

The above is a basic example of what can be done with the hooks and AIO Tests APIs. It is recommended to add appropriate error handling and enhance it based on your automation requirements.

 

For further queries and suggestions, feel free to reach out to our customer support via help@aiotests.com.