...
Generating the Junit report from Playwright tests and uploading it to AIO Tests.
Generating Cucumber reports with Playwright + Cucumber and uploading it in AIO Tests
Using AIO Tests REST APIs to report results and much more, using the Playwright framework hooks.
Table of Contents |
---|
Playwright + JUnit
The demo example is based on the Getting Started project of Playwright.
Required Playwright Setup
npm init playwright@latest
. On prompt, select the tests folderFor reporting results
For JUnit report :
PLAYWRIGHT_JUNIT_OUTPUT_NAME=results.xml npx playwright test --reporter=junit
For reporting results via AIO Tests API :
npm install axios
(This can be replaced with any library to make API calls)
Mapping and Running your tests
The sample test generated by Playwright is as follows.
...
To trigger the Playwright tests, use:
npx playwright test
Reporting results via JUnit file
Using the native JUnit reporter
Playwright Test comes with a few built-in reporters for different needs and ability to provide custom reporters. The easiest way to try out built-in reporters is to pass --reporter
.
...
Expand | ||
---|---|---|
| ||
|
Uploading results to AIO Tests
Post execution of a suite, the TEST-<xxx>.xml file can be uploaded either via
...
If the same file is uploaded again, the cases will be identified using the automation key (
classname.name )
and would be updated, instead of creating new cases.
Status Mapping JUnit → AIO Tests
JUnit XML | Description | AIO Tests Mapping |
---|---|---|
No tag inside <testcase> means Passed | Passed case | Passed |
</skipped> | Skipped case either by @Ignore or others | Not Run |
</failure> |
| Failed |
</error> |
| Failed |
Reporting results via Playwright Hooks and AIO Tests REST APIs
AIO Tests provides a rich set of APIs for Execution Management, using which users can not only report execution status, but also add effort, actual results, comments, defects and attachments to runs as well as steps.
AIO Tests also provides APIs to create cycles and to add cases to cycles for execution planning.
The basic sample below will show how Mocha Hooks in WebdriverIO can leverage the AIO Tests REST APIs to report results. In the wdio.conf.js, the afterTest
method can be used to make AIO API call.
Mocha aftestTest method
The afterTest call is defined as below: [ref https://webdriver.io/docs/configurationfile ]
Code Block |
---|
/** * Function to be executed after a test (in Mocha/Jasmine only) * @param {Object} test test object * @param {Object} context scope object the test was executed with * @param {Error} result.error error object in case the test fails, otherwise `undefined` * @param {Any} result.result return object of test function * @param {Number} result.duration duration of test * @param {Boolean} result.passed true if test has passed, otherwise false * @param {Object} result.retries informations to spec related retries, e.g. `{ attempts: 0, limit: 0 }` */ afterTest: function (test, context, { error, result, duration, passed, retries }) { }, |
Establish a convention for AIO Tests Case keys
For the purpose of the example, we have established a convention to map cases - the AIO Tests case key is the prefix to the case title eg. it('NVPROJ-TC-11: should login with valid credentials'
contains NVPROJ-TC-11, which is the AIO Tests Case key.
...
Code Block |
---|
describe('My Login application', () => { it('NVPROJ-TC-11: should login with valid credentials', async () => { await LoginPage.open(); await LoginPage.login('tomsmith', 'SuperSecretPassword!'); await expect(SecurePage.flashAlert).toBeExisting(); await expect(SecurePage.flashAlert).toHaveTextContaining( 'You logged into a secure area!'); }); it('NVPROJ-TC-12: should login with invalid credentials : ', async () => { await LoginPage.open(); await LoginPage.login('tomsmith', 'SuperSecretPassword1'); await expect(SecurePage.flashAlert).toBeExisting(); await expect(SecurePage.flashAlert).toHaveTextContaining( 'You logged into a secure area!'); }); }); |
Reporting result via API
Playwright provides a way to develop custom reporters.
...
Info |
---|
The above is a basic example of what can be done with the hooks and AIO Tests APIs. It is recommended to add appropriate error handling and enhance it based on your automation requirements. |
Playwright + Cucumber Setup
npm init playwright@latest
. On prompt, select the tests foldernpm install @cucumber/cucumber
.
Mapping and Running your tests
Cucumber generates a cucumber.json report which can be directly imported in AIO Tests.
...
Code Block | ||
---|---|---|
| ||
[ { "description": "", "elements": [ { "description": "", "id": "minimal-scenario;justified-reasoning", "keyword": "Scenario", "line": 5, "name": "Justified reasoning", "steps": [ { "arguments": [], "keyword": "Given ", "line": 6, "name": "If there is \"here\" and \"there\"", "match": { "location": "features/support/steps.js:9" }, "result": { "status": "passed", "duration": 1546712 } }, { "arguments": [], "keyword": "Then ", "line": 7, "name": "I attain nirvana", "match": { "location": "features/support/steps.js:14" }, "result": { "status": "passed", "duration": 154070 } } ], "tags": [ { "name": "@P0", "line": 1 }, { "name": "@NVTES-TC-33", "line": 4 } ], "type": "scenario" } ], "id": "minimal-scenario", "line": 2, "keyword": "Feature", "name": "Minimal Scenario", "tags": [ { "name": "@P0", "line": 1 } ], "uri": "features/AIODemo.feature" } ] |
Uploading results to AIO Tests
Post execution of a suite, the cucumber.json file can be uploaded either via
...
If the same file is uploaded again, the cases will be identified using the automation key (
scenario.id)
and would be updated, instead of creating new cases.
Reporting results via Cucumber Hooks and AIO Tests REST APIs
AIO Tests provides a rich set of APIs for Execution Management, using which users can not only report execution status, but also add effort, actual results, comments, defects and attachments to runs as well as steps.
AIO Tests also provides APIs to create cycles and to add cases to cycles for execution planning.
...