TestNG
- APPWRK Writer
AIO Tests allows easy reporting of TestNG results by supporting importing the testng-results.xml. This article discusses how to import TestNG results and map them to tests existing in AIO Tests.
In this documentation, you’ll understand:
TestNG
TestNG is a Java testing framework inspired by JUnit. TestNG strives to cover a wider range of test categories from unit to integration to end-to-end testing.
It is often, but not solely, combined with Selenium/Webdriver to drive Java-based tests. With its powerful features of data-driven tests, parallel runs, flexible setup/teardown, and inbuilt reporting, it has become one of the most used Java frameworks alongside JUnit.
Sample TestNG Report
TestNG offers an XML reporter capturing TestNG-specific information like groups and data provider data.
Sample TestNG Report of a simple single test. The report gives information on total tests, groups of the tests, their statuses, and duration.
Generating TestNG XML Report
The XMLReporter is enabled by default in TestNG and no extra effort is required.
The report “testng-results.xml” is generated by default in the standard output directory, defaults to “test-output”.
If running through an IDE, check the settings for the TestNG configuration and enable default reporters if not enabled.
Status Mapping TestNG → AIO Tests
TestNG | AIO Tests |
|---|---|
PASS | Passed |
FAIL | Failed |
SKIP | Not Run |
SKIP due to retry | (n-1) runs - Marked as failed [Note: TestNG shows them as skipped] |
Mapping Automated TestNG Tests to AIO Tests
In going with the simplicity advantage of AIO Tests, the TestNG integration has been designed as a simple and non-intrusive integration, without any extra dependency or coding required.
Groups of TestNG should be used to specify the key existing in AIO Tests.
However, there are many variations available in TestNG, which are captured in the XML report.
Below is a list of various scenarios that TestNG offers and a way to capture those in AIO Tests:
1. Normal Standalone Case
Groups can be used to specify the AIO Tests key. In the below example,
“AT-TC-25” has been added as a group, which maps to an existing case in AIO Tests.
public class ForestCreatorTests {
@Test(groups = {"P0","AT-TC-25"}, description = "Verify positive case - all trees get planted")
public void testPlantTree() {
ForestCreator fc = new ForestCreator();
int plantedTrees = fc.plantTrees(10);
Assert.assertEquals(plantedTrees, 10, "Verify all trees get planted");
}
}2. Mapping an automated case to multiple manual tests
Multiple manual case keys (in eg. below: "AT-TC-26", "AT-TC-27", "AT-TC-28") can be specified if an automated case covers more than one scenario. 3 cases in AIO Tests will be updated with the test result of this single test.
@Test(groups = {"P0","AT-TC-26","AT-TC-27","AT-TC-28"},
description = "Verify plant is healthy with correct process")
public void testCreateForest() {
ForestCreator fc = new ForestCreator();
fc.plantTrees(1);
fc.waterThePlants();
fc.removeWeeds();
Assert.assertEquals(fc.getHealthyTreeCount(), 1, "Verify end to end plant growth");
}3. Data Provider driven cases
There can be many variations to data provider cases:
a. Each data in dataprovider is one manual case. TestNG runs the below test 3 times with different data supplied by the Data Provider method. In this case, with each data, the corresponding manual case key needs to be specified eg. Below shows the dataprovider providing AT-TC-29 to 31.
c. Mixed Bag Some data point to the same case, whereas some point to individual manual cases.
In this case, a combination of groups and data provider keys can be used. In the example below, AT-TC-33 has been added to @Test and AT-TC-34 has been added with dataprovider data. In this case, Baobab and Banyan would report results for AT-TC-33, and “Palm” would report results for AT-TC-34.
3.d. Dataprovider case mapping to multiple manual cases
Comma-separated case keys can be provided to map a single data provider-driven case to multiple AIO tests. In the example below, the result of “Baobab” would update two cases - AT-TC-40, and AT-TC-42.
In total, these 3 tests would translate to 4 tests in Cycle executions.
4. Multiple invocation Counts
The same case gets executed multiple times. In the below example, based on the option selected while uploading results, 5 runs can be created for AT-TC-25 or the last result can be captured in one run.
Mapping a Case to a Folder
Users have the option to designate a default folder for newly created cases. They can also define a mapping between a case and a folder by simply adding a group starting with the prefix @AIO-FOLDER- and mentioning the folder hierarchy. Folder hierarchy can be separated using / , for example - @AIO-FOLDER-Reports/Audit log.
This functionality is applicable only:
If a new case is generated through the import process.
If the folder structure specified in the tag already exists.
Additionally, if configured, this will supersede the default folder setting.
Mapping using the default folder option
Mapping each case using tags
Importing Results
Post execution of a TestNG suite, the testng-results.xml file can be uploaded either via
Import Results- REST API - REST API call using multipart form-data to upload file
Import Results- Cycle Tab - Importing results file from the Cycles tab of the AIO Tests app
Please follow the above links to continue to import results using either of the options.
For further queries and suggestions, please feel free to reach out to our customer support service via help@aiotests.com.