Our documentation site has been updated to AIO Tests Knowledge Base
TestNG
- Noopur Varshney [AIO Tests]
- Samyak Awale
AIO Tests allows easy reporting of TestNG results by supporting importing the testng-results.xml.
This article discusses how to import TestNG results and map it to tests existing in AIO Tests.
TestNG
TestNG is a Java testing framework inspired by JUnit. TestNG strives to cover a wider range of test categories from unit to integration to end-to-end testing.
It is often, but not solely, combined with Selenium/Webdriver to drive Java based tests. With its powerful features of data driven test, parallel runs, flexible setup/teardown and inbuilt reporting, it has become one of the most used Java frameworks alongside JUnit.
Sample TestNG Report
TestNG offers an XML reporter capturing TestNG specific information like groups and data provider data.
Sample TestNG Report of a simple single test. The report gives information of total tests, groups of the tests, their statuses and duration.
Generating TestNG XML Report
The XMLReporter is enabled by default in TestNG and no extra effort is required.
The report “testng-results.xml” is generated by default in the standard output directory, defaults to “test-output”.
If running through an IDE, check the settings for the TestNG configuration and enable default reporters if not enabled.
Status Mapping TestNG → AIO Tests
TestNG | AIO Tests |
|---|---|
PASS | Passed |
FAIL | Failed |
SKIP | Not Run |
SKIP due to retry | (n-1) runs - Marked as failed [Note: TestNG shows them as skipped] |
Mapping automated TestNG tests to AIO Tests
In going with the simplicity advantage of AIO Tests, the TestNG integration has been designed as a simple and non-intrusive integration, without any extra dependency or coding required.
Groups of TestNG should be used to specify the key existing in AIO Tests.
However, there are many variations available in TestNG, which are captured in the xml report.
Below is a list of various scenarios that TestNG offers and a way to capture those in AIO Tests
1. Normal standalone case
Groups can be used to specify the AIO Tests key. In below example,
“AT-TC-25” has been added as a group, which maps to an existing case in AIO Tests.
public class ForestCreatorTests {
@Test(groups = {"P0","AT-TC-25"}, description = "Verify positive case - all trees get planted")
public void testPlantTree() {
ForestCreator fc = new ForestCreator();
int plantedTrees = fc.plantTrees(10);
Assert.assertEquals(plantedTrees, 10, "Verify all trees get planted");
}
}2. Mapping an automated case to multiple manual tests
Multiple manual case keys (in eg. below : "AT-TC-26","AT-TC-27","AT-TC-28") can be specified if an automated case covers more than one scenario. 3 cases in AIO Tests will be updated with the test result of this single test.
@Test(groups = {"P0","AT-TC-26","AT-TC-27","AT-TC-28"},
description = "Verify plant is healthy with correct process")
public void testCreateForest() {
ForestCreator fc = new ForestCreator();
fc.plantTrees(1);
fc.waterThePlants();
fc.removeWeeds();
Assert.assertEquals(fc.getHealthyTreeCount(), 1, "Verify end to end plant growth");
}3. Data Provider driven cases
There can be many variations to data provider cases:
3.a. Each data in dataprovider is one manual case. TestNG runs the below test 3 times with different data supplied by the Data Provider method. In this case, with each data, the corresponding manual case key needs to be specified eg. below shows dataprovider providing AT-TC-29 to 31.
public class ForestCreatorTests {
@DataProvider(name = "saplingSupplier")
public Object[][] testDataProvider() {
return new Object[][] {
{ "Baobab", "AT-TC-29"},
{ "Palm" , "AT-TC-30"},
{ "Banyan", "AT-TC-31"},
};
}
@Test(dataProvider = "saplingSupplier")
public void testSpecificTreePlantation(String treeName, String aioCaseKey) {
//Your test goes here
ForestCreator fc = new ForestCreator();
fc.plantSpecificTree(treeName);
Assert.assertTrue(true, "Tree successfully planted");
}
}3.b. All data point to same manual case. In this case, the AIO case key need not be sent with data, but can be added similar to normal cases in groups. In example below, AT-TC-33 has been added to @Test
AIO APIs expose a parameter, while uploading results, to specify whether multiple results of a case in a results.xml should be captured as separate runs of the same case or just capture the last run’s result. To know more, jump to Importing Results.
public class ForestCreatorTests {
@DataProvider(name = "saplingSupplier")
public Object[][] testDataProvider() {
return new Object[][] {
{ "Baobab"},
{ "Palm" },
{ "Banyan"},
};
}
@Test(dataProvider = "saplingSupplier", groups = "AT-TC-33")
public void testSpecificTreePlantation(String treeName) {
//Your test goes here
ForestCreator fc = new ForestCreator();
fc.plantSpecificTree(treeName);
Assert.assertTrue(true, "Tree successfully planted");
}
}3.c. Mixed Bag Some data point to the same case, whereas some point to individual manual cases.
In this case, a combination of groups and data provider keys can be used. In example below, AT-TC-33 has been added to @Test and AT-TC-34 has been added with dataprovider data. In this case, Baobab and Banyan would report results for AT-TC-33 and “Palm” would report result to AT-TC-34
public class ForestCreatorTests {
@DataProvider(name = "saplingSupplier")
public Object[][] testDataProvider() {
return new Object[][] {
{ "Baobab", ""},
{ "Palm", "AT-TC-34" },
{ "Banyan",""},
};
}
@Test(dataProvider = "saplingSupplier", groups = "AT-TC-33")
public void testSpecificTreePlantation(String treeName, String aioCaseKey ) {
//Your test goes here
ForestCreator fc = new ForestCreator();
fc.plantSpecificTree(treeName);
Assert.assertTrue(true, "Tree successfully planted");
}
}3.d. Dataprovider case mapping to multiple manual case
Comma separated case keys can be provided to map a single data provider driven case to multiple AIO tests. In example below, the result of “Baobab” would update two cases - AT-TC-40, AT-TC-42.
In total, these 3 tests would translate to 4 tests in Cycle executions.
public class ForestCreatorTests {
@DataProvider(name = "saplingSupplier")
public Object[][] testDataProvider() {
return new Object[][] {
{ "Baobab", "AT-TC-40, AT-TC-42"},
{ "Palm", "AT-TC-34" },
{ "Banyan",""},
};
}
@Test(dataProvider = "saplingSupplier", groups = "AT-TC-33")
public void testSpecificTreePlantation(String treeName, String aioCaseKey ) {
//Your test goes here
ForestCreator fc = new ForestCreator();
fc.plantSpecificTree(treeName);
Assert.assertTrue(true, "Tree successfully planted");
}
}4. Multiple invocation Counts
Same case gets executed multiple times. In below example, based on the option selected while uploading results, 5 runs can be created for AT-TC-25 or the last result can be captured in one run.
@Test(groups = {"P2","AT-TC-25"}, invocationCount = 5, description = "Verify no
random failures while planting trees")
public void testPlantTreeSeq() {
ForestCreator fc = new ForestCreator();
int plantedTrees = fc.plantTrees(1);
Assert.assertEquals(plantedTrees, 1, "Verify tree got planted successfully");
}Mapping a case to a folder
Users have the option to designate a default folder for newly created cases. They can also define a mapping between a case and a folder by simply adding a group starting with the prefix @AIO-FOLDER- and mentioning the folder hierarchy. Folder hierarchy can be separated using / , for example - @AIO-FOLDER-Reports/Audit log
Note- This functionality is applicable only:
if a new case is generated through the import process
if the folder structure specified in the tag already exists.
Additionally, if configured, this will supersede the default folder setting.
Mapping using the default folder option
Mapping each case using tags
@Test(groups = "@AIO-FOLDER-Imports/TestNg")
public void testGoogle() {
DriverManager.get().get("https://www.google.com");
SoftAsserts.getSoftAssert().assertEquals(2,2, "Verify that you are on google");
SoftAsserts.getSoftAssert().assertEquals(true,false, "Verify that there is a search button");
}
Importing Results
Post execution of a TestNG suite, the testng-results.xml file can be uploaded either via
Import Results - REST API - REST API call using multipart form-data to upload file
Import Results- Cycle Tab - Importing results file from Cycles tab of AIO Tests app
Please follow the above links to continue to import results using either of the options.