{"type":"doc","content":[{"type":"paragraph","content":[{"text":"AIO Tests allows easy reporting of TestNG results by supporting importing the testng-results.xml.","type":"text"}]},{"type":"paragraph","content":[{"text":"This article discusses how to import TestNG results and map it to tests existing in AIO Tests.","type":"text"},{"type":"hardBreak"}]},{"type":"extension","attrs":{"layout":"default","extensionType":"com.atlassian.confluence.macro.core","extensionKey":"toc","parameters":{"macroParams":{},"macroMetadata":{"macroId":{"value":"24f6fef5-25a7-4435-be28-edadf8d25687"},"schemaVersion":{"value":"1"},"title":"Table of Contents"}},"localId":"135dca2c-9dc6-482d-92db-8a64b41cbe00"}},{"type":"heading","attrs":{"level":2},"content":[{"text":"TestNG","type":"text","marks":[{"type":"strong"}]},{"text":" ","type":"text"}]},{"type":"paragraph","content":[{"text":"TestNG is a Java testing framework inspired by JUnit. TestNG strives to cover a wider range of test categories from unit to integration to end-to-end testing.","type":"text"}]},{"type":"paragraph","content":[{"text":"It is often, but not solely, combined with Selenium/Webdriver to drive Java based tests. With its powerful features of data driven test, parallel runs, flexible setup/teardown and inbuilt reporting, it has become one of the most used Java frameworks alongside JUnit.","type":"text"}]},{"type":"heading","attrs":{"level":3},"content":[{"text":"Sample TestNG Report","type":"text","marks":[{"type":"strong"}]}]},{"type":"paragraph","content":[{"text":"TestNG offers an XML reporter capturing TestNG specific information like groups and data provider data.","type":"text"}]},{"type":"paragraph","content":[{"text":"Sample TestNG Report of a simple single test. The report gives information of total tests, groups of the tests, their statuses and duration.","type":"text"}]},{"type":"expand","attrs":{"title":"Sample testng-results.xml"},"content":[{"type":"codeBlock","attrs":{"language":"xml"},"content":[{"text":"\n\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n","type":"text"}]}]},{"type":"heading","attrs":{"level":3},"content":[{"text":"Generating TestNG XML Report","type":"text"}]},{"type":"paragraph","content":[{"text":"The XMLReporter is enabled by default in TestNG and no extra effort is required.","type":"text"}]},{"type":"paragraph","content":[{"text":"The report ","type":"text"},{"text":"“testng-results.xml”","type":"text","marks":[{"type":"strong"},{"type":"em"}]},{"text":" is generated by default in the standard output directory, ","type":"text"},{"text":"defaults to “test-output”.","type":"text","marks":[{"type":"strong"},{"type":"em"}]}]},{"type":"paragraph","content":[{"text":"If running through an IDE, check the settings for the TestNG configuration and enable default reporters if not enabled.","type":"text"}]},{"type":"paragraph","content":[{"text":" ","type":"text"}]},{"type":"expand","attrs":{"title":"IntelliJ example to enable default reporters"},"content":[{"type":"mediaSingle","attrs":{"layout":"align-start","width":65.0},"content":[{"type":"media","attrs":{"width":563,"alt":"IntelliJ settings","id":"d36dd46b-4ab6-48cd-9fe7-2be16ce0ddfa","collection":"contentId-1900879485","type":"file","height":494}}]}]},{"type":"paragraph","content":[{"text":" ","type":"text"}]},{"type":"heading","attrs":{"level":2},"content":[{"text":"Status Mapping TestNG → AIO Tests","type":"text"}]},{"type":"table","attrs":{"layout":"default","width":1800.0,"localId":"74b114a8-29c7-45d4-9529-9226841f935d"},"content":[{"type":"tableRow","content":[{"type":"tableHeader","attrs":{"colspan":1,"rowspan":1,"colwidth":[182.0]},"content":[{"type":"paragraph","content":[{"text":"TestNG","type":"text","marks":[{"type":"strong"}]}]}]},{"type":"tableHeader","attrs":{"colspan":1,"rowspan":1,"colwidth":[578.0]},"content":[{"type":"paragraph","content":[{"text":"AIO Tests","type":"text","marks":[{"type":"strong"}]}]}]}]},{"type":"tableRow","content":[{"type":"tableCell","attrs":{"colspan":1,"rowspan":1,"colwidth":[182.0]},"content":[{"type":"paragraph","content":[{"text":"PASS","type":"text"}]}]},{"type":"tableCell","attrs":{"colspan":1,"rowspan":1,"colwidth":[578.0]},"content":[{"type":"paragraph","content":[{"text":"Passed","type":"text"}]}]}]},{"type":"tableRow","content":[{"type":"tableCell","attrs":{"colspan":1,"rowspan":1,"colwidth":[182.0]},"content":[{"type":"paragraph","content":[{"text":"FAIL","type":"text"}]}]},{"type":"tableCell","attrs":{"colspan":1,"rowspan":1,"colwidth":[578.0]},"content":[{"type":"paragraph","content":[{"text":"Failed","type":"text"}]}]}]},{"type":"tableRow","content":[{"type":"tableCell","attrs":{"colspan":1,"rowspan":1,"colwidth":[182.0]},"content":[{"type":"paragraph","content":[{"text":"SKIP","type":"text"}]}]},{"type":"tableCell","attrs":{"colspan":1,"rowspan":1,"colwidth":[578.0]},"content":[{"type":"paragraph","content":[{"text":"Not Run","type":"text"}]}]}]},{"type":"tableRow","content":[{"type":"tableCell","attrs":{"colspan":1,"rowspan":1,"colwidth":[182.0]},"content":[{"type":"paragraph","content":[{"text":"SKIP due to retry","type":"text"}]}]},{"type":"tableCell","attrs":{"colspan":1,"rowspan":1,"colwidth":[578.0]},"content":[{"type":"paragraph","content":[{"text":"(n-1) runs - Marked as failed [Note: TestNG shows them as skipped]","type":"text"},{"type":"hardBreak"},{"text":"n","type":"text"},{"text":"th","type":"text","marks":[{"type":"subsup","attrs":{"type":"sup"}}]},{"text":" run - Status based on above 3 statuses.","type":"text"}]}]}]}]},{"type":"heading","attrs":{"level":2},"content":[{"text":"Mapping automated TestNG tests to AIO Tests","type":"text"}]},{"type":"paragraph","content":[{"text":"In going with the simplicity advantage of AIO Tests, the TestNG integration has been designed as a simple and non-intrusive integration, without any extra dependency or coding required.","type":"text"}]},{"type":"mediaSingle","attrs":{"layout":"center"},"content":[{"type":"media","attrs":{"width":820,"id":"ebfdffd0-e0f4-45eb-9373-731fb60e24a3","collection":"contentId-1900879485","type":"file","height":116}}]},{"type":"paragraph","content":[{"text":"Groups of TestNG should be used to specify the key existing in AIO Tests.","type":"text","marks":[{"type":"strong"},{"type":"em"}]},{"type":"hardBreak"},{"text":"However, there are many variations available in TestNG, which are captured in the xml report.","type":"text"}]},{"type":"paragraph","content":[{"text":"Below is a list of various scenarios that TestNG offers and a way to capture those in AIO Tests","type":"text"}]},{"type":"heading","attrs":{"level":4},"content":[{"text":"1. Normal standalone case","type":"text","marks":[{"type":"strong"},{"type":"em"}]}]},{"type":"paragraph","content":[{"text":"Groups can be used to specify the AIO Tests key. In below example, ","type":"text"},{"type":"hardBreak"},{"text":"“AT-TC-25”","type":"text","marks":[{"type":"strong"},{"type":"em"}]},{"text":" has been added as a group, which maps to an existing case in AIO Tests.","type":"text"}]},{"type":"codeBlock","content":[{"text":"public class ForestCreatorTests {\n \n @Test(groups = {\"P0\",\"AT-TC-25\"}, description = \"Verify positive case - all trees get planted\")\n public void testPlantTree() {\n ForestCreator fc = new ForestCreator();\n int plantedTrees = fc.plantTrees(10);\n Assert.assertEquals(plantedTrees, 10, \"Verify all trees get planted\");\n }\n}","type":"text"}]},{"type":"heading","attrs":{"level":4},"content":[{"text":"2. Mapping an automated case to multiple manual tests","type":"text","marks":[{"type":"strong"},{"type":"em"}]}]},{"type":"paragraph","content":[{"text":"Multiple manual case keys (in eg. below : ","type":"text"},{"text":"\"AT-TC-26\",\"AT-TC-27\",\"AT-TC-28\"","type":"text","marks":[{"type":"code"}]},{"text":") can be specified if an automated case covers more than one scenario. 3 cases in AIO Tests will be updated with the test result of this single test.","type":"text"}]},{"type":"codeBlock","content":[{"text":" @Test(groups = {\"P0\",\"AT-TC-26\",\"AT-TC-27\",\"AT-TC-28\"}, \n description = \"Verify plant is healthy with correct process\")\n public void testCreateForest() {\n ForestCreator fc = new ForestCreator();\n fc.plantTrees(1);\n fc.waterThePlants();\n fc.removeWeeds();\n Assert.assertEquals(fc.getHealthyTreeCount(), 1, \"Verify end to end plant growth\");\n }","type":"text"}]},{"type":"heading","attrs":{"level":4},"content":[{"text":"3. Data Provider driven cases","type":"text","marks":[{"type":"strong"},{"type":"em"}]}]},{"type":"paragraph","content":[{"text":"There can be many variations to data provider cases:","type":"text"}]},{"type":"paragraph","content":[{"text":"3.a. ","type":"text"},{"text":"Each data in dataprovider is one manual case","type":"text","marks":[{"type":"strong"}]},{"text":". TestNG runs the below test 3 times with different data supplied by the Data Provider method. In this case, with each data, the corresponding manual case key needs to be specified eg. below shows dataprovider providing AT-TC-29 to 31.","type":"text"}]},{"type":"codeBlock","content":[{"text":"public class ForestCreatorTests {\n\n @DataProvider(name = \"saplingSupplier\")\n public Object[][] testDataProvider() {\n return new Object[][] {\n { \"Baobab\", \"AT-TC-29\"},\n { \"Palm\" , \"AT-TC-30\"},\n { \"Banyan\", \"AT-TC-31\"},\n };\n }\n\n @Test(dataProvider = \"saplingSupplier\")\n public void testSpecificTreePlantation(String treeName, String aioCaseKey) {\n //Your test goes here\n ForestCreator fc = new ForestCreator();\n fc.plantSpecificTree(treeName);\n Assert.assertTrue(true, \"Tree successfully planted\");\n }\n}","type":"text"}]},{"type":"paragraph","content":[{"text":"3.b. ","type":"text"},{"text":"All data point to same manual case","type":"text","marks":[{"type":"strong"}]},{"text":". In this case, the AIO case key need not be sent with data, but can be added similar to normal cases in groups. In example below, AT-TC-33 has been added to @Test","type":"text"}]},{"type":"panel","attrs":{"panelType":"info"},"content":[{"type":"paragraph","content":[{"text":"AIO APIs expose a parameter, while uploading results, to specify whether multiple results of a case in a results.xml should be captured as separate runs of the same case or just capture the last run’s result. To know more, jump to Importing Results.","type":"text"}]}]},{"type":"codeBlock","content":[{"text":"public class ForestCreatorTests {\n\n @DataProvider(name = \"saplingSupplier\")\n public Object[][] testDataProvider() {\n return new Object[][] {\n { \"Baobab\"},\n { \"Palm\" },\n { \"Banyan\"},\n };\n }\n\n @Test(dataProvider = \"saplingSupplier\", groups = \"AT-TC-33\")\n public void testSpecificTreePlantation(String treeName) {\n //Your test goes here\n ForestCreator fc = new ForestCreator();\n fc.plantSpecificTree(treeName);\n Assert.assertTrue(true, \"Tree successfully planted\");\n }\n}","type":"text"}]},{"type":"paragraph","content":[{"text":"3.c. ","type":"text"},{"text":"Mixed Bag","type":"text","marks":[{"type":"strong"}]},{"text":" Some data point to the same case, whereas some point to individual manual cases.","type":"text"}]},{"type":"paragraph","content":[{"text":"In this case, a combination of groups and data provider keys can be used. In example below, AT-TC-33 has been added to @Test and AT-TC-34 has been added with dataprovider data. In this case, Baobab and Banyan would report results for AT-TC-33 and “Palm” would report result to AT-TC-34","type":"text"}]},{"type":"codeBlock","content":[{"text":"public class ForestCreatorTests {\n\n @DataProvider(name = \"saplingSupplier\")\n public Object[][] testDataProvider() {\n return new Object[][] {\n { \"Baobab\", \"\"},\n { \"Palm\", \"AT-TC-34\" },\n { \"Banyan\",\"\"},\n };\n }\n\n @Test(dataProvider = \"saplingSupplier\", groups = \"AT-TC-33\")\n public void testSpecificTreePlantation(String treeName, String aioCaseKey ) {\n //Your test goes here\n ForestCreator fc = new ForestCreator();\n fc.plantSpecificTree(treeName);\n Assert.assertTrue(true, \"Tree successfully planted\");\n }\n}","type":"text"}]},{"type":"paragraph","content":[{"text":"3.d. ","type":"text"},{"text":"Dataprovider case mapping to multiple manual case","type":"text","marks":[{"type":"strong"}]}]},{"type":"paragraph","content":[{"text":"Comma separated case keys can be provided to map a single data provider driven case to multiple AIO tests. In example below, the result of “Baobab” would update two cases - AT-TC-40, AT-TC-42.","type":"text"}]},{"type":"blockquote","content":[{"type":"paragraph","content":[{"text":"In total, these 3 tests would translate to 4 tests in Cycle executions.","type":"text"}]}]},{"type":"codeBlock","content":[{"text":"public class ForestCreatorTests {\n\n @DataProvider(name = \"saplingSupplier\")\n public Object[][] testDataProvider() {\n return new Object[][] {\n { \"Baobab\", \"AT-TC-40, AT-TC-42\"},\n { \"Palm\", \"AT-TC-34\" },\n { \"Banyan\",\"\"},\n };\n }\n\n @Test(dataProvider = \"saplingSupplier\", groups = \"AT-TC-33\")\n public void testSpecificTreePlantation(String treeName, String aioCaseKey ) {\n //Your test goes here\n ForestCreator fc = new ForestCreator();\n fc.plantSpecificTree(treeName);\n Assert.assertTrue(true, \"Tree successfully planted\");\n }\n}","type":"text"}]},{"type":"heading","attrs":{"level":4},"content":[{"text":"4. Multiple invocation Counts","type":"text","marks":[{"type":"strong"},{"type":"em"}]}]},{"type":"paragraph","content":[{"text":"Same case gets executed multiple times. In below example, based on the option selected while uploading results, 5 runs can be created for AT-TC-25 or the last result can be captured in one run.","type":"text"}]},{"type":"codeBlock","content":[{"text":"@Test(groups = {\"P2\",\"AT-TC-25\"}, invocationCount = 5, description = \"Verify no \n random failures while planting trees\")\npublic void testPlantTreeSeq() {\n ForestCreator fc = new ForestCreator();\n int plantedTrees = fc.plantTrees(1);\n Assert.assertEquals(plantedTrees, 1, \"Verify tree got planted successfully\");\n}","type":"text"}]},{"type":"heading","attrs":{"level":2},"content":[{"text":"Mapping a case to a folder","type":"text"}]},{"type":"paragraph","content":[{"text":"Users have the option to designate a default folder for newly created cases. They can also define a mapping between a case and a folder by simply adding a group starting with the prefix ","type":"text"},{"text":"@AIO-FOLDER-","type":"text","marks":[{"type":"code"}]},{"text":" and mentioning the folder hierarchy. Folder hierarchy can be separated using ","type":"text"},{"text":"/","type":"text","marks":[{"type":"code"}]},{"text":" , for example - @AIO-FOLDER-Reports/Audit log","type":"text"},{"type":"hardBreak"},{"text":"Note- This functionality is applicable only:","type":"text"}]},{"type":"bulletList","content":[{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"if a new case is generated through the import process","type":"text"}]}]},{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"if the folder structure specified in the tag already exists. ","type":"text"}]},{"type":"paragraph","content":[{"text":"Additionally, if configured, this will supersede the default folder setting.","type":"text"}]}]}]},{"type":"orderedList","attrs":{"order":1},"content":[{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"Mapping using the default folder option","type":"text","marks":[{"type":"strong"}]}]}]}]},{"type":"mediaSingle","attrs":{"layout":"align-start","width":273,"widthType":"pixel"},"content":[{"type":"media","attrs":{"width":517,"alt":"image-20240214-054843.png","id":"e5150385-eb1e-49ac-9c1d-e3df6338672f","collection":"contentId-1900879485","type":"file","height":646}}]},{"type":"orderedList","attrs":{"order":2},"content":[{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"Mapping each case using tags","type":"text","marks":[{"type":"strong"}]}]}]}]},{"type":"codeBlock","content":[{"text":"@Test(groups = \"@AIO-FOLDER-Imports/TestNg\")\n public void testGoogle() {\n DriverManager.get().get(\"https://www.google.com\");\n SoftAsserts.getSoftAssert().assertEquals(2,2, \"Verify that you are on google\");\n SoftAsserts.getSoftAssert().assertEquals(true,false, \"Verify that there is a search button\");\n }","type":"text"}]},{"type":"paragraph"},{"type":"heading","attrs":{"level":2},"content":[{"text":"Importing Results","type":"text"}]},{"type":"paragraph","content":[{"text":"Post execution of a TestNG suite, the testng-results.xml file can be uploaded either via","type":"text"}]},{"type":"bulletList","content":[{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"Import Results - REST API","type":"text","marks":[{"type":"link","attrs":{"href":"/wiki/spaces/NAT/pages/1900880129"}}]},{"text":" - REST API call using multipart form-data to upload file","type":"text"}]}]},{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"Import Results- Cycle Tab","type":"text","marks":[{"type":"link","attrs":{"href":"/wiki/spaces/NAT/pages/1900879982"}}]},{"text":" - Importing results file from Cycles tab of AIO Tests app ","type":"text"}]}]}]},{"type":"paragraph","content":[{"text":"Please follow the above links to continue to import results using either of the options.","type":"text"}]}],"version":1}

Browser not supported