Jenkins Plugin

In recent years, Continuous Integration (CI) has become a best practice for software development. Build automation and automated testing form the core of CI.

AIO Tests supports automated testing with AIO public APIs and integration with frameworks like JUnit and TestNG.

The AIO Tests Jenkins plugin provides another easy-to-use interface to report test results right after the automated tests have run, thus helping in CI. The plugin wraps the AIO public APIs to import test results from Cucumber (via cucumber.json), TestNG or JUnit result XML files. The plugin is available for all Cloud, Server and Data Center instances.

In this documentation, you’ll understand:

AIO APIs and Jenkins plugin can be used to publish results from any framework capable of generating JUnit files like Robot/Cypress/NUnit amongst many others.

Installing the Plugin

AIO Tests Jenkins plugin can be installed from the Jenkins marketplace as below.

Login to your Jenkins instance.
Go to Manage Jenkins > Manage Plugins > Available
Now search for AIO Tests and click Install

Requirements

Jenkins 2.235.1 or higher
Jira cloud/server/data centre instance with AIO Tests for Jira installed

Reporting Results

The plugin works in the post build actions step after the build steps are run.
Assuming that the tests have run and a results file (Cucumber.json/Junit/TestNG/any XUnit framework which generates a JUnit XML report) has been generated, the AIO Tests plugin can be used as below to report results of cases in the report to Jira. For information on how to map cases with AIO Tests for different frameworks, please refer to Test Framework Integration.

Go to project -> Configure
Click on the Post Build Actions -> Select Publish Results to AIO Tests

The following options need to be configured for publishing results to AIO Tests
1. Cloud Authentication- Use the API Token generated from Access Token.
2. Server/DataCenter Authentication- Use Jira server/DC username and password with basic authentication
Choose if the results should be published synchronously or in batches.

Specify the Project for which run results need to be published eg. SCRUM
Choose a framework and specify the path of the file - relative to the workspace eg. /target/surefire-reports/testng-results.xml

For frameworks other than JUnit/TestNG/NUnit/Cucumber, if results file is being generated in JUnit format, please select JUnit.

The below table defines the rest of the field options:

	Field	Details

	Field	Details
1	Cycle Preference	Select between existing, new cycle or search for an existing cycle by name and create if not found
2	Cycle Prefix	In case of new cycle, Cycle prefix will be used to prefix the cycle title
3	Cycle Key	If existing cycle is selected, specify AIO Tests cycle key eg. SCRUM-CY-121
4	Cycle Name	Uses the cycle name to search for an existing cycle with an exact match. If cycle is found, then the cycle is updated. If it is not found, then a new cycle is created.
5	Add Case	If checked, cases not already in cycle are added, else the cases are reported as errors
6	Create New Run	If checked, new run is created for each case execution, else existing run is overwritten
7	Create Case	If checked and no case exists with a matching case key or automation key, a new case is created
8	Update BDD Case Steps [Cucumber Only]	If checked, if the automated case steps are different from the steps in AIO Tests, then the automated steps will update the steps in the AIO Tests cases
9	Force Update Case	If checked, updates case steps from steps in automaton results files. (Works for robot only)
10	Hide Publish Result Details	If checked, it would hide testcase wise details
11	Fail build if result publishing fails	If checked, the build result will be updated to failed, in case publishing of results to AIO Tests fails

Reporting Results in the Pipeline

The AIO Tests Jenkins plugin has support for being run as a pipeline task.
For the security of API Key/Jira Password, secret text credentials can be used to avoid putting the key in plain text. A plain text API Key would work as well.

Cloud Example

pipeline {
       agent any
       environment {
           AIO_TESTS_API_KEY = credentials('aioTestApiKey')
       }
       stages {
           stage('SCM') {
               steps {
                   git 'https://github.com/org/sample-testng-tests.git'                       
               }
           }
           stage('Build') {
               steps {
                   sh 'mvn test'
               }
           }
       }
       post {
           always {
                   aioImport jiraInstanceType: 'cloud',
                         frameworkType : 'TestNG',
                         addCaseToCycle :true,
                         createCase :true,
                         createNewRun: true,
                         bddForceUpdateCase: true,
                         entry: [$class: 'NewCycle', cyclePrefix: 'Regression Run V1.0'],
                        //For existing cycles : entry: [$class: 'ExistingCycle', cycleKey: 'SCRUM-CY-191'],
                         apiKey : hudson.util.Secret.fromString(env.AIO_TESTS_API_KEY),
                         resultsFilePath : '/target/surefire-reports/testng-results.xml',
                         projectKey: 'SCRUM',
                         hideDetails: false
           }
       }
   }

Server/Data Center Example

pipeline {
       agent any
       environment {
           AIO_TESTS_PWD = credentials('JIRA_PWD')
       }
       stages {
           stage('SCM') {
               steps {
                   git url:'https://github.com/yourco/testng-report-samples.git',
                       branch: 'main'
               }
           }
           stage('Build') {
               steps {
                   sh 'mvn test'}
           }
       }
       post {
           always {
                   aioImport jiraInstanceType: 'server',
                         jiraPassword: hudson.util.Secret.fromString(env.AIO_TESTS_PWD),
                         jiraServerUrl: '<your jira server/DC url>',
                         jiraUsername: '<jira username>',
                         frameworkType : 'TestNG',
                         addCaseToCycle :true,
                         createNewRun: true,
                         createCase :true,
                         bddForceUpdateCase: true,
                         apiKey: null, //mandatory to set null 
                         entry: [$class: 'NewCycle', cyclePrefix: 'Regression Run V1.0'],
                        //For existing cycles : entry: [$class: 'ExistingCycle', cycleKey: 'SCRUM-CY-191'],
                         resultsFilePath : '/src/testng-results.xml',
                         projectKey: 'NV',
                         hideDetails: false
               
           }
       }
   }

Parameters

If the job is a parameterized build, the plugin supports parametrization for the project key, results file path, cycle prefix, existing cycle key, and API key fields.

Trouble seeing AIO Tests Plugin Options?

Jenkins version 2.263.x version had an issue with elements related to radio blocks being hidden [Jenkins Issue- 64040, issue 64136]. Upgrading to the latest version resolves this issue.

For further queries and suggestions, please feel free to reach out to our customer support service via help@aiotests.com.