Import Results - Automated upload using REST API

AIO Tests supports importing of the automation test execution reports through the Import Results API. This article covers the options available while importing results.

After mapping automated tests to AIO tests and executing the test suite, the results file is generated.
This results file can then be imported by passing the file as part of a multipart/form-data request.

Try it out:
AIO Tests Swagger

In this documentation, you’ll understand:

1 API Details
- 1.1 Overview
- 1.2 Key Capabilities
- 1.3 How It Works
- 1.4 Typical CI/CD Workflow
- 1.5 TestNG
- 1.6 JUnit
- 1.7 NUnit
- 1.8 Cucumber
- 1.9 Robot
2 Explanation of Parameters
3 Explanation of Response
4 Reporting results as a Batch
- 4.1 Batch Processing Behavior
- 4.2 Monitoring Batch Status
- 4.3 When to Use This API
5 Known Errors List (as part of errors.msg above):
6 Response Errors

API Details

Endpoint

POST /project/{projectKey}/testcycle/{cycleKey}/import/results

Overview

The Import Results API allows automation frameworks to upload execution results in bulk to a test cycle in AIO Tests. Instead of updating runs individually, this API processes a test report file (such as JUnit or TestNG XML) and automatically maps the results to the corresponding test cases in AIO.

This API is typically used in CI/CD pipelines where automated test suites generate execution reports that need to be imported into AIO Tests for tracking and reporting.

Key Capabilities

Supports standard test report formats such as TestNG, JUnit, Cucumber, NUnit, Robot, Newman.
Automatically maps test cases based on identifiers in the report.
Can create new runs if a matching run does not exist.
Optionally create new test cases or add them to the cycle if they are missing.

Reporting can be controlled by multiple parameters, specific to each framework, as listed in Explanation of Parameters.

How It Works

The automation framework executes tests and generates a test report file.
The report file is uploaded using this API.
AIO parses the report and:
- Identifies test cases based on keys or identifiers.
- Creates or updates test runs in the specified test cycle.
Execution statuses such as Passed, Failed, or Skipped are recorded against the corresponding test runs.

Typical CI/CD Workflow

Tests run in an automation pipeline (e.g., Selenium, Cucumber, Robot).
Framework generates a XML or JSON reports.
Pipeline uploads the report to AIO using the Import Results API.
AIO automatically updates execution results in the specified test cycle.

Note that the API can also be invoked from teardown hooks in frameworks.

The API assumes the cycle to be pre-existing, it can either be created from the AIO Tests interface or by using the Create Cycle API.

TestNG

curl -X POST "https://tcms.aiojiraapps.com/aio-tcms/api/v1/project/AT/testcycle/AT-CY-Adhoc/import/results?type=Testng" 
-H "accept: application/json;charset=utf-8" 
-H "Content-Type: multipart/form-data"
 -F "createNewRun=true" 
 -F "addCaseToCycle=true"
 -F "createCase=true"
 -F "file=@testng-results.xml;type=text/xml"

JUnit

curl -X POST "https://tcms.aiojiraapps.com/aio-tcms/api/v1/project/AT/testcycle/AT-CY-Adhoc/import/results?type=Junit" 
-H "accept: application/json;charset=utf-8" 
-H "Content-Type: multipart/form-data"
 -F "createNewRun=true" 
 -F "addCaseToCycle=true" 
 -F "createCase=true" 
 -F "file=@TEST-featureTest.xml;type=text/xml"

NUnit

curl -X POST "https://tcms.aiojiraapps.com/aio-tcms/api/v1/project/AT/testcycle/AT-CY-Adhoc/import/results?type=NUnit" 
-H "accept: application/json;charset=utf-8" 
-H "Content-Type: multipart/form-data"
 -F "createNewRun=true" 
 -F "addCaseToCycle=true" 
 -F "createCase=true" 
 -F "file=<filename>;type=text/xml"

Cucumber

curl -X POST "https://tcms.aiojiraapps.com/aio-tcms/api/v1/project/AT/testcycle/AT-CY-Adhoc/import/results?type=Cucumber" 
-H "accept: application/json;charset=utf-8" 
-H "Content-Type: multipart/form-data"
 -F "createNewRun=true" 
 -F "addCaseToCycle=true" 
 -F "createCase=true" 
 -F "file=<filename>;type=application/json"

Robot

curl -X POST "https://tcms.aiojiraapps.com/aio-tcms/api/v1/project/AT/testcycle/AT-CY-Adhoc/import/results?type=Robot" 
-H "accept: application/json;charset=utf-8" 
-H "Content-Type: multipart/form-data"
 -F "createNewRun=true" 
 -F "addCaseToCycle=true" 
 -F "createCase=true" 
 -F "file=<resultsfile>;type=text/xml"

Explanation of Parameters

Form Data parameters	Description	Default

Form Data parameters	Description	Default
file	File [results.xml]	Mandatory
createNewRun	Helps control the creation of new runs for repeating tests in a suite execution. Repeating tests may be due to data driven cases or multiple invocation cases. One may choose to disable it for e.g. in retry cases, where only the last status matters and in between runs don’t matter	true
addCaseToCycle	If mapped case is not already existing in the cycle being executed, adds the case to the cycle and reports results for the case. If set to false, such cases are reported as errors in case of TestNG AIO Tests only considers mappings which start with “JiraProjName-TC”- and not all groups	true
createCase	For cases where no key or automation key is found in AIO Tests, if createCase is set to true, a new case is created and added to cycle. If set to false, such a case is reported as an error
updateDatasets	Only impacts tests based on datasets. If set to true, it adds/removes datasets from data driven tests. Can be set to false in retries where subset of datasets only may have been executed.
bddForceUpdateCase	Only applies to BDD cases. Set to true to update case steps with those found in scenario. Use if automated scenarios get updated and changes need to be percolated to AIO Tests cases.	false
forceUpdateCase	Only applies to Robot, JUnit, TestNg and NUnit. Set to false to prevent case from being updated from results file. Helpful if users dont want to update the case from the file..	false
updateOnlyRunStatus	Affects Cucumber, Robot, and Newman. Only impacts tests with steps. Set to true to update test run-level details while preventing steps from being updated from the results file. Helpful for users who want to bypass step comparison and update only run-level details.	false
ignoreClassInAutoKey	Only applies to JUnit. Form data: Set to true to ignore classname and use only methodname from Junit file for automation key. Use when classname does not guarantee uniqueness or has a irrelevant value.	false
defaultFolder	Assist users in configuring the folder ID to designate the default folder for cases generated via import.

Explanation of Response

Sample Response

{
    "status": "SUCCESS_WITH_ERROR",
    "requestCount": 4,
    "successCount": 3,
    "errorCount": 1,
    "errors": {
        "SCRUM-TC-2092": {
            "message": "Case does not exist in cycle.  Set addCaseToCycle if you wish to add.",
            "cause": null,
            "info": "Consider setting form data addCaseToCycle=true to add case to cycle."
        }
    },
    "processedData": {
        "SCRUM-TC-2090": {
            "tcKey": "SCRUM-TC-2090",
            "className": "com.aio.tests.tng.SkippedTests2",
            "name": "failedTest",
            "status": "Failed",
            "duration": 0,
            "finishedAt": 1595440959000,
            "runId": 2374
        },
        "SCRUM-TC-2093": {
            "tcKey": "SCRUM-TC-2093",
            "className": "com.aio.tests.tng.SkippedTests2",
            "name": "method1",
            "status": "Passed",
            "duration": 0,
            "finishedAt": 1595440959000,
            "runId": 2375
        },
        "SCRUM-TC-2091": {
            "tcKey": "SCRUM-TC-2091",
            "className": "com.aio.tests.tng.SkippedTests2",
            "name": "skippedTest",
            "status": "Not Run",
            "duration": 0,
            "finishedAt": 1595440959000,
            "runId": 2376
        }
    }
}

Reporting results as a Batch

Endpoint

POST /project/{projectKey}/testcycle/{cycleKey}/import/results/batch

The Batch Import Results API allows automation frameworks or CI/CD pipelines to upload execution results in bulk to a test cycle in AIO Tests.

Unlike synchronous APIs that update runs immediately, this API creates a batch process that runs asynchronously in the background.

Batch Processing Behavior

When this API is invoked:

A batch process is created to handle the import.
The batch is placed in a scheduled job queue for processing.
The system parses the uploaded report and updates the corresponding test runs in the cycle.
Execution statuses such as Passed, Failed, or Skipped are recorded for the mapped test cases.

Because the operation may involve large automation result files, it is processed asynchronously to avoid blocking the API request.

Monitoring Batch Status

The progress and outcome of the import can be monitored in the UI:

Navigate to Cycles
Open the relevant Cycle
Go to the Batches tab

The Batches tab displays:

Batch ID
Batch type (Import Results)
Status (Queued, Processing, Completed, Failed)
Number of processed records
Any errors encountered during processing

When to Use This API

Use this API when:

Reporting large volumes of automated test results.
Exiting CI/CD workflows faster, instead of waiting for the non-batch API to finish.

Known Errors List (as part of errors.msg above):

Invalid xml: Unable to parse file. Please check if it is a valid results.xml file.
<Case key> is in Under Review/Draft status. Only published cases can be added to cycle.
<Case key> not found or you don't have privileges to add this case.
Case does not exist in the cycle. Set addCaseToCycle if you wish to add.
No case found with this automation key. Either map automation key or set createCase to true.
Unable to update automation data in the case. Insufficient permissions to edit cases in the project.

Response Errors

For further queries and suggestions, please feel free to reach out to our customer support service via help@aiotests.com.