Page Comparison

AIO Tests supports importing Behave results through it’s support for JUnit reports or results can also be reported using the hooks from the Behave framework and calling AIO Tests REST APIs.

Behave is a tool used for Behaviour Behavior driven development (BDD) in Python, that is very similar to Cucumber in Java/Ruby or Specflow for .Net. It uses tests written in a natural language style i.e Gherkin, backed up by Python code. Test scenarios are written in Gherkin “.feature” files. BDD Keyword based steps are glued to a step definition, which is a Python function decorated by a matching string in a step definition module. The feature files act like test scripts. 
Behave provides multiple Hooks at various points of execution, which  can insert helper logic for test execution. It also provides an out of the box JUnit file, which can simply be generated using a flag.

...

• Generating the Junit report from Behave tests and uploading it to AIO Tests.

• Using AIO Tests REST APIs to report results and much more, using the Behave hooks.

Required Setup

1. pip install behave

2. pip install requests (Optional, required only if reporting results via APIs)

Reporting results via JUnit file

...

Post execution of a suite, the TEST-<xxx>.xml file can be uploaded either via

• AIO Tests REST API ​call using multipart form-data to upload file

• Cycle options in Cycles Tab inside AIO Tests

• via AIO Tests Jenkins Plugin

...

Uploading the above file for the first time will

1. create new cases in the system if there is no key found in the classname.name. The new case is created with
   - title as the name value from <testcase> tag of the JUnit report
   - automation key as classname.name from the JUnit report.
   - status as Published
   - automation status as Automated
   - automation owner as user uploading the results

2. Add the newly created case to the cycle being uploaded to

3. Mark the details of the run

   1. Execution mode is set to Automated

   2. Duration of run is set to Actual Effort

   3. Status of run is set based on status mapping table below

   4. Failures and errors are reported as Run Level comments

If the same file is uploaded again, the cases will be identified using the automation key (classname.name )and would be updated, instead of creating new cases.

Status Mapping JUnit → AIO Tests

Reporting results via Behave Hooks and AIO Tests REST APIs

AIO Tests provides a rich set of APIs for Execution Management, using which users can not only report execution status, but also add effort, actual results, comments, defects and attachments to runs as well as steps.
AIO Tests also provides APIs to create cycles and to add cases to cycles for execution planning.

The basic sample below will show how Behave Hooks can leverage the AIO Tests REST APIs to report results. In the environment.py the before_scenario and after_scenario methods can be used to make AIO API calls.

...

In the example above, the add_run method uses requests to make an HTTP call.

1. It uses the scenario tags to identify the case key [ based on the convention we established]

2. Create a POST request

   1. URL : For cloud the url host would be https://tcms.

...

1. 1. aiojiraapps.com/aio-tcms/api/v1. For Jira server, it would be the native Jira server hostname.

   2. Authorization : Please refer to Rest API Authentication to understand how to authorize users. The authentication information goes in the headers: {'Authorization': '<Auth based on Jira Cloud/Jira Server>'},

   3. POST Body : The body consists of data from the test object.

   4. If required, the basic example can be extended to upload attachments against the case using the upload attachment API.