Karate Framework

AIO Tests supports importing Karate Framework results through its support for Cucumber reports.

Karate is an open-source general-purpose test-automation framework that can script calls to HTTP end-points and assert that the responses are valid. It also supports UI Automation. Karate is implemented in Java but test scripts are written in Gherkin since Karate was originally an extension of the Cucumber framework.

Karate can generate Cucumber reports, which can then be imported into AIO Tests, to report results from Karate runs to AIO Tests. This document provides an overview of how you can generate the Cucumber report from Karate and upload it to AIO Tests.

In this documentation, you’ll understand:

1 Required Karate Setup
- 1.1 Sample Case
- 1.2 Running the Sample Case and Generating Cucumber JSON Report
2 Mapping Cases with AIO Tests
3 Importing Results

Required Karate Setup

Java 8 + Maven
Use the karate archetype to generate a sample project.
mvn archetype:generate \ -DarchetypeGroupId=com.intuit.karate \ -DarchetypeArtifactId=karate-archetype \ -DarchetypeVersion=1.2.0 \ -DgroupId=com.aiotests \ -DartifactId=aio-karate-tests

To start using Karate, please follow the Get Started documentation.

Sample Case

This document will work on the below sample case, which demos a simple AIO Tests API test.

Feature: AIO Tests Demo Karate file

  Scenario: get all users and then get the first user by id
    Given url baseUrl
    Given header Authorization = auth
    Given path '/project/',projectKey,'/config/testcase/status'
    When method get
    Then status 200
    * def first = response[0]
    And print first
    Then assert first.name == 'Draft'
    Then assert first.description == 'The test is being worked upon and'

The archetype also generates a Karate configurations file (karate-config.js), which can hold variables per environment or define actions to be executed in different environments. Global variables accessible to all cases can be defined in this file.
The sample case above is using the below variables:

function fn() {
//Setting showLog to false, to avoid cluttering response docstrings
// with response objects
 karate.configure('report',{ showLog: false, showAllSteps: true }
  var config = {
    baseUrl: "https://tcms.aiojiraapps.com/aio-tcms/api/v1",
    auth: 'AioAuth <yourauth>',
    projectKey: 'NVTES'
  }
  
  return config;
}

Running the Sample Case and Generating Cucumber JSON Report

For running the cases and generating the cucumber report, the following class identifies what needs to run.

Note the addition of .outputCucumberJson(true) to generate the cucumber.report.

import com.intuit.karate.Results;
import com.intuit.karate.Runner;
import static org.junit.jupiter.api.Assertions.*;
import org.junit.jupiter.api.Test;

class AIOAPITests {

    @Test
    void testParallel() {
        Results results = Runner.path("classpath:examples")
                .outputCucumberJson(true)
                .parallel(5);
        assertEquals(0, results.getFailCount(), results.getErrorMessages());
    }

}

Running the above generates the following results file:


[
  {
    "line": 1,
    "elements": [
      {
        "start_timestamp": "2022-07-21T07:39:44.005Z",
        "line": 4,
        "name": "get all users and then get the first user by id",
        "description": "",
        "id": "get-all-users-and-then-get-the-first-user-by-id",
        "type": "scenario",
        "keyword": "Scenario",
        "steps": [
          {
            "name": "url baseUrl",
            "result": {
              "duration": 800214,
              "status": "passed"
            },
            "match": {
              "location": "karate",
              "arguments": [
                
              ]
            },
            "keyword": "Given",
            "line": 5
          },
          {
            "name": "header Authorization = auth",
            "result": {
              "duration": 263396,
              "status": "passed"
            },
            "match": {
              "location": "karate",
              "arguments": [
                
              ]
            },
            "keyword": "Given",
            "line": 6
          },
          {
            "name": "path '\/project\/',projectKey,'\/config\/testcase\/status'",
            "result": {
              "duration": 10339103,
              "status": "passed"
            },
            "match": {
              "location": "karate",
              "arguments": [
                
              ]
            },
            "keyword": "Given",
            "line": 7
          },
          {
            "name": "method get",
            "result": {
              "duration": 1397104805,
              "status": "passed"
            },
            "match": {
              "location": "karate",
              "arguments": [
                
              ]
            },
            "keyword": "When",
            "line": 8,
            "doc_string": {
              "content_type": "",
              "value": "13:09:44.361 request:\n1 > GET https:\/\/tcms.aiojiraapps.com\/aio-tcms\/api\/v1\/project\/NVTES\/config\/testcase\/status\n1 > Authorization: AioAuth auth \n1 > Host: tcms.aiojiraapps.com\n1 > Connection: Keep-Alive\n1 > User-Agent: Apache-HttpClient\/4.5.13 (Java\/11.0.15)\n1 > Accept-Encoding: gzip,deflate\n\n\n13:09:45.522 response time in milliseconds: 1142\n1 < 200\n1 < content-type: application\/json;charset=utf-8\n1 < date: Thu, 21 Jul 2022 07:39:45 GMT\n1 < strict-transport-security: max-age=31536000; includeSubDomains;\n1 < cache-control: no-store\n1 < pragma: no-cache\n1 < x-xss-protection: 1; mode=block;\n1 < x-content-type-options: nosniff\n1 < transfer-encoding: chunked\n1 < vary: Accept-Encoding\n1 < Set-Cookie: JSESSIONID=s2~63888A01EE477F43FC7632C29C7A9798; Domain=tcms.aiojiraapps.com; Secure\n[{\"ID\":1465,\"name\":\"Draft\",\"description\":\"The test is being worked upon\"},{\"ID\":1466,\"name\":\"Under Review\",\"description\":\"The test is under review\"},{\"ID\":1467,\"name\":\"Published\",\"description\":\"The test is ready for execution\"},{\"ID\":1468,\"name\":\"Deprecated\",\"description\":\"The test is no longer valid\"}]\n\n",
              "line": 8
            }
          },
          {
            "name": "status 200",
            "result": {
              "duration": 18013,
              "status": "passed"
            },
            "match": {
              "location": "karate",
              "arguments": [
                
              ]
            },
            "keyword": "Then",
            "line": 9
          },
          {
            "name": "def first = response[0]",
            "result": {
              "duration": 18558295,
              "status": "passed"
            },
            "match": {
              "location": "karate",
              "arguments": [
                
              ]
            },
            "keyword": "*",
            "line": 10
          },
          {
            "name": "print first",
            "result": {
              "duration": 36007471,
              "status": "passed"
            },
            "match": {
              "location": "karate",
              "arguments": [
                
              ]
            },
            "keyword": "And",
            "line": 11,
            "doc_string": {
              "content_type": "",
              "value": "13:09:45.613 [print] {\n  \"ID\": 1465,\n  \"name\": \"Draft\",\n  \"description\": \"The test is being worked upon\"\n}\n \n",
              "line": 11
            }
          },
          {
            "name": "assert first.name == 'Draft'",
            "result": {
              "duration": 10294256,
              "status": "passed"
            },
            "match": {
              "location": "karate",
              "arguments": [
                
              ]
            },
            "keyword": "Then",
            "line": 12
          },
          {
            "name": "assert first.description == 'The test is being worked upon and'",
            "result": {
              "duration": 559825,
              "error_message": "did not evaluate to 'true': first.description == 'The test is being worked upon and'\nclasspath:examples\/aiodemo\/demo.feature:13",
              "status": "failed"
            },
            "match": {
              "location": "karate",
              "arguments": [
                
              ]
            },
            "keyword": "Then",
            "line": 13,
            "doc_string": {
              "content_type": "",
              "value": "13:09:45.626 classpath:examples\/aiodemo\/demo.feature:13\nThen assert first.description == 'The test is being worked upon and'\ndid not evaluate to 'true': first.description == 'The test is being worked upon and'\nclasspath:examples\/aiodemo\/demo.feature:13\n",
              "line": 13
            }
          }
        ],
        "tags": [
          {
            "name": "@NVTES-TC-13241",
            "line": 3
          }
        ]
      }
    ],
    "name": "examples\/aiodemo\/demo.feature",
    "description": "AIO Tests Demo Karate file",
    "id": "aio-tests-demo-karate-file",
    "keyword": "Feature",
    "uri": "examples\/aiodemo\/demo.feature"
  }
]

Mapping Cases with AIO Tests

AIO Tests supports creating tests using the cucumber reports as well as mapping existing cases. Cases can simply be mapped using the AIO Tests Case keys as tags on features.

The mapping of cases follows the same convention as Cucumber. To learn more, click here.

Importing Results

Post execution of a suite, the TEST-<xxx>.xml file can be uploaded either via

AIO Tests REST API calls using multipart form data to upload file
Cycle options in the Cycles Tab inside AIO Tests
via AIO Tests Jenkins Plugin

Please follow the above links to continue to import results using either of the options.

For further queries and suggestions, feel free to reach out to our customer support via help@aiotests.com.