Our documentation site has been updated to AIO Tests Knowledge Base
NUnit
- Noopur Varshney [AIO Tests]
AIO Tests allows easy reporting of NUnit results by supporting the import of the NUnit results xml.
With AIO Tests NUnit integration, developers and testers can report their execution results to existing AIO Tests or directly create testcases which do not yet exist in AIO Tests.
This article discusses how to import NUnit XML results and how to map existing cases or create new ones from NUnit results.
NUnit
NUnit is an open source unit-testing framework for all .Net languages. It has strong support for parallel runs and data driven tests, with a rich set of assertions to support all kinds of verifications. NUnit uses a rich set of custom attributes to identify tests. Attributes also help to achieve flexible setup and teardown for the tests. All NUnit attributes are contained in the NUnit.Framework namespace.
NUnit tests can be run via command line or via GUI. NUnit results are captured in a xml file, which can provide information on the status, duration and failures of the tests.
NUnit Reporting
NUnit 3.0 report structure can be found @ Test Result XML Format | NUnit Docs. AIO Tests supports the latest 3.0 version of the report and does not support any previous versions.
Sample NUnit XML Report
Below is a sample NUnit 3.0 XML report with just one testcase, which gives an idea of the structure of ther report. It captures a failing test, along with the failure and it’s stacktrace.
<test-run id="2" duration="0.038897" testcasecount="1" total="1" passed="0" failed="1" inconclusive="0" skipped="0" result="Failed" start-time="2022-06-13T 12:10:58Z" end-time="2022-06-13T 12:10:59Z">
<test-suite type="Assembly" name="nunit-aiotests-demo.dll" fullname="xxx" total="1" passed="0" failed="1" inconclusive="0" skipped="0" result="Failed" start-time="2022-06-13T 12:10:58Z" end-time="2022-06-13T 12:10:58Z" duration="0.038897">
<test-suite type="TestSuite" name="Money" fullname="Money" total="1" passed="0" failed="1" inconclusive="0" skipped="0" result="Failed" start-time="2022-06-13T 12:10:58Z" end-time="2022-06-13T 12:10:58Z" duration="0.038897">
<test-suite type="TestFixture" name="MoneyTest" fullname="Money.MoneyTest" classname="Money.MoneyTest" total="1" passed="0" failed="1" inconclusive="0" skipped="0" result="Failed" start-time="2022-06-13T 12:10:58Z" end-time="2022-06-13T 12:10:58Z" duration="0.038897">
<test-case name="BagMultiply" fullname="Money.MoneyTest.BagMultiply" methodname="BagMultiply" classname="MoneyTest" result="Failed" start-time="2022-06-13T 12:10:58Z" end-time="2022-06-13T 12:10:58Z" duration="0.038897" asserts="0" seed="1998700055">
<properties>
<property name="Category" value="SingleTest" />
</properties>
<failure>
<message> Expected: <{[24 CHF][15 USD]}>
But was: <{[24 CHF][14 USD]}>
</message>
<stack-trace> at Money.MoneyTest.BagMultiply() in /Users/varshneyn/aiotcms/automation-demo-repos/nunit-aiotests-demo/nunit-aiotests-demo/MoneyTest.cs:line 47
</stack-trace>
</failure>
</test-case>
</test-suite>
</test-suite>
<errors />
</test-suite>
</test-run>Status Mapping NUnit → AIO Tests
NUnit XML result value in test-case | Description | AIO Tests Mapping |
|---|---|---|
Passed | Passed case | Passed |
Failed | Failed | Not Run |
Inconclusive |
| Skipped |
Skipped |
| Skipped |
Mapping automated NUnit tests to AIO Tests
In going with the simplicity advantage of AIO Tests, the NUnit integration has been designed as a simple and non-intrusive integration, without any extra dependency or coding required, utilising the core NUnit features of Attributes.
There are 3 ways that a NUnit case is mapped to an AIO Case.
NUnit Property Attribute: If a case has been created in AIO Tests, The Property attribute can be used with AIO property name :
aioCaseKey[Test] [Property("aiocasekey", "RAM-TC-1")] [Category("SimpleTests")] public void PassingTest() { Assert.Pass(); }Testcase key in Underscores in test name : Since many languages do not allow hyphens in their method signatures, AIO Tests allows case keys with underscores as valid mapping. The AIO Test key can be used in the method name by replacing the hyphen with underscores. eg. if AT-TC-123 is being automated by a test, the test method can be named as
verifyNotNullType_AT_TC_123and the results of this method will be marked against AT-TC-123.Automation Key: The mapping of an AIO Case to an automated NUnit case can happen through a field Automation Key, which can be specified in the Case in AIO Tests app as shown below. The automation key is the fully qualified name of the test method i.e fullname from the results xml.
This can be useful at places where you do not want to add AIO case keys in the name or annotations or where automation is happening first without a manual case being created in AIO Tests.
Examples
Below are few examples of how to achieve mapping in various cases:
1. Normal standalone case - Use property aioCaseKey
[Test]
[Property("aiocasekey", "RAM-TC-1")]
[Category("SimpleTests")]
public void PassingTest()
{
Assert.Pass();
}
[Test]
[Property("aiocasekey", "SCRUM-TC-421")] //cross-project case
[Category("SimpleTests")]
public void Test_Failed_Retry()
{
Assert.Fail("I failed coz I wanted to fail");
}2. Normal standalone case - Use case key in method name
The below case will update the status of SCRUM-TC-22 in AIO Tests as passed.
[Test]
[Category("LongRun")]
public void MyTest_SCRUM_TC_22()
{
Assert.Pass();
}3. Data driven cases [ using TestCaseSource attribute]
Below example shows how data driven cases can be mapped to different cases in AIO Tests.
[TestFixture]
[Category("DDTests")]
public class DataDrivenTests {
[TestCaseSource(typeof(MyDataClass), nameof(MyDataClass.TestCases))]
public int DivideTest(int n, int d, String tcId)
{
return n / d;
}
}
public class MyDataClass {
public static IEnumerable TestCases
{
get
{
yield return new TestCaseData(12, 3,"SCRUM-TC-15").Returns(4);
yield return new TestCaseData(12, 2,"SCRUM-TC-423").Returns(6);
yield return new TestCaseData(12, 4,"SCRUM-TC-1615").Returns(3);
}
}
}4. Data driven cases [ using TestCase attribute]
Below example shows how data driven cases can be mapped to different cases in AIO Tests with TestCase attribute
[TestFixture]
[Category("DDTests")]
public class DataDrivenTests {
[TestCase(12, 3, 4, "SCRUM-TC-13194")]
[TestCase(12, 2, 1, "SCRUM-TC-13195")]
[TestCase(12, 4, 3, "SCRUM-TC-13196")]
public void DivideTest(int n, int d, int q, String tcId)
{
Assert.AreEqual(q, n / d);
}
}If Case key is not provided at TestCase level and instead, is provided as Property aioCaseKey, then depending upon user’s choice at time of import, either one run would be created per data set or the same run would be updated thrice based on status and the last status update would remain.
5. Mapping case to multiple cases
Below example shows how to map a case if an automated case covers multiple cases
[Test]
[Category("Multitest")]
[Property("aiocasekey", "RAM-TC-2,RAM-TC-3")]
public void TestEquality()
{
String eq1 = "AIO";
String eq2 = "aio";
Assert.AreEqual(eq1, eq2);
if (eq1 != null && eq2 != null)
Assert.AreEqual(eq1.GetHashCode(), eq2.GetHashCode());
}Importing Results
Post execution of a NUnit suite, the xml file can be uploaded either via
Import Results - REST API - REST API call using multipart form-data to upload file
Import Results- Cycle Tab - Importing results file from Cycles tab of AIO Tests app
Please follow the above links to continue to import results using either of the options.