GO-vatar series: Integrate Cucumber scenario with JIRA Xray

https://marketplace-cdn.atlassian.com/files/images/df86e405-35d4-42da-87a6-42d467b03103.png

Anyone of you know or remember about https://github.com /mpermperpisang/golang-automation?

Yeah that’s my own automation testing framework with Golang. At first I made it very simple. But now it’s already integrated with JIRA, to be more spesific with Xray as a test documentation tool. To shorten the time, let me explain to you how to integrate automation report with Xray.

Before we start a how to integrate the Xray, I want to quote the types of issues that are in Xray. This already explained when we open the Xray settings at https://<your_JIRA_name>.atlassian.net/plugins/servlet/ac/com.xpandit.plugins.xray/xray-project-settings-all?board.id=2&project.key=MPER&project.id=10001&board.type=kanban (I’m using Kanban board here).

Xray Issue Type

• Test Plan

This is the Xray Test Plan Issue Type. Used to define the scope of test cases for a given test campaign and to aggregate all executions for those tests displaying the latest result for each test case.

• Test Execution

This is the Xray Test Execution Issue Type. Used to execute test cases already defined.

• Test Set

This is the Xray Test Set Issue Type. Creates a group of test cases. Used to associate all included Tests with other Xray issue types like Test Execution and Test Plan. A Test Set can also be associated with a requirement issue to provide coverage and test status.

• Test

This is the Xray Test Issue Type. Used to define test cases of different types that can be executed multiple times using Test Execution issues.

• Precondition

This is the Xray Precondition Issue Type. Used to abstract common actions that must be ensured before the test case execution. A Precondtion can be associated with multiple test cases.

Let’s go next !!!

JIRA Apps Installation

There are 2 types of Xray in JIRA. Based on https://confluence.xpand-it.com/display/public/ADDONS/Xpand+IT+Apps :

1. Server
2. Cloud

I’ll show you the Cloud version (because I’m using one in my daily task).

Make sure we have an administration access for JIRA.

1. Go to Apps menu.
2. Search Xray.
3. Click Install.

Please note that Xray give a Free Trial so we don’t have to purchase anything in the beginning.

Left image to access Apps Menu. Right image to search the apps.

Left image showing Xray search result. Right image after Xray installation.

API Token

Make sure you have an administration access and then go to this URL https://<your_JIRA_name>.atlassian.net/plugins/servlet/ac/com.xpandit.plugins.xray/xray-global-settings-api-keys?s=com.xpandit.plugins.xray__xray-global-settings-api-keys.

Save the Client Id & Client Secret in auth.json.

Please note that the user we assign will be recorded as the executor every time we import the automation run result

API Endpoint

There is a simple way to do Xray integration. These 3 steps are an example that we can use to integrate Xray with our automation scenario.

1. Endpoint for authentication.
2. Endpoint to export the Cucumber scenario.
3. Endpoint to import the automation run result.

But actually, there are more endpoint that we can use. You can see it here.

Xpand IT Documentation Portal

Next~~~

Authentication

Xpand IT Documentation Portal

See the usage example at https://github.com/mpermperpisang/golang-automation/blob/master/xray#L13.

Export Scenario

Exporting Cucumber Tests - REST

See the usage example at https://github.com/mpermperpisang/golang-automation/blob/master/xray#L17.

Import Automation Run Result

Import Execution Results - REST

See the usage example at https://github.com/mpermperpisang/golang-automation/blob/master/xray#L27.

We can use curl in shell script or net/http from Go library to hit those endpoint. In my case, I’m using shell script. When I send command in terminal ./xray -e -x <TEST_EXECUTION_ID> it will download the scenario as .zip file and then extract it into features/scenario/xray folder (https://github.com/mpermperpisang/golang-automation/blob/master/xray#L22).

Explanation for Xray shell command :

• ./xray. The xray shell file.
• -e is for export. Generate Cucumber scenario from Xray to our local environment.
• -f is for feature . Test execution ID that will become the feature tag.

We can see the generated scenario in https://github.com/mpermperpisang/golang-automation/tree/master/features/scenarios/xray. Then we can run it as usual but wait! We need the report result to be in Cucumber format. How to do that with Godog?

Try to type godog --help .

Godog already has a format for Cucumber test output. So what we have to do next is to add the additional format to the run script. See the example at https://github.com/mpermperpisang/golang-automation/blob/master/godog#L22. After the test run, it will generate cucumber report json (FYI, we can also generate to yaml format if we want to do so).

Report Json Example

That json file is what we will use to import the test result go to JIRA. I already write the shell command to export the Cucumber scenario. For import, we can use ./xray -i -j cucumber_report.json.

Explanation for Xray shell command :

• ./xray. The xray shell file.
• -i is for import. Send the automation run result to JIRA.
• -j is for json. Cucumber report json file name.

You can see all of those command above at this file https://github.com/mpermperpisang/golang-automation/blob/master/godog.

Congrats ! The result has imported successfully to JIRA.

I want to inform you something. If you have another scenario outside the exported Test Execution, then you’ll have to delete all of those scenario first (https://github.com/mpermperpisang/golang-automation/blob/master/xray#L55) because the import process will not success if the generated json including report outside Xray report.

https://docs.eazybi.com/eazybijira/files/13205655/43516054/4/1539983863000/xray.png

Many of us using test documentation tools besides Xray. What is your eXperience? — MperMperPisang

References :

• https://confluence.xpand-it.com/display/public/ADDONS/Xpand+IT+Apps