Introduction

One of the key goals of Continuous Integration is to support a shift left testing philosophy which provides Developers with fast feedback on the quality of their code changes. This is achieved, in part, by the use of fast, tightly-scoped tests, triggered quickly after a change is detected. As such, the speed of the Continuous Integration testing process is of paramount importance. This speed can be optimised using a number of methods, the most effective of which are…

• Minimise the scope of the tests being performed (e.g. by avoiding the dynamic provisioning of environments or sizeable volumes of data), and

• Execute the CI tests as quickly as possible, running testing steps in parallel where at all possible.

This page describes how to configure the latter in Jenkins.

MettleCI Use Cases

In MettleCI the obvious candidates for parallel test execution are:

• Compliance Testing (for Warnings),

• Compliance Testing (for Failures), and

• Unit Tests

Each of these can be performed concurrently without affecting the other as the Compliance processes run independently in memory using a data from supplied files, while the Unit Test runs in the DataStage environment as one or more executing Jobs.

Parallelism in Jenkins

At which pipeline level? Stages, which (remember) can be nested.

Our use cases: Compliance (WARN), Compliance (FAIL), and Unit Test steps (term?) concurrently.

See Parallel stages with Declarative Pipeline 1.2 (jenkins.io)

Defining Parallel Stages in a Jenkins Pipeline

Here’s a pseudocode representation of parallel Stages defined in a Jenkins Pipeline.

pipeline {
    stages {
        stage('Do These Concurrently') {
            parallel {
                stage('Test On Windows') {
                    agent {
                        label "windows"
                    }
                    steps {
                        bat "run-tests.bat"
                    }
                    post {
                        always {
                            junit "**/TEST-*.xml"
                        }
                    }
                }
                stage('Test On Linux') {
                    agent {
                        label "linux"
                    }
                    steps {
                        sh "run-tests.sh"
                    }
                    post {
                        always {
                            junit "**/TEST-*.xml"
                        }
                    }
                }
            }
        }
    }
}