1

Diags 11.7

various

Uses the Build Agent to log environment variables and variable groups for diagnostics purposes.

1

Create DataStage Project

mettleci datastage create-project (documentation)

Verify that the CI project exists (otherwise, create it).

2

Substitute parameters in DataStage config

mettleci properties config (documentation)

Replace variables in the DSParams file, any Parameter Sets files, and any user-define shell scripts (from repository path /datastage) with values defined in the var.ci properties file. Processed files are stored in a config directory created by this command in the current workspace on the agent.

This example operates on *.sh files in the 'datastage' base directory, the DSParams file, and all Parameter Sets

using values from the file 'var.ci' (as determined by the environmentId parameter)

// Substituted files are created in a new temporary build directory on your build agent host called 'config'

3

Cleanup temporary files from previous builds

mettleci remote execute (documentation)

Execute your datastage/cleanup.sh script. This script removes the directory containing previous pipeline artefacts on the datastage engine, ensuring contains only artefacts created for this execution are present.

4

Transfer DataStage config and filesystem assets

mettleci remote upload (documentation)

Upload the config directory containing files modified by the Configure Properties step and your repository’s filesystem directory from the build agent to your CI environment.

5

Deploy DataStage config and file system assets

mettleci remote execute (documentation)

Execute your config/deploy.sh script (previously generated by the Configure properties task from your repository's datastage/deploy.sh file) on your CI environment. This user-supplied script performs solution-specific deployment processes required by the assets in the filesystem directory.

6

Deploy DataStage project

mettleci datastage deploy (documentation)

Perform an incremental deployment of your DataStage assets to the CI project.

7

Cleanup

mettleci remote execute (documentation)

Execute the script config/cleanup.sh (previously generated by the Configure properties task from your repository's datastage/cleanup.sh file) in the CI environment. This script removes files which are no longer required from your build agent’s disk.

8

Find files in the workspace

Inline script

Detects and records (in a build pipeline variable) whether the Job compilation performed by the Deploy DataStage assets task produced an output JUnit .xml file. Note that the incremental deployment approach used by MettleCI may mean that a test output file would not be generated in the event that the deployment process does not detect any changes.

9

Archive JUnit-formatted test results

Build tool specific

If a JUnit .xml file was detected by the Find files in the workspace step then publish it to the build tool for use in test suite management.

1

Check out from version control

Build tool specific

Make the Compliance repository (separate to the DataStage asset repository, where this pipeline definition is stored) accessible to the build agent.

2

Compliance Test - Warnings

mettleci compliance test -ignore-test-failures (documentation)

Invokes MettleCI Compliance checks for changes jobs. Note that in this mode (including the -ignore-test-failures option) a failed Compliance check will not cause the entire build pipeline to fail.

3

Find files in the workspace

Build tool specific

Detects and records (in a build pipeline variable) whether the Compliance check performed by the Run Compliance task produced an output JUnit .xml file. Note that the incremental deployment approach used by MettleCI may mean that a test output file would not be generated in the event that the deployment process does not detect any changes.

4

Archive JUnit-formatted test results

Build tool specific

If a JUnit .xml file was detected by the Check for results step then publish it to the build tool for use in test suite management.

1

Check out from version control

Build tool specific

Make the Compliance repository (separate to the DataStage asset repository, where this pipeline definition is stored) accessible to the build agent.

2

Compliance Test - Errors

mettleci compliance test (documentation)

Invoke MettleCI Compliance checks for changes jobs. Note that in this mode (omitting the -ignore-test-failures option) a failed Compliance check will cause the entire build pipeline to fail.

3

Find files in the workspace

Inline script

Detects and records (in a build pipeline variable) whether the Compliance check performed by the Run Compliance task produced an output JUnit .xml file. Note that the incremental deployment approach used by MettleCI may mean that a test output file would not be generated in the event that the deployment process does not detect any changes.

4

Archive JUnit-formatted test results

Build tool specific

If a JUnit .xml file was detected by the Check for results step then publish it to the build tool for use in test suite management.

Configure Properties

mettleci properties config (documentation)

Replace variables in the cleanup_unittest.sh file, with values defined in the var.ci properties file.

// Substituted files are created in a new temporary build directory on your build agent host called 'config'.

7

Cleanup

mettleci remote execute (documentation)

Execute the script stored in the repository file config/cleanup_unittest.sh (previously generated by the Configure properties task) in the CI environment. This script removes unit test results, to ensure a clean environment for the next execution.

2

Upload unit test specs

mettleci remote upload (documentation)

Upload your repository’s unittest directory from the build agent to your CI environment.

4

Create unit test report dir

mkdir unittest-reports

Create directory to receive unit test outputs.

5

Run Unit Tests

mettleci unittest test (documentation)

Invoke the Unit Tests defined in your unittest directory. Note that MettleCI will use its incremental detection functionality to only execute Unit Tests where either the Unit Test case or associated Job have changed.

6

Download unit test reports

mettleci remote download (documentation)

Download any JUnit .xml files produced by the Run Unit Tests task from your CI environment to the build agent.

8

Find files in the workspace

Inline script

Detects and records (in a build pipeline variable) whether the Unit Test execution performed by the Run Unit Tests task produced an output JUnit .xml file. Note that the incremental deployment approach used by MettleCI may mean that a Unit Test output file would not be generated in the event that the deployment process does not detect any changes.

9

Archive JUnit-formatted test results

Build tool specific

If a JUnit .xml file was detected by the Check for results step then publish it to the build tool for use in test suite management

1

Tag current build

various

Adds a tag to the commit that triggered the successful build, thereby identifying it as a candidate for deployment into the QA environment. The tag is in the format ci-{build_number}.

1

Tag current build

various

Calls upon the the user to choose the target environment, and then choose the desired tag from the list of approved candidates.

1

Create DataStage Project

mettleci datastage create-project (documentation)

Verify that the CI project exists (otherwise, create it).

2

Substitute parameters in DataStage config

mettleci properties config (documentation)

Replace variables in the DSParams file, any Parameter Sets files, and any user-define shell scripts (from repository path /datastage) with values defined in the var.ci properties file. Processed files are stored in a config directory created by this command in the current workspace on the agent.

This example operates on *.sh files in the 'datastage' base directory, the DSParams file, and all Parameter Sets

using values from the file 'var.ci' (as determined by the environmentId parameter)

// Substituted files are created in a new temporary build directory on your build agent host called 'config'

3

Cleanup temporary files from previous builds

mettleci remote execute (documentation)

Execute your datastage/cleanup.sh script. This script removes the directory containing previous pipeline artefacts on the datastage engine, ensuring contains only artefacts created for this execution are present.

4

Transfer DataStage config and filesystem assets

mettleci remote upload (documentation)

Upload the config directory containing files modified by the Configure Properties step and your repository’s filesystem directory from the build agent to your CI environment.

5

Deploy DataStage config and file system assets

mettleci remote execute (documentation)

Execute your config/deploy.sh script (previously generated by the Configure properties task from your repository's datastage/deploy.sh file) on your CI environment. This user-supplied script performs solution-specific deployment processes required by the assets in the filesystem directory.

6

Deploy DataStage project

mettleci datastage deploy (documentation)

Perform an incremental deployment of your DataStage assets to the CI project.

7

Cleanup

mettleci remote execute (documentation)

Execute the script config/cleanup.sh (previously generated by the Configure properties task from your repository's datastage/cleanup.sh file) in the CI environment. This script removes files which are no longer required from your build agent’s disk.

8

Find files in the workspace

Inline script

Detects and records (in a build pipeline variable) whether the Job compilation performed by the Deploy DataStage assets task produced an output JUnit .xml file. Note that the incremental deployment approach used by MettleCI may mean that a test output file would not be generated in the event that the deployment process does not detect any changes.

9

Archive JUnit-formatted test results

Build tool specific

If a JUnit .xml file was detected by the Find files in the workspace step then publish it to the build tool for use in test suite management.

1

Tag current build

various

Adds a tag to the commit that triggered the successful build, thereby identifying it as a candidate for deployment into the next environment. ie: if the current run successfully deployed a “ci-” tag into QA, the commit the tag was related to also receives a “qa-” tag, identifying it as a potential candidate for deployment into the Perf environment.

1

Building Jenkins-DevOps-Hotfix-Create

various

If the current deployment was into the Production environment, run a separate Jenkins pipeline that updates a special hotfix DataStage project to always be in line with the Production project, and also create a separate branch in the git repository to manage any hotfix changes that may be made.