1

Diagnostics v11.5

various

Uses the legacy Build Agent to log environment variables and variable groups for diagnostics purposes.

1

Diagnostics v11.7

various

Uses the target Build Agent to log environment variables and variable groups for diagnostics purposes.

1

Fetch template DSParams

mettleci remote download (documentation)

Fetch the template DSParams file from the legacy DataStage engine.

2

Create artefacts directory

Inline script

Verify that the build directory artefactsexists (otherwise, create it).

3

Generate diff

mettle dsparams diff (documentation)

Generate a DSParams.diff file of differences between template DSParams and actual DSParams.

4

Publish diff artefact

Build tool specific

Define the DSParams.diff file as an artefact to make it available to other pipeline stages.

1

Download artefact

Build tool specific

Download the previously-created DSParams.diff file to the agent’s local directory.

2

Create project

mettleci datastage create-project (documentation)

Verify that the legacy CI project exists (otherwise, create it).

3

Configure properties

mettleci properties config (documentation)

Replace variables in the DSParams file, any Parameter Sets files, and any user-define shell scripts (from repository path /datastage) with values defined in the var.ci properties file. Processed files are stored in a config directory created by this command in the current workspace on the agent.

4

Upload config files

mettleci remote upload (documentation)

Upload the config directory containing files modified by the Configure Properties step and your repository’s filesystem directory from the build agent to your legacy CI environment.

5

Execute deploy script

mettleci remote execute (documentation)

Execute your config/deploy.sh script (previously generated by the Configure properties task from your repository's datastage/deploy.sh file) on your legacy CI environment. This user-supplied script performs solution-specific deployment processes required by the assets in the filesystem directory.

6

Deploy DataStage assets

mettleci datastage deploy (documentation)

Perform an incremental deployment of your DataStage assets to the legacy CI project.

7

Cleanup

mettleci remote execute (documentation)

Execute the script config/cleanup.sh (previously generated by the Configure properties task from your repository's datastage/cleanup.sh file) in the legacy CI environment. This script removes files which are no longer required from your build agent’s disk.

8

Check for results

Inline script

Detects and records (in a build pipeline variable) whether the Job compilation performed by the Deploy DataStage assets task produced an output JUnit .xml file. Note that the incremental deployment approach used by MettleCI may mean that a test output file would not be generated in the event that the deployment process does not detect any changes.

9

Publish compilation results

Build tool specific

If a JUnit .xml file was detected by the Check for results step then publish it to the build tool for use in test suite management.

1

Checkout Compliance repository

Build tool specific

Make the Compliance repository (separate to the DataStage asset repository, where this pipeline definition is stored) accessible to the build agent.

2

Run Compliance Warnings

mettleci compliance test -ignore-test-failures (documentation)

Invokes MettleCI Compliance checks for changes jobs. Note that in this mode (including the -ignore-test-failures option) a failed Compliance check will not cause the entire build pipeline to fail.

3

Check for results

Build tool specific

Detects and records (in a build pipeline variable) whether the Compliance check performed by the Run Compliance task produced an output JUnit .xml file. Note that the incremental deployment approach used by MettleCI may mean that a test output file would not be generated in the event that the deployment process does not detect any changes.

4

Publish compliance results

Build tool specific

If a JUnit .xml file was detected by the Check for results step then publish it to the build tool for use in test suite management.

1

Checkout Compliance repository

Build tool specific

Make the Compliance repository (separate to the DataStage asset repository, where this pipeline definition is stored) accessible to the build agent.

2

Run Compliance (Legacy CI FAIL)

mettleci compliance test (documentation)

Invoke MettleCI Compliance checks for changes jobs. Note that in this mode (omitting the -ignore-test-failures option) a failed Compliance check will cause the entire build pipeline to fail.

3

Check for results

Inline script

Detects and records (in a build pipeline variable) whether the Compliance check performed by the Run Compliance task produced an output JUnit .xml file. Note that the incremental deployment approach used by MettleCI may mean that a test output file would not be generated in the event that the deployment process does not detect any changes.

4

Publish compliance results

Build tool specific

If a JUnit .xml file was detected by the Check for results step then publish it to the build tool for use in test suite management.

1

Upload config files

mettleci remote upload (documentation)

Upload the file datastage/cleanup_unittest.sh from the build agent to your legacy CI environment.

2

Upload unit test specs

mettleci remote upload (documentation)

Upload your repository’s unittest directory from the build agent to your legacy CI environment.

3

Configure properties

mettleci properties config (documentation)

Replace variables in the datastage/cleanup_unittest.sh file with values defined in the var.ci properties file and store the processed file in the config directory.

4

Create unit test report dir

mkdir unittest-reports

Create directory to receive unit test outputs.

5

Run Unit Tests

mettleci unittest test (documentation)

Invoke the Unit Tests defined in your unittest directory. Note that MettleCI will use its incremental detection functionality to only execute Unit Tests where either the Unit Test case or associated Job have changed.

6

Download unit test reports

mettleci remote download (documentation)

Download any JUnit .xml files produced by the Run Unit Tests task from your legacy CI environment to the build agent.

7

Cleanup

mettleci remote execute (documentation)

Execute the script stored in the repository file config/cleanup_unittest.sh (previously generated by the Configure properties task) in the legacy CI environment. This script removes unit test results, to ensure a clean environment for the next execution.

8

Check for results

Inline script

Detects and records (in a build pipeline variable) whether the Unit Test execution performed by the Run Unit Tests task produced an output JUnit .xml file. Note that the incremental deployment approach used by MettleCI may mean that a Unit Test output file would not be generated in the event that the deployment process does not detect any changes.

9

Publish compilation results

Build tool specific

If a JUnit .xml file was detected by the Check for results step then publish it to the build tool for use in test suite management

1

Download artefact

Build tool specific

Download the previously-created DSParams.diff file to the agent’s local working directory.

2

Create project

mettleci datastage create-project (documentation)

Verify that the target CI project exists (otherwise, create it)

3

Fetch template DSParams

mettleci remote download (documentation)

Download the target DataStage environment’s template DSParams file (typically located at <install-directory>/IBM/InformationServer/Server/Template/DSParams) to the build agent’s working directory.

4

Create artefacts directory

Inline script

Verify that the build directory artefactsexists (otherwise, create it).

5

Fetch diff

Build tool specific

Fetch the previously-generated DSParams.diff into the build agent’s working directory.

6

Generate merged DSParams file

mettleci dsparams merge (documentation)

Replace variables in the DSParams template file with values defined in the DSParams.diff file.

7

Configure properties

mettleci properties config (documentation)

Replace variables in the DSParams file, any Parameter Sets files, and any user-define shell scripts (from repository paths /datastage or /filesystem) with values defined in the var.ci properties file. Processed files are stored in a newly-created config directory.

8

Upload config files

mettleci remote upload (documentation)

Upload the config directory containing files modified by the Configure Properties step and your repository’s filesystem directory from the build agent to your target CI environment.

9

Execute deploy script

mettleci remote execute (documentation)

Execute the script config/deploy.sh on your target CI environment. This user-supplied script performs solution-specific deployment processes required by the assets in the filesystem directory.

10

Deploy DataStage assets

mettleci datastage deploy (documentation)

Perform an incremental deployment of your DataStage assets to the target CI project.

11

Cleanup

mettleci remote execute (documentation)

Execute the cleanup.sh (previously generated by the Configure properties task) in the target CI environment. This script removes files which are no longer required from your build agent’s disk.

12

Check for results

Inline script

Detects and records (in a build pipeline variable) whether the Job compilation performed by the Deploy DataStage assets task produced an output JUnit .xml file. Note that the incremental deployment approach used by MettleCI may mean that a test output file would not be generated in the event that the deployment process does not detect any changes.

13

Publish compilation results

Build tool specific

If a JUnit .xml file was detected by the Check for results step then publish it to the build tool for use in test suite management.

1

Download artefact

Build tool specific

Downloads any artefacts declared within this build pipeline in case they are required by subsequent steps. This task is often automatically introduced by the build tool.

2

Run CCMT Upgrade Stages

mettleci datastage ccmt

(documentation)

The MettleCI datastage ccmt command…

• Invokes the IBM CCMT command line utility to incrementally convert deprecated legacy Database Connector stages to modern equivalents,

• Interprets the output of the CCMT utility,

• Compiles any jobs that were updated by the CCMT utility,

• Produces, if appropriate, a JUnit .xml results file to enable your build pipeline tool to respond to the outcome of the CCMT upgrade.

3

Check for results

Inline script

Detects and records (in a build pipeline variable) whether the upgrade performed by the Run CCMT Upgrade Stages task produced an output JUnit .xml file. Note that the CCMT may have not found any relevant artefacts to convert in which case a test output file would not be generated.

4

Publish CCMT upgrade results

Build tool specific

If a JUnit .xml file was detected by the Check for results step then publish it to the build tool for use in test suite management.

1

Upload config files

mettleci remote upload (documentation)

Upload the file datastage/cleanup_unittest.sh from the build agent to your target CI environment.

2

Upload unit test specs

mettleci remote upload (documentation)

Upload your repository’s unittest directory from the build agent to your target CI environment.

3

Configure properties

mettleci properties config (documentation)

Replace variables in the cleanup_unittest.sh file with values defined in the var.ci properties file and store the processed file in the config directory.

4

Create unit test report dir

mkdir unittest-reports

Create directory to receive unit test outputs.

5

Run MettleCI Unit Tests

mettleci unittest test (documentation)

Invoke the Unit Tests defined in your unittest directory. Note that MettleCI will use its incremental detection functionality to only execute Unit Tests where either the Unit Test case or associated Job have changed.

6

Download unit test reports

mettleci remote download (documentation)

Download any JUnit .xml files produced by the Run Unit Tests task from your target CI environment to the build agent.

7

Cleanup

mettleci remote execute (documentation)

Execute the script stored in the repository file config/cleanup_unittest.sh (previously generated by the Configure properties task) in the target CI environment. This script removes unit test results, to ensure a clean environment for the next execution.

8

Check for results

Inline script

Detects and records (in a build pipeline variable) whether the Compliance check performed by the Run Compliance task produced an output JUnit .xml file. Note that the incremental deployment approach used by MettleCI may mean that a test output file would not be generated in the event that the deployment process does not detect any changes.

9

Publish compilation results

Build tool specific

If a JUnit .xml file was detected by the Check for results step then publish it to the build tool for use in test suite management