Document toolboxDocument toolbox

Using SSH Keys in MettleCI CI/CD Pipelines

By default, the example MettleCI CI/CD plans use passwords to perform remote actions, such as uploading or downloading files between the agent machine and the DataStage engine, or remote execution of scripts on the DataStage engine (all enabled by commands from within the MettleCI CLI Remote Namespace).

MettleCI may also use a SSH key (with or without a passphrase) instead of a password. The main steps you need to follow to use SSH keys in your CI/CD pipelines are:

  • Add two more variables to your CI/CD system. These variables should be stored in your CI/CD system in the same way as other sensitive credentials (e.g. in. a secrets store, etc.)

  • Expose those two new variables in a manner that enables them to be referenced within your pipeline.

  • Update the commands from the Remote Namespace to use those environment variables.

Example

To make this change in Jenkins, for example…

  • Add Jenkins credentials (as Secret Text, like our other credentials) for each SSH in use:

    • the path (on the Jenkins Agent) of the SSH key file. Note that there is a reference to the associated public key in the .ssh/authorized_keys file on the associated DataStage Engine host)

    • the passphrase, if one is being used

  • Add environment variables to relevant agents, that refer to the SSH credentials in the same manner as the example pipelines already do with IIS and MCI passwords, eg:

    • SSHKEYPATH for the path of the SSH key file

    • SSHKEYPHRASE for the SSH key passphrase, if used

  • In the JenkinsFile, add calls for SSHKEYPATH and SSHKEYPHRASE to withCredentials calls for:

    • the stage titled "Create DSParams Diff Artifact", where the remote download command fetches the template DSPARAMS file from the source DataStage instance

    • all stages that reference mci_deploy or mci_unittest

      For example:

      withCredentials([ string(credentialsId: "${env.IISPASSWORD}", variable: 'IISPASSWORD'), string(credentialsId: "${env.MCIPASSWORD}", variable: 'MCIPASSWORD'), string(credentialsId: "${env.SSHKEYPATH}", variable: 'SSHKEYPATH'), string(credentialsId: "${env.SSHKEYPHRASE}", variable: 'SSHKEYPHRASE') ]) { <commands here> }

 

  • Update all calls to Mettleci remote commands, replacing -password %MCIPASSWORD% with -privateKey \"%SSHKEYPATH%\", and also include -passphrase \"%SSHPASSPHRASE\" if a passphrase is required for the SSH key:

    • the remote download command in JenkinsFile, in the stage titled "Create DSParams Diff Artifact"

    • all MettleCI remote calls (remote upload, remote download, remote execute) inside the mci_deploy or mci_unittest shared library functions

      For example:

      bat label: 'Cleanup', script: ''' %AGENTMETTLECMD% remote execute ^ -host %IISENGINENAME% ^ -username %MCIUSERNAME% -password %MCIPASSWORD% ^ -script "config/cleanup_unittest.sh" '''

      becomes

      bat label: 'Cleanup', script: ''' %AGENTMETTLECMD% remote execute ^ -host %IISENGINENAME% ^ -username %MCIUSERNAME% -privateKey \"%SSHKEYPATH%\" ^ -passphrase \"%SSHKEYPHRASE%\" ^ -script "config/cleanup_unittest.sh" '''

 

For other CI/CD platforms, the way in which we organise settings will differ, but in each case the way we add settings for the SSH key will be the same as for how we currently manage passwords.

Updating the MettleCI remote calls to replace the -password argument with -privateKey and -passphrase is the single element that will be consistent for all platforms. Of course, the exact syntax used is specific to the platform, but the style of updates will be consistent with what is already coded.

© 2015-2024 Data Migrators Pty Ltd.