A MettleCI-enabled build and deployment GitLab pipeline MettleCI depends on a GitLab Runner. This is the agent that GitLab pipelines use to invoke the MettleCI command line. It must be installed as a Service on a Microsoft Windows host because the DataStage compilation process (v11.7 and earlier) still requires the use of Windows-based components. Refer to the diagram in https://datamigrators.atlassian.net/wiki/spaces/MCIDOC/pages/373424141 for details of the agent relationship to the overall architecture. Note that you must use a self-managed runner, as GitLab.com runners will not have the necessary prerequisite software.

The Runner install and configuration process has changed over time so this page doesn’t reproduce GitLab’s documentation. Instead it provides an overview of the process and some things to take into account.

This material is abridged from the GitLab documentation, which is authoritative.

Process Overview

The process breaks down into two major phases, 1) preparing the Runner’s Windows host and 2) installing/configuring the Runner

Setting up the Runner’s environment on a Windows Server host (all GitLab versions)

The first part, preparing the Runner’s Windows host, will be similar regardless of how you carry out the installation/configuration, and has the following steps:

• Configure and provision the agent Windows host machine (virtual or physical). This will typically use a Windows Server operating system.

• Determine or create the account under which you will run the agent, and if necessary (in the case of a user account) obtain the credentials for it. It should be possible to run as a user or system account.

• Install and test Git (See: https://git-scm.com/download/win ).

• Install and test the MettleCI CLI. (See: https://datamigrators.atlassian.net/wiki/spaces/MCIDOC/pages/488898631 ).

Installing and configuring the Runner

This process has changed a number of times over the last several releases, so these installation notes only discuss areas of particular concern for Runners used by MettleCI enabled pipelines.

A GitLab Runner is an kind of agent, and like any other agent, it is a service running on a (typically) remote machine that is in communication with the ALM server and carries out tasks as directed. Like any other agent, a GitLab Runner needs software installed on the host, although how it gets there varies by release and user preference. Like any other agent, the GitLab runner needs to be associated with the GitLab server that will be issuing requests for task execution, although how this registration happens varies from release to release. You should start at the top of the documentation tree (See: ) and follow the instructions carefully, while keeping the following points in mind. (even the order of what you do, install software and register the Runner, can vary from one version to the next, so these points are deliberately vague)

• Ensure that you obtain the Windows version of the Runner software image. Other types of runners (Linux, container hosted etc) are not suitable.

• Create the home location for your runner, such as C:\GitLab-Runner, with appropriate permissions, and place the binary in this location. If necessary, rename or copy the downloaded file from gitlab-runner-windows-amd64.exe (or whatever it may be, as it may have version information embedded in the name as well) to gitlab-runner.exe before running the other commands. (the rest of this material and the source GitLab documentation both assume you have done this). Note that you will need to create additional directories if you have more than one runner on the same machine.

• It is convenient, if possible, to access the GitLab download page, and the GitLab Server GUI from the Windows Server machine where you will host the Runner, as that way the binary doesn’t have to be copied to elsewhere, and the instructions that follow can be directly executed.

• When obtaining a token, use the newer “Runner Authentication Token” rather than the older “Registration Token”. The newest versions will not even offer you Registration Tokens as they have been obsoleted. For this reason, stick to the newer one to keep your instance future-proof. You will typically obtain the token by visting the GitLab console’s Admin Area > Runners page (for example: https://gitlab.mycorp.com/admin/runners ) Make sure you record the token you obtained.

• During the registration process, you may asked for various items of information:

  • If you are asked what type of runner executor you want, choose shell as this supports running MettleCI CLI commands . A subsequent configuration step will allow you to choose Bash, PowerShell or Batch. (there are several container executors available but these are untested with the MettleCI CLI)

  • If you are prompted for tags, we suggest you give the Runner some meaningful tags. The reference implementation pipelines we supply references these tags: mettleci, datastage11.7 so those (or similar) are good choices.

  • If you are prompted for a description, we suggest you enter meaningful text. (take into account what other runners you have already) We suggest you do not encode version or machine location/name information here.

  • You may be asked what scope you want the runner to have. There are three types of runners, with varying scope; instance, group and project. You may wish to start with instance runners, as these are the most flexible and can be used by any project or pipeline. We suggest you only add more specific scope runners if your workload increases substantially or you need to segregate operations.

• Once the Runner has been registered, check that you can see the runner status from the GitLab console. Use the commands provided in the documentation to install and start it.

• You should be able to see the runner installed as a service on the Windows Server, and to see the runner status from the GitLab console. You can now use the runner in your pipelines.

If you have an existing Gitlab installation, you probably have some Runners already. You’ll want to disambiguate things so that the pipeline runs in the correct place, on the runner that has the MettleCI CLI and is compatible with the destination version of DataStage (for RDU pipelines you will have at least two runners, one for the legacy system and one for the target, on different servers). Tags are used to do this. and you added some tags during the install process. You can use the Runner admin page (for example, this URL: http://gitlab.mycorp.com/admin/runners ) or your project’s CI/CD settings to configure this as necessary.

You can also narrow the scope of a Runner from global (“instance” or shared with all projects on the server) to group, or to project. Note that once you make a Runner “specific” you cannot make it shared again, you can only narrow the scope.

If you need to delete a runner, it can also be accomplished from the runner admin page ( for example, this URL: http://gitlab.mycorp.com/admin/runners ).