MettleCI works with assets that are kept in Git repositories, notably the repository associated with a DataStage project and the repository associated with the compliance rules. Achieving usable a MettleCI includes the configuration of these repositories within GitLab. Note: GitLab differs from similar tools in its application of meanings to common terms.
Avoiding confusion over common terms in GitHub, Bitbucket, and GitLab
Despite Git being a well established standard, different vendors' Git-based version control systems can use terminology like Repository and Project differently, creating the potential for confusion.
In other platforms such as GitHub and Bitbucket, “repositories” contain the Git code repository, and other project-related assets such as issues, contribution metrics, etc. However GitHub users often use the terms repository and project interchangeably. In GitLab, the overarching container is called a Project. A Project includes the Git repository, issues, merge requests, milestones, and more.
It's important to make this distinction because you import a Project in GitLab, regardless of whether that is called a Repository elsewhere. In GitLab, the Repository is a component part of a Project. Thus GitLab tightly couples project and repository, and each project typically has at most one Git repository.
Note: One may keep MettleCI Compliance rules in the same repository as the DataStage assets but it’s generally recommended that a repository be dedicated to those Compliance rules to be used by multiple pipelines (across all your DataStage related GitLab projects). That means GitLab pipelines must be configured to explicitly clone that Compliance repository so it’s available when the pipeline reaches the Compliance execution step. The example GitLab pipeline provided with MettleCI reflects this dependency.
Steps
Create a blank GitLab project by selecting new project from your GitLab dashboard, or visiting http://your.gitlab.server/projects/new. The recommended practice is to create a new GitLab project for each DataStage project, plus one for your Compliance rules. You may want to collect your GitLab projects into groups. See GitLab - Create Project. <<< Broken link.
Clone the empty repository to somewhere convenient, such as your local DataStage client machine. Create a README file in the root folder of the repository which ensures the repository is initialized, has a default branch, and can be cloned. A sample README file is supplied in the template repository supplied with MettleCI.
For integration with MettleCI Workbench, with CI/CD pipelines, and with external entities in general, you will need to set up access to the repository associated with the project
Decide what credentials type you will use: userid/password, userid/PAT, deploy keys (a kind of SSH key) or regular SSH. SSH is recommended and this page has more details: Configuring MettleCI Workbench SSH Authentication to GitLab
Create a public/private key pair and then add it to the project. SeeAdding public keys to GitLab for more details <<< Redundant, given 2.a.?
Set up access between GitLab and the MettleCI Workbench service (typically running on your DataStage Engine host)
Obtain the “clone URL” for the project
Configure the project in MettleCI Workbench. See MettleCI Workbench Configure First Project for more details.
Test this setup by using MettleCI Workbench to run a compliance check (this tests read only access to Compliance) and perform a commit (this tests write access to the Project repository)
Set up access between GitLab and your build server. The easiest way to do this is to create a (project-specific or public) deploy key. Public keys can be used across multiple GitLab projects, once the project administrator grants access, while private keys only grant access to one project. You can also use a deploy token instead.
This page, while targeted at MettleCI Workbench setup, also covers how to create Deploy keys (Configuring MettleCI Workbench SSH Authentication to GitLab ). They are created the same way you create other SSH keys, and are actually a pair of keys, public and private.
Once you have a deploy key you will place the public part in the http://your.gitlab.server/your_project/-/settings/repository repository portion of the project settings page.
Deploy tokens are generated for you by GitLab at the same project repository settings page and you must record them for later use, you only have one chance to see them. This makes them both more secure and less convenient. See GitLab deploy tokens for more information.