Preparing your Azure environment

MettleCI ships with a sample Azure DevOps repository which includes a number of example pipelines definitions as well as a shell script which demonstrates how to use the Azure command line to establish the requires assets in your Azure environment. This page describes the contents of that file.

Collecting your required information

1. Connectivity

   export ADO_USERNAME=myuser.name@mycorp.com # Your Azure DevOps username export ADO_PASSWORD=MyPassword # Your Azure DevOps password export ADO_ORGANIZATION_URL=https://dev.azure.com/mettleci # Your Azure DevOps organization's URL export ADO_PROJECT=MyProject # Your Azure DevOps project name (this project doesn't need to pre-exist)

2. Git repository references

   export ADO_REPOSITORY_DATASTAGE=MyAzureDataStageRepo export ADO_GIT_SOURCE_URL_DATASTAGE=https://dev.azure.com/myorg/my-template-project/_git/MyAzureDataStageRepo export ADO_REPOSITORY_COMPLIANCE=MyAzureComplianceRepo export ADO_GIT_SOURCE_URL_COMPLIANCE=https://dev.azure.com/myorg/my-template-project/_git/MyAzureComplianceRepo

3. Variable groups

   export ADO_VARGROUP_NONPROD=MyProject_NONPROD # The name of the variable group for variables describing your non-production environment export ADO_VARGROUP_PROD=MyProject_PROD # The name of the variable group for variables describing your production environment

Create your Azure Project and configure your CLI environment

1. Connect to your Azure DevOps instance by opening a browser-based interactive authentication session:

2. Create an Azure DevOps project to host your DataStage assets and Azure build pipelines:

3. Set the default organisation and project for the Azure CLI environment:

Agents and Agent Pools

Agent pools can be created easily using the Azure DevOps UI. Neither the number of pools you create, nor the names you give them, are relevant to your MettleCI-enabled pipelines as jobs are automatically assigned to agents by Azure DevOps which matches the demands of each of your pipelines' steps with the capabilities advertised by your agents.

The definition of Agents requires you to install one or more self-hosted Azure agents on a suitably equipped host (see https://learn.microsoft.com/en-us/azure/devops/pipelines/agents/windows-agent?view=azure-devops) and associate the agent(s) with a relevant agent pool.

1. List your agent pools:

2. Select (or create) your preferred agent pool and assign its ID to $ADO_AGENT_POOL_id (for subsequent use):

3. List your agent pools' constituent agents:

Open

Environments

The creation of Deployment Environments is not currently supported by the Azure CLI. Environments are created by the supplied pipelines as they are references. i.e., if you try and run a MettleCI deployment to an environment called MyQualityAssurance then environment of that name will be automatically created. Once a deploy environment has been created (either manually using the Azure UI or automatically by reference from a pipeline) you can then configure its Approvals and checks settings to restrict deployment to that environment as required.

Repositories

1. Create DataStage repository:

2. Import DataStage repository from source repository ($ADO_GIT_SOURCE_URL_DATASTAGE):

3. Create Compliance repository:

4. Create Compliance repository & import from source repository ($ADO_GIT_SOURCE_URL_COMPLIANCE):

Variable Groups

This example creates two variable groups: one containing all the variables required to specify the details of your non-production environment a similar one for your production environment. You’ll need one of these variable groups for each of your distinct DataStage environments. Each variable group also makes use of passwords stored secret variables.

Pipelines

1. Create a DevOps Continuous Integration pipeline:

2. Create a Hot Fix Continuous Integration pipeline:

3. Create a Hot Fix Deploy pipeline:

Next Steps

1. Register your DataStage development project in the MettleCI Workbench referencing your Azure DevOps DataStage and Compliance repositories.

2. Test your Compliance connection by running compliance using MettleCI Workbench.

3. Commit a DataStage asset to your new DataStage repository using MettleCI Workbench. This should trigger your new MyDataStage-DevOps-CI pipeline.