Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Gliffy
imageAttachmentIdatt1144487979
macroIdaf0f0b4e-f3db-4baa-b899-aa780f91a860
baseUrlhttps://datamigrators.atlassian.net/wiki
nameMettleCI Topology (Jenkins)
diagramAttachmentIdatt1146126343
containerId1085767681

The above diagram is one possible configuration, and is presented as an example only.

  1. The MettleCI Workbench application running on your DataStage Engine tier needs to perform a dynamic lookup of Work items when displaying the Git Commit page.

  2. The MettleCI Workbench application running on your DataStage Engine tier needs to commit to your Git platform.

  3. The Jenkins Host the Git repository need to communicate in order to configure Git commits to automatically trigger Jenkins pipelines. Jenkins then needs access to your Git repository to source the artefacts for build and deployment. The Jenkins pipeline definitions ('Jenkinsfiles') are themselves stored and managed in a Git repository.

  4. The Developer Workstation provides data engineers with access to the Azure user interface via a supported web browser.

  5. Your Azure Pipelines perform their duties via an agent installed on the MettleCI-dedicated DataStage Client.

  6. The Developer Workstation provides a DataStage client tier with access to the development environment's DataStage Engine and Services tiers.

  7. The Developer Workstation provides data engineers with access to the following components via a supported web browser:

    1. OPTIONAL The MettleCI Scheduler (Azkaban) user interface

    2. OPTIONAL The MettleCI Wallboard  

  8. The MettleCI Host requires regular access to the development environment's DataStage Engine and Services tiers, identical to that of Developer Workstations.

  9. The MettleCI Host requires regular access to the downstream test environments' DataStage Engine and Services tiers, to affect automated deployment.

#

Summary

Description

Technical Notes

1

MettleCI Workbench Service (Engine Tier) ↔︎ Your Work item Management system

The MettleCI workbench is configured to perform a dynamic lookup of work items as part of the Git Commit process. Currently supported work item management systems are listed here. <where?>

The Workbench service on your DataStage Engine tier needs to be able to use your Work Item Management system’s query API, over whatever protocols/ports it requires.

2

MettleCI Workbench Service (Engine Tier) ↔︎ Your Git Repository

The MettleCI Workbench application running on your DataStage Engine tier needs to performs a commit to your Git platform

3

Jenkins Host ↔︎ Your Git repositoryJenkins can keep pipeline definitions (as Jenkinsfile assets) in a repository so they can be versioned the same as the artefacts.

4

Jenkins Host ↔︎ Developer Workstation DataStage designer client

Web access to Jenkins by developers. Similar web access should exist to the work item management and git repository

5

Jenkins Host ↔︎ MettleCi Host DataStage designer client

Jenkins controller issues commands to the Jenkins Agent to perform MettleCI related tasks such as code movement, and DataStage deployment (project manipulation, job compilation)

6

Developer Workstation ↔︎ IS/DS Service/Engine tiers

Normal developer activity such as editing jobs, test compilations, etc

7

Mettle CI Host ↔︎ DataStage Development Environment engine and service tiers

The MettleCI Host requires regular access to the development environment's DataStage Engine tier for initiating compilations, project creation, job compilation and <?> and access to the Services Tier for authentication

8

Developer Workstation DataStage designer client ↔︎ MettleCI Host

The (optional) plugins effect checkin, test case specification generation, and <?> via the Mettle CI host, further the developer can access the (optional) Azkaban scheduler and wallboard components via a supported web browser

9

Mettle CI Host ↔︎ DataStage downstream (test, prod, etc) engine and service tiers

The MettleCI Host requires regular access to the downstream test environments' DataStage Engine and Services tiers, to effect automated deployment (and possibly execute unit tests)

...