The following diagram is one possible configuration, and is presented as an example for further architecting. The number of IS environments, for example, and how many tiers they use, is notional
| Gliffy | imageAttachmentId | att1088356353
|---|
| Table of Contents |
Topology Components
Jenkins Host: Where the Jenkins controller (providing build/deploy services as part of ALM) is run
Application Lifecycle Management Tools: One or more hosts running Work Item Management, and Git services
Developer's Workstation: Where the Windows DataStage Designer client is typically run
Information Server Development Environment: Your development instance of Information Server, which may be deployed in any topology, and on any number of hosts.
Other Information Server Environment(s): Downstream Information Server environments, including testing and (optionally) Production. These environments can be MettleCI deployment targets without requiring the deployment of any MettleCI components.
MettleCI Agent Host: A MettleCI-dedicated Windows server hosting an IBM DataStage Client tier which is used by your Build system's agent, in conjunction with the MettleCI Command Line Interface, to automate build and deployment activities.
Topology Connections
The following high-level MettleCI architecture shows the key software components to be installed, and their communications between hosts.
| Gliffy | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
...
MettleCI Workbench Service (Engine Tier) ↔︎ Your Work item Management system
...
|
...
#
...
Summary
...
Description
...
Technical
...
1
|
The above diagram is one possible configuration, and is presented as an example only.
The MettleCI Workbench application running on your DataStage Engine tier needs to perform a dynamic lookup of
...
Work items when displaying the Git Commit
...
page.
The MettleCI Workbench
...
application running on your DataStage Engine tier needs to
...
commit to
...
2
...
MettleCI Workbench Service (Engine Tier) ↔︎ Your Git Repository
...
3
...
Jenkins Host ↔︎ Your Git repository
...
Jenkins can keep pipeline definitions (as jenkinsfile files) in a repo so they can be versioned the same as the artifacts
...
4
...
Jenkins Host ↔︎ DataStage designer client (developer workstation)
...
Web access to Jenkins by developers
...
5
...
Jenkins Host ↔︎ DataStage designer client (MettleCI host)
...
Jenkins controller issues commands to the Jenkins Agent to perform MettleCI related tasks such as code movement, and DataStage deployment (project manipulation, job compliation)
...
6
...
Developer Workstation ↔︎ IS/DS Service/Engine tiers
...
Normal developer activity such as editing jobs, test compilations, etc
...
7
...
8
...
9
...
your Git platform.
The Jenkins Host and your Git repository need to communicate in order to configure Git commits to automatically trigger Jenkins pipelines. Jenkins then needs access to your Git repository to source the artefacts for build and deployment. The Jenkins pipeline definitions ('Jenkinsfiles') are themselves stored and managed in a Git repository.
The Developer Workstation provides data engineers with access to your ALM tools' user interfaces via a supported web browser.
The Developer Workstation provides data engineers with access to your Jenkins user interfaces via a supported web browser.
The Developer Workstation provides a DataStage client tier with access to the development environment's DataStage Engine and Services tiers.
Your Jenkins pipeline perform their duties via an agent installed on the MettleCI Host. (Read about Jenkins ports and services here)
The MettleCI Host requires regular access to the development environment's DataStage Engine and Services tiers, identical to that of Developer Workstations.
The MettleCI Host requires regular access to the downstream test environments' DataStage Engine and Services tiers, to affect automated deployment.
Using Multiple Engines and Multiple Jenkins Agents
In real-world scenarios it’s possible that security restrictions or network zoning prohibit a single MettleCI Agent Host from communicating with DataStage environments across development, testing, and production domains. In this case you may need to consider a deployment topology using multiple Jenkins Agents, each running on a dedicated MettleCI Agent Host.
| Gliffy | ||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
The diagram above (explained in more detail on this page) describes how you might deploy Jenkins Agents in complex multi-environment DataStage topologies.
This is by no means the only way to support multiple DataStage environments, but is certainly one of the easiest solutions to understand and maintain.