MettleCI provides build and deployment components that can query IBM Information Governance Catalog to understand the dependencies between your DataStage components. This dependency graph is then used to construct an execution schedule which can be expressed in a range of formats, each supporting different execution technologies. Job execution schedules can be generated as Shell scripts, Control-M schedules, DataStage Job Sequences, or IBM Workload Scheduler job stream, for example.
The existing IWS plugin only exposes a limited set of DataStage execution features to users. In particular, the inability to specify the DataStage Workload Manager queue on a job-by-job basis makes managing the workload on the DataStage server more problematic.
Data Migrators can supply an enhanced IWS plugin to integrate Information Server in a more capable fashion. Below is a comparison of the capabilities of IBM's existing IWS DataStage plugin against Data Migrators' offering.
DataStage Job Invocation Options | Supported in IBM's Existing IWS Plugin | Supported in Data Migrators' New IWS Plugin | Description |
|---|---|---|---|
mode NORMAL | Y | Y | NORMAL starts a normal job run (the default). |
| mode RESET | Y | Y | RESET resets the job. Note that the 'Reset if required |
| mode VALIDATE | - | Y | VALIDATE validates the job. |
| mode RESTART | - | Y | RESTART resumes a restartable job sequence from the last checkpoint by using the original job parameter values. |
Reset if required then run | N | Y | This option (unlike the others in this table) is not available as an option on the dsjob command line. It is currently implemented in the DataStage BASIC automatically generated when using a Job Activity in a DataStage Job Sequence . Data Migrators strongly recommend replicating this logic in the replacement IWS DataStage Plugin. |
| stop | N | Y | Stops a running job. This has a questionable use case in IWS, but should nevertheless be considered for support. |
| param name=value | Y | Y | |
| paramfile filename | Y | Y | |
| warn n | Y | Y | |
| rows n | Y | Y | |
| wait | Not applicable | Not applicable | wait - Waits for the job to complete. jobstatus - Waits for the job to complete, then returns an exit code that is derived from the job status. The existing IWS plugin always captures DataStage job return codes and automatically maps them to appropriate equivalent IWS job return codes. The new plugin should replicate this functionality. |
| jobstatus | Y | Y | |
| userstatus | N | N | This is a mechanism for returning non-standard completion codes for a job, but IWS mapping logic would interpret this as a failure. Data Migrators recommend that support for this feature is deliberately omitted to prevent the misuse of this legacy status-signalling mechanism. |
| local | N | N | Use this to permit the job to utilise environment variables defined as part of the shell configuration of the project engine. Data Migrators recommend that support for this feature is deliberately omitted to prevent the introduction of hard dependencies to environmental configuration., |
| opmetadata [ TRUE | FALSE ] | Y | Y | |
| disableprjhandler | Y | Y | |
| disablejobhandler | Y | Y | |
| queue | - | Y | The name of the workload management queue to which the job will be submitted. If omitted, the project default queue is used. If the job is a sequence job, all jobs in the sequence job are submitted to this queue. |
| useid | - | Y | Specify this if you intend to use a job alias (jobid) rather than a job name (job) to identify the job. The following command can be used to specify your own ID for an InfoSphere DataStage job: dsjob -jobid [my_ID] project job To identify a job invocation of a multi-instance job, use the format job.invocation_id. |
| project | Y | Y | |
| job | job_id | Y | Y |