...
Each format occupies varying amounts of disk space, with the design-only ISX format occupying the least. By way of example here is a set of exports (of a simple 3-stage Parallel Job) using various formats, order by increasing file size:
...
You’ll see that the ISX design-only format occupies the least space.
...
Why does MettleCI use
...
ISX files?
When performing a Git commit MettleCI uses the ‘default’ flavour of the ISX format with only design information. Executable information is deliberately excluded. (I’ll link to the “But how do we do binary-only deployments to PROD?” FAQ here) MettleCI provides the mechanisms for committing DataStage artefacts into to a Git repository, testing them against Compliance Rules and Unit Tests, and deploying them to downstream environments, including the swapping of . It’s also possible to swap Jobs' parameter values to those appropriate to each target environment. All Many of these operations relay on the unique capabilities provided by the ISX format. When performing a Git commit MettleCI commits and pushes files using the ‘default’ flavour of the ISX format, and which only contain design information. Executable information is deliberately excluded but can be reconstituted further down the delivery pipeline for organisations who have a mandate which prevents the compilation of code in Production environments. See Deploying DataStage Binaries for more details.
We want to be able to DIFF DSXs to compare versions in version control, or between environments. WE can’t do that with ISXs
The format in which DataStage artefacts are stored in Git only matters to a customer when they want to do something with Does the re-compilation of jobs in each environment compromise good deployment governancethose artefacts outside of what can be achieved using MettleCI’s capabilities, most commonly accessed using its Command Line Interface. Many users have an historical attachment to DSX files for a variety of reasons: DSX is ASCII based which, despite being
Benefit | Reason | MettleCI Mitigation |
|---|---|---|
ASCII is easily 'inspectable' | ||
ASCII format means DSX files can be subjected to traditional test-based diffing techniques to identify the changes between code versions | While DSX’s may be human readable they are certainly not human parsable. | MettleCI |
Can be modified by scripts (DSX Split?) | ||
Can be modified by scripts (Parameter updates) | ||
Can be parsed for anti-patterns or security violations | MettleCI provides a library of Compliance Rules to scan for coding anti-patterns, standards non-compliance, security violations, and code maintainability issues. these rules are easily modified or extended as necessary. | |
The perception that ISX files are opaque binary files to which existing build and deployment processes and can include the compiled version of the job.
File Contents
Treat the export as opaque
...
Git will still be able to perform a merge, but if two different versions of the same DataStage export are detected Git will report a conflict and the versions will need to be manually merged. Since DataStage does not include any tools for merging two DataStage exports the only way to resolve a conflict is for a developer to ‘eye-ball’ each version in the DataStage user interface and manually construct the merged version. This process is time consuming, tedious, and error prone.
A desire to use DSXs stems from:
The perception that ISX files are opaque binary files to which existing build and deployment processes and can include the compiled version of the job.
We use a custom DSX processing script to split DSXs or to swap parameter values appropriate to the environments into which they’re being imported.
We want to be able to DIFF DSXs to compare versions in version control, or between environments. WE can’t do that with ISXsMettleCI’s use of design only assets Some customers have security constraints meaning that only executable assets can be
Conclusions
MettleCI means that the file format used to store your jobs in Git version control doesn’t matter. This is because:
...