While most DataStage jobs can be tested by using MettleCI Unit Testing to replace input and output stages, some job designs will necessitate a more advanced Unit Testing configuration. This page outlines Unit Testing patterns that can be applied to test these job designs.
Input stage with rejects
The Input stage can be Unit Tested by including both read and reject links in the given clause of the Unit Test Spec. MettleCI Workbench will automatically detect this Unit Testing pattern during creation of new Unit Tests.
The CSV data specified for the rejects link should contain records used for Unit Testing the flow of records through the reject path(s) of the job.
given:
- stage: Input
link: Read
path: Input-Read.csv
- stage: Input
link: Rejects
path: Input-Rejects.csv
when:
...
Output stage with rejects
The output stage can be Unit Tested by including the write link in the then clause of the Unit Test Spec and the reject in the given clause of the Unit Test Spec. MettleCI Workbench will automatically detect this Unit Testing pattern during creation of new Unit Tests.
The CSV data specified for the rejects link should contain records used for Unit Testing the flow of records through the reject path(s) of the job.
given:
- stage: Output
link: Rejects
path: Output-Rejects.csv
when:
...
then:
- stage: Output
link: Write
path: Output-Write.csv
Stored Procedure stage
A Stored Procedure Stage will not only connect to an external Database for processing but it will also produce output records which are not deterministic. To Unit Test this job design, the Stored Procedure Stage needs to be removed from the job and replaced with Unit Test Data. This is done by adding the input link in the then clause of the Unit Test Spec and the output link in the given clause of the Unit Test Spec.
The CSV data specified by the given clause contains the data that will test the flow of records from the Stored Procedure stage. The data could simulate what would be produced by the real stored procedure if it had processed the Unit Test input records, however this is not a requirement.
given:
- stage: StoredProcedure
link: Output
path: StoredProcedure-Output.csv
when:
...
then:
- stage: StoredProcedure
link: Input
path: StoredProcedure-Input.csv
Classic Surrogate Key Generator stage
The classic Surrogate Key Generator stage will generate sequential keys from a given start value which is typically set via a Job Parameter. To ensure that the same surrogate key values are generated during unit test execution will always match the expected values, add a fixed value for the start value Job Parameter in the when clause of the Unit Test Spec.
given:
...
when:
job: KeyGeneratorExample
parameters:
START_KEY: 100
then:
...
Database or Flat File backed Surrogate Key Generator stage
TBD
Sparse Lookup stage
TBD