Why don't Before/After job routine failures cause a Unit Test failure?

DataStage jobs may have Before or After routines associated with them which, as the name suggests, are executed either before or after the job’s ETL process is performed. These routines may serve a variety of reasons, such as establishing one or more pre-conditions for the job, or logging details about the job’s execution, for example. These routines generate a return code which indicates its success or failure - for example, the file system used to write job logs is full, or the invoking user doesn’t have the requisite permissions.

If a job is invoked in MettleCI’s unit test mode (whether invoked manually from Designer or by a build tool) then the result of the unit test is considered successful if the job produces output data that matches the expected output data defined in the test, regardless of the success (or otherwise) of any Before and After routines configured on the job.

Why does the MettleCI Unit Test Harness ignore Before and After routines when determining the unit test result?

To understand why, it is instructive to consider the components involved in executing a parallel job, and their interrelationship. UniVerse (the DataStage BASIC engine) orchestrates the execution of a DataStage job and invokes any Before job routines (whether built-in or user defined) first. Once all Before routines complete successfully, UniVerse then invokes OSH - the Orchestrate Shell - to execute the ETL job itself. This is the point at which the Unit Test Harness gains control. It checks whether the job is being run in one of the testing related modes (intercept or test) and, if so, modifies the job’s OSH script to insert the components appropriate to the mode being specified, then executes the job by passing control back to the OSH process. After the job finishes, OSH returns control to UniVerse which, if the job didn’t abort, will invoke any specified After job routines.

Note that the success or failure of a job (did the job complete without aborting?) is different to the success or failure of a unit test (did the job transform the test’s inputs to the expected outputs correctly?) It will sometimes be the case that a job completes execution successfully but that one or more of its unit tests fail. In some cases a Before job routine will cause a job abort and the unit tests may never get a chance to execute at all. Alternatively, the Before job routines may succeed, the unit test OSH process runs uneventfully, but an After job routine aborts - again causing the overall jobs to fail, giving rise to a situation in which the Unit Test succeeded but the job itself did not.

Because of the scope within which the Unit Test Harness exists - as one of the children of the UniVerse orchestration process - it can have no awareness or control of what UniVerse’s other child processes are doing. The ‘unit’ being tested is the job itself, not the overall execution package.

MettleCI Documentation

Why don't Before/After job routine failures cause a Unit Test failure?

Why does the MettleCI Unit Test Harness ignore Before and After routines when determining the unit test result?

For More Information