Summary

A ‘reject row’ is an incoming row which meets one or more conditions which are identified by the Transformer and cause it to be propagated to one or more dedicated reject links. The configuration of reject links in the Transformer Stage differs significantly between DataStage’s Server and Parallel Jobs and depends greatly upon the reason for the row being rejected. Rejects are identified by expressions in the constraint field of each of a Transformer’s output links. Each type of reject, and how they’re identified, is summarised below. Note that in these cases the ‘reject link’ must be identified in the Transformer by selecting the checkbox in the output link’s Reject Row column.

The manner by which these conditions are detected and handled in Server Jobs is described in detail on this IBM documentation page. The Parallel engine, however, provides alternative mechanisms for handling each of these these rejects scenarios, some of which require a degree of manual remediation. Server jobs which use reject links, and will consequently require some manual effort, are identified for you by the Transformer Has Reject Links asset query.

Example Manual Remediation

The following Server Job (called ‘Transformer_with_Rejects’) demonstrates two types of reject link behaviour in a server job. In Server Job, downstream stages which have the capacity to reject data - the Oracle Connectors in the example below - will notify their upstream Transformer of the rejection. This Transformer Transformer_1 then provides the rejection handling mechanism which, in this case, propagates those rows to the Rejects link. Note that in this example the reject link Rejects propagates not only rows flagged as rejected by the downstream Connectors but also rows rejected for non-technical reasons identified within the Transformer, perhaps because the row fails some business rule.

Open

The Parallel engine does not support the handling of reject links in the same manner as the Server engine so Jobs that contain this logic need to be identified and manually updated after conversion to a Parallel Job . In this example we’ll start by identifying those Jobs which utilise a reject-handling design pattern which requires restructuring for compatibility with the Parallel Engine. These jobs can be identified using the S2PX Analysis process:

Reviewing the analysis output spreadsheet tells us that the reject link rejects in our example job is configured to output data rows which have been rejected by two of the Transformer’s other output links:

Examining Transformer Transformer_1 we see the constraints against each output link:

Open

We can see that dataOut1 and dataOut2 each have business rule constraints determining their output, and a constraint determining which rows are propagated to the rejects link:

dataOut1.REJECTED - any row rejected due to…

• failing the business-rule constraint or

• rejected due to a write error in the downstream Connector

dataOut2.REJECTEDCODE - any row rejected due to…

• rejected due to a write error in the downstream Connector

There are two types of rejects being propagated, each necessitating different methods of remediation in the generated Parallel Job:

For the example Job shown above S2PX would deliver a Parallel Job with an identical appearance to the original Server Job:

Note that the following logic transformations have been performed:

• String comparisons in the output link constraints have been updated to ensure they replicate the behaviour of the original Server Job

• The constraint on the Reject link has been automatically converted by S2PX in an effort to render an expression compatible with the DataStage Parallel engine.

Despite the constraint on the Reject link being a valid Parallel expression the Job fails to compile because the .REJECTED and .REJECTEDCODE return codes are not recognised by the Parallel engine. These were deliberately retained by S2PX as a reminder that manual remediation needs to be performed.

To update the Parallel Job design to handle the reject rows we’ll modify the Job so that rather than configure the Transformer to identify and generate all rejects we’ll allow the Connectors which interface to the database to handle the rejects relevant to them.

Remediation Steps

Start by Creating new reject links from each of the output connectors which are detecting and generating reject rows:

Next, select the database error codes you wish to track (selecting all error codes is recommended), and append ERRORCODE and ERRORMSG to the fields in the link to improve future error handling diagnosis.

Update the original Rejects link with a constraint complementary to dataOut1, i.e.: StrCmp(dataIn.a, "Apple") <> 0. This now becomes a standard link that specifically outputs the rows that dataOut1 filters out.

With these changes applied the job will now behave as designed.

See also

• Transformer Has Reject Links asset query

• Defining Constraints and Handling Rejects - IBM Documentation

• Connector-specific documentation

  • Db2 - Configuring a reject link - IBM Documentation

  • ODBC - Rejecting records that contain errors - IBM Documentation

  • Oracle - Rejecting records that contain errors - IBM Documentation

  • Teradata - Rejecting records that contain errors - IBM Documentation