WIP - Avoiding NullPointerException when processing Compliance Rules 🔒

MettleCI Compliance Rules are designed for use against compiled DataStage Jobs, and are not designed to identify reasons why your incomplete or incorrect Job does not compile. Jobs which are malformed do not provide the traversable graph structure upon which Compliance Rules rely. Nevertheless, users sometimes inadvertently run Compliance against non-compiling jobs, and this will often produce a NullPointerException error message.

The following example produces an exception when accessing stage.XMLProperties.Usage.Session.SchemaReconciliation.FailOnTypeMismatch

def sFOTM = stage.XMLProperties.Usage.Session.SchemaReconciliation.FailOnTypeMismatch.toString();
if (sFOTM.substring(1,2) == "0") { // schema recon off
  return true
}

The underlying issue is that stage.XMLProperties is null, so evaluating null.Usage triggers a NullPointerException.

NullPointerExceptions can be prevented by ensuring the code does not access null values. There are scenarios, like the one illustrated above, where the DataStage Designer can produce a Database Connector stage with an incomplete XMLProperties field, which the out0of-the-box MettleCI Compliance Rules do not handle. Later versions of MettleCI Compliance Rules will handle this situation to make it clear that the Compliance system is behaving as expected, and that the error is caused by an incomplete jobs definition.

There are a few tools available within Compliance Rules which make it easy to avoid this type of problem:

• Change your graph traversal to ignore stages with a null XMLProperty. e.g. item.graph.V.stage.has('XMLProperties', T.ne, null).???

• From within a Pipe closure (e.g. sideEffect{}, filter{}, etc.), you can use the Groovy safe navigation operator to cancel the evaluation and immediately return null.

  • e.g. stage?.XMLProperties.Usage.Session.SchemaReconciliation.FailOnTypeMismatch.toString() would return null when XMLProperties is null.

• From within a Pipe closure you can use the Groovy Elvis operator to default a null value to something more usable. Using stage.XMLProperties?:'' will return an empty string whenXMLProperties is null. This could be combined with the Safe Navigation operator described above to produce a reasonable default string:  stage?.XMLProperties.Usage.Session.SchemaReconciliation.FailOnTypeMismatch.toString()?:''

In the example given above, calling stage.XMLProperties.Usage.Session.SchemaReconciliation.FailOnTypeMismatch will return either a single value or an array of values if it contains more than one value. This is why calling .toString() on a property with more than one entry results in a string like [item1,item2,item3]. Both arrays and single values can be accessed using the subscript [] operator.

The example given above would therefore be better implemented using the following expressions:

def sFOTM = stage?.XMLProperties.Usage.Session.SchemaReconciliation.FailOnTypeMismatch[0]
if (sFOTM == "0") {
   return true
}

The example above also appends a space at the start and end of the string, and uses a .toString() and sFOTM.substring(1,2) to strip off the square brackets ([]) which are present when an XML Property is an array.

Given that, we’d suggest:

def sFOTM = stage?.XMLProperties.Usage.Session.SchemaReconciliation.FailOnTypeMismatch
if (sFOTM == "0") {
   return true
}