...

Each MettleCI Compliance rule can include extra metadata using a set of user-specified values which are referred to as ‘tags’. Each Compliance rule may specify zero or more ‘tags’ which are free-form text labels associated with each rule.

Tagging supports a number of functions:

1. Identifies which asset type(s) each rule applies to. Currently a single rule (e.g. ‘Database Connect Not Auto Generate SQL’) needs to exist as four files within the Compliance repository (‘.pjb.grm’, ‘psc.grm’,. ‘sjb.grm’, and ‘ssc.grm’) with each file effectively repeating the rule but with a different file extension to associate it with the asset to which it can be applied. Tagging will replace this mechanism by moving the name of the asset type(s) to which a file can be applied from file extension to tags defined within the rule, thereby removing the need for multiple, redundant versions of the same file to be maintained within the Compliance repository. This capability also easily enables the future application of Compliance to non-DataStage asset types.

2. Identifies the ‘severity’ of each rule. The MettleCI tools that use Compliance results (the Workbench user interface and the mettleci compliance test CLI command) need to know how to respond to Compliance Rule breaches. Whether a rule’s breach should produce an abort (Fail) or informational (Warning) in Workbench or CI pipeline is currently defined (somewhat awkwardly) by the folder within which the rule exists in the repository. For NextGen this metadata would be better defined using tagging.

3. Groups Compliance Rules into ‘bundles’ of functionally related rules. This could enable users to report or test by functional area. Functional groups into which Compliance Rules could be bundled might be Performance, Security, Maintainability, etc. Tagging also permits a single rule to be associated with multiple functional areas, if required.

...

...

Each rule specifies zero or more free form ‘tags’

Rule Tagging

...

provides improved rule metadata management

...

in support of a number of use cases:

• It enables the fine-grained sharing of rules across teams within organisations (i.e. tags could be used to identify which DataStage

  projects

  teams they apply to)

• Define rules’ behaviour in different environments (e.g. Workbench vs. CI/CD plans)

• The CLI (for use in build and deployment pipelines) also permits the selection of rules using positive and negative rule tag values

• This has been retrofitted to MettleCI for standalone DataStage

  • The Workbench permits the selection of rules using positive and negative rule tag values

  • Legacy CLI also updated to support tags

  • This sets the expectation that this capability is also available in Cloud Pak!

• Define rules’ behaviour in different environments (e.g. Workbench vs. CI/CD plans)

• Updated pipeline examples demonstrate the use of tags to identify which rules inhibit the successful completion of CI.

...

• Tags are case insensitive.

• We strongly recommend that the values your use for Tags employ only alphanumeric characters (0-9, a-Z, A-Z) as we can’t guarantee the support for non-alphanumeric tags across all potential platforms and use cases.

• The default behaviour of not specifying any ‘Include’ tags is that everything is included.

• The default behaviour of not specifying any ‘Exclude’ tags is that nothing is excluded.

• When no ‘Include’ and ‘Exclude’ tags are specified Compliance Rules are not scanned recursively (existing behaviour).  When at least one tag is specified (either Include or Exclude), rules are scanned recursively.

• You can use * to match all tags so you can use * to include everything and leave Exclude tags blank. This will trigger recursive scanning of rules (different from leaving both blank which results in the existing, shallow, non-recursive behaviour).

Include and Exclude options for Compliance operations

The various MettleCI tools which use the Compliance Rule library permit the filtering of the rules which they use by ‘positive’ and 'negative' tag inclusion.

You can use the The Compliance Test command (available within the Compliance namespace of the MettleCI Command Line Interface) permits you to use -include-tag and -exclude-tag options to filter the rules which will are used in your call to the compliance test by the command. The way the tags are considered that MettleCI interprets these options means that set of include tags is used before first, then the set of negative exclude tags is removed. In the diagram above, only the rules with tags in set A and NOT in B will be used to select Compliance Rules (formally referred to as the 'relative complement of B in A')

Similarly, the MettleCI Workbench permits you to specify which include and exclude tags will be used when testing your Job’s Compliance interactively from within the Workbench interface.