Each Compliance rule can be augmented with additional rule metadata through the use of Attributes and Tags, each of which is described here.

Rule Tags

Each MettleCI Compliance rule can include extra metadata using a set of user-specified values which are referred to as ‘tags’. Each Compliance rule may specify zero or more ‘tags’ which are free-form text labels associated with each rule.

Rule Tagging provides improved rule metadata management in support of a number of use cases:

• Identifying the ‘severity’ of each rule. The MettleCI tools that use Compliance results (the Workbench user interface and the mettleci compliance test CLI command) need to know how to respond to Compliance Rule breaches. Whether a rule’s breach should produce an abort (Fail) or informational (Warning) in Workbench or CI pipeline is currently defined (somewhat awkwardly) by the folder within which the rule exists in the repository. For NextGen this metadata would be better defined using tagging.

• Grouping Compliance Rules into ‘bundles’ of functionally related rules. This could enable users to report or test by functional area. Functional groups into which Compliance Rules could be bundled might be Performance, Security, Maintainability, etc. Tagging also permits a single rule to be associated with multiple functional areas, if required.

• Enabling the fine-grained sharing of rules across teams within organisations (i.e. tags could be used to identify which DataStage teams they apply to)

• Defining rules’ behaviour in different environments (e.g. Workbench vs. CI/CD plans)

Some notes on Tag behaviour:

• Tags are case insensitive.

• We strongly recommend that the values your use for Tags employ only alphanumeric characters (0-9, a-Z, A-Z) as we can’t guarantee the support for non-alphanumeric tags across all potential platforms and use cases.

• The default behaviour of not specifying any ‘Include’ tags is that everything is included.

• The default behaviour of not specifying any ‘Exclude’ tags is that nothing is excluded.

• When no ‘Include’ and ‘Exclude’ tags are specified Compliance Rules are not scanned recursively (existing behaviour).  When at least one tag is specified (either Include or Exclude), rules are scanned recursively.

• You can use * to match all tags so you can use * to include everything and leave Exclude tags blank. This will trigger recursive scanning of rules (different from leaving both blank which results in the existing, shallow, non-recursive behaviour).

The example CI/CD build pipelines that ship with MettleCI demonstrate the use of tags to identify which rules inhibit the successful completion of CI.

Rule Attributes

Here’s an example of a rule definition which incorporates some Tags.

# Rule attributes
package datamigrators

# Rule tags (effectively user-defined, free-form attributes)
@Tag(“security”)            # This rule identifies a potential security vulnerability
@Tag(“portability”)         # This rule identifies a issues with assets' portability between environments
@Tag(“maintainability”)     # This rule identifies a potential maintainability issue
@Tag(“CorpDataWarehouse”)   # This rule is specific to the 'CorpDataWarehouse' team
@Tag(“fail-ci”)             # This rule is mandatory and so should fail continuous integration if breached

# Rule definition
<blah blah blah>

Attributes are used to add extra information to the Rule which are used by the various MettleCI tools (Workbench, CLI) to change their behaviour.

Include and Exclude options for Compliance operations

The various MettleCI tools which use the Compliance Rule library permit the filtering of the rules which they use by ‘positive’ and 'negative' tag inclusion.

The Compliance Test command (available within the Compliance namespace of the MettleCI Command Line Interface) permits you to use -include-tag and -exclude-tag options to filter the rules which will are used by the command. The way that MettleCI interprets these options means that set of include tags is used first, then the set of exclude tags is removed. In the diagram above, only the rules with tags in set A and NOT in B will be used to select Compliance Rules (formally referred to as the 'relative complement of B in A')

Similarly, the MettleCI Workbench permits you to specify which include and exclude tags will be used when testing your Job’s Compliance interactively from within the Workbench interface.