MettleCI DataStage Transmuters

This MettleCI capability takes a specified set of input ISX files and modifies them based on instructions provided in a supplied YAML file. The first public interface to the Transmuter function will be made available from the MettleCI command line.

Command

From command-shell, run asset-transformer transform with these parameters:

isx: path to an ISX archive or directory containing archives
user: DataStage user to credit changes to
transforms: path to a YAML file describing transforms to perform

YAML file

This file should contain a list at the top level.
Each item in the list should have a property transform, containing the name of
a transform, and other properties required by that transform.

E.g:

 - transform: AppendToAttribute
   xpath: //contains_JobObject[@stageType]/@name
   appendString: _I_AM_STAGE

 - transform: PrependToAttribute
   xpath: //contains_JobObject[@stageType]/@name
   prependString: MY_NAME_IS_

Transforms

The functions from which Recipes will be built are described below:

Transform	Description	Parameters
AddJobParameter	Adds a named parameter to one or more specified jobs	Larry Pieniazek (Unlicensed) to confirm
RemoveJobParameter	Removes a named parameter from one or more specified jobs	parameterName: The name of the parameter to remove
AppendToAttribute	Append a string value to an attribute.	xpath: Path to the attribute in the XML to modify appendString: String to append to the current value of the attribute
PrependToAttribute	Prepend a string value to an attribute.	xpath: Path to the attribute in the XML to modify prependString: String to prepend to the current value of the attribute
SearchAndReplace	Perform a regex search/replace on a text node (i.e. attribute or element with text content). If an element with child elements is provided by the xpath, the child elements will all be removed by this operation.	xpath: Path to the text node in the XML to apply search and replace searchString: Regex search string replaceString: Regex replace string
InsertElement	Adds an XML element as a child to a specified parent element.	xpath: Path to the parent node. The provided element will be a child of this node element: XML element to insert (as a string)
InsertAttribute	Adds an attribute to an XML element. If the attribute already exists it will be overwritten.	xpath: Path to the element to add attribute to attributeName: name of the attribute to add attributeValue: value of the attribute to add
RemoveElement	Removes an XML element from the document.	xpath: Path to the element to remove.
RemoveAttribute	Removes an attribute from an existing element.	xpath: Path to the element the attribute is in. attributeName: Name of the attribute to remove.

Recipes

Name	Config File	Description
Change prefetch memory	`recipes/change_prefetchmemory.yaml`	Change the prefetch memory setting for an ORAOCI9 stage to 102,400,000
Add unit test parameter	`recipes/add_unittest_parameter.yaml`	Add the $DM_ENABLE_UNIT_TESTING parameter to all jobs in the target folder structure
Remove unit test parameter	`recipes/remove_unittest_parameter.yaml`	Remove the $DM_ENABLE_UNIT_TESTING parameter from jobs in the target folder structure that have it
Change library path and file type	`recipes/changeStaticLibToDynamicLibFilename.yaml`	Search the modulePath attribute (of parallel routine definitions, typically) for a given pattern, and replace it with a different directory path, keeping the base name of the file but changing the extension from .o to .so. This changes a statically linked routine to dynamic, helping ensure coherence of routine definitions and routine implementations. Shows how to use the SearchAndReplace transform.
Fix ODBC SQL placeholder issue	`recipes/fix_odbc_sql_parameter_issue.yaml`	This recipe fixes an issue where the following warning is displayed when migrating an ODBC stage using CCMT: [WARN] WARNING: The number of placeholders in the UPDATE statement does not match the number of key flow variables so placeholder substitution will not occur. NOTE: This warning can occur due to a genuine bug in the job in which case this recipe will not fix it. This is a fix for a specific, unusual scenario (described in the yaml file).