Because Hashed File stages are not supported in Parallel jobs an alternative solution must be provided to support the use of Hashed File-stored data in a Parallel job. For each Hashed File stage encountered in a Server job S2PX will:

• Translate the Hashed File stage to its Parallel equivalent (a DRS stage), and

• Generate a new, entirely separate utility Server job which is intended for the one-off loading of Hashed File data into a DRS-compatible database table.

This page discusses the structure and usage of these Hashed File migration jobs.

Behaviour

These migration jobs will read data from Hashed Files and write them to database tables (the job will drop the table if it already exists, then create the table). There will be a one-to-one relationship between Hashed Files and tables. There will also be a one-to-one relationship between original jobs (that include Hashed File stages) and migration jobs.

See https://datamigrators.atlassian.net/wiki/spaces/S2PX/pages/2048131114 to understand how the database tables will be named and the structure of the data in the tables.

Each migration job will be wrapped in a Sequence that will include a single Activity Stage. The reason for this Sequence is to receive the parameters that the original job would have received and then supply the necessary parameters to the migration job. For example, the Sequence will generate and provide a couple of parameters to the migration job, which the job will use to set the database table name.

Structure

Migration job

Each migration job will comprise one or more separate ‘data flows’. Each flow will comprise a Hashed File stage with one or more output links; each output link will be followed by a Transformer (to massage the data being written into the S2PX format used for storage in Hashed File database tables) and then a DRS Connector. Here’s an example of one of these flows:

Open

Each Hashed file stage (with at least one input link) in the original job will cause the generation of a Hashed File stage (of the same name) in the migration job. The number of input links that the original Hashed File stage has will be the number of output links the Hashed file stage has in the migration job.

Sequence

The wrapper Sequence will simply include one Activity Stage that will call the migration job.

Open

Example Sequence Activity Stage for a Hashed File migration job

Usage

To migrate the Hashed Files each Sequence should be called in the same manner as the original job. If, for example, the original job was invoked once, with a single set of parameter values (or none at all), then the migration Sequence should be called once with the same set of parameter values. If, on the other hand, the original Server job was invoked multiple times, with differing sets of parameter values for each invocation, then the equivalent Sequence should be invoked once for each set of parameter values. This will ensure that all Hashed Files are migrated appropriately.

Examples