Multiple Processes
Unlike Parallel Jobs, input and output (passive) stages can act as both an input and output in the one Server job. When a job contains such a design, the Server engine will synchronize execution so that any load processes are completed before performing the read processes. For example, consider the following job design:
The processes used by the server job will be synchronized such to ensure that the stages grouped by the yellow annotation on the left are fully executed before starting the stages grouped by the annotation on the right.
To convert this job design to run on the PX engine, we’ll need to create a parallel job representing each of the yellow annotations and a sequence which runs each job in to mimic the synchronization provided by the Server Engine:
A similar challenge exists for shared containers that include stages that act as both a source and target. The same principles apply here but need to split the shared containers:
Jobs that use DSJobName
and DSJobController
macros will no longer produce the same output after applying this transformation. Any easy solution would be to have the sequence capture these values and pass them to the child jobs as parameters. Any existing use of DSJobName
and DSJobController
would then be updated to use the parameters