Issue

The Sequential File stage in Server offers far more granular support for missing or incomplete options than is possible in Parallel, The default value causes the job to abort, which is the most conservative default option to choose in Parallel. Any field that is set to a different option is listed in the result, leading to a review of the initial requirement and/or manual remediation.

Actions

Understanding your source data is key. As the available actions to handle incomplete data are less granular in Parallel jobs compared to Server jobs, decisions on how to handle data issues need to be made on a per-table level.

The default mode of operation on Parallel is to reject any rows containing incomplete data, which is the same as setting the Reject Mode property to Discard and Warn in the Server version (set at a Stage level.)

Adding the environment variable $APT_IMPORT_HANDLE_SHORT as a job property changes this behaviour to the Parallel equivalent of a Server job’s Retain and Warn setting, but affects the entire job.

See also

• Sequential File stage