Document toolboxDocument toolbox

Sequential File Stage

Summary

The Sequential File stage reads data from or writes data to a sequential file.

IBM Documentation

Server Jobs: Sequential File Stages - IBM Documentation

Parallel Stage: Sequential file stage - IBM Documentation

Conversion Notes

  • The Server Sequential File provides an option to handle fixed-width files with spaces between columns. S2PX translates this as a Parallel Sequential File with a custom delimiter defined as the appropriate number of spaces.

  • Note that if the last row in a Sequential file does not end with the specified row delimiter then the generated Parallel Job will unavoidably produce one or more Warnings in its Job log.

Structural changes

  • Split Inputs (?): A Sequential File input (read) stages with multiple output links will be converted into multiple read stages, each with a single output link

  • Split Outputs(?): A Sequential File output (write) stages with multiple input links will be converted into multiple write stages, each with a single input link

See Parallel Job Structural Differences.

Server features not supported

Feature

Asset Query (?)

Comment

Feature

Asset Query (?)

Comment

Named Pipes

Sequential File Uses Named Pipes

Not supported.

This property specifies that the stage uses the named piped facility. Named pipes can improve the performance of jobs with a large degree of inherent parallelism. This performance optimisation strategy is not relevant to the Parallel engine which is already inherently … Parallel.

 

See IBM documentation here.

Suppress row truncation warnings

Sequential File with Suppress Truncation Warnings

Not supported.

Parallel doesn’t support the logging of truncation warnings. The Parallel job will not show truncation warnings, even if the original Server job did.

Missing columns action

Sequential File Incomplete Column Handling

Server and Parallel Sequential File stages differ in their behaviour when the file they read doesn't have all of the available file columns defined in the stage’s output tab.

In the Server Sequential File stage the Columns → Outputs link tab permits the setting of an ‘Incomplete Column’ for each each column.

Two of the options also make use of the additional ‘Missing columns action’ attribute to allow Job designers to provide further detail on desired stage behaviour.

In its default state, the Parallel version of the Sequential File stage discards rows with short or missing fields. This is the the equivalent of the ‘Discard and Warn' action, except it applies for every field in the Parallel stage, rather than on a field-by-field basis like it does in Server stages..

This query, therefore, identifies those columns in the output link of a Sequential File where the ‘Incomplete Column’ settings could produce results which differ from those of the Parallel Sequential File stage’s default settings.

Backup existing file

Sequential File with Backup

Not supported.

Not natively supported by the Parallel Engine and any no generic workaround available that would suit every use case.

Omit last new-line

Sequential File with Omit Last NewLine

Not supported.

Not natively supported by the Parallel Engine.

First line is column names

Sequential File incorrect column headers

Supported with caveat.
Column names will be listed, there will however be a slight difference with how the header is displayed.

Quotation marks will be around all column names rather than only around string based type names.

Server Header: DBL_COL,"STR_COL","TS_COL","DT_COL",DEC_COL,"CHR_COL"

Parallel Header: "DBL_COL","STR_COL","TS_COL","DT_COL","DEC_COL","CHR_COL"

 

© 2015-2024 Data Migrators Pty Ltd.