Summary

The Complex Flat File ('CFF') Stage reads data from or writes data to a complex sequential file.

IBM Documentation

Server Jobs: Complex Flat File Stage - IBM Documentation

Parallel Stage: Complex Flat File stage - IBM Documentation

Conversion Notes

Supported CFF Version

Version 1 of the CFF stage is not supported by S2PX. Users will need to upgrade their Server Jobs to use the newer version of the CFF stage before using S2PX to convert Jobs container Server CFF stages to their Parallel equivalent.

Structural changes

No structural changes are introduced during Complex Flat File conversion.

See Parallel Job Structural Differences.

Data differences

Empty row in the CRLF data

When an ASCII CRLF data has an empty row, Server will output an empty record with empty data. PxCFF will skip the empty row. e.g.,

Source Data

ABC123

Server CFF Output

Col1	Col2

Col1	Col2
ABC	123

PxCFF Output

Col1	Col2

Col1	Col2
ABC	123

Incorrect Source Data

When source data is in invalid format, Server CFF will still show results, PxCFF will reject the data and return default value instead. Refer to Native Type differences section for more details

Native Type differences

Native Type	Server CFF	PxCFF

Native Type	Server CFF	PxCFF
BINARY	Columns with `NativeType` of `BINARY` are read by Server CFF stage using `Big Endian` byte order.	Byte order needs set to `Big Endian` to read `BINARY` data
CHARACTER	When source data does not match the native type setting, Server CFF will return still return row with incorrect data Example: using source data, `ABC` on PxCFF with 2 columns CHARACTER(3) & CHARACTER(4). CFF will return ABC for first column, no value for second column	When source data does not match the native type setting, PxCFF will reject the whole record Example: using source data, `ABC` on PxCFF with 2 columns CHRACTER(3) & CHARACTER(4). PxCFF will reject the row.
DECIMAL	-	-
DISPLAY_DECIMAL	When source data is in invalid format, Server CFF will return still return row with incorrect data Example: using source data of `123.45` on a DISPLAY_DECIMAL(5,2) column. Server CFF will return `+123..4`	When source data is in invalid format, PxCFF will reject the data, return default value (based on user setting) Example: using source data of `123.45` on a DISPLAY_DECIMAL(5,2) column. PxCFF will return `000.00` (if default is set to 0)
FLOAT	Columns with `NativeType` of `FLOAT` are read in `Little Endian` byte order	Byte order needs set to `Little Endian` to read `FLOAT` data
GROUP	-	-
GRAPHIC_N	-	-
GRAPHIC_G	-	-
NATIVE_BINARY	Refer to `BINARY`	Refer to `BINARY`
VARCHAR	-	-
VARGRAPHIC_N	-	-
VARGRAPHIC_G	-	-

Server features not supported

Feature	Asset Query (?)	Comment

Feature	Asset Query (?)	Comment
Select Columns not subset of Source Columns	CFF Select Columns Not Subset of Source Columns	Not Supported. When Select Columns not a subset of Source Columns. Server Job can still compile and run, but Parallel Engine will either fail or returning different results.
Source Columns with both `BINARY` and `FLOAT` native type in the same CFF stage	CFF Using Binary and Float Native Types Concurrently	Not Supported. Server CFF reads `BINARY` and `FLOAT` using different byte orders. PxCFF does not allow the configuration of per-column byte-order.
Selection Criteria with Value Preceded by `&`	CFF Using Selection Criteria with Value Preceded by Ampersand	Not Supported. Not natively supported by the Parallel Engine
FLOAT Native Type	CFF Using FLOAT Native Type	Supported with caveat. Server CFF stages use a unique Float/Double to String conversion routine which is not replicated by S2PX. This can result in S2PX converted jobs producing floating point columns with less significant digits compared to Server. The difference should be negligible since the difference is usually 15 significant digits (PX) compare to up to 18 significant digits (Server)
Using `ANYSIGN_DECIMAL` `UNSIGNED_DECIMAL` `OCCURS_COUNTERS` in Meta Data Description	CFF Using Special Value in Decription Field	Not Supported. Not natively supported by the Parallel Engine
Verify sign value in DECIMAL COMP-3 data	CFF Using Verify sign value in DECIMAL COMP-3 data	Not Supported. The `Verify sign value in DECIMAL COMP-3 data` option in Server is ignored entirely. PxCFF will strictly verify input data for all types (not just COMP-3)
Variable Record Length with Record Style = CRLF	CFF With Record Length Shorter then Defined Columns	Supported with caveat. PxCFF does not have equivalent functionality but S2PX attempts to emulate it when Record Length is set. However, variable length input records (defined by the CR/LF option) that smaller than the defined record metadata will result in dropped records in PX