Introduction

When S2PX encounters Server Routine names it takes the following approach:

Look for an appropriately-named mapping in the mapping section of the config.yml file and substitute the Server function name for the defined Parallel expression. This expression may include one or more references to new Parallel functions, defined in the config.yml file's libraries section.
If a mapping is not found then the Server function name is retained, unaltered, in the generated Parallel job, and S2PX assumes that an identically-name Parallel function is available. A set of new Parallel functions to support this behaviour, and which are named after Server functions, is suppled by IBM’s recent Fix Packs.

Generated Parallel Function Definitions

Where a custom Parallel function is defined in your config.yml file's libraries section then S2PX will create a Parallel Routine asset in your DataStage project which will provide DataStage’s interface into that external function.

By way of example, the function defined in the example config.yaml file above would be made accessible in DataStage using the S2PX-generated files described here.

The following config.yml configuration file section…

customRoutines:
  generateCHeaders: true
  mapping:
    AddOneToNumber: "{0} + 1"
    GetBalanceInUSD: "{0} * 0.75"

  libraries:
    -
      name: test                               # FUNCTION LIBRARY 'test'
      path: /path/to/lib/
      type: SHARED_LIBRARY
      
      routines:
        ACustomRoutine:                        # DATASTAGE FUNCTION 'a_custom_function(FLOAT, STRING, STRING)'
          type: STRING
          externalName: "a_custom_function"    # Wraps C function 'a_custom_function(float arg1, char * arg2, char * a)'
          arguments:
            -
              name: arg1
              type: FLOAT
            - 
              name: arg2
              type: STRING
            -
              name: a
              type: STRING
    -
      name: test2                               # FUNCTION LIBRARY 'test2'
      path: /path/to/lib/again/
      type: SHARED_LIBRARY

      routines:
        ConvertSomething:                       # DATASTAGE FUNCTION 'convert_something(INT)'
          type: UINT
          externalName: "convert_something"     # Wraps C function 'convert_something(int a)''
          arguments:
            -
              name: a
              type: INT

Produces an output DSX file which included Parallel Routine definitions that look like this:

These Parallel Routine definitions are implemented by C/C++ external functions for which S2PX will generate code templates that contain the type signature of the required functions. The configuration file above, for example will cause the generation of these files…

File name

File contents

test.h

extern char * a_custom_function(float arg1, char * arg2, char * a);

test.cpp

#include <apt_util/logmsg.h>
#include <apt_util/errlog.h>
#include <apt_util/errind.h>
#include <stdlib.h>

#include "test.h"


/*
 * Utilities used by generated parallel stub functions
 */
static APT_Error::SourceModule APT_s2pxId("S2PX");
static APT_Error::SourceModule &APT_localErrorSourceModule = APT_s2pxId;


static void failNotImplemented(const char *function)
{                                                                                                                                                                                                                                                                              
	APT_ErrorLog log(APT_localErrorSourceModule);
	*log << "Custom PX Routine " << function << " has not been implemented";                                                                                   
	log.logError(1);
	log.dumpAll();

	abort();
}


/*
 *  Generated parallel stub functions
 */

char * a_custom_function(float arg1, char * arg2, char * a)
{
	failNotImplemented(__func__);
	return 0;
}

… and:

File name

File Contents

test2.h

extern unsigned int convert_something(int a);

test2.cpp

#include <apt_util/logmsg.h>
#include <apt_util/errlog.h>
#include <apt_util/errind.h>
#include <stdlib.h>

#include "test2.h"


/*
 * Utilities used by generated parallel stub functions
 */
static APT_Error::SourceModule APT_s2pxId("S2PX");
static APT_Error::SourceModule &APT_localErrorSourceModule = APT_s2pxId;


static void failNotImplemented(const char *function)
{                                                                                                                                                                                                                                                                              
	APT_ErrorLog log(APT_localErrorSourceModule);
	*log << "Custom PX Routine " << function << " has not been implemented";                                                                                   
	log.logError(1);
	log.dumpAll();

	abort();
}


/*
 *  Generated parallel stub functions
 */

unsigned int convert_something(int a)
{
	failNotImplemented(__func__);
	return 0;
}

Due to the dependency on the apt_util classes (logmsg, errlog and errind) the generated files need to be linked against the appropriate liborch<arch>.so file (where <arch> is the relevant host architecture. e.g. liborchx86_64.so) which is located in /opt/IBM/InformationServer/Server/PXEngine/lib .

Compiling Your Custom Parallel Routines

A detailed description of how you can compile your custom Parallel Routines, ready for use by DataStage, can be found here.

Custom Routines

Introduction

Generated Parallel Function Definitions

Compiling Your Custom Parallel Routines