Document toolboxDocument toolbox

Can MettleCI work with Talend?

We do receive enquiries from customers who use Talend, or other ETL tools such as Informatica, alongside DataStage! They often wonder whether MettleCI’s unique capabilities could be brought to bear on both tools.

MettleCI doesn't yet support Talend, although it's completely feasible and, indeed, something that we have in our medium-term product roadmap. Talend data integration jobs are still represented as directed acyclic graphs and are exportable as XML-formatted '.item' files, very similar to DataStage's ISX files, so we expect our Compliance Checking solution to be very quick and easy to port to Talend. It's unlikely that MettleCI's current library of ~50 DataStage Job Compliance rules could be used without at least some modification, but that's not too complex a task.

MettleCI’s automated Unit Testing solution may take slightly more work, but we don't currently foresee any roadblocks. Talend is effectively a Java source code generator, and Java is actually what we used to create our DataStage Unit Testing framework, so we believe the integration should be relatively straightforward.

Talend already has tight Git integration, so we'd (initially, at least) seek to re-use that to trigger our CI/CD build pipeline components. Those components can continue to use their existing logic and would require some simple re-plumbing to utilise the Talend command line and REST API's, rather than those of DataStage as they currently do.



© 2015-2024 Data Migrators Pty Ltd.