...
Trunk Based Development is a source-control branch model where developers frequently integrate code change into a single branch referred to as the “Trunk” (using Git, Trunk is typically called main or master or main). This branch model encourages developers to integrate their changes many times a day and is a key enabler of Continuous Integration (CI) and, by extension, Continuous Delivery (CD). This is in stark contrast to feature based branching models which encourage developers to work in isolated, long lived feature branches that are only integrated into the “Trunk” when work on a feature is complete.
...
Given the benefits of CI/CD and the merging constraints imposed by DataStage development, it is strong recommended that MettleCI users adopt the use of Trunk Based Development. This guide will provide an overview of key considerations and implementation choices when adopting Trunk Base -Based Development.
...
Release-ability of work in progress
A key behavioral change when using Trunk Based Development (compared to what most developers are used to) is the idea that changes are will be integrated into the Trunk when progress has been made on a feature. There is no expectation that the feature be is complete, just that there has been a worthwhile amount of change to the codebase and that the change won't cause a CI/CD pipeline failure. Developers should never have more than a day’s work that has not been integrated with the Trunk. In practice, developers familiar with working this this way will integrate changes many times a day, happy to integrate an hour’s worth of work or less.
As a consequence of this behavioral change, developers need to get used to the idea of reaching frequent integration points with a partially-built feature. Features which require an extended amount of work need to be decomposed into small pieces of work that can be integrated with Trunk without exposing a partially-built feature or impacting undermining the stability of a running system. When building ETL software with DataStage, this is usually pretty easy: if adding a new column or table to a Data Warehouse, start with the source and progressively add the required calculations but don’t actually load the new column or table until the feature is complete. As a result, the calculations will be performed during transformation but discarded prior to loading the data. New DataStage jobs can also be integrated into Trunk and deployed to production but they are completely inert until they are executed as part of the ETL Batch schedule.
If there is no way to easily hide a feature which is only partially built, feature toggles can be used. Not only can feature toggles hide a partially-built feature, they also allow developers to complete development and testing of new functionality but defer its activation until a specific date. ETL solutions typically integrate source and target systems which are not within the control of the development team. Changes to source and target schemas have to be carefully coordinated to ensure that ETL processes are compatible. Using feature toggles, DataStage developers can build, test and deploy changes for source and target changes but only switch them on in production when the source/target changes are deployed.
...