Versions Compared

Old Version 6

changes.mady.by.user John McKeever

Saved on

compared with

New Version 7

changes.mady.by.user John McKeever

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

The MettleCI Jenkins project DataStage repository and shared library repository templates make use of environment variables defined for each Jenkins Node and credentials to store passwords.The Jenkins API includes calls that can reduce require the specification of Jenkins nodes, environment variables, and credentials. Some DataStage organisations have multiple development teams working on multiple DataStage development projects, each of which would typically require a dedicated Git repository associated with a dedicated Jenkins project and all its attendant Jenkins artefacts. Creating all of these artefacts manually for a large number of DataStage delivery streams could be very onerous and error prone. A better solution might be to automate this process using the the Jenkins Remote Access API which provides calls that can minimise the effort of creating multiple Jenkins Jobs projects (projects“Jobs”), Nodes, and Credentials when.

Guides

Table of Contents
minLevel1
maxLevel1
excludeGuides

See Authenticating scripted clients (jenkins.io).

Prerequisites

In order to successfully make calls to the Jenkins REST API, you require:

  • A user with permissions to create pipelines, nodes and credentials,

  • An access token for that user (may be created as required), and

  • A session token, called a “Jenkins crumb”'Jenkins Crumb'.

Generating the Jenkins Crumb

The following request returns the Crumb Request Field and the Crumb itself:

Code Block
languagebash
$> wget -q --auth-no-challenge \
   --user <JENKINS_USER> --password <ACCESS_TOKEN> \
   --output-document - '<JENKINS_URL>:<JENKINS_PORT>/crumbIssuer/api/xml?xpath=concat(//crumbRequestField,":",//crumb)'

...

This JSON is passed into the REST API call:

Code Block
languagebash
$> curl -L -s \
   -o /dev/null \
   -w "%{http_code}" \
   -u <JENKINS_USER>:<ACCESS_TOKEN> \
   -H "Content-Type:application/x-www-form-urlencoded" \
   -H '<JENKINS_CRUMB_REQUEST_FIELD>:<JENKINS_CRUMB>' \
   -X POST \
   -d "json=<JSON_TEXT>" \
   '<JENKINS_URL>:<JENKINS_PORT>/computer/doCreateItem?name=<NODE_NAME>&type=hudson.slaves.DumbSlave'

...

Code Block
languagexml
<flow-definition plugin="workflow-job@1186.v8def1a_5f3944">
	<actions>
		<org.jenkinsci.plugins.pipeline.modeldefinition.actions.DeclarativeJobAction plugin="pipeline-model-definition@2.2086.v12b_420f036e5"/>
		<org.jenkinsci.plugins.pipeline.modeldefinition.actions.DeclarativeJobPropertyTrackerAction plugin="pipeline-model-definition@2.2086.v12b_420f036e5">
			<jobProperties/>
			<triggers/>
			<parameters/>
			<options/>
		</org.jenkinsci.plugins.pipeline.modeldefinition.actions.DeclarativeJobPropertyTrackerAction>
	</actions>
	<description>PIPELINE_DESCRIPTION</description>
	<keepDependencies>false</keepDependencies>
	<properties>
		<hudson.plugins.jira.JiraProjectProperty plugin="jira@3.7.1"/>
		<org.jenkinsci.plugins.workflow.job.properties.DisableConcurrentBuildsJobProperty>
			<abortPrevious>false</abortPrevious>
		</org.jenkinsci.plugins.workflow.job.properties.DisableConcurrentBuildsJobProperty>
		<org.jenkinsci.plugins.workflow.job.properties.PipelineTriggersJobProperty>
			<triggers>
				<hudson.triggers.SCMTrigger>
					<spec>SCM_POLLING_SCHEDULE</spec>
					<ignorePostCommitHooks>false</ignorePostCommitHooks>
				</hudson.triggers.SCMTrigger>
			</triggers>
		</org.jenkinsci.plugins.workflow.job.properties.PipelineTriggersJobProperty>
	</properties>
	<definition class="org.jenkinsci.plugins.workflow.cps.CpsScmFlowDefinition" plugin="workflow-cps@2725.v7b_c717eb_12ce">
		<scm class="hudson.plugins.git.GitSCM" plugin="git@4.11.3">
			<configVersion>2</configVersion>
			<userRemoteConfigs>
				<hudson.plugins.git.UserRemoteConfig>
<url>
GIT					<url>GIT_PROJECT_REPOSITORY_URL
<URL</url>
					<credentialsId>GIT_PROJECT_REPOSITORY_CRED</credentialsId>
				</hudson.plugins.git.UserRemoteConfig>
			</userRemoteConfigs>
			<branches>
				<hudson.plugins.git.BranchSpec>
					<name>*/master</name>
				</hudson.plugins.git.BranchSpec>
			</branches>
			<doGenerateSubmoduleConfigurations>false</doGenerateSubmoduleConfigurations>
			<submoduleCfg class="empty-list"/>
			<extensions/>
		</scm>
		<scriptPath>Jenkinsfile</scriptPath>
		<lightweight>true</lightweight>
	</definition>
	<triggers/>
	<disabled>false</disabled>
</flow-definition>

...

Here is a cURL call to create the new Jenkins Pipeline:

Code Block
$> curl -X POST -L -s \
   -o /dev/null -w "%{http_code}" \
   -H "Content-Type:text/xml" \
   -H '<JENKINS_CRUMB_REQUEST_FIELD>:<JENKINS_CRUMB>' \
   -X POST \
   -u '<JENKINS_USER>:<ACCESS_TOKEN>' \
   '<JENKINS_URL>:<JENKINS_PORT>/createItem?name=<PIPELINE_NAME>' -H "Content-Type:text/xml" -H '<JENKINS_CRUMB_REQUEST_FIELD>:<JENKINS_CRUMB>'\
   --data-binary @NewPipeline.xml

...

When setting up the Jenkins Nodes used by the MettleCI pipelines, there are two environment variables for storing passwords:

  • IISPASSWORD: the Information Server user password

  • MCIPASSWORD the MettleCI user password

...

The cURL call to make is

Code Block
languagebash
$> curl -L -s \
   -o /dev/null -w "%{http_code}" \
   -u <JENKINS_USER>:<ACCESS_TOKEN> \
   -H "Content-Type:application/x-www-form-urlencoded" \
   -H '<JENKINS_CRUMB_REQUEST_FIELD>:<JENKINS_CRUMB>' \
   -X POST \
   -d "json=<JSON_TEXT>" \
   '<JENKINS_URL>:<JENKINS_PORT>credentials/store/system/domain/_/createCredentials'

...