TO DO - Setting up a GitLab Project/Repository

MettleCI works with assets that are kept in Git repositories, notably the repository associated with a DataStage project and the repository associated with the compliance rules. So part of the set up and integration process involves configure these within GitLab. Unfortunately GitLab uses terms in somewhat different ways than other similar tools which can lead to confusion.

Thus GitLab tightly couples project and repository, and each project typically has at most one git repository.

Recommended practice is to create a new GitLab project for each DataStage project. You may want to collect your projects into groups. Begin by selecting “new project” from your dashboard, or visiting http://your.gitlab.server/projects/new as explained in GitLab - create project and create a blank project. Since you will be cloning the project in order to populate it, you should create a README file, which ensures the Git repository is initialised, has a default branch, and can be cloned.

After you create the project you will want to set up access to it for MettleCI and your CI server. The most common way to do this is to create a (project specific or public) deploy key. Detailed nuances of the differences are out of scope, you may want to refer to GitLab - deploy keys for more information, but public keys can be used across projects, after the project administrator grants access, while private keys only grant access to one project.

= = = = = = =

couples
All Gitlab projects come wIn Gitlab thIn Gitlab thereTo Do

Rough notes
In conformance with the naming conventions used with Jenkins, names onn the GitLab server (http://test1-gitlab.datamigrators.io/) are

• Repository - wwi_gitlab_ds117_ds117 (same as project)

• Pipeline - wwi_gitlab_ds117

Hazy if this naming works out, we will see

This page in admin area is how to create new project

http://test1-gitlab.datamigrators.io/projects/new

Project is here http://test1-gitlab.datamigrators.io/lar/wwi_gitlab_ds117_ds117

after creating it reminds we might need a key http://test1-gitlab.datamigrators.io/profile/keys

This page goes into gory detail http://test1-gitlab.datamigrators.io/projects/new

also can use this cmd from (somewhere)?

git push --set-upstream git@ec2-3-94-105-174.compute-1.amazonaws.com:lar/$(git rev-parse --show-toplevel | xargs basename).git $(git rev-parse --abbrev-ref HEAD)

after creating it, clone it on a machine where you can seed it

git@ec2-3-94-105-174.compute-1.amazonaws.com:lar/wwi_gitlab_ds117_ds117.git or http://ec2-3-94-105-174.compute-1.amazonaws.com/lar/wwi_gitlab_ds117_ds117.git

I need to do this setup

git config --global user.name "Larry Pieniazek"
git config --global user.email "larry.pieniazek@datamigrators.com"

Create a new repository

git clone http://ec2-3-94-105-174.compute-1.amazonaws.com/lar/wwi_gitlab_ds117_ds117.git
cd wwi_gitlab_ds117_ds117
touch README.md
git add README.md
git commit -m "add README"
git push -u origin master

May need to create and register an ssh key

C:\data\DM_temp\git\test1-gitlab>ssh-keygen -t rsa -b 4096 -C "lar@test1-gitlab.datamigrators.io"
Generating public/private rsa key pair.
Enter file in which to save the key (C:\Users\lpieniaz/.ssh/id_rsa): C:\data\DM_temp\git\test1-gitlab\id_rsa
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in C:\data\DM_temp\git\test1-gitlab\id_rsa.
Your public key has been saved in C:\data\DM_temp\git\test1-gitlab\id_rsa.pub.
The key fingerprint is:
SHA256:X5rl0YnMs/w9zUzcm1KyGLv957gI4C7dJgtMwj0T4YY lar@test1-gitlab.datamigrators.io
The key's randomart image is:
+---[RSA 4096]----+
| . |
| o . |
| E + |
| . o . o o . |
| o = S O o..|
| + + o O = .+|
| o...= B +++|
| .oo o+.= =|
| .o+ .o.+=|
+----[SHA256]-----+

C:\data\DM_temp\git\test1-gitlab>dir
Volume in drive C is Windows
Volume Serial Number is 6CA6-6484

Directory of C:\data\DM_temp\git\test1-gitlab

01/12/2021 04:02 PM <DIR> .
01/12/2021 04:02 PM <DIR> ..
01/12/2021 04:02 PM 3,247 id_rsa
01/12/2021 04:02 PM 760 id_rsa.pub
2 File(s) 4,007 bytes
2 Dir(s) 50,928,996,352 bytes free

C:\data\DM_temp\git\test1-gitlab>mkdir .ssh

C:\data\DM_temp\git\test1-gitlab>move id* .ssh
C:\data\DM_temp\git\test1-gitlab\id_rsa
C:\data\DM_temp\git\test1-gitlab\id_rsa.pub
2 file(s) moved.

C:\data\DM_temp\git\test1-gitlab>dir
Volume in drive C is Windows
Volume Serial Number is 6CA6-6484

Directory of C:\data\DM_temp\git\test1-gitlab

01/12/2021 04:02 PM <DIR> .
01/12/2021 04:02 PM <DIR> ..
01/12/2021 04:02 PM <DIR> .ssh
0 File(s) 0 bytes
3 Dir(s) 50,928,996,352 bytes free

C:\data\DM_temp\git\test1-gitlab>cd .ssh

C:\data\DM_temp\git\test1-gitlab.ssh>dir
Volume in drive C is Windows
Volume Serial Number is 6CA6-6484

Directory of C:\data\DM_temp\git\test1-gitlab.ssh

01/12/2021 04:02 PM <DIR> .
01/12/2021 04:02 PM <DIR> ..
01/12/2021 04:02 PM 3,247 id_rsa
01/12/2021 04:02 PM 760 id_rsa.pub
2 File(s) 4,007 bytes
2 Dir(s) 50,929,061,888 bytes free

C:\data\DM_temp\git\test1-gitlab.ssh>

actually used my existing one

Directory of C:\Users\lpieniaz.ssh

11/26/2020 07:13 PM <DIR> .
11/26/2020 07:13 PM <DIR> ..
11/26/2020 07:13 PM 1,675 id_rsa
11/26/2020 07:13 PM 416 id_rsa.pub
10/19/2020 04:39 PM 604 known_hosts
3 File(s) 2,695 bytes
2 Dir(s) 50,917,089,280 bytes free

C:\Users\lpieniaz.ssh>

Sorted this out after some trouble.

installed it here

http://test1-gitlab.datamigrators.io/profile/keys

then cloned the repo.

C:\data\DM_temp\git\test1-gitlab>git clone git@test1-gitlab.datamigrators.io:lar/wwi_gitlab_ds117_ds117.git
Cloning into 'wwi_gitlab_ds117_ds117'...
warning: You appear to have cloned an empty repository.

Then used the template to set up the repo files along with my project’s files

C:\data\DM_temp\git\test1-gitlab\wwi_gitlab_ds117_ds117>git status
On branch master
Your branch is up to date with 'origin/master'.

Untracked files:
(use "git add <file>..." to include in what will be committed)
.gitattributes
.gitlab-ci.yml
compliance/
datastage/
filesystem/
unittest/
var.ci
var.develop
var.hf_ci
var.hf_develop
var.perf
var.prod
var.qa

nothing added to commit but untracked files present (use "git add" to track)

C:\data\DM_temp\git\test1-gitlab\wwi_gitlab_ds117_ds117>git add *
warning: CRLF will be replaced by LF in compliance/WORKBENCH/Adjacent Transformers.pjb.grm.
The file will have its original line endings in your working directory
…
warning: CRLF will be replaced by LF in var.qa.
The file will have its original line endings in your working directory

C:\data\DM_temp\git\test1-gitlab\wwi_gitlab_ds117_ds117>

then commit and push

C:\data\DM_temp\git\test1-gitlab\wwi_gitlab_ds117_ds117

C:\data\DM_temp\git\test1-gitlab\wwi_gitlab_ds117_ds117>git commit -m "seed the repository with GitLab template and WWI project files
[master 982d4ab] seed the repository with GitLab template and WWI project files
147 files changed, 117738 insertions(+)
create mode 100644 .gitattributes
create mode 100644 .gitlab-ci.yml
create mode 100644 compliance/.DS_Store
create mode 100644 compliance/WORKBENCH/Adjacent Transformers.pjb.grm
…
create mode 100644 var.qa

C:\data\DM_temp\git\test1-gitlab\wwi_gitlab_ds117_ds117>

C:\data\DM_temp\git\test1-gitlab\wwi_gitlab_ds117_ds117>git push
Enumerating objects: 161, done.
Counting objects: 100% (161/161), done.
Delta compression using up to 8 threads
Compressing objects: 100% (155/155), done.
Writing objects: 100% (160/160), 2.21 MiB | 1.93 MiB/s, done.
Total 160 (delta 17), reused 0 (delta 0), pack-reused 0
remote: Resolving deltas: 100% (17/17), done.
To test1-gitlab.datamigrators.io:lar/wwi_gitlab_ds117_ds117.git
569988e..982d4ab master -> master

C:\data\DM_temp\git\test1-gitlab\wwi_gitlab_ds117_ds117>