DevOps principles are becoming very popular in the networking space. In this post, I will demonstrate how to setup a GitLab pipeline using your own GitLab Runner. By default, GitLab does have public runners that perform the same tasks.
Overview
In GitLab, a pipeline is a series of stages and jobs defined in a .gitlab-ci.yml
file that outline the tasks to be executed in a CI/CD process. When you push changes to your repository or trigger a pipeline manually, GitLab reads the .gitlab-ci.yml
file and executes the defined stages and jobs within a pipeline.
Runners
Runners are agents that execute jobs in a pipeline. GitLab provides shared runners that you can use, or you can set up your own dedicated runners. Runners can be configured with specific tags, allowing you to assign jobs to specific runners based on their capabilities or characteristics.
Shared Runners
- Provided by GitLab
- Used by multiple projects within a GitLab instance
- Convenient way to execute CI/CD pipelines without the need to set up and manage your own dedicated runners
- Can be configured to run jobs on specific platforms or with specific tags
Group Runners
- Specific to a group in GitLab
- Can be shared across multiple projects within that group
- Useful when you have a set of projects that require similar configurations or share common resources.
Specific Runners
- Dedicated to individual projects
- Registered and configured to work exclusively for a single project
- Suitable when you have specific requirements or dependencies that are unique to a project
Documents
Installation
The GitLab runner needs to be installed somewhere, there are multiple OS options. In this example, I have used Linux.
The GitLab runner should be installed as a service this way, check that it is runningsudo gitlab-runner status
Register Runner
Currently, the runner has been installed on my Ubuntu VM. This needs to be connected to GitLab. To do this, navigate to the project in GitLab that you want to use this runner for. I have chosen to use my Terraform project as an example.
Under Project Overview
navigate to settings
>> CI\CD
>> Runners
>> Expand

Create a new Project Runner

Options to select for me will be;
- Platform: Linux
- Tags: run untagged jobs
- Create Runner

Once Create runner
has been clicked, the next screen shows the details to connect the runner.

A few notes about registering the runner, there are multiple parameters to configure on the GitLab runner itself. The main one is the executor.
The GitLab Runner executor is responsible for executing the jobs defined in the .gitlab-ci.yml
file within a CI/CD pipeline. The executor determines how the jobs are executed and provides the necessary environment for running the commands and scripts specified in the pipeline configuration.
There are multiple different options. I will only go over Docker in this example. The Docker executor runs each job within a separate Docker container. It provides better isolation between jobs and allows you to use specific Docker images for each job. The Docker executor is useful when you need reproducible and isolated environments for running your jobs.
The docker container by default is Ruby. You can choose what you want this to be. I have picked alpine
docker image. This image does not have Terraform installed by default, but that is OK, as this can be changed in the .gitlab-ci.yml
file.
Back on Linux, we will register the runner using the details provided from GitLab
URL: https://gitlab.com
Token: lrt-r1sRA3q8snV6QvaAF7Nz
When I registered this runner with the above commands, it could not connect to any local resources. There is an option to add to the config.toml
file after the runner has been registered, but again I had difficulty having these changes take effect./etc/gitlab-runner/config.toml
[runners.docker]
network_mode = "gitlab-runner-net"
To get around this, I removed and re-registered the runner, adding in the docker network. To create the docker network, I used docker network create -d bridge gitlab-runner-net
. This creates a new bridge network that can be viewed with sudo docker network ls
.
Always run as sudo
otherwise it will save and run as the user it was registered under. This caused me a lot of confusion, so sudo
seems best to me.
The runner registration command creates a file /etc/gitlab-runner/config.toml
. Inside this file are the settings.
Run the runner, and back on that same GitLab page, a notification will be shown that states the runner has been connected. The runner will also be shown as green in the settings page.


Create a Test Job
Now that the runner has been connected, a test job can be created. Navigate back to the repository and a file named .gitlab-ci.yml
needs to be created. This file is what controls the runner. Whenever there is a change to this file, it will be executed on the GitLab runner.
As a basic test, I have created the following job to display “Hello Terraform User”

The output on the runner is
The job has been run and can be seen in the GitLab interface.


Create a Second Test Job
I’m going to make this change from my machine, and commit to ensure that the job runs as expected.
For this, I need to create a GitLab SSH Key, found in this post. Once that has been setup and tested, I can download the repo and make some changes.
In the following steps, I am going to (For more info on Git commands, see this post)
- Clone the repo
- Create a new branch
- Add a second job to the
.gitlab-ci.yml
- Push the new branch to the repo
- Merge the new branch with the main repo
- Push the merged main branch upto GitLab
Back in the GitLab interface, navigate to: Build
>> Pipelines
to see the status of the jobs. Clicking on job02
will show the details, and in there is the echo Hello Terraform Users, from VS Code


Merging in GitLab – Creating a Third Job
In the previous example, the merge request was performed in the Git CLI. When committing a branch, the merge request can be raised through GitLab.
If I make a change to the .gitlab-ci.yml
file, I can demonstrate this.
In the above ouput for the pushing of the branch to GitLab, there is a link https://gitlab.com/ntwklab1/Terraform/-/merge_requests/new?merge_request%5Bsource_branch%5D=changetoci
This is the link that if clicked will take you to create a merge request in GitLab

In the GitLab interface, more details can be added. There is more that can be done here, such as having a merge request approved before being merged.

Now the merge request has been made it can be approved (optional in this example), and merged.

Once merged, the pipeline will then run automatically. This again is something that can be changed.

Navigating over to Build
>> Jobs
and selecting the latest job which will be job02
. In the output is the echo command, shown, $ echo "Hello Terraform Users, from VS Code, changed for merge request"
