Get started
Language / Region
© 2022 Yandex.Cloud LLC
Practical guidelines

Continuous deployment of containerized applications using GitLab

Written by

GitLab is a continuous integration tool. This scenario describes how to build an application in a Docker container and deploy it from the container on a Kubernetes cluster with GitLab using Yandex Cloud tools. After each commit, a script runs in GitLab that describes the steps for building a Docker image and applying a new Kubernetes cluster configuration, which specifies the application to be deployed. To configure the infrastructure required for storing source code, building Docker images, and deploying applications, follow these steps:

If you no longer need the VM and cluster, delete them.

Before you start

  1. Go to the management console. Then log in to Yandex Cloud or sign up if don't already have an account.
  2. On the billing page, make sure you linked a billing account, and it has the ACTIVE or TRIAL_ACTIVE status. If you don't have a billing account, create one.

If you have an active billing account, you can create or select a folder to run your VM in from the Yandex Cloud page.

Learn more about clouds and folders.

The cost of this infrastructure includes:

Install additional dependencies

To run the script, install the following in the local environment:

Create a VM from the GitLab image

Run GitLab on a VM with a public IP address:

  1. On the folder page in the management console, click Create resource and select Virtual machine.

  2. In the Name field, enter a name for the VM: ci-tutorial-gitlab.

  3. Select the availability zone to host the VM in.

  4. Under Images, click Choose.

  5. In the window that opens, go to the DevTools tab.

  6. Choose the GitLab image.

  7. Under Computing resources, specify the following configuration:

    • vCPU: 2
    • Guaranteed vCPU share: 100%
    • RAM: 2 GB

  8. Under Network settings, select the subnet to connect the VM to. If you don't have a network or subnet, create them right on the VM creation page.

  9. In the Public address field, choose Auto.

  10. Specify data required for accessing the VM:

    • Enter the username in the Login field.
    • In the SSH key field, copy the contents of the public key file. You need to create a key pair for SSH connection yourself. To generate keys, use third-party tools, such as ssh-keygen utilities on Linux and macOS or PuTTygen on Windows.

  11. Click Create VM.

Creating the VM may take several minutes. When the VM status changes to RUNNING and GitLab is run, you can start configuring it.

Create a registry in Container Registry

To store Docker images, you need a registry in Container Registry.

To create a registry:

  1. On the folder page in the management console, select Container Registry.
  2. Click Create registry.
  3. Specify a name for the registry.

Save the registry ID: you'll need it for the next steps.

Configure GitLab

To configure GitLab and enable Continuous Integration (CI), create a new project and enter the necessary GitLab CI authorization parameters.

  1. On the Compute Cloud page, select the created VM and find its public IP.
  2. In the browser, open a link in the format http://<public VM IP address> . The GitLab admin panel opens.
  3. Set the administrator password and click Change your password.
  4. Enter the username root and administrator password. Click Sign in.
  5. Select Create a project.
  6. Set the project name: gitlab-test.
  7. Click Create project.

Create a test application

Create a test application that can be deployed in a Kubernetes cluster:

  1. Add the Docker image configuration.

    1. In the left panel of GitLab, go to Project and select Details.

    2. On the project page, click New file.

    3. Name the file Dockerfile. Add the Docker image configuration to it:

      FROM alpine:3.10
CMD echo "Hello"

    4. Add a comment to the commit in the Commit message field: Dockerfile for test application.

    5. Click Commit changes.

  2. Add the configuration for deploying the Docker image in the Kubernetes cluster:

    1. In the left panel, go to Project and select Details.

    2. To the right of the project name, click + and select New file from the drop-down menu.

    3. Name the file k8s.yaml. Add the configuration for creating a namespace and deployment to it:

      apiVersion: v1
kind: Namespace
metadata:
  name: hello-world
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: hello-world-deployment
  namespace: hello-world
spec:
  replicas: 1
  selector:
    matchLabels:
      app: hello
  template:
    metadata:
      namespace: hello-world
      labels:
        app: hello
    spec:
      containers:
        - name: hello-world
          image: cr.yandex/<registry ID>/hello:__VERSION__
          imagePullPolicy: Always

    4. In the given file, replace the <registry ID> with the ID of your previously created registry.

    5. Add a comment to the commit in the Commit message field: Docker image deployment config.

    6. Click Commit changes.

Create Kubernetes resources

Create the Kubernetes resources necessary to run the scripts: a cluster and node group.

  1. If you don't have a network, create one.

  2. If you don't have any subnets, create them in the availability zones where you will create a Kubernetes cluster and node group.

  3. Create service accounts:

    • A service account for resources with the editor role for the folder where the Kubernetes cluster will be created. The resources that the Kubernetes cluster needs will be created on behalf of this account.
    • A service account for nodes with the container-registry.images.pusher role for the folder containing the Docker image registry. The nodes push the Docker images built in GitLab to the registry on behalf of this service account.

    You can use the same service account for both operations.

Create a cluster

  1. In management console, select the folder where you want to create your Kubernetes cluster.

  2. In the list of services, select Managed Service for Kubernetes.

  3. Click Create cluster.

  4. Enter a name and description for the Kubernetes cluster. The cluster name must be unique within Yandex Cloud.

  5. Specify a service account for the resources. This is used to create the resources.

  6. Specify a service account for nodes. The nodes use this service account to access the Docker image registry.

  7. (Optional) Specify the Encryption key that will be used for encrypting secrets.

    Warning

    You cannot edit this setting after you create a cluster.

  8. Specify a release channel.

  9. Under Master configuration:

    • In the Kubernetes version field, select the Kubernetes version to be installed on the master.

    • In the Public IP field, choose a method for assigning an IP address:

      • Auto: Assign a random IP address from the Yandex Cloud IP pool.
      • No address: Don't assign a public IP address.

    • In the Master type field, select the type of master:

      • Zonal: A master created in a subnet in one availability zone.
      • Regional: A master created and distributed in three subnets in each availability zone.

    • Select the availability zone where you want to create the master node address.

      This step is only available for the zonal master.

    • In the Cloud network field, select the network to create the master in.

    • In the Subnet field, select the subnet to create the master in.

      For the regional master, specify a subnet in each availability zone.

    • Select security groups to control the cluster's network traffic.

      Warning

      The configuration of security groups determines cluster performance, availability, and services running in the cluster.

  10. Under Maintenance window settings:

    • In the Maintenance frequency / Disable field, configure the maintenance window:
      • Disabled: Automatic updates are disabled.
      • Anytime: Maintenance is allowed at any time.
      • Daily: Maintenance is performed in the time interval specified in the Time (UTC) and duration field.
      • On selected days: Maintenance is performed in the time interval specified in the Weekly schedule field.

  11. Under Cluster network settings:

    • (optional) Select the network policy controller:
      • Enable network policies to activate Calico.
      • Enable tunneling mode to use Cilium.
    • Specify the cluster CIDR, which is a range of IP addresses for allocating pod addresses.
    • Specify the service CIDR, which is a range of IP addresses for allocating service addresses.
    • Set the node subnet mask and the maximum number of pods in a node.

  12. Click Create cluster.

Save the cluster ID: you need it for the next steps.

Create a node group

It may take several minutes to create the Kubernetes cluster. When the cluster's status changes to RUNNING, you can start creating a node group in this cluster.

To create a node group:

  1. In the management console, select the folder where you want to create your Kubernetes cluster.

  2. In the list of services, select Managed Service for Kubernetes.

  3. Select the Kubernetes cluster to create a node group for.

  4. On the Kubernetes cluster page, go to the Node groups tab.

  5. Click Create node group.

  6. Enter a name and description for the node group.

  7. Specify the Kubernetes version for the node.

  8. Specify the number of nodes in the group.

  9. Under Scaling, select a type:

    • Fixed: The number of nodes in the group remains unchanged. Specify the number of nodes in the group.

    • Automatic: The number of nodes in the group can be controlled using automatic cluster scaling.

      As a result, the following settings become available:

      • Minimum number of nodes.
      • Maximum number of nodes.
      • Initial number of nodes with which the group will be created.

    Warning

    You can't change the scaling type after you create a node group.

  10. Under Allow when creating and updating, specify the maximum number of instances that you can exceed and reduce the size of the group by.

  11. Under Computing resources:

  12. (optional) Under Placement, enter a name for a placement group for your nodes. This setting cannot be edited after the group is created.

  13. Under Storage:

    • Specify the Disk type:

      • HDD: Standard network drive. Network block storage on an HDD.

      • SSD: Fast network drive. Network block storage on an SSD.

      • Non-replicated SSD: Improved-performance network drive. You can only change the size of this type of disk in 93 GB increments.

        Alert

        Non-replicated disks have no redundancy. If a disk fails, its data will be irretrievably lost. For more information, see Non-replicated disk limitations.

    • Specify the disk size.

  14. Under Network settings:

    • In the Public IP field, choose a method for assigning an IP address:

      • Auto: Assign a random IP address from the Yandex Cloud IP pool.
      • No address: Don't assign a public IP address.

    • Select the security groups.

      Warning

      The configuration of security groups determines cluster performance, availability, and services running in the cluster.

    • Specify how nodes should be distributed across availability zones and networks.

    • (optional) Click Add location and specify an additional availability zone and network to create nodes in different zones.

  15. Under Access, specify the information required to access the node:

    • Enter the username in the Login field.
    • In the SSH key field, paste the contents of the public key file.

  16. Under Maintenance window settings:

    • In the Maintenance frequency / Disable field, choose the maintenance window:
      • Disabled: Automatic updates are disabled.
      • Anytime: Maintenance is allowed at any time.
      • Daily: Maintenance is performed during the interval specified in the Time (UTC) and duration field.
      • On selected days: Maintenance is performed during the interval specified in the Schedule by day field.

  17. In the Advanced section:

    • To be able to edit the unsafe kernel parameters on the group's nodes, click Add variable. To enter the name of each kernel parameter, create a separate field.
    • To set up taint policies for nodes, use the Add policy button. Enter the key, value, and effect of each taint policy in a separate set of fields.
    • To set up node labels for a node group, click Add label. Enter the key and value of each label in a separate set of fields.

  18. Click Create node group.

Get a Kubernetes service account token to authenticate with GitLab

Note

Please note that a service account in Kubernetes is not an IAM service account.

To get the Kubernetes service account token:

  1. Configure the local environment to work with the created Kubernetes cluster. To do this, run the command:

    $ yc managed-kubernetes cluster get-credentials <cluster-id> --external

  2. Save the configuration for creating a Kubernetes service account to a YAML file named gitlab-admin-service-account.yaml:

    apiVersion: v1
kind: ServiceAccount
metadata:
  name: gitlab-admin
  namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
  name: gitlab-admin
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin
subjects:
- kind: ServiceAccount
  name: gitlab-admin
  namespace: kube-system

  3. Run the command:

    $ kubectl apply -f gitlab-admin-service-account.yaml

  4. Get the token using the kubectl get secrets command:

    $ kubectl -n kube-system get secrets -o json | \
jq -r '.items[] | select(.metadata.name | startswith("gitlab-admin")) | .data.token' | \
base64 --decode

  5. Save the token: you need it for the next steps.

Connect the Kubernetes cluster to the GitLab build runners

To run your build tasks on a Kubernetes cluster, connect the cluster in the GitLab settings.

  1. In the browser, open a link in the format http://<public GitLab VM IP address>/root.

  2. Select the project named gitlab-test.

  3. In the window that opens on the left, click Operations and select Kubernetes.

  4. Click Add Kubernetes cluster.

  5. In the window that opens click Add existing cluster.

  6. In the Kubernetes cluster name field, enter the cluster name.

  7. In the URL API field, enter the master node address. Retrieve it using the command:

    $ yc managed-kubernetes cluster get <cluster-id> --format=json \
| jq -r .master.endpoints.external_v4_endpoint

  8. In the CA Certificate field, enter the master certificate. Retrieve it using the command:

    $ yc managed-kubernetes cluster get <cluster-id> --format=json \
| jq -r .master.master_auth.cluster_ca_certificate

  9. In the Service Token field, enter the token GitLab will use to create the Kubernetes resources.
    Use the token you received before you started.

  10. Click Add Kubernetes cluster.

  11. Install the applications required for proper operation of GitLab Runner on the Kubernetes cluster:

    • Click Install next to Helm Tiller.
    • Click Install next to GitLab Runner.

Now you can run automated builds inside your Kubernetes cluster.

Read more about Kubernetes cluster connection settings for GitLab builds in the GitLab documentation.

Configure a Docker image build and deployment from CI

  1. Create the GitLab environment variables.

    1. In GitLab, go to Settings in the left panel and select CI/CD from the drop-down list.
    2. Click Expand next to Variables.
    3. Add two environment variables:

      • KUBE_URL: The master node address. Retrieve it using the command:

        $ yc managed-kubernetes cluster get <cluster-id> --format=json \
| jq -r .master.endpoints.external_v4_endpoint

      • KUBE_TOKEN: The token GitLab will use to apply the configuration. Use the token received when getting started.

    4. Click Save variables.

  2. With GitLab, you can configure your build scripts in a YAML file. Create a configuration file named .gitlab-ci.yml:

    1. In GitLab, go to Project in the left panel and select Details.
    2. To the right of the project name, click + and select New file from the drop-down menu.
    3. Name the file .gitlab-ci.yml. Add the steps to build and push a Docker image and update the Kubernetes cluster:
    stages:
  - build
  - deploy

build:
  stage: build
  variables:
    DOCKER_DRIVER: overlay2
    DOCKER_TLS_CERTDIR: ""
    DOCKER_HOST: tcp://localhost:2375/
  image: cr.yandex/yc/metadata-token-docker-helper:0.2
  services:
    - docker:19.03.1-dind
  script:
    - docker build . -t cr.yandex/<registry ID>/hello:gitlab-$CI_COMMIT_SHORT_SHA
    - docker push cr.yandex/<registry ID>/hello:gitlab-$CI_COMMIT_SHORT_SHA

deploy:
  image: gcr.io/cloud-builders/kubectl:latest
  stage: deploy
  script:
    - kubectl config set-cluster k8s --server="$KUBE_URL" --insecure-skip-tls-verify=true
    - kubectl config set-credentials admin --token="$KUBE_TOKEN"
    - kubectl config set-context default --cluster=k8s --user=admin
    - kubectl config use-context default
    - sed -i "s/__VERSION__/gitlab-$CI_COMMIT_SHORT_SHA/" k8s.yaml
    - kubectl apply -f k8s.yaml
    1. In the given file, replace the <registry ID> with the ID of your previously created registry.
    2. Add a comment to the commit in the Commit message field: CI scripts.
    3. Click Commit changes.

    Note

    In the .gitlab-ci.yml file, the following two steps of project build are described:

    • Build a Docker image using the Dockerfile from the previous step and push the image to Container Registry.
      • For this step, use the Docker image build container and run the Docker server as a GitLab service.
      • For Container Registry authentication, use a service account linked to the Kubernetes nodes. When getting started, this account was assigned the container-registry.images.pusher role.
      • To get authentication data from VM metadata, use an auxiliary public Docker image cr.yandex/yc/metadata-token-docker-helper:0.2. It runs Docker credential helper, which gets an IAM token from the metadata service.
    • Set up an environment to work with Kubernetes and apply k8s.yaml configurations to Kubernetes clusters. This way the application is deployed on the previously created cluster.

  3. After saving the file, the build script should start. You can view its progress on the CI/CD tab on the left by selecting Pipelines from the drop-down menu. Wait until both build steps are complete.

  4. Check the results in the container logs in the Kubernetes cluster:

$ kubectl logs deployment/hello-world-deployment -n hello-world

Hello

Delete the created resources

If you no longer need the deployed applications and Kubernetes cluster:

See also

Language / Region
© 2022 Yandex.Cloud LLC
In this article: