Get started
Solutions

Continuous deployment of containerized applications using GitLab

GitLab is a continuous integration tool. This scenario describes how to build an application in a Docker container and deploy it from the container on a Kubernetes cluster with GitLab using Yandex.Cloud tools. After each commit, a script runs in GitLab that describes the steps for building a Docker image and applying a new Kubernetes cluster configuration, which specifies the application to be deployed. To configure the infrastructure required for storing source code, building Docker images, and deploying applications, follow these steps:

If you no longer need the VM and cluster, delete them.

Before you start

  1. Go to the management console. Then log in to Yandex.Cloud or sign up if don't already have an account.
  2. On the billing page, make sure you linked a billing account, and it has the ACTIVE or TRIAL_ACTIVE status. If you don't have a billing account, create one.

If you have an active billing account, you can create or select a folder to run your VM in from the Yandex.Cloud page.

Learn more about clouds and folders.

The cost of this infrastructure includes:

Install additional dependencies

To run the script, install the following in the local environment:

Create a VM from the GitLab image

Run GitLab on a VM with a public IP address:

  1. On the folder page in the management console, click Create resource and select Virtual machine.

  2. In the Name field, enter a name for the VM: ci-tutorial-gitlab.

  3. Select the availability zone to host the VM in.

  4. Under Images, click Choose.

  5. In the window that opens, go to the DevTools tab.

  6. Choose the GitLab image.

  7. Under Computing resources, specify the following configuration:

    • vCPU: 2
    • Guaranteed vCPU share: 100%
    • RAM: 2 GB

  8. Under Network settings, select the subnet to connect the VM to. If you don't have a network or subnet, create them right on the VM creation page.

  9. In the Public address field, choose Auto.

  10. Specify data required for accessing the VM:

    • Enter the username in the Login field.
    • In the SSH key field, copy the contents of the public key file. You need to create a key pair for SSH connection yourself. To generate keys, use third-party tools, such as ssh-keygen utilities on Linux and macOS or PuTTygen on Windows.

  11. Click Create VM.

Creating the VM may take several minutes. When the VM status changes to RUNNING and GitLab is run, you can start configuring it.

Create a registry Container Registry

To store Docker images, you need a registry in Container Registry.

To create a registry:

  1. On the folder page in the management console, select Container Registry.
  2. Click Create registry.
  3. Specify a name for the registry.

Save the registry ID: you'll need it for the next steps.

Configure GitLab

To configure GitLab and enable Continuous Integration (CI), create a new project and enter the necessary GitLab CI authorization parameters.

  1. On the Compute Cloud page, select the created VM and find its public IP.
  2. In the browser, open a link in the format http://<public VM IP address> . The GitLab admin panel opens.
  3. Set the administrator password and click Change your password.
  4. Enter the username root and administrator password. Click Sign in.
  5. Select Create a project.
  6. Set the project name: gitlab-test.
  7. Click Create project.

Create a test application

Create a test application that can be deployed in a Kubernetes cluster:

  1. Add the Docker image configuration

    1. In the left panel of GitLab, go to Project and select Details.

    2. On the project page, click New file.

    3. Name the file Dockerfile. Add the Docker image configuration to it:

      FROM alpine:3.10
CMD echo "Hello"

    4. Add a comment to the commit in the Commit message field: Dockerfile for test application.

    5. Click Commit changes.

  2. Add the configuration for deploying the Docker image in the Kubernetes cluster:

    1. In the left panel, go to Project and select Details.

    2. To the right of the project name, click + and select New file from the drop-down menu.

    3. Name the file k8s.yaml. Add the configuration for creating a namespace and deployment to it:

      apiVersion: v1
kind: Namespace
metadata:
  name: hello-world
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: hello-world-deployment
  namespace: hello-world
spec:
  replicas: 1
  selector:
    matchLabels:
      app: hello
  template:
    metadata:
      namespace: hello-world
      labels:
        app: hello
    spec:
      containers:
        - name: hello-world
          image: cr.yandex/<registry ID>/hello:__VERSION__
          imagePullPolicy: Always

    4. In the given file, replace the <registry ID> with the ID of your previously created registry.

    5. Add a comment to the commit in the Commit message field: Docker image deployment config.

    6. Click Commit changes.

Create resources Kubernetes

Create the Kubernetes resources necessary to run the scripts: a cluster and node group.

  1. If you don't have a network yet, create one.

  2. If you don't have any subnets yet, create them in the availability zones where the Kubernetes cluster and node group will be created.

  3. Create service accounts:

    • A service account for resources with the editor role for the folder where the Kubernetes cluster will be created. The resources that the Kubernetes cluster needs will be created on behalf of this account.
    • A service account for nodes with the container-registry.images.pusher role for the folder containing the Docker image registry. The nodes push the Docker images built in GitLab to the registry on behalf of this service account.

    You can use the same service account for both operations.

Create a cluster

To create a Kubernetes cluster:

  1. In the management console, select the folder where you want to create your Kubernetes cluster.
  2. In the list of services, select Managed Service for Kubernetes.
  3. Click Create cluster.
  4. Enter a name and description for the Kubernetes cluster.
  5. Specify a service account for the resources. This is used to create the resources.
  6. Specify a service account for nodes. The nodes use this service account to access the Docker image registry.
  7. Specify a release channel.
  8. Under Master configuration:

    • In the Kubernetes version field, select the Kubernetes version to be installed on the master.

    • In the Public IP field, choose a method for assigning an IP address:

      • Auto: Assign a random IP address from the Yandex.Cloud IP pool.
      • No address: Don't assign a public IP address.

    • In the Master type field, select the type of master:

      • Zonal: A master created in a subnet in one availability zone.
      • Regional: A master created and distributed in three subnets in each availability zone.

    • Select the availability zone where you want to create the master node address.

      This step is only available for the zonal master.

    • In the Network format field, choose how networks are displayed:

      • List: Available networks are displayed as a list. If you don't have a cloud network, click Create network:
        • In the window that opens, enter a name for the network. Specify that subnets should be created. Click Create network. By default, networks are created with subnets in each availability zone.
      • ID: Enter the unique ID of the required networks.

    • In the Cloud network field, select the network to create the master in.

    • In the Subnet field, select the subnet to create the master in.

      For the regional master, specify three subnets in each availability zone.

  9. Under Maintenance window settings:
    • In the Maintenance frequency / Disable field, configure the maintenance window:
      • Disabled: Automatic updates are disabled.
      • Anytime: Maintenance is allowed at any time.
      • Daily: Maintenance is performed in the time interval specified in the Time (UTC) and duration field.
      • On selected days: Maintenance is performed in the time interval specified in the Time (UTC) and duration field.
  10. Under Cluster network settings:
    • Specify the cluster CIDR, which is a range of IP addresses for allocating pod addresses.
    • Specify the service CIDR, which is a range of IP addresses for allocating service addresses.
  11. Click Create cluster.

Save the cluster ID: you need it for the next steps.

Create a node group

It may take several minutes to create the Kubernetes cluster. When the status of the cluster changes to RUNNING, you can start creating a node group in this cluster.

To create a node group:

  1. In the management console, select the folder where you want to create your Kubernetes cluster.
  2. In the list of services, select Managed Service for Kubernetes.
  3. Select the Kubernetes cluster to create a node group for.
  4. On the Kubernetes cluster page, go to the Node groups tab.
  5. Click Create node group.
  6. Enter a name and description for the node group.
  7. Specify the number of nodes in the node group.
  8. Specify the Kubernetes version for the nodes.
  9. Under Computing resources:
  10. Under Storage:
    • Specify the Disk type:
      • HDD: A standard network drive. Network block storage on an HDD.
      • SSD: A fast network drive. Network block storage on an SSD.
    • Specify the disk size.
  11. Under Network settings:
    • In the Public IP field, choose a method for assigning an IP address:
      • Auto: Assign a random IP address from the Yandex.Cloud IP pool.
      • No address: Don't assign a public IP address.
    • Specify how nodes should be distributed across availability zones and networks.
    • (optional) Click Add location and specify an additional availability zone and network to create nodes in different zones.
  12. Under Access, specify the information required to access the node:
    • Enter the username in the Login field.
    • In the SSH key field, paste the contents of the public key file.
  13. Under Maintenance settings, UTC:
    • In the Maintenance window field, select your preferred maintenance window:
      • Disabled: Automatic updates are disabled.
      • Anytime: Maintenance is allowed at any time.
      • Daily: Maintenance is performed during the interval specified in the Time and duration field.
      • On selected days: Maintenance is performed during the interval specified in the Time and duration field.
  14. Click Create node group.

Get a Kubernetes service account token to authenticate with GitLab

Note

Please note that a service account in Kubernetes is not an IAM service account.

To get the Kubernetes service account token:

  1. Configure the local environment to work with the created Kubernetes cluster. To do this, run the command:

    $ yc managed-kubernetes cluster get-credentials <cluster-id> --external

  2. Save the configuration for creating a Kubernetes service account to a YAML file named gitlab-admin-service-account.yaml:

    apiVersion: v1
kind: ServiceAccount
metadata:
  name: gitlab-admin
  namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
  name: gitlab-admin
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin
subjects:
- kind: ServiceAccount
  name: gitlab-admin
  namespace: kube-system

  3. Run the command:

    $ kubectl apply -f gitlab-admin-service-account.yaml

  4. Get the token using the kubectl get secrets command:

    $ kubectl -n kube-system get secrets -o json | \
jq -r '.items[] | select(.metadata.name | startswith("gitlab-admin")) | .data.token' | \
base64 --decode

  5. Save the token: you need it for the next steps.

Connect the Kubernetes cluster to the GitLab builds

To run your build tasks on a Kubernetes cluster, connect the cluster in the GitLab settings.

  1. In the browser, open a link in the format http://<public GitLab VM IP address>/root.

  2. Select the project named gitlab-test.

  3. In the window that opens on the left, click Operations and select Kubernetes.

  4. Click Add Kubernetes cluster.

  5. In the window that opens click Add existing cluster.

  6. In the Kubernetes cluster name field, enter the cluster name.

  7. In the URL API field, enter the master node address. Retrieve it using the command:

    $ yc managed-kubernetes cluster get <cluster-id> --format=json \
| jq -r .master.endpoints.external_v4_endpoint

  8. In the CA Certificate field, enter the master certificate. Retrieve it using the command:

    $ yc managed-kubernetes cluster get <cluster-id> --format=json \
| jq -r .master.master_auth.cluster_ca_certificate

  9. In the Service Token field, enter the token GitLab will use to create the Kubernetes resources.
    Use the token you received before you started.

  10. Click Add Kubernetes cluster.

  11. Install the applications required for proper operation of GitLab Runner on the Kubernetes cluster:

    • Click Install next to Helm Tiller.
    • Click Install next to GitLab Runner.

Now you can run automated builds inside your Kubernetes cluster.

Read more about Kubernetes cluster connection settings for GitLab builds in the GitLab documentation.

Configure building and deploying a Docker image from CI

  1. Create the GitLab environment variables.

    1. In GitLab, go to Settings in the left panel and select CI/CD from the drop-down list.
    2. Click Expand next to Variables.
    3. Add two environment variables:

      • KUBE_URL: The master node address. Retrieve it using the command:

        $ yc managed-kubernetes cluster get <cluster-id> --format=json \
| jq -r .master.endpoints.external_v4_endpoint

      • KUBE_TOKEN: The token GitLab will use to apply the configuration. Use the token received when getting started.

    4. Click Save variables.

  2. With GitLab, you can configure your build scripts in a YAML file. Create a configuration file named .gitlab-ci.yml:

    1. In GitLab, go to Project in the left panel and select Details.
    2. To the right of the project name, click + and select New file from the drop-down menu.
    3. Name the file .gitlab-ci.yml. Add the steps to build and push a Docker image and update the Kubernetes cluster:
    stages:
  - build
  - deploy

build:
  stage: build
  variables:
    DOCKER_DRIVER: overlay2
    DOCKER_TLS_CERTDIR: ""
    DOCKER_HOST: tcp://localhost:2375/
  image: cr.yandex/yc/metadata-token-docker-helper:0.2
  services:
    - docker:19.03.1-dind
  script:
    - docker build . -t cr.yandex/<registry ID>/hello:gitlab-$CI_COMMIT_SHORT_SHA
    - docker push cr.yandex/<registry ID>/hello:gitlab-$CI_COMMIT_SHORT_SHA

deploy:
  image: gcr.io/cloud-builders/kubectl:latest
  stage: deploy
  script:
    - kubectl config set-cluster k8s --server="$KUBE_URL" --insecure-skip-tls-verify=true
    - kubectl config set-credentials admin --token="$KUBE_TOKEN"
    - kubectl config set-context default --cluster=k8s --user=admin
    - kubectl config use-context default
    - sed -i "s/__VERSION__/gitlab-$CI_COMMIT_SHORT_SHA/" k8s.yaml
    - kubectl apply -f k8s.yaml
    1. In the given file, replace the <registry ID> with the ID of your previously created registry.
    2. Add a comment to the commit in the Commit message field: CI scripts.
    3. Click Commit changes.

    Note

    In the .gitlab-ci.yml file, the following two steps of project build are described:

    • Build a Docker image using the Dockerfile from the previous step and push the image to Container Registry.
      • For this step, use the Docker image build container and run the Docker server as a GitLab service.
      • For Container Registry authentication, use a service account linked to the Kubernetes nodes. When getting started, this account was assigned the container-registry.images.pusher role.
      • To get authentication data from VM metadata, use an auxiliary public Docker image cr.yandex/yc/metadata-token-docker-helper:0.2. It runs Docker credential helper, which gets an IAM token from the metadata service.
    • Set up an environment to work with Kubernetes and apply k8s.yaml configurations to Kubernetes clusters. This way the application is deployed on the previously created cluster.

  3. After saving the file, the build script should start. You can view its progress on the CI/CD tab on the left by selecting Pipelines from the drop-down menu. Wait until both build steps are complete.

  4. Check the results in the container logs in the Kubernetes cluster:

$ kubectl logs deployment/hello-world-deployment -n hello-world

Hello

Delete the created resources

If you no longer need the deployed applications and Kubernetes cluster:

See also