Yandex.Cloud
  • Services
  • Why Yandex.Cloud
  • Pricing
  • Documentation
  • Contact us
Get started
Use cases
  • Web service
    • All use cases
    • Static website in Object Storage
    • Website on LAMP or LEMP stack
    • Fault-tolerant website with load balancing from Yandex Network Load Balancer
    • Fault-tolerant website using DNS load balancing
    • Joomla-based website with PostgreSQL
    • WordPress website
    • WordPress website on a MySQL database
    • 1C-Bitrix website
  • Online stores
    • All use cases
    • 1C-Bitrix online store
    • Opencart online store
  • Data archive
    • All use cases
    • Single-node file server
    • Configuring an SFTP server on Centos 7
    • Backup to Object Storage via Acronis Backup
    • Backup to Object Storage via CloudBerry Desktop Backup
    • Backup to Object Storage via Duplicati
    • Backup to Object Storage via Bacula
    • Digitizing archives in Yandex Vision
  • Test environment
    • All use cases
    • Testing applications with GitLab
    • Creating test VMs using GitLab CI
    • High-performance computing on preemptible VMs
  • Infrastructure management
    • All use cases
    • Getting started with Terraform
    • Uploading Terraform states to Object Storage
    • Getting started with Packer
    • VM images building automation using Jenkins
    • Continuous deployment of containerized applications using GitLab
    • Creating a cluster of 1C:Enterprise Linux servers with a Managed Service for PostgreSQL cluster
    • Creating a cluster of 1C:Enterprise Windows servers with MS SQL Server
    • Migrating to Yandex.Cloud using Hystax Acura
    • Emergency recovery in Yandex.Cloud using Hystax Acura
    • Configuring a fault-tolerant architecture in Yandex.Cloud
  • Windows in Yandex.Cloud
    • All use cases
    • Deploying Active Directory
    • Deploying Microsoft Exchange
    • Deploying Remote Desktop Services
    • Deploying an Always On availability group
    • Deploying an Always On availability group with an internal network load balancer
  • Network routing
    • All use cases
    • Routing through a NAT instance
    • Creating a VPN tunnel
    • Installing a Cisco CSR1000v virtual router
    • Installing a Mikrotik CHR virtual router
    • Creating a VPN connection using OpenVPN
  • Data visualization and analytics
    • All use cases
    • Visualizing data from a CSV file
    • Visualizing data from a ClickHouse database
    • Visualizing data from Yandex.Metrica
    • Visualizing data from Yandex.Metrica Logs API
    • Publishing a chart with a map from a CSV file to DataLens Public
    • Visualizing data from AppMetrica
    • Visualizing geodata from a CSV file
  • Internet of things
    • Use cases for the internet of things
    • Status monitoring of geographically distributed devices
    • Monitoring sensor readings and event notifications
  1. Infrastructure management
  2. Continuous deployment of containerized applications using GitLab

Continuous deployment of containerized applications using GitLab

  • Before you start
    • Required paid resources
    • Install additional dependencies
  • Create a VM from the GitLab image
  • Create a registry Container Registry
  • Configure GitLab
  • Create a test application
  • Create resources Kubernetes
    • Create a cluster
    • Create a node group
    • Get a Kubernetes service account token to authenticate with GitLab
  • Connect the Kubernetes cluster to the GitLab builds
  • Configure building and deploying a Docker image from CI
  • Delete the created resources

GitLab is a continuous integration tool. This scenario describes how to build an application in a Docker container and deploy it from the container on a Kubernetes cluster with GitLab using Yandex.Cloud tools. After each commit, a script runs in GitLab that describes the steps for building a Docker image and applying a new Kubernetes cluster configuration, which specifies the application to be deployed. To configure the infrastructure required for storing source code, building Docker images, and deploying applications, follow these steps:

  • Before you start
    • Required paid resources
    • Install additional dependencies
  • Create a VM from the GitLab image
  • Configure GitLab
  • Create a Container Registry resource
  • Create Kubernetes resources
    • Create a cluster
    • Create a node group
  • Connect the Kubernetes cluster to the GitLab runners
  • Configure a Docker image build and deployment from CI

If you no longer need the VM and cluster, delete them.

Before you start

  1. Go to the management console. Then log in to Yandex.Cloud or sign up if don't already have an account.
  2. On the billing page, make sure you linked a billing account, and it has the ACTIVE or TRIAL_ACTIVE status. If you don't have a billing account, create one.

If you have an active billing account, you can create or select a folder to run your VM in from the Yandex.Cloud page.

Learn more about clouds and folders.

Required paid resources

The cost of this infrastructure includes:

  • A fee for the disks and continuously running VMs (see Yandex Compute Cloud pricing).
  • A fee for using a dynamic public IP address (see Yandex Virtual Private Cloud pricing).
  • A fee for storing created Docker images (see Yandex Container Registry pricing).
  • A fee for using the Kubernetes master (see Yandex Managed Service for Kubernetes pricing).

Install additional dependencies

To run the script, install the following in the local environment:

  • The Yandex.Cloud command line interface.
  • jq.
  • kubectl.

Create a VM from the GitLab image

Run GitLab on a VM with a public IP address:

  1. On the folder page in the management console, click Create resource and select Virtual machine.

  2. In the Name field, enter a name for the VM: ci-tutorial-gitlab.

  3. Select the availability zone to host the VM in.

  4. Under Images, click Choose.

  5. In the window that opens, go to the DevTools tab.

  6. Choose the GitLab image.

  7. Under Computing resources, specify the following configuration:

    • vCPU: 2
    • Guaranteed vCPU share: 100%
    • RAM: 2 GB
  8. Under Network settings, select the subnet to connect the VM to. If you don't have a network or subnet, create them right on the VM creation page.

  9. In the Public address field, choose Auto.

  10. Specify data required for accessing the VM:

    • Enter the username in the Login field.
    • In the SSH key field, copy the contents of the public key file. You need to create a key pair for SSH connection yourself. To generate keys, use third-party tools, such as ssh-keygen utilities on Linux and macOS or PuTTygen on Windows.
  11. Click Create VM.

Creating the VM may take several minutes. When the VM status changes to RUNNING and GitLab is run, you can start configuring it.

Create a registry Container Registry

To store Docker images, you need a registry in Container Registry.

To create a registry:

  1. On the folder page in the management console, select Container Registry.
  2. Click Create registry.
  3. Specify a name for the registry.

Save the registry ID: you'll need it for the next steps.

Configure GitLab

To configure GitLab and enable Continuous Integration (CI), create a new project and enter the necessary GitLab CI authorization parameters.

  1. On the Compute Cloud page, select the created VM and find its public IP.
  2. In the browser, open a link in the format http://<public VM IP address> . The GitLab admin panel opens.
  3. Set the administrator password and click Change your password.
  4. Enter the username root and administrator password. Click Sign in.
  5. Select Create a project.
  6. Set the project name: gitlab-test.
  7. Click Create project.

Create a test application

Create a test application that can be deployed in a Kubernetes cluster:

  1. Add the Docker image configuration

    1. In the left panel of GitLab, go to Project and select Details.

    2. On the project page, click New file.

    3. Name the file Dockerfile. Add the Docker image configuration to it:

      FROM alpine:3.10
      CMD echo "Hello"
      
    4. Add a comment to the commit in the Commit message field: Dockerfile for test application.

    5. Click Commit changes.

  2. Add the configuration for deploying the Docker image in the Kubernetes cluster:

    1. In the left panel, go to Project and select Details.

    2. To the right of the project name, click + and select New file from the drop-down menu.

    3. Name the file k8s.yaml. Add the configuration for creating a namespace and deployment to it:

      apiVersion: v1
      kind: Namespace
      metadata:
        name: hello-world
      ---
      apiVersion: apps/v1
      kind: Deployment
      metadata:
        name: hello-world-deployment
        namespace: hello-world
      spec:
        replicas: 1
        selector:
          matchLabels:
            app: hello
        template:
          metadata:
            namespace: hello-world
            labels:
              app: hello
          spec:
            containers:
              - name: hello-world
                image: cr.yandex/<registry ID>/hello:__VERSION__
                imagePullPolicy: Always
      
    4. In the given file, replace the <registry ID> with the ID of your previously created registry.

    5. Add a comment to the commit in the Commit message field: Docker image deployment config.

    6. Click Commit changes.

Create resources Kubernetes

Create the Kubernetes resources necessary to run the scripts: a cluster and node group.

  1. If you don't have a network yet, create one.

  2. If you don't have any subnets yet, create them in the availability zones where the Kubernetes cluster and node group will be created.

  3. Create service accounts:

    • A service account for resources with the editor role for the folder where the Kubernetes cluster will be created. The resources that the Kubernetes cluster needs will be created on behalf of this account.
    • A service account for nodes with the container-registry.images.pusher role for the folder containing the Docker image registry. The nodes push the Docker images built in GitLab to the registry on behalf of this service account.

    You can use the same service account for both operations.

Create a cluster

  1. In management console, select the folder where you want to create your Kubernetes cluster.
  2. In the list of services, select Managed Service for Kubernetes.
  3. Click Create cluster.
  4. Enter a name and description for the Kubernetes cluster.
  5. Specify a service account for the resources. This is used to create the resources.
  6. Specify a service account for nodes. The nodes use this service account to access the Docker image registry.
  7. Specify a release channel.
  8. Under Master configuration:
    • In the Kubernetes version field, select the Kubernetes version to be installed on the master.

    • In the Public IP field, choose a method for assigning an IP address:

      • Auto: Assign a random IP address from the Yandex.Cloud IP pool.
      • No address: Don't assign a public IP address.
    • In the Master type field, select the type of master:

      • Zonal: A master created in a subnet in one availability zone.
      • Regional: A master created and distributed in three subnets in each availability zone.
    • Select the availability zone where you want to create the master node address.

      This step is only available for the zonal master.

    • In the Network format field, choose how networks are displayed:

      • List: Available networks are displayed as a list. If you don't have a cloud network, click Create network:
        • In the window that opens, enter a name for the network. Specify that subnets should be created. Click Create network. By default, networks are created with subnets in each availability zone.
      • ID: Enter the unique ID of the required networks.
    • In the Cloud network field, select the network to create the master in.

    • In the Subnet field, select the subnet to create the master in.

      For the regional master, specify three subnets in each availability zone.

  9. Under Maintenance window settings:
    • In the Maintenance frequency / Disable field, configure the maintenance window:
      • Disabled: Automatic updates are disabled.
      • Anytime: Maintenance is allowed at any time.
      • Daily: Maintenance is performed in the time interval specified in the Time (UTC) and duration field.
      • On selected days: Maintenance is performed in the time interval specified in the Weekly schedule field.
  10. Under Cluster network settings:
    • Specify the cluster CIDR, which is a range of IP addresses for allocating pod addresses.
    • Specify the service CIDR, which is a range of IP addresses for allocating service addresses.
  11. Click Create cluster.

Save the cluster ID: you need it for the next steps.

Create a node group

It may take several minutes to create the Kubernetes cluster. When the status of the cluster changes to RUNNING, you can start creating a node group in this cluster.

To create a node group:

  1. In management console, select the folder where you want to create your Kubernetes cluster.
  2. In the list of services, select Managed Service for Kubernetes.
  3. Select the Kubernetes cluster to create a node group for.
  4. On the Kubernetes cluster page, go to the Node groups tab.
  5. Click Create node group.
  6. Enter a name and description for the node group.
  7. Specify the Kubernetes version for the nodes.
  8. Specify the number of nodes in the node group.
  9. Under Scalability:
    • Select the scaling policy type.
    • Specify the number of nodes in the node group.
  10. Under Allow when creating and updating, specify the maximum number of instances that you can exceed and reduce the size of the group by.
  11. Under Computing resources:
    • Choose a platform.
    • Specify the required number of vCPUs, guaranteed vCPU performance, and RAM.
    • (optional) Specify that the VM must be preemptible.
  12. Under Storage:
    • Specify the Disk type:
      • HDD: Standard network drive. Network block storage on an HDD.
      • SSD: Fast network drive. Network block storage on an SSD.
    • Specify the disk size.
  13. Under Network settings:
    • In the Public IP field, choose a method for assigning an IP address:
      • Auto: Assign a random IP address from the Yandex.Cloud IP pool.
      • No address: Don't assign a public IP address.
    • Specify how nodes should be distributed across availability zones and networks.
    • (optional) Click Add location and specify an additional availability zone and network to create nodes in different zones.
  14. Under Access, specify the information required to access the node:
    • Enter the username in the Login field.
    • In the SSH key field, paste the contents of the public key file.
  15. Under Maintenance window settings:
    • In the Maintenance frequency / Disable field, choose the maintenance window:
      • Disabled: Automatic updates are disabled.
      • Anytime: Maintenance is allowed at any time.
      • Daily: Maintenance is performed during the interval specified in the Time (UTC) and duration field.
      • On selected days: Maintenance is performed during the interval specified in the Schedule by day field.
  16. Click Create node group.

Get a Kubernetes service account token to authenticate with GitLab

Note

Please note that a service account in Kubernetes is not an IAM service account.

To get the Kubernetes service account token:

  1. Configure the local environment to work with the created Kubernetes cluster. To do this, run the command:

    Bash
    $ yc managed-kubernetes cluster get-credentials <cluster-id> --external
    
  2. Save the configuration for creating a Kubernetes service account to a YAML file named gitlab-admin-service-account.yaml:

    apiVersion: v1
    kind: ServiceAccount
    metadata:
      name: gitlab-admin
      namespace: kube-system
    ---
    apiVersion: rbac.authorization.k8s.io/v1beta1
    kind: ClusterRoleBinding
    metadata:
      name: gitlab-admin
    roleRef:
      apiGroup: rbac.authorization.k8s.io
      kind: ClusterRole
      name: cluster-admin
    subjects:
    - kind: ServiceAccount
      name: gitlab-admin
      namespace: kube-system
    
  3. Run the command:

    Bash
    $ kubectl apply -f gitlab-admin-service-account.yaml
    
  4. Get the token using the kubectl get secrets command:

    Bash
    $ kubectl -n kube-system get secrets -o json | \
    jq -r '.items[] | select(.metadata.name | startswith("gitlab-admin")) | .data.token' | \
    base64 --decode
    
  5. Save the token: you need it for the next steps.

Connect the Kubernetes cluster to the GitLab builds

To run your build tasks on a Kubernetes cluster, connect the cluster in the GitLab settings.

  1. In the browser, open a link in the format http://<public GitLab VM IP address>/root.

  2. Select the project named gitlab-test.

  3. In the window that opens on the left, click Operations and select Kubernetes.

  4. Click Add Kubernetes cluster.

  5. In the window that opens click Add existing cluster.

  6. In the Kubernetes cluster name field, enter the cluster name.

  7. In the URL API field, enter the master node address. Retrieve it using the command:

    Bash
    $ yc managed-kubernetes cluster get <cluster-id> --format=json \
    | jq -r .master.endpoints.external_v4_endpoint
    
  8. In the CA Certificate field, enter the master certificate. Retrieve it using the command:

    Bash
    $ yc managed-kubernetes cluster get <cluster-id> --format=json \
    | jq -r .master.master_auth.cluster_ca_certificate
    
  9. In the Service Token field, enter the token GitLab will use to create the Kubernetes resources.
    Use the token you received before you started.

  10. Click Add Kubernetes cluster.

  11. Install the applications required for proper operation of GitLab Runner on the Kubernetes cluster:

    • Click Install next to Helm Tiller.
    • Click Install next to GitLab Runner.

Now you can run automated builds inside your Kubernetes cluster.

Read more about Kubernetes cluster connection settings for GitLab builds in the GitLab documentation.

Configure building and deploying a Docker image from CI

  1. Create the GitLab environment variables.

    1. In GitLab, go to Settings in the left panel and select CI/CD from the drop-down list.
    2. Click Expand next to Variables.
    3. Add two environment variables:
      • KUBE_URL: The master node address. Retrieve it using the command:

        Bash
        $ yc managed-kubernetes cluster get <cluster-id> --format=json \
        | jq -r .master.endpoints.external_v4_endpoint
        
      • KUBE_TOKEN: The token GitLab will use to apply the configuration. Use the token received when getting started.

    4. Click Save variables.
  2. With GitLab, you can configure your build scripts in a YAML file. Create a configuration file named .gitlab-ci.yml:

    1. In GitLab, go to Project in the left panel and select Details.
    2. To the right of the project name, click + and select New file from the drop-down menu.
    3. Name the file .gitlab-ci.yml. Add the steps to build and push a Docker image and update the Kubernetes cluster:
    stages:
      - build
      - deploy
    
    build:
      stage: build
      variables:
        DOCKER_DRIVER: overlay2
        DOCKER_TLS_CERTDIR: ""
        DOCKER_HOST: tcp://localhost:2375/
      image: cr.yandex/yc/metadata-token-docker-helper:0.2
      services:
        - docker:19.03.1-dind
      script:
        - docker build . -t cr.yandex/<registry ID>/hello:gitlab-$CI_COMMIT_SHORT_SHA
        - docker push cr.yandex/<registry ID>/hello:gitlab-$CI_COMMIT_SHORT_SHA
    
    deploy:
      image: gcr.io/cloud-builders/kubectl:latest
      stage: deploy
      script:
        - kubectl config set-cluster k8s --server="$KUBE_URL" --insecure-skip-tls-verify=true
        - kubectl config set-credentials admin --token="$KUBE_TOKEN"
        - kubectl config set-context default --cluster=k8s --user=admin
        - kubectl config use-context default
        - sed -i "s/__VERSION__/gitlab-$CI_COMMIT_SHORT_SHA/" k8s.yaml
        - kubectl apply -f k8s.yaml
    
    1. In the given file, replace the <registry ID> with the ID of your previously created registry.
    2. Add a comment to the commit in the Commit message field: CI scripts.
    3. Click Commit changes.

    Note

    In the .gitlab-ci.yml file, the following two steps of project build are described:

    • Build a Docker image using the Dockerfile from the previous step and push the image to Container Registry.
      • For this step, use the Docker image build container and run the Docker server as a GitLab service.
      • For Container Registry authentication, use a service account linked to the Kubernetes nodes. When getting started, this account was assigned the container-registry.images.pusher role.
      • To get authentication data from VM metadata, use an auxiliary public Docker image cr.yandex/yc/metadata-token-docker-helper:0.2. It runs Docker credential helper, which gets an IAM token from the metadata service.
    • Set up an environment to work with Kubernetes and apply k8s.yaml configurations to Kubernetes clusters. This way the application is deployed on the previously created cluster.
  3. After saving the file, the build script should start. You can view its progress on the CI/CD tab on the left by selecting Pipelines from the drop-down menu. Wait until both build steps are complete.

  4. Check the results in the container logs in the Kubernetes cluster:

Bash
$ kubectl logs deployment/hello-world-deployment -n hello-world

Hello

Delete the created resources

If you no longer need the deployed applications and Kubernetes cluster:

  • Delete the node group and the Kubernetes cluster.
  • Delete the created VMs.
  • Delete the created Docker images and the registry in Container Registry.
  • Delete the created subnets and networks.
  • Delete the created service accounts.

See also

  • Creating test VMs using GitLab CI
In this article:
  • Before you start
  • Required paid resources
  • Install additional dependencies
  • Create a VM from the GitLab image
  • Create a registry Container Registry
  • Configure GitLab
  • Create a test application
  • Create resources Kubernetes
  • Create a cluster
  • Create a node group
  • Get a Kubernetes service account token to authenticate with GitLab
  • Connect the Kubernetes cluster to the GitLab builds
  • Configure building and deploying a Docker image from CI
  • Delete the created resources
Language
Careers
Privacy policy
Terms of use
© 2021 Yandex.Cloud LLC