Get started
Language / Region
© 2022 Yandex.Cloud LLC
Yandex Application Load Balancer

Enabling blue-green and canary deployment of web service versions

Written by

In this tutorial, you'll configure your web service architecture to switch between versions using the commonly adopted deployment models: blue-green deployment and canary deployment.

Both models use two backends: a "blue" and a "green" one. First you deploy a stable version generally available to users on one backend (let it be the blue one). Then you use the other backend (the green one) to test the next version. When the testing is complete, the backends switch roles:

  • With a blue-green deployment, all user traffic switches from one backend to the other right away.
  • With canary deployment, the traffic is switched over gradually, starting with some part of your user base.

After that, the green backend becomes the primary one, and you can use the "blue" backend to test your next version. As long as your previous version runs on the blue backend, you can roll the service back to it by switching the backends back.

In this tutorial, we use Yandex Object Storage buckets as backends with the Yandex Application Load Balancer L7 load balancer switching traffic between them. User requests are transmitted to the load balancer via the Yandex Cloud CDN content delivery network (CDN) that reduces the time of content delivery.

In the tutorial, we'll use the domain names cdn.yandexcloud.example and cdn-staging.yandexcloud.example as examples.

To complete the tutorial, use the supported tools.

To build an architecture for the blue-green and canary deployment:

  1. Before you start.
  2. Create a cloud network and subnets.
  3. Create buckets in Object Storage.
  4. Upload the service files to buckets.
  5. Create Application Load Balancer backend groups.
  6. Create an HTTP router and virtual hosts.
  7. Create an L7 load balancer.
  8. Create a CDN resource.
  9. Configure DNS for the service.
  10. Run a health check and test the switching between service versions.

If you no longer need the created resources, delete them.

Supported tools

Most of the steps in the tutorial can be completed in any standard tool: management console, command line interfaces (CLI) Yandex Cloud, AWS, Terraform, and APIs Yandex Cloud. Each step lists tools supported for it.

Some steps don't support certain tools:

Before you start

Before working, you need to register in Yandex Cloud and create a billing account:

  1. Go to the management console. Then log in to Yandex Cloud or sign up if don't already have an account.
  2. On the billing page, make sure you linked a billing account, and it has the ACTIVE or TRIAL_ACTIVE status. If you don't have a billing account, create one.

If you have an active billing account, you can create or select a folder to run your VM in from the Yandex Cloud page.

Learn more about clouds and folders.

This use case uses a folder named example-folder as an example.

The cost of this infrastructure includes:

Create a cloud network and subnets

All resources you have created in the tutorial belong to the same cloud network.

To create a network and subnets:

  1. In the management console, select the example-folder folder.
  2. In the list of services, select Virtual Private Cloud.
  3. Click Create network.
  4. Specify the Name of the network: canary-network.
  5. In the Advanced field, select Create subnets.
  6. Click Create network.

If you don't have the Yandex Cloud command line interface yet, install and initialize it.

The folder specified in the CLI profile is used by default. You can specify a different folder using the --folder-name or --folder-id parameter.

  1. Create a network named canary-network:

    yc vpc network create canary-network

    Command output:

    id: enptrcle5q3d3ktd33hj
folder_id: b1g9hv2loamqfnbul7d9
created_at: "2021-11-03T09:25:03Z"
name: canary-network
default_security_group_id: enpbsnnop4akg7ng70ll

    For more information about the yc vpc network create command, see the CLI reference.

  2. Create subnets in all availability zones:

    • In ru-central1-a:

      yc vpc subnet create canary-subnet-ru-central1-a \
  --zone ru-central1-a \
  --network-name canary-network \
  --range 10.1.0.0/16

      Command output:

      id: e9bnnssj8sc8mjhat9qk
folder_id: b1g9hv2loamqfnbul7d9
created_at: "2021-11-03T09:27:00Z"
name: canary-subnet-ru-central1-a
network_id: enptrcle5q3d3ktd33hj
zone_id: ru-central1-a
v4_cidr_blocks:
- 10.1.0.0/16

    • In ru-central1-b:

      yc vpc subnet create canary-subnet-ru-central1-b \
  --zone ru-central1-b \
  --network-name canary-network \
  --range 10.2.0.0/16

      Command output:

      id: e2lghukd9iqo4haidjbt
folder_id: b1g9hv2loamqfnbul7d9
created_at: "2021-11-03T09:27:39Z"
name: canary-subnet-ru-central1-b
network_id: enptrcle5q3d3ktd33hj
zone_id: ru-central1-b
v4_cidr_blocks:
- 10.2.0.0/16

    • In ru-central1-c:

      yc vpc subnet create canary-subnet-ru-central1-c \
  --zone ru-central1-c \
  --network-name canary-network \
  --range 10.3.0.0/16

      Command output:

      id: b0c3pte4o2kn4v12o05p
folder_id: b1g9hv2loamqfnbul7d9
created_at: "2021-11-03T09:28:08Z"
name: canary-subnet-ru-central1-c
network_id: enptrcle5q3d3ktd33hj
zone_id: ru-central1-c
v4_cidr_blocks:
- 10.3.0.0/16

    For more information about the yc vpc subnet create command, see the CLI reference.

If you don't have Terraform, install it and configure the Yandex Cloud provider.

  1. In the configuration file, describe the parameters of canary-network and its subnets: canary-subnet-ru-central1-a, canary-subnet-ru-central1-b, and canary-subnet-ru-central1-c:

    resource "yandex_vpc_network" "canary-network" {
  name = "canary-network"
}

resource "yandex_vpc_subnet" "canary-subnet-a" {
  name           = "canary-subnet-ru-central1-a"
  zone           = "ru-central1-a"
  network_id     = "${yandex_vpc_network.canary-network.id}"
  v4_cidr_blocks = ["10.1.0.0/16"]
}

resource "yandex_vpc_subnet" "canary-subnet-b" {
  name           = "canary-subnet-ru-central1-b"
  zone           = "ru-central1-b"
  network_id     = "${yandex_vpc_network.canary-network.id}"
  v4_cidr_blocks = ["10.2.0.0/16"]
}

resource "yandex_vpc_subnet" "canary-subnet-c" {
  name           = "canary-subnet-ru-central1-c"
  zone           = "ru-central1-c"
  network_id     = "${yandex_vpc_network.canary-network.id}"
  v4_cidr_blocks = ["10.3.0.0/16"]
}

    For more information, see the yandex_vpc_network and yandex_vpc_subnet resource descriptions in the Terraform provider documentation.

  2. Make sure that the configuration files are correct.

    1. In the command line, go to the directory where you created the configuration file.

    2. Run the check using the command:

      terraform plan

    If the configuration is described correctly, the terminal displays a list of created resources and their parameters. If there are errors in the configuration, Terraform points them out.

  3. Deploy the cloud resources.

    1. If the configuration doesn't contain any errors, run the command:

      terraform apply

    2. Confirm that you want to create the resources.

  1. Create the canary-network network using the gRPC API NetworkService/Create call or the REST API create method.
  2. Create the canary-subnet-ru-central1-a, canary-subnet-ru-central1-b, and canary-subnet-ru-central1-c subnets in the three availability zones by calling the gRPC API SubnetService/Create or the REST API create method.

Create buckets in Object Storage

Create two buckets: canary-bucket-blue and canary-bucket-green:

  1. In the management console, select the example-folder folder.

  2. In the list of services, select Object Storage.

  3. Create a bucket named canary-bucket-blue:

    1. Click Create bucket.
    2. Specify the bucket Name: canary-bucket-blue.
    3. In the Object read access and Object listing access fields, select Public.
    4. Click Create bucket.

  4. Similarly create a bucket named canary-bucket-green.

  1. Create a bucket named canary-bucket-blue:

    aws --endpoint-url https://storage.yandexcloud.net \
  s3 mb s3://canary-bucket-blue

    Command output:

    make_bucket: s3://canary-bucket-blue

  2. Enable public access to reading objects and their list:

    aws --endpoint-url https://storage.yandexcloud.net \
  s3api put-bucket-acl \
  --bucket canary-bucket-blue \
  --acl public-read

  3. In a similar way, create a bucket named canary-bucket-green and enable public access to it.

  1. Add the parameters of the canary-bucket-blue and canary-bucket-green buckets to the configuration file:

    ...

resource "yandex_storage_bucket" "canary-bucket-blue" {
  bucket = "canary-bucket-blue"
  acl    = "public-read"
}

resource "yandex_storage_bucket" "canary-bucket-green" {
  bucket = "canary-bucket-green"
  acl    = "public-read"
}

    For more information about the yandex_storage_bucket resource, see the Terraform provider documentation.

  2. Make sure that the configuration files are correct.

    1. In the command line, go to the directory where you created the configuration file.

    2. Run the check using the command:

      terraform plan

    If the configuration is described correctly, the terminal displays a list of created resources and their parameters. If there are errors in the configuration, Terraform points them out.

  3. Deploy the cloud resources.

    1. If the configuration doesn't contain any errors, run the command:

      terraform apply

    2. Confirm that you want to create the resources.

Use the create REST API method.

Upload the files of your service to the buckets

  1. Create two files named index.html. They will represent two service versions: version 1 and version 2.

    Sample file index.html version 1
    <!DOCTYPE html>
<html>
  <head>
    <title>Version 1</title>
  </head>
  <body>
    <p>Version 1 is working</p>
  </body>
</html>
    Sample file index.html version 2
    <!DOCTYPE html>
<html>
  <head>
    <title>Version 2</title>
  </head>
  <body>
    <p>Version 2 is working</p>
  </body>
</html>

  2. Upload files to buckets:

    1. In the management console, select the example-folder folder.
    2. In the list of services, select Object Storage.
    3. In the bucket list, select canary-bucket-blue.
    4. Click Upload and select the index.html file for version 1.
    5. Similarly upload to the canary-bucket-green bucket the index.html file for version 2.

    1. To the canary-bucket-blue bucket, upload the index.html file version 1:

      aws --endpoint-url https://storage.yandexcloud.net \
  s3 cp v1/index.html s3://canary-bucket-blue/index.html

      Command output:

      upload: v1/index.html to s3://canary-bucket-blue/index.html

    2. Upload to the canary-bucket-green bucket the index.html file version 2:

      aws --endpoint-url https://storage.yandexcloud.net \
  s3 cp v2/index.html s3://canary-bucket-green/index.html

      Command output:

      upload: v2/index.html to s3://canary-bucket-green/index.html

    1. To the configuration file, add the parameters of the v1/index.html and v2/index.html files uploaded to canary-bucket-blue and canary-bucket-green, respectively:

      ...

resource "yandex_storage_object" "canary-bucket-blue-index" {
  bucket = "canary-bucket-blue"
  key    = "index.html"
  source = "v1/index.html"
}

resource "yandex_storage_bucket" "canary-bucket-green-index" {
  bucket = "canary-bucket-green"
  key    = "index.html"
  source = "v2/index.html"
}

      For more information about the yandex_storage_object resource, see the Terraform provider documentation.

    2. Make sure that the configuration files are correct.

      1. In the command line, go to the directory where you created the configuration file.

      2. Run the check using the command:

        terraform plan

      If the configuration is described correctly, the terminal displays a list of created resources and their parameters. If there are errors in the configuration, Terraform points them out.

    3. Deploy the cloud resources.

      1. If the configuration doesn't contain any errors, run the command:

        terraform apply

      2. Confirm that you want to create the resources.

    Use the upload REST API method.

Create a security group

Note

Security groups are at the Preview stage. If they aren't available on your network, all incoming and outgoing traffic will be enabled for the resources and no additional setup is required.

Security groups contain rules that allow the L7 load balancer to receive incoming traffic and send it to backend buckets.

To create security groups:

  1. In the management console, open Virtual Private Cloud.

  2. Click the Security groups tab.

  3. Click Create group.

  4. Enter the Name of the group: canary-sg.

  5. Select the Network canary-network.

  6. Under Rules, create the following rules using the instructions below the table:

    Traffic
    direction    		 Description Port
    range    		 Protocol Source/
    destination type    		 Source /
    destination
    Outgoing any All Any CIDR 0.0.0.0/0
    Incoming ext-http 80 TCP CIDR 0.0.0.0/0
    Incoming ext-https 443 TCP CIDR 0.0.0.0/0
    Incoming healthchecks 30080 TCP CIDR 198.18.235.0/24
    198.18.248.0/24

    1. Go to the Outgoing traffic or Incoming traffic tab.

    2. Click Add rule.

    3. In the Port range field of the window that opens, specify a single port or a range of ports that traffic will come to or from.

    4. In the Protocol field, specify the desired protocol or leave Any to allow traffic transmission over any protocol.

    5. In the Purpose or Source field, select the purpose of the rule:

      • CIDR: The rule will apply to the range of IP addresses. In the CIDR blocks field, specify the CIDR and masks of subnets that traffic will come to or from. To add multiple CIDRs, click Add CIDR.
      • Security group: The rule will apply to the VMs from the current group or the selected security group.

    6. Tap Save. Repeat the steps to create all rules from the table.

  7. Tap Save.

Run the following command:

yc vpc security-group create canary-sg \
  --network-name canary-network \
  --rule direction=egress,port=any,protocol=any,v4-cidrs=[0.0.0.0/0] \
  --rule direction=ingress,port=80,protocol=tcp,v4-cidrs=[0.0.0.0/0] \
  --rule direction=ingress,port=443,protocol=tcp,v4-cidrs=[0.0.0.0/0] \
  --rule direction=ingress,port=30080,protocol=tcp,v4-cidrs=[198.18.235.0/24,198.18.248.0/24]

Command output:

id: enpd133ngcnrgc8475cc
folder_id: b1g9hv2loamqfnbul7d9
created_at: "2021-11-03T10:26:16Z"
name: canary-sg
network_id: enptrcle5q3d3ktd33hj
status: ACTIVE
rules:
- id: enpkgrpi2gsibdm6aotd
  direction: EGRESS
  protocol_name: ANY
  protocol_number: "-1"
  cidr_blocks:
    v4_cidr_blocks:
    - 0.0.0.0/0
- id: enpgssij0i168jknb85r
  direction: INGRESS
  ports:
    from_port: "80"
    to_port: "80"
  protocol_name: TCP
  protocol_number: "6"
  cidr_blocks:
    v4_cidr_blocks:
    - 0.0.0.0/0
- id: enp0bft67j9lrlnhdur5
  direction: INGRESS
  ports:
    from_port: "443"
    to_port: "443"
  protocol_name: TCP
  protocol_number: "6"
  cidr_blocks:
    v4_cidr_blocks:
    - 0.0.0.0/0
- id: enpmorcimu65fk4oaanm
  direction: INGRESS
  ports:
    from_port: "30080"
    to_port: "30080"
  protocol_name: TCP
  protocol_number: "6"
  cidr_blocks:
    v4_cidr_blocks:
    - 198.18.235.0/24
    - 198.18.248.0/24

For more information about the yc vpc security-group create command, see the CLI reference.

  1. Add the canary-sg security group parameters to the configuration file:

    resource "yandex_vpc_security_group" "canary-sg" {
  name       = "canary-sg"
  network_id = "${yandex_vpc_network.canary-network.id}"

  egress {
    protocol       = "ANY"
    port           = "ANY"
    v4_cidr_blocks = ["0.0.0.0/0"]
  }

  ingress {
    protocol       = "TCP"
    port           = 80
    v4_cidr_blocks = ["0.0.0.0/0"]
  }

  ingress {
    protocol       = "TCP"
    port           = 443
    v4_cidr_blocks = ["0.0.0.0/0"]
  }

  ingress {
    protocol       = "TCP"
    port           = 30080
    v4_cidr_blocks = ["198.18.235.0/24", "198.18.248.0/24"]
  }
}

    For more information about the yandex_vpc_security_group resource, see the Terraform provider documentation.

  2. Make sure that the configuration files are correct.

    1. In the command line, go to the directory where you created the configuration file.

    2. Run the check using the command:

      terraform plan

    If the configuration is described correctly, the terminal displays a list of created resources and their parameters. If there are errors in the configuration, Terraform points them out.

  3. Deploy the cloud resources.

    1. If the configuration doesn't contain any errors, run the command:

      terraform apply

    2. Confirm that you want to create the resources.

Use the SecurityGroupService/Create gRPC API call or the create REST API method.

Create Application Load Balancer backend groups

  1. Create a backend group named canary-bg-production with the canary-backend-blue and canary-backend-green backends:

    1. In the management console, select the example-folder folder.
    2. In the list of services, select Application Load Balancer and go to the Backend groups tab.
    3. Click Create backend group.
    4. Enter the backend group name: canary-bg-production.
    5. Under Backends, click Add. Specify the backend settings:
      1. Enter the backend name: canary-backend-blue.
      2. set the weight of the backend: 100.
      3. Select Bucket as the backend type.
      4. In the Bucket field, enter the bucket name: canary-bucket-blue.
    6. Click Add and similarly enter settings for the canary-backend-green backend with the weight of 0 and the canary-bucket-green bucket.
    7. Click Create.

  2. In a similar way, create a backend group named canary-bg-staging. For the canary-backend-blue backend, set the weight to 0, for canary-backend-green, set the weight to 100.

  3. If you are going to complete the next steps in Terraform, copy the IDs of the backend groups canary-bg-production and canary-bg-staging from the Backend groups tab.

Use the BackendGroupService/Create gRPC API call or the REST API create method.

Create an HTTP router and virtual hosts

Create an HTTP router with two virtual hosts: cdn.mywebsite.com and cdn-staging.mywebsite.com:

  1. In the management console, select the example-folder folder.

  2. In the list of services, select Application Load Balancer and go to the HTTP routers tab.

  3. Click Create HTTP router.

  4. Enter the router name: canary-router.

  5. Create a virtual host named canary-vh-production:

    1. Under Add virtual host, click Add host.
    2. Enter the host name: canary-vh-production.
    3. Specify the value for Authority: cdn.yandexcloud.example
    4. Click Add route.
    5. Enter the Name: canary-route-production.
    6. In the Path field, select Starts with and specify the path /.
    7. In the HTTP methods list, select GET.
    8. In the Action field, leave the value Routing.
    9. In the Backend group list, select canary-bg-production.

  6. In a similar way, create a virtual host named canary-vh-staging with the following parameters:

    • Authority: cdn-staging.yandexcloud.example.
    • Route name: canary-route-staging.
    • Backend group: canary-bg-staging.
    • The other parameters are the same as for canary-vh-production.

  7. Leave the other settings as they are and click Create.

  1. Create the canary-router HTTP Router:

    yc alb http-router create canary-router

    Command output:

    id: ds7qd0vj01djuu3c6f8q
name: canary-router
folder_id: b1g9hv2loamqfnbul7d9
created_at: "2021-11-03T10:31:41.027649223Z"

    For more information about the yc alb http-router create command, see the CLI reference.

  2. Create a virtual host named canary-vh-production:

    yc alb virtual-host create canary-vh-production \
  --http-router-name canary-router \
  --authority cdn.yandexcloud.example

    Command output:

    done (1s)
name: canary-vh-production
authority:
- cdn.yandexcloud.example

    For more information about the yc alb virtual-host create command, see the CLI reference.

  3. Create a route named canary-route-production in the canary-vh-production virtual host:

    yc alb virtual-host append-http-route canary-route-production \
  --http-router-name canary-router \
  --virtual-host-name canary-vh-production \
  --prefix-path-match "/" \
  --backend-group-name canary-bg-production

    Command output:

    done (1s)
name: canary-vh-production
authority:
- cdn.yandexcloud.example
routes:
- name: canary-route-production
  http:
    match:
      path:
        prefix_match: /
    route:
      backend_group_id: ds7pbm5fj2v09ptnn29p

    For more information about the yc alb virtual-host append-http-route command, see the CLI reference.

  4. Create a virtual host named canary-vh-staging:

    yc alb virtual-host create canary-vh-staging \
  --http-router-name canary-router \
  --authority cdn-staging.yandexcloud.example

    Command output:

    done (1s)
name: canary-vh-staging
authority:
- cdn-staging.yandexcloud.example

  5. Create a route named canary-route-staging in the canary-vh-staging virtual host:

    yc alb virtual-host append-http-route canary-route-staging \
  --http-router-name canary-router \
  --virtual-host-name canary-vh-staging \
  --prefix-path-match "/" \
  --backend-group-name canary-bg-staging

    Command output:

    done (1s)
name: canary-vh-staging
authority:
- cdn-staging.yandexcloud.example
routes:
- name: canary-route-staging
  http:
    match:
      path:
        prefix_match: /
    route:
      backend_group_id: ds765atleotaiui5pqeu

  1. To the configuration file, add parameters of the canary-router HTTP router, its virtual hosts and routes:

    ...

resource "yandex_alb_http_router" "canary-router" {
  name = "canary-router"
}

resource "yandex_alb_virtual_host" "canary-vh-production" {
  name           = "canary-vh-production"
  http_router_id = ${yandex_alb_http_router.canary-router.id}
  authority      = "cdn.yandexcloud.example"

  route {
    name = "canary-route-production"
    http_route {
      http_route_action {
        backend_group_id = "<ID of the canary-bg-production backend group>"
      }
    }
  }  
}

resource "yandex_alb_virtual_host" "canary-vh-staging" {
  name           = "canary-vh-staging"
  http_router_id = ${yandex_alb_http_router.canary-router.id}
  authority      = "cdn-staging.yandexcloud.example"

  route {
    name = "canary-route-staging"
    http_route {
      http_route_action {
        backend_group_id = "<ID of the canary-bg-staging backend group>"
      }
    }
  }  
}

    For more information, see the yandex_alb_http_router and yandex_alb_virtual_host resource descriptions in the Terraform provider documentation.

  2. Make sure that the configuration files are correct.

    1. In the command line, go to the directory where you created the configuration file.

    2. Run the check using the command:

      terraform plan

    If the configuration is described correctly, the terminal displays a list of created resources and their parameters. If there are errors in the configuration, Terraform points them out.

  3. Deploy the cloud resources.

    1. If the configuration doesn't contain any errors, run the command:

      terraform apply

    2. Confirm that you want to create the resources.

  1. Create the canary-router HTTP router using the gRPC API HttpRouterService/Create call or the create REST API method.
  2. Create the canary-vh-production and canary-vh-staging virtual hosts linked to the router, then create their routes using the gRPC API VirtualHostService/Create call or the create REST API method.

Create an L7 load balancer

  1. In the management console, select the example-folder folder.

  2. In the list of services, select Application Load Balancer, then click Load balancers.

  3. Click Create L7 load balancer.

  4. Enter the load balancer name: canary-balancer.

  5. Under Network settings:

    1. Select the Network canary-network.
    2. Select the Security group canary-sg. If this field is omitted, any incoming and outgoing traffic is allowed for the load balancer.

  6. Under Allocation, select three subnets for the load balancer nodes: canary-subnet-ru-central1-a, canary-subnet-ru-central1-b, and canary-subnet-ru-central1-c, then enable traffic to these subnets.

  7. Click Add listener under Listeners. Set the listener settings:

    1. Enter the listener name: canary-listener.
    2. Under Public IP address settings, enable traffic.
    3. Set the port to 80.
    4. In the Assign IP address field, select Automatically.

  8. In the HTTP router field, select canary-router.

  9. Click Create.

  1. Get the IDs of subnets for canary-network:

    yc vpc network list-subnets canary-network

    Command output:

    +----------------------+-----------------------------+----------------------+----------------------+----------------+---------------+---------------+
|          ID          |            NAME             |      FOLDER ID       |      NETWORK ID      | ROUTE TABLE ID |     ZONE      |     RANGE     |
+----------------------+-----------------------------+----------------------+----------------------+----------------+---------------+---------------+
| e9bnnssj8sc8mjhat9qk | canary-subnet-ru-central1-c | b1g9hv2loamqfnbul7d9 | enptrcle5q3d3ktd33hj |                | ru-central1-c | [10.1.0.0/16] |
| e2lghukd9iqo4haidjbt | canary-subnet-ru-central1-b | b1g9hv2loamqfnbul7d9 | enptrcle5q3d3ktd33hj |                | ru-central1-b | [10.2.0.0/16] |
| b0c3pte4o2kn4v12o05p | canary-subnet-ru-central1-a | b1g9hv2loamqfnbul7d9 | enptrcle5q3d3ktd33hj |                | ru-central1-a | [10.3.0.0/16] |
+----------------------+-----------------------------+----------------------+----------------------+----------------+---------------+---------------+

    For more information about the yc vpc network list-subnets command, see the CLI reference.

  2. Get the canary-sg security group ID:

    yc vpc security-group get canary-sg | grep "^id"

    Command output:

    id: enpd133ngcnrgc8475cc

    For more information about the yc vpc security-group get command, see the CLI reference.

  3. Create a load balancer named canary-balancer:

    yc alb load-balancer create canary-balancer \
  --network-name canary-network \
  --security-group-id <ID of the canary-sg security group> \
  --location zone=ru-central1-a,subnet-id=<ID of the canary-subnet-ru-central1-a subnet> \
  --location zone=ru-central1-b,subnet-id=<ID of the canary-subnet-ru-central1-b subnet> \
  --location zone=ru-central1-c,subnet-id=<ID of the canary-subnet-ru-central1-c subnet>

    Command output:

    done (3m0s)
id: ds77q7v39b4ubg8ta2n4
name: canary-balancer
folder_id: b1g9hv2loamqfnbul7d9
status: ACTIVE
region_id: ru-central1
network_id: enptrcle5q3d3ktd33hj
allocation_policy:
  locations:
  - zone_id: ru-central1-c
    subnet_id: b0c3pte4o2kn4v12o05p
  - zone_id: ru-central1-b
    subnet_id: e2lghukd9iqo4haidjbt
  - zone_id: ru-central1-a
    subnet_id: e9bnnssj8sc8mjhat9qk
log_group_id: ckg23vr4dlkse3hvq0kc
security_group_ids:
- enpd133ngcnrgc8475cc
created_at: "2021-11-03T10:55:49.134935148Z"

    For more information about the yc alb load-balancer create command, see the CLI reference.

  4. Add a listener to the load balancer:

    yc alb load-balancer add-listener \
  --name canary-balancer \
  --listener-name canary-listener \
  --external-ipv4-endpoint port=80 \
  --http-router-name canary-router

    Command output:

    done (43s)
id: ds77q7v39b4ubg8ta2n4
name: canary-balancer
folder_id: b1g9hv2loamqfnbul7d9
status: ACTIVE
region_id: ru-central1
network_id: enptrcle5q3d3ktd33hj
listeners:
- name: canary-listener
  endpoints:
  - addresses:
    - external_ipv4_address:
        address: 84.252.133.149
    ports:
    - "80"
  http:
    handler:
      http_router_id: ds7qd0vj01djuu3c6f8q
allocation_policy:
  locations:
  - zone_id: ru-central1-c
    subnet_id: b0c3pte4o2kn4v12o05p
  - zone_id: ru-central1-b
    subnet_id: e2lghukd9iqo4haidjbt
  - zone_id: ru-central1-a
    subnet_id: e9bnnssj8sc8mjhat9qk
log_group_id: ckg23vr4dlkse3hvq0kc
security_group_ids:
- enpd133ngcnrgc8475cc
created_at: "2021-11-03T10:55:49.134935148Z"

    For more information about the yc alb load-balancer add-listener command, see the CLI reference.

  1. Add the parameters of the canary-balancer L7 load balancer to the configuration file:

    ...

resource "yandex_alb_load_balancer" "canary-balancer" {
  name               = "canary-balancer"
  network_id         = ${yandex_vpc_network.canary-network.id}
  security_group_ids = [ ${yandex_vpc_security_group.canary-sg.id} ]

  allocation_policy {
    location {
      zone_id   = "ru-central1-a"
      subnet_id = ${yandex_vpc_subnet.canary-subnet-ru-central1-a.id}
    }

    location {
      zone_id   = "ru-central1-b"
      subnet_id = ${yandex_vpc_subnet.canary-subnet-ru-central1-b.id}
    }

    location {
      zone_id   = "ru-central1-c"
      subnet_id = ${yandex_vpc_subnet.canary-subnet-ru-central1-c.id}
    }
  }

  listener {
    name = "canary-listener"
    endpoint {
      address {
        external_ipv4_address {
        }
      }
      ports = [80]
    }
    http {
      handler {
        http_router_id = ${yandex_alb_http_router.canary-router.id}
      }
    }
  }
}

    For more information about the yandex_alb_load_balancer resource, see the Terraform provider documentation.

  2. Make sure that the configuration files are correct.

    1. In the command line, go to the directory where you created the configuration file.

    2. Run the check using the command:

      terraform plan

    If the configuration is described correctly, the terminal displays a list of created resources and their parameters. If there are errors in the configuration, Terraform points them out.

  3. Deploy the cloud resources.

    1. If the configuration doesn't contain any errors, run the command:

      terraform apply

    2. Confirm that you want to create the resources.

Use the LoadBalancerService/Create gRPC API call or the REST API create method.

Create a CDN resource

  1. In the management console, select the example-folder folder.

  2. In the list of services, select Cloud CDN.

  3. If the CDN provider hasn't been activated yet, click Activate provider.

  4. Create a CDN resource:

    1. On the CDN resources tab, click Create resource.

    2. Set up the main parameters of the CDN resource as follows:

      • Content query: From one origin.

      • Source type: L7 load balancer.

      • L7 load balancer: canary-balancer.

      • IP address: The IP address assigned to the load balancer (the only one in the list).

      • Domain names for content distribution: cdn.yandexcloud.example and cdn-staging.yandexcloud.example.

        Alert

        The first domain name cdn.yandexcloud.examplewill become the primary one, and you won't be able to edit it after you create a CDN resource.

      • In the Advanced section:

        • In the Source protocol field, select HTTP.
        • In the Redirect clients field, select HTTP to HTTPS.
        • Enable End-user access to content.
        • In the Certificate type field, select Let's Encrypt® to automatically issue a certificate for the cdn.yandexcloud.example and cdn-staging.yandexcloud.example domain names after creating the CDN resource.
        • In the Host header field, select HTTP and HTTPS.

    3. Click Create.

    4. Enable CDN caching:

      1. In the list of CDN resources, select the resource with the cdn.yandexcloud.example primary domain name.
      2. Go to the Caching tab.
      3. Click Edit.
      4. Enable CDN caching.
      5. Tap Save.

Use the gRPC API ResourceService/Create call or the REST API create method.

Configure DNS for the service

The domain names cdn.yandexcloud.example and cdn-staging.yandexcloud.example must be linked to the CDN resource using DNS records.

To configure DNS:

  1. Get the domain name of the CDN load balancer:

    1. In the management console, select the example-folder folder.
    2. In the list of services, select Cloud CDN.
    3. In the list of CDN resources, select the resource with the cdn.yandexcloud.example primary domain name.
    4. From DNS settings, copy the domain name that has the format cl-....gcdn.co.

  2. On the site of your DNS hosting provider, go to the DNS settings.

  3. Create or edit CNAME records for cdn.yandexcloud.example and cdn-staging.yandexcloud.example to link them to the copied domain name:

    cdn CNAME cl-....gcdn.co
cdn-staging CNAME cl-....gcdn.co

    If you use Cloud DNS, follow these instructions to configure the record:

    Instruction for configuring DNS records for Cloud DNS

    1. In the management console, open Cloud DNS.

    2. If you don't have a public DNS zone, create one:

      1. Click Create zone.
      2. In the Zone field, enter the site's domain name with a trailing dot: yandexcloud.example.
      3. Select the zone Type: Public.
      4. Specify the Name of the zone: canary-dns-zone.
      5. Click Create.

    3. In the zone, create a CNAME record for cdn.yandexcloud.example:

      1. In the list of zones, click canary-dns-zone.
      2. Click Create record.
      3. In the Name field, enter cdn.
      4. Select the record Type: CNAME.
      5. In the Value field, paste the copied value in the cl-....gcdn.co format.
      6. Click Create.

    4. In a similar way, create in the same zone a CNAME record for cdn-staging.yandexcloud.example. In the Name field, specify cdn-staging.

    1. If you don't have a public DNS zone, create one:

      yc dns zone create \
  --name canary-dns-zone \
  --zone yandexcloud.example. \
  --public-visibility

      Command output:

      id: dns4rq4tadddth4h20qm
folder_id: b1g9hv2loamqfnbul7d9
created_at: "2021-11-03T11:03:28.847Z"
name: canary-dns-zone
zone: yandexcloud.example.
public_visibility: {}

      For more information about the yc dns zone create command, see the CLI reference.

    2. In the zone, create CNAME records for cdn.yandexcloud.example and cdn-staging.yandexcloud.example with a copied value in the cl-....gcdn.co format:

      yc dns zone add-records \
  --name canary-dns-zone \
  --record "cdn CNAME cl-....gcdn.co" \
  --record "cdn-staging CNAME cl-....gcdn.co"

      For more information about the yc dns zone add-records command, see the CLI reference.

    1. Add the parameters of the canary-dns-zone DNS zone and its CNAME records to the configuration file:

      ...

resource "yandex_dns_zone" "canary-dns-zone" {
  zone   = "yandexcloud.example."
  name   = "canary-dns-zone"
  public = true
}

resource "yandex_dns_recordset" "canary-recordset-production" {
  zone_id = ${yandex_dns_zone.canary-dns-zone.id}
  name    = "cdn"
  type    = "CNAME"
  data    = ["<copied value in the format cl-....gcdn.co>"]
}

resource "yandex_dns_recordset" "canary-recordset-staging" {
  zone_id = ${yandex_dns_zone.canary-dns-zone.id}
  name    = "cdn-staging"
  type    = "CNAME"
  data    = ["<copied value in the format cl-....gcdn.co>"]
}

      For more information, see the descriptions of the yandex_dns_zone and yandex_dns_recordset resources in the Terraform provider documentation.

    2. Make sure that the configuration files are correct.

      1. In the command line, go to the directory where you created the configuration file.

      2. Run the check using the command:

        terraform plan

      If the configuration is described correctly, the terminal displays a list of created resources and their parameters. If there are errors in the configuration, Terraform points them out.

    3. Deploy the cloud resources.

      1. If the configuration doesn't contain any errors, run the command:

        terraform apply

      2. Confirm that you want to create the resources.

    1. Create a DNS zone named canary-dns-zone using the gRPC API DnsZoneService/Create call or the REST API create method.
    2. Add the cdn and cdn-staging CNAME records to the zone, copying the cl-....gcdn.co value with the gRPC API DnsZoneService/UpdateRecordSets call or the REST API updateRecordSets method.

Wait 15 to 20 minutes after setting up the DNS to check that the service is up and running.

Run a health check and test the switching between versions

Check one

Check that the domain name cdn.yandexcloud.example corresponds to version 1 and cdn-staging.yandexcloud.example corresponds to version 2:

  1. Open a browser and go to https://cdn.yandexcloud.example/index.html. You should see a page indicating version 1.

  2. Delete index.html from the CDN resource cache:

    1. In the management console, select the example-folder folder.
    2. In the list of services, select Cloud CDN.
    3. Select the created CDN resource (the list of resources will contain its primary domain name: cdn.yandexcloud.example).
    4. Go to the Content tab.
    5. Click Purge cache.
    6. Select the purge type: Selective.
    7. Enter the path to the uploaded file: /index.html.
    8. Click Purge cache.

    1. Get the ID of the CDN resource that you created:

      yc cdn resource list

      Command output:

      +----------------------+--------------------------+--------------------------------+--------------------------------+--------+-------------------------------------------+
|          ID          |          CNAME           |           CREATED AT           |           UPDATED AT           | ACTIVE |                  OPTIONS                  |
+----------------------+--------------------------+--------------------------------+--------------------------------+--------+-------------------------------------------+
| bc837xptmpkhbc7xwioa | cdn.yandexcloud.example  | seconds:1637235693             | seconds:1637235693             | true   | edge_cache_settings:{enabled:true         |
|                      |                          | nanos:434085000                | nanos:434115000                |        | default_value:345600}                     |
|                      |                          |                                |                                |        | cache_http_headers:{value:"accept-ranges" |
|                      |                          |                                |                                |        | value:"cache-control" value:"connection"  |
|                      |                          |                                |                                |        | value:"content-encoding"                  |
|                      |                          |                                |                                |        | value:"content-length"                    |
|                      |                          |                                |                                |        | value:"content-type"                      |
|                      |                          |                                |                                |        | value:"date" value:"etag"                 |
|                      |                          |                                |                                |        | value:"expires" value:"keep-alive"        |
|                      |                          |                                |                                |        | value:"last-modified" value:"server"      |
|                      |                          |                                |                                |        | value:"vary"} stale:{enabled:true         |
|                      |                          |                                |                                |        | value:"error" value:"updating"}           |
|                      |                          |                                |                                |        | allowed_http_methods:{value:"GET"         |
|                      |                          |                                |                                |        | value:"POST" value:"HEAD"                 |
|                      |                          |                                |                                |        | value:"OPTIONS"}                          |
+----------------------+--------------------------+--------------------------------+--------------------------------+--------+-------------------------------------------+

    2. Delete the file from the cache:

      yc cdn cache purge \
  --resource-id <ID of the CDN resource> \
  --path "/index.html"
    1. Get the ID of the CDN resource that you created using the gRPC API ResourceService/List call or the REST API list method.
    2. Delete the index.html file from the cache using the gRPC API CacheService/Purge call or the purge method.

  3. Open the browser and go to https://cdn-staging.yandexcloud.example/index.html. You should see a page indicating version 2.

Canary deployment of version 2

  1. Disable caching of the CDN resource and delete the index.html file from the cache:

    1. In the management console, select the example-folder folder.
    2. In the list of services, select Cloud CDN.
    3. Select the created CDN resource (the list of resources will contain its primary domain name: cdn.yandexcloud.example).
    4. Go to the Caching tab
    5. Click Edit.
    6. Disable CDN caching.
    7. Tap Save.
    1. Get the ID of the CDN resource that you created using the gRPC API ResourceService/List call or the REST API list method.
    2. Disable caching using the gRPC API ResourceService/Update call or the REST API list method.

  2. Delete index.html from the cache:

    1. In the management console, select the example-folder folder.
    2. In the list of services, select Cloud CDN.
    3. Select the created CDN resource (the list of resources will contain its primary domain name: cdn.yandexcloud.example).
    4. Go to the Content tab.
    5. Click Purge cache.
    6. Select the purge type: Selective.
    7. Enter the path to the uploaded file: /index.html.
    8. Click Purge cache.

    1. Get the ID of the CDN resource that you created:

      yc cdn resource list

      Command output:

      +----------------------+--------------------------+--------------------------------+--------------------------------+--------+-------------------------------------------+
|          ID          |          CNAME           |           CREATED AT           |           UPDATED AT           | ACTIVE |                  OPTIONS                  |
+----------------------+--------------------------+--------------------------------+--------------------------------+--------+-------------------------------------------+
| bc837xptmpkhbc7xwioa | cdn.yandexcloud.example  | seconds:1637235693             | seconds:1637235693             | true   | edge_cache_settings:{enabled:true         |
|                      |                          | nanos:434085000                | nanos:434115000                |        | default_value:345600}                     |
|                      |                          |                                |                                |        | cache_http_headers:{value:"accept-ranges" |
|                      |                          |                                |                                |        | value:"cache-control" value:"connection"  |
|                      |                          |                                |                                |        | value:"content-encoding"                  |
|                      |                          |                                |                                |        | value:"content-length"                    |
|                      |                          |                                |                                |        | value:"content-type"                      |
|                      |                          |                                |                                |        | value:"date" value:"etag"                 |
|                      |                          |                                |                                |        | value:"expires" value:"keep-alive"        |
|                      |                          |                                |                                |        | value:"last-modified" value:"server"      |
|                      |                          |                                |                                |        | value:"vary"} stale:{enabled:true         |
|                      |                          |                                |                                |        | value:"error" value:"updating"}           |
|                      |                          |                                |                                |        | allowed_http_methods:{value:"GET"         |
|                      |                          |                                |                                |        | value:"POST" value:"HEAD"                 |
|                      |                          |                                |                                |        | value:"OPTIONS"}                          |
+----------------------+--------------------------+--------------------------------+--------------------------------+--------+-------------------------------------------+

    2. Delete the file from the cache:

      yc cdn cache purge \
  --resource-id <ID of the CDN resource> \
  --path "/index.html"
    1. Get the ID of the CDN resource that you created using the gRPC API ResourceService/List call or the REST API list method.
    2. Delete the index.html file from the cache using the gRPC API CacheService/Purge call or the purge method.

  3. Configure the canary-bg-production so that 20% of the traffic coming to the cdn.yandexcloud.example domain name is processed by the canary-backend-green backend that runs version 2:

    1. In the management console, select the example-folder folder.

    2. In the list of services, select Application Load Balancer and go to the Backend groups tab.

    3. Select canary-bg-production in the backend group list.

    4. For the canary-backend-blue backend, set the weight to 80 instead of 100:

      1. In the Backends section, find canary-backend-blue, then click → Edit.
      2. In the Weight field, enter 80.
      3. Tap Save.

    5. Similarly set the weight to 20 instead of 0 for canary-backend-green.

    6. Tap Save.

    1. For the canary-backend-blue backend, set the weight to 80 instead of 100:

      yc alb backend-group update-http-backend \
  --backend-group-name canary-bg-production \
  --name canary-backend-blue \
  --weight 80

      Command output:

      done (1s)
id: ds7l9puc18c9b40cd359
name: canary-bg-production
folder_id: b1g9hv2loamqfnbul7d9
http:
  backends:
  - name: canary-backend-blue
    backend_weight: "80"
    storage_bucket:
      bucket: canary-bucket-blue
created_at: "2021-11-03T10:28:47.680825561Z"

      For more information about the yc alb backend-group update-http-backend command, see the CLI reference.

    2. Set the weight to 20 instead of 0 for canary-backend-green:

      yc alb backend-group update-http-backend \
  --backend-group-name canary-bg-production \
  --name canary-backend-green \
  --weight 20

      Command output:

      done (1s)
id: ds7l9puc18c9b40cd359
name: canary-bg-production
folder_id: b1g9hv2loamqfnbul7d9
http:
  backends:
  - name: canary-backend-green
    backend_weight: "20"
    storage_bucket:
      bucket: canary-bucket-green
created_at: "2021-11-03T10:28:47.680825561Z"

    Use the gRPC API BackendGroupService/UpdateBackend call of the REST API updateBackend method.

  4. Open the browser and go to https://cdn.yandexcloud.example/index.html, refreshing the page several times. In about 20% of cases, you should see a page indicating version 2, in the other cases, version 1.

  5. Similarly to steps 1–2, configure and check the following traffic allocations between the backends:

    1. In the canary-bg-production backend group: 50%-50% traffic distribution between backends.
    2. In the canary-bg-production backend group, forward all traffic to canary-backend-green.
    3. In the canary-bg-staging backend group (with the domain name of cdn-staging.yandexcloud.example), allocate all traffic to canary-backend-blue.

  6. Re-enable caching:

    1. In the management console, select the example-folder folder.
    2. In the list of services, select Cloud CDN.
    3. Select the created CDN resource (the list of resources will contain its primary domain name: cdn.yandexcloud.example).
    4. Go to the Caching tab
    5. Click Edit.
    6. Enable CDN caching.
    7. Tap Save.
    1. Get the ID of the CDN resource that you created using the gRPC API ResourceService/List call or the REST API list method.
    2. To enable caching, use the gRPC API ResourceService/Update call or the REST API list method.

Blue-green deployment for rolling back to version 1

  1. Disable caching of the CDN resource and delete the index.html file from the cache:

    1. In the management console, select the example-folder folder.
    2. In the list of services, select Cloud CDN.
    3. Select the created CDN resource (the list of resources will contain its primary domain name: cdn.yandexcloud.example).
    4. Go to the Caching tab
    5. Click Edit.
    6. Disable CDN caching.
    7. Tap Save.
    1. Get the ID of the CDN resource that you created using the gRPC API ResourceService/List call or the REST API list method.
    2. Disable caching using the gRPC API ResourceService/Update call or the REST API list method.

  2. Delete index.html from the cache:

    1. In the management console, select the example-folder folder.
    2. In the list of services, select Cloud CDN.
    3. Select the created CDN resource (the list of resources will contain its primary domain name: cdn.yandexcloud.example).
    4. Go to the Content tab.
    5. Click Purge cache.
    6. Select the purge type: Selective.
    7. Enter the path to the uploaded file: /index.html.
    8. Click Purge cache.

    1. Get the ID of the CDN resource that you created:

      yc cdn resource list

      Command output:

      +----------------------+--------------------------+--------------------------------+--------------------------------+--------+-------------------------------------------+
|          ID          |          CNAME           |           CREATED AT           |           UPDATED AT           | ACTIVE |                  OPTIONS                  |
+----------------------+--------------------------+--------------------------------+--------------------------------+--------+-------------------------------------------+
| bc837xptmpkhbc7xwioa | cdn.yandexcloud.example  | seconds:1637235693             | seconds:1637235693             | true   | edge_cache_settings:{enabled:true         |
|                      |                          | nanos:434085000                | nanos:434115000                |        | default_value:345600}                     |
|                      |                          |                                |                                |        | cache_http_headers:{value:"accept-ranges" |
|                      |                          |                                |                                |        | value:"cache-control" value:"connection"  |
|                      |                          |                                |                                |        | value:"content-encoding"                  |
|                      |                          |                                |                                |        | value:"content-length"                    |
|                      |                          |                                |                                |        | value:"content-type"                      |
|                      |                          |                                |                                |        | value:"date" value:"etag"                 |
|                      |                          |                                |                                |        | value:"expires" value:"keep-alive"        |
|                      |                          |                                |                                |        | value:"last-modified" value:"server"      |
|                      |                          |                                |                                |        | value:"vary"} stale:{enabled:true         |
|                      |                          |                                |                                |        | value:"error" value:"updating"}           |
|                      |                          |                                |                                |        | allowed_http_methods:{value:"GET"         |
|                      |                          |                                |                                |        | value:"POST" value:"HEAD"                 |
|                      |                          |                                |                                |        | value:"OPTIONS"}                          |
+----------------------+--------------------------+--------------------------------+--------------------------------+--------+-------------------------------------------+

    2. Delete the file from the cache:

      yc cdn cache purge \
  --resource-id <ID of the CDN resource> \
  --path "/index.html"
    1. Get the ID of the CDN resource that you created using the gRPC API ResourceService/List call or the REST API list method.
    2. Delete the index.html file from the cache using the gRPC API CacheService/Purge call or the purge method.

  3. Forward all traffic of the cdn.yandexcloud.example domain name back to the canary-backend-blue backend running version 1:

    1. In the management console, select the example-folder folder.

    2. In the list of services, select Application Load Balancer and go to the Backend groups tab.

    3. Select canary-bg-production in the backend group list.

    4. For the canary-backend-blue backend, set the weight to 100 instead of 0:

      1. In the Backends section, find canary-backend-blue, then click → Edit.
      2. In the Weight field, enter 100.
      3. Tap Save.

    5. Similarly, set the weight to 0 instead of 100 for canary-bucket-green.

    6. Tap Save.

    1. For the canary-backend-blue backend, set the weight to 100 instead of 0:

      yc alb backend-group update-http-backend \
  --backend-group-name canary-bg-production \
  --name canary-backend-blue \
  --weight 100

      Command output:

      done (1s)
id: ds7l9puc18c9b40cd359
name: canary-bg-production
folder_id: b1g9hv2loamqfnbul7d9
http:
  backends:
  - name: canary-backend-blue
    backend_weight: "100"
    storage_bucket:
      bucket: canary-bucket-blue
created_at: "2021-11-03T10:28:47.680825561Z"

    2. For canary-backend-green, set the weight to 0 instead of 100:

      yc alb backend-group update-http-backend \
  --backend-group-name canary-bg-production \
  --name canary-backend-green \
  --weight 0

      Command output:

      done (1s)
id: ds7l9puc18c9b40cd359
name: canary-bg-production
folder_id: b1g9hv2loamqfnbul7d9
http:
  backends:
  - name: canary-backend-green
    backend_weight: "0"
    storage_bucket:
      bucket: canary-bucket-green
created_at: "2021-11-03T10:28:47.680825561Z"

    Use the gRPC API BackendGroupService/UpdateBackend call of the REST API updateBackend method.

  4. Open the browser and go to https://cdn.yandexcloud.example/index.html, refreshing the page several times. In all other cases, you should see a page indicating version 1.

  5. Similarly to steps 1–2, switch all the traffic for the cdn-staging.yandexcloud.example domain name to canary-backend-green running version 2 and check the switching in the browser.

  6. Re-enable caching:

    1. In the management console, select the example-folder folder.
    2. In the list of services, select Cloud CDN.
    3. Select the created CDN resource (the list of resources will contain its primary domain name: cdn.yandexcloud.example).
    4. Go to the Caching tab
    5. Click Edit.
    6. Enable CDN caching.
    7. Tap Save.
    1. Get the ID of the CDN resource that you created using the gRPC API ResourceService/List call or the REST API list method.
    2. To enable caching, use the gRPC API ResourceService/Update call or the REST API list method.

Delete the resources you created

To shut down the infrastructure and stop paying for the created resources:

  1. If you set up CNAME records in Cloud DNS, delete canary-dns-zone.
  2. Delete the CDN resource with the primary cdn.yandexcloud.example domain name.
  3. Delete the canary-balancer L7 load balancer.
  4. Delete all objects from the canary-bucket-blue and canary-bucket-green buckets.
  5. Delete the canary-bucket-blue and canary-bucket-green buckets.
  6. Delete the canary-subnet-ru-central1-a, canary-subnet-ru-central1-b, and canary-subnet-ru-central1-c subnets.
  7. Delete canary-network.
Language / Region
© 2022 Yandex.Cloud LLC
In this article: