Get started
Yandex Cloud CDN

Publishing game updates using Yandex Cloud CDN

Written by

Create and configure a Yandex Cloud CDN CDN resource to host content that is expected to handle a large number of requests over a short period of time, such as game update files (patches, DLC, and so on). To prevent CDN servers from increasing the workload on the content origins during this period, files will be preloaded to the server cache once.

We assume a patch is a single file named ycgame-update-v1.1.exe. It will be uploaded to a Yandex Object Storage bucket.

Note

We don't recommend preloading files smaller than 200 MB or larger than 5 GB.

To create a CDN infrastructure:

  1. Before you start.
  2. Create buckets in Object Storage.
  3. Enable logging for the bucket with files.
  4. Upload a file to the bucket.
  5. Create a CDN resource and enable caching.
  6. Set up DNS for your domain.
  7. Preload content to the cache of CDN servers.
  8. Test the CDN.

If you no longer need these resources, delete them.

Before you begin

Before working, you need to register in Yandex Cloud and create a billing account:

  1. Go to the management console. Then log in to Yandex Cloud or sign up if don't already have an account.
  2. On the billing page, make sure you linked a billing account, and it has the ACTIVE or TRIAL_ACTIVE status. If you don't have a billing account, create one.

If you have an active billing account, you can create or select a folder to run your VM in from the Yandex Cloud page.

Learn more about clouds and folders.

Make sure that you have a domain name and can access the DNS settings on the site of the company that provides DNS hosting to you. This is usually the company that registered your domain.

The cost of supporting the CDN infrastructure includes:

  • A fee for outgoing traffic from CDN servers (see Cloud CDN pricing).
  • A fee for data storage in Object Storage, operations with data, and outgoing traffic (see Object Storage pricing).
  • A fee for public DNS queries and DNS zones if you use Yandex Cloud DNS (see Cloud DNS pricing).

Create buckets in Object Storage

You must create two buckets: one, ycprojektblue-storage, will store files, and the other, ycprojektblue-logs, request logs for the first one.

  1. In the management console, select Object Storage.

  2. Create a bucket for files:

    1. Click Create bucket.
    2. Enter the bucket Name: ycprojektblue-storage.
    3. In the Object read access and Object listing access fields, select Public.
    4. Click Create bucket.

  3. Create a bucket for logs:

    1. Click Create bucket.
    2. Enter the bucket Name: ycprojektblue-logs.
    3. Click Create bucket.

  1. Create a bucket for files:

    aws --endpoint-url=https://storage.yandexcloud.net \
  s3api create-bucket \
  --bucket ycprojektblue-storage \
  --acl public-read

    Result:

    {
    "Location": "/ycprojektblue-storage"
}

  2. Create a bucket for logs:

    aws --endpoint-url=https://storage.yandexcloud.net \
  s3api create-bucket \
  --bucket ycprojektblue-logs

    Result:

    {
    "Location": "/ycprojektblue-logs"
}

If you don't have Terraform, install it and configure the Yandex Cloud provider.

Before you start, retrieve the static access keys: a secret key and a key ID used for authentication in Object Storage.

  1. In the configuration file, describe the bucket parameters:

    provider "yandex" {
  token     = "<OAuth>"
  cloud_id  = "<cloud ID>"
  folder_id = "<folder ID>"
  zone      = "ru-central1-a"
}

resource "yandex_storage_bucket" "storage" {
  access_key = "<static key ID>"
  secret_key = "<secret key>"
  bucket     = "ycprojektblue-storage"
  acl        = "public-read"
}

resource "yandex_storage_bucket" "logs" {
  access_key = "<static key ID>"
  secret_key = "<secret key>"
  bucket     = "ycprojektblue-logs"
}

Where:

  • access_key: The ID of the static access key.
  • secret_key: The value of the secret access key.
  • bucket: The name of the created bucket (ycprojektblue-storage).

  1. Make sure that the configuration files are correct:

    1. In the command line, go to the directory where you created the configuration file.

    2. Run the check using the command:

      terraform plan

    If the configuration is described correctly, the terminal displays the parameters of the bucket being created. If the configuration contain errors, Terraform will point them out.

  2. Deploy the bucket:

    1. If the configuration doesn't contain any errors, run the command:

      terraform apply

    2. Confirm that you want to create the bucket.

Use the API create method.

Enable logging for the bucket with files

You need to check that, when user requests are made, files are downloaded from the CDN server cache rather than directly from the bucket. To do this, enable bucket logging.

Use the API method putBucketLogging for bucket ycprojektblue-storage. HTTP request body:

<BucketLoggingStatus xmlns="http://doc.s3.amazonaws.com/2006-03-01">
  <LoggingEnabled>
    <TargetBucket>ycprojektblue-logs</TargetBucket>
  </LoggingEnabled>
</BucketLoggingStatus>

Where TargetBucket is the name of the bucket to write logs to (ycprojektblue-logs).

Upload a file to the bucket

  1. In the management console, select Object Storage.
  2. Select ycprojektblue-storage.
  3. Click Upload.
  4. In the window that opens, select the ycgame-update-v1.1.exe patch file and click Open.
  5. Click Upload.

Run the command:

aws --endpoint-url=https://storage.yandexcloud.net \
  s3 cp \
  <path to the file ycgame-update-v1.1.exe> \
  s3://ycprojektblue-storage/ycgame-update-v1.1.exe

Result:

upload: <path to ycgame-update-v1.1.exe> to s3://ycprojektblue-storage/ycgame-update-v1.1.exe

  1. Add the parameters of the object to upload to the configuration file you created in the bucket creation step:

    ...

resource "yandex_storage_object" "patch-v1-1" {
  access_key = "<static key ID>"
  secret_key = "<secret key>"
  bucket = "ycprojektblue-storage"
  key    = "ycgame-update-v1.1.exe"
  source = "<file path>/ycgame-update-v1.1.exe"
}

    Where:

    • bucket: The name of the bucket to add the object to (ycprojektblue-storage).
    • key: The name of the object in the bucket (ycgame-update-v1.1.exe). Required parameter.
    • source: A relative or absolute path to the file that you upload as an object.

  2. Make sure that the configuration files are correct.

    1. Using the command line, change to the directory with the configuration file.

    2. Run the check using the command:

      terraform plan

    If the configuration is described correctly, the terminal displays a list of created resources and their parameters. If the configuration contain errors, Terraform will point them out.

  3. Deploy the cloud resources.

    1. If the configuration doesn't contain any errors, run the command:
    terraform apply
    1. Confirm that you want to create the object.

Use the API upload method.

Create a CDN resource and enable caching

  1. In the management console, select Cloud CDN.

  2. If the CDN provider hasn't been activated yet, click Activate provider.

  3. Create a CDN resource:

    1. On the CDN resources tab, click Create resource.

    2. Set up the main parameters of the CDN resource as follows:

      • Content query: From one origin.

      • Origin type: Bucket.

      • Bucket: ycprojektblue-storage.

      • Domain names for content distribution: The primary domain name that you'll use to publish patches, such as cdn.ycprojectblue.example.

        Alert

        You can't change the primary domain name used for content distribution after you create a CDN resource.

      • In the Advanced section:

        • In the Origin request protocol field, select HTTPS.
        • In the Redirect clients field, select HTTP to HTTPS.
        • Enable End-user access to content.
        • In the Certificate type field, select Let's Encrypt® to automatically issue a certificate for the cdn.ycprojektblue.example domain name after creating your CDN resource.
        • In the Host header field, select Custom. In the Header value field, specify the origin domain name, ycprojektblue-storage.storage.yandexcloud.net, so that the source bucket responds correctly to CDN server requests.

    3. Click Create.

  4. Enable caching on CDN servers for the resource:

    1. Select the created CDN resource (the list of resources will contain its primary domain name: cdn.ycprojektblue.example).
    2. Go to the Caching tab.
    3. Click Edit.
    4. Enable CDN caching.
    5. Click Save.

If you don't have the Yandex Cloud command line interface yet, install and initialize it.

The folder specified in the CLI profile is used by default. You can specify a different folder using the --folder-name or --folder-id parameter.

  1. If the CDN provider hasn't been activated yet, run the command:

    yc cdn provider activate --folder-id <folder ID> --type gcore

  2. Create a CDN resource:

    yc cdn resource create \
  --cname cdn.ycprojectblue.example \
  --origin-bucket-source ycprojektblue-storage.storage.yandexcloud.net \
  --origin-bucket-name ycprojektblue-storage \
  --origin-protocol https \
  --lets-encrypt-gcore-ssl-cert \
  --host-header ycprojektblue-storage.storage.yandexcloud.net \
  --redirect-http-to-https

    Result:

    id: bc8e3l7s4dhadigoh3jr
folder_id: b1g86q4m5vej8lkljme5
cname: cdn.ycprojektblue.example
...
active: true
...

    For more information about the yc cdn resource create command, see the CLI reference.

  1. Add parameters of CDN resources to the configuration file:

    ...

resource "yandex_cdn_origin_group" "my_group" {
  name     = "updates-origin-group"
  use_next = true
  origin {
    source = "ycprojektblue-storage.storage.yandexcloud.net"
  }
}

resource "yandex_cdn_resource" "my_resource" {
  cname               = "cdn.ycprojectblue.example"
  active              = true
  origin_protocol     = "https"
  origin_group_id     = yandex_cdn_origin_group.my_group.id
  options {
    redirect_https_to_http = true
    custom_host_header     = "ycprojektblue-storage.storage.yandexcloud.net"
  }
  ssl_certificate {
    type = "lets_encrypt_gcore"
  }
}

    For more information, see the descriptions of the yandex_cdn_origin_group and yandex_cdn_resource resources in the Terraform provider documentation.

  2. Make sure that the configuration files are correct.

    1. In the command line, go to the directory where you created the configuration file.

    2. Run the check using the command:

      terraform plan

    If the configuration is described correctly, the terminal displays a list of created resources and their parameters. This is a test step. No resources are created. If the configuration contain errors, Terraform will point them out.

  3. Apply the configuration changes:

    1. If the configuration doesn't contain any errors, run the command:

      terraform apply

    2. Confirm the resource creation: type yes in the terminal and press Enter.

    Afterwards, all the necessary resources are created in the specified folder. You can check that the resources are there with the correct settings using the management console.

Use the gRPC API ResourceService/Create call or the REST API create method. To enable caching on CDN servers, add the edge_cache_settings field to the request body.

Set up DNS for your domain

  1. Get a domain name generated for the created CDN resource on the .edgecdn.ru domain:

    1. In the management console, select Cloud CDN.
    2. Select the created CDN resource (the list of resources will contain its primary domain name: cdn.ycprojektblue.example).
    3. On the Overview tab, under DNS settings, copy the name generated on the .edgecdn.ru domain to the clipboard.

  2. Go to your domain's DNS settings on the site of your DNS hosting provider.

  3. Edit the CNAME record for cdn so that it points to the previously copied URL on the .edgecdn.ru domain, for example:

    cdn CNAME cl-.....6bb.edgecdn.ru.

    If you use Cloud DNS, follow these instructions to configure the record:

    Instructions for configuring DNS records for Cloud DNS

    1. In the management console, select Cloud DNS.

    2. If you don't have a public DNS zone, create one:

      1. Click Create zone.
      2. Name the zone: cdn-dns-a.
      3. In the Zone field, specify your domain with a dot at the end: ycprojektblue.example.
      4. Select the zone Type: Public.
      5. Click Create.

    3. Create a record in the zone:

      1. In the list of zones, click cdn-dns-a.
      2. Click Create record.
      3. In the Name field, specify cdn to make the record match the cdn.ycprojektblue.example domain name.
      4. Select the record Type: CNAME.
      5. In the Value field, paste the URL you copied on the .edgecdn.ru domain with a dot at the end.
      6. Click Create.

    1. If you don't have a public DNS zone, create one:

      yc dns zone create --name cdn-dns-a --zone ycprojektblue.example. --public-visibility

      Where:

      • --name: Zone name.
      • --zone: Domain zone (your domain with a dot at the end).
      • --public-visibility: Zone visibility (indicates if it's public).

      Result:

      id: aetuvdw77q61dwbl1z2d
folder_id: aoewzf73jwdlm1xtp4dd
created_at: "2021-09-28T10:33:31.917Z"
name: cdn-zone-a
zone: ycprojektblue.example.
public_visibility: {}

    2. Create a record in the zone:

      yc dns zone add-records --name cdn-dns-a --record "cdn CNAME cl-.....6bb.edgecdn.ru."

      Where:

      • --name: Zone name.
      • --record: Resource record.

    3. Check that the record was created:

      yc dns zone list-records --name cdn-dns-a

      Result:

      +----------------------------+------+-------+--------------------------------+
|            NAME            | TTL  | TYPE  |              DATA              |
+----------------------------+------+-------+--------------------------------+
| ycprojektblue.example.     | 3600 | NS    | ns1.yandexcloud.net.           |
|                            |      |       | ns2.yandexcloud.net.           |
| ycprojektblue.example.     | 3600 | SOA   | ns1.yandexcloud.net.           |
|                            |      |       | mx.cloud.yandex.net. 1 10800   |
|                            |      |       | 900 604800 86400               |
| cdn.ycprojektblue.example. |  600 | CNAME | cl-.....6bb.edgecdn.ru.           |
+----------------------------+------+-------+--------------------------------+

      The list should contain a record named cdn.ycprojektblue.example.

    1. If you do not have a public DNS zone, create one using a gRPC API call to DnsZoneService/Create or the REST API create method. To make the zone public, add the public_visibility (gRPC) or publicVisibility (REST) field to the request body.
    2. In the zone, create a record named cdn CNAME cl-.....6bb.edgecdn.ru. using a gRPC API DnsZoneService/UpdateRecordSets call or the REST API updateRecordSets method.

Preload content to the cache of CDN servers

  1. In the management console, select Cloud CDN.

  2. Select the created CDN resource (the list of resources will contain its primary domain name: cdn.ycprojektblue.example).

  3. Go to the Content tab.

  4. Click Preload content.

  5. In the File path field, specify the path to the file stored in the origin, omitting the domain name:

    /ycgame-update-v1.1.exe

  6. Click Preload content.

If you don't have the Yandex Cloud command line interface yet, install and initialize it.

The folder specified in the CLI profile is used by default. You can specify a different folder using the --folder-name or --folder-id parameter.

Specify the path to the file to pre-fetch:

yc cdn cache prefetch --resource-id <resource ID> \
  --path /ycgame-update-v1.1.exe

For more information about the yc cdn cache prefetch command, see the CLI reference.

Use the gRPC API CacheService/Prefetch call or the prefetch REST API method.

Test the CDN

  1. Wait for the DNS records to be updated and the file to be preloaded to CDN servers.

  2. Download the file at the new URL:

    https://cdn.ycprojektblue.example/ycgame-update-v1.1.exe

  3. Get the logs of requests to the ycprojektblue-storage bucket:

    1. In the management console, select Object Storage.
    2. Select ycprojektblue-logs.
    3. Click on the name of the object corresponding to the ycgame-update-v1.1.exe download time.
    4. Click Download.

    1. Get a list of objects with logs:

      aws --endpoint-url=https://storage.yandexcloud.net \
  s3 ls s3://ycprojektblue-logs

      Result:

      2021-10-01 08:37:53         10 2021-10-01-08-37-53-631E0FC3B732AEDD
2021-10-01 09:38:05         62 2021-10-01-09-38-05-436E6CDC832A20EF
2021-10-01 10:38:01         23 2021-10-01-10-38-01-7F65EF1A6366FB18
2021-10-01 11:37:57         41 2021-10-01-11-37-57-6F31613427A7DB9A
2021-10-01 12:38:02         58 2021-10-01-12-38-02-AB893E6148AFDC55
2021-10-01 13:38:02         73 2021-10-01-13-38-02-E69EAEC1C9083756

    2. In the resulting list, find the object with the log saved after downloading ycgame-update-v1.1.exe and download it:

      aws --endpoint-url=https://storage.yandexcloud.net \
  s3 cp s3://ycprojektblue-logs/2021-10-01-13-38-02-E69EAEC1C9083756 \
  2021-10-01-13-38-02-E69EAEC1C9083756

      Result:

      download: s3://ycprojektblue-logs/2021-10-01-13-38-02-E69EAEC1C9083756 to 2021-10-01-13-38-02-E69EAEC1C9083756

    1. Get a list of objects with logs in the ycprojektblue-logs bucket using the listObjects.

    2. In the resulting list, find the object whose log was saved after downloading ycgame-update-v1.1.exe and download it using the API get method.

  4. Check the logs of requests to the source bucket to make sure that the CDN servers did not download the file from the origin after your request. For more information about log contents, see the Format of the log object section of the Object Storage documentation.

How to delete created resources

To shut down your CDN resource and stop paying for the created resources:

  1. Disable the created resource.
  2. Delete the ycgame-update-v1.1.exe object from the ycprojektblue-storage bucket.
  3. Delete the ycprojektblue-storage bucket.
  4. Delete all objects from the ycprojektblue-logs bucket.
  5. Delete the ycprojektblue-logs bucket.
  6. Delete the DNS zone if you used it during DNS setup.
In this article: