Yandex Cloud
Search
Contact UsGet started
  • Blog
  • Pricing
  • Documentation
Yandex project
© 2023 Intertech Services AG
Yandex Object Storage
  • Getting started
  • Log reference
  • Access management
  • Questions and answers
  1. Step-by-step guides
  2. Buckets
  3. Enabling logging

Logging actions with the bucket

Written by
Yandex Cloud
,
improved by
Mikhail P.
  • Enable logging
  • Get the logging settings
  • Get the logs
  • Disable logging

To track operations with the bucket, enable logging. Detailed information about requests to the source bucket will be saved in an object in the target bucket. However, Object Storage does not guarantee that the logs are saved in a complete and timely manner.

By default, logging is disabled. After you enable it, Object Storage will save information about actions with the bucket once an hour.

Enable loggingEnable logging

To log requests to the bucket:

  1. Use an existing target bucket or create a new one. This is the bucket your logs will be written to.

    How to create a bucket

    The minimum role required to create a bucket is storage.editor. See the role description for details.

    Management console
    Yandex Cloud CLI
    AWS CLI
    Terraform
    API
    1. In the management console, select the folder where you want to create a bucket.
    2. Select Object Storage.
    3. Click Create bucket.
    4. On the bucket creation page:
      1. Enter a name for the bucket according to the naming requirements.

        By default, a bucket with a dot in the name is only available over HTTP. To provide HTTPS support for your bucket, upload your own security certificate to Object Storage.

      2. Limit the maximum bucket size, if required.

        If the value is 0, the maximum size is not limited and is similar to the enabled No limit option.

      3. Select the type of access.

      4. Select the default storage class.

      5. Add labels, if needed:

        1. Click Add label.
        2. Enter a label in key: value format.
        3. Click Enter.
      6. Click Create bucket to complete the operation.

    If you don't have the Yandex Cloud command line interface yet, install and initialize it.

    The folder specified in the CLI profile is used by default. You can specify a different folder using the --folder-name or --folder-id parameter.

    1. View a description of the CLI command to create a bucket:

      yc storage bucket create --help
      
    2. Create a bucket in the default folder:

      yc storage bucket create \
        --name <bucket_name> \
        --default-storage-class <storage_class> \
        --max-size <maximum_bucket_size> \
        --public-read \
        --public-list \
        --public-config-read
      

      Where:

      • --name: Bucket name. For more information, see Bucket naming rules.

        By default, a bucket with a dot in the name is only available over HTTP. To provide HTTPS support for your bucket, upload your own security certificate to Object Storage.

      • --default-storage-class: Storage class. The possible values include:

        • standard: Standard storage
        • cold: Cold storage
        • ice: Ice storage
      • --max_size: Maximum bucket size (bytes) 0: No limit

      • --public-read: Flag to enable public read access to bucket objects.

      • --public-list: Flag to enable public access to view the list of bucket objects.

      • --public-config-read: Flag to enable public read access to the bucket configuration.

      The name parameter is required. Other parameters are optional. By default, the --max-size parameter is set to 0 and public access to the bucket is disabled.

      Result:

      name: <bucket_name>
      folder_id: b1gmit33ngp6********
      anonymous_access_flags:
        read: false
        list: false
      default_storage_class: STANDARD
      versioning: VERSIONING_DISABLED
      max_size: "53687091200"
      acl: {}
      created_at: "2022-12-16T14:05:12.196007Z"
      

    If you do not have the AWS CLI yet, install and configure it.

    In the terminal, run the command below by specifying the bucket name and the Object Storage endpoint:

    aws --endpoint-url=https://storage.yandexcloud.net \
      s3 mb s3://<bucket_name>
    

    Note

    By default, a bucket with a dot in the name is only available over HTTP. To provide HTTPS support for your bucket, upload your own security certificate to Object Storage. For more information, see Bucket naming rules.

    If you don't have Terraform, install it and configure the Yandex Cloud provider.

    1. In the configuration file, describe the parameters of the resources you want to create:

      terraform {
        required_providers {
          yandex = {
            source = "yandex-cloud/yandex"
          }
        }
        required_version = ">= 0.13"
      }
      
      provider "yandex" {
        token     = "<IAM_or_OAuth_token>"
        cloud_id  = "<cloud_ID>"
        folder_id = "<folder_ID>"
        zone      = "ru-central1-a"
      }
      
      resource "yandex_iam_service_account" "sa" {
        name = "<service_account_name>"
      }
      
      // Assigning roles to the service account
      resource "yandex_resourcemanager_folder_iam_member" "sa-editor" {
        folder_id = "<folder_ID>"
        role      = "storage.editor"
        member    = "serviceAccount:${yandex_iam_service_account.sa.id}"
      }
      
      // Creating a static access key
      resource "yandex_iam_service_account_static_access_key" "sa-static-key" {
        service_account_id = yandex_iam_service_account.sa.id
        description        = "static access key for object storage"
      }
      
      // Creating a bucket using the key
      resource "yandex_storage_bucket" "test" {
        access_key = yandex_iam_service_account_static_access_key.sa-static-key.access_key
        secret_key = yandex_iam_service_account_static_access_key.sa-static-key.secret_key
        bucket     = "<bucket_name>"
      }
      

      Where:

      • yandex_iam_service_account: Description of the service account that will create and use a bucket:
        • name: Service account name
      • yandex_storage_bucket: Bucket description:
        • bucket: Bucket name

      For more information about resources you can create using Terraform, see the provider documentation.

    2. Make sure the configuration files are valid.

      1. In the command line, go to the directory where you created the configuration file.
      2. Run a check using this command:
        terraform plan
        

      If the configuration is described correctly, the terminal will display a list of created resources and their parameters. If the configuration contains any errors, Terraform will point them out.

    3. Deploy cloud resources.

      1. If the configuration does not contain any errors, run this command:

        terraform apply
        
      2. Confirm that you want to create the resources.

      All the resources you need will then be created in the specified folder. You can check the new resources and their configuration using the management console.

    To create a bucket, use the create REST API method for the Bucket resource, the BucketService/Create gRPC API call, or the create S3 API method.

Note

To write logs to the target bucket with an access policy configured, add there a rule to allow any account to perform the PutObject action, and specify the log object key prefix, e.g., logs/, as a resource.

  1. Enable logging in the source bucket that you want to track.

    AWS CLI
    Terraform
    API

    To enable logging via the AWS CLI:

    1. Create a file with logging settings in JSON format, e.g.:

      {
         "LoggingEnabled": {
            "TargetBucket": "<target_bucket_name>",
            "TargetPrefix": "<key_prefix>"
         }
      }
      

      Where:

      • TargetBucket: Name of the target bucket to write logs to.
      • TargetPrefix: Prefix of the key used for log objects, e.g., logs/.
    2. Enable logging in the bucket:

      aws s3api put-bucket-logging \
        --bucket <source_bucket_name> \
        --endpoint-url https://storage.yandexcloud.net \
        --bucket-logging-status file://<path_to_settings_file>
      

      Where:

      • --bucket: Name of the source bucket to enable action logging for.
      • --bucket-logging-status: Path to the logging settings file.

    With Terraform, you can quickly create a cloud infrastructure in Yandex Cloud and manage it using configuration files. They store the infrastructure description in HashiCorp Configuration Language (HCL). Terraform and its providers are distributed under the Business Source License.

    For more information about the provider resources, see the documentation on the Terraform website or mirror website.

    If you change the configuration files, Terraform automatically determines which part of your configuration is already deployed and what should be added or removed.

    If you don't have Terraform, install it and configure the Yandex Cloud provider.

    To enable logging for a bucket that you wish to monitor:

    1. Open the Terraform configuration file and add a section called logging to the fragment describing the bucket.

      resource "yandex_storage_bucket" "log_bucket" {
        access_key = "<static_key_ID>"
        secret_key = "<private_key>"
        bucket     = "<name_of_bucket_to_store_logs>"
      }
      
      resource "yandex_storage_bucket" "bucket" {
        access_key = "<static_key_ID>"
        secret_key = "<private_key>"
        bucket     = "<source_bucket_name>"
        acl        = "private"
      
        logging {
          target_bucket = yandex_storage_bucket.log_bucket.id
          target_prefix = "log/"
        }
      }
      

      Where:

      • access_key: ID of the static access key.
      • secret_key: Value of the secret access key.
      • target_bucket: Reference to the bucket that will store logs.
      • target_prefix: Prefix of the key used for log objects, e.g., logs/.

      For more information about yandex_storage_bucket resource parameters in Terraform, see the provider documentation.

    1. In the terminal, change to the folder where you edited the configuration file.

    2. Make sure the configuration file is correct using the command:

      terraform validate
      

      If the configuration is correct, the following message is returned:

      Success! The configuration is valid.
      
    3. Run the command:

      terraform plan
      

      The terminal will display a list of resources with parameters. No changes are made at this step. If the configuration contains errors, Terraform will point them out.

    4. Apply the configuration changes:

      terraform apply
      
    5. Confirm the changes: type yes in the terminal and press Enter.

    All the resources you need will then be created in the specified folder. You can check the new resources and their configuration using the management console.

    To enable logging for your bucket, use the putBucketLogging S3 API method.

    Example of the HTTP request body:

    <BucketLoggingStatus xmlns="http://doc.s3.amazonaws.com/2006-03-01">
      <LoggingEnabled>
        <TargetBucket>target bucket name</TargetBucket>
        <TargetPrefix>logs/</TargetPrefix>
      </LoggingEnabled>
    </BucketLoggingStatus>
    

    Where:

    • <TargetBucket>: Target bucket name.
    • <TargetPrefix>: Prefix of the key used for log objects, e.g., logs/. This is an optional parameter.

Get the logging settingsGet the logging settings

To get the name of the target bucket and the prefix of the key for the log object, follow these steps:

AWS CLI
API

To retrieve the logging settings via the AWS CLI:

  1. Run this command:

    aws s3api get-bucket-logging \
      --bucket <bucket_name> \
      --output json \
      --endpoint-url https://storage.yandexcloud.net
    

    Result:

    {
       "LoggingEnabled": {
          "TargetBucket": "<bucket_name>",
          "TargetPrefix": "<key_prefix>"
       }
    }
    

Use the getBucketLogging S3 API method.

Example of the HTTP response body:

HTTP/1.1 200
<?xml version="1.0" encoding="UTF-8"?>
<BucketLoggingStatus>
   <LoggingEnabled>
      <TargetBucket>target bucket name</TargetBucket>
      <TargetPrefix>logs/</TargetPrefix>
   </LoggingEnabled>
</BucketLoggingStatus>

Get the logsGet the logs

To get logs, download the object prefixed with logs/ from the target bucket:

Management console
  1. In the management console, select the folder where the target bucket with logs is located.
  2. Select Object Storage.
  3. Click the name of the bucket with the logs.
  4. Click on the object name with the logs/ prefix.
  5. Click Download.

Disable loggingDisable logging

To disable logging, follow these steps:

Terraform
API

With Terraform, you can quickly create a cloud infrastructure in Yandex Cloud and manage it using configuration files. They store the infrastructure description in HashiCorp Configuration Language (HCL). Terraform and its providers are distributed under the Business Source License.

For more information about the provider resources, see the documentation on the Terraform website or mirror website.

If you change the configuration files, Terraform automatically determines which part of your configuration is already deployed and what should be added or removed.

If you don't have Terraform, install it and configure the Yandex Cloud provider.

To disable the logging mechanism:

  1. Open the Terraform configuration file and delete the logging section from the fragment describing the bucket.

    Example bucket description in a Terraform configuration
    ...
    resource "yandex_storage_bucket" "log_bucket" {
      access_key = "<static_key_ID>"
      secret_key = "<private_key>"
      bucket     = "<name_of_bucket_to_store_logs>"
    }
    
    resource "yandex_storage_bucket" "bucket" {
      access_key = "<static_key_ID>"
      secret_key = "<private_key>"
      bucket     = "<source_bucket_name>"
      acl        = "private"
    
      logging {
        target_bucket = yandex_storage_bucket.log_bucket.id
        target_prefix = "log/"
      }
    }
    ...
    
  1. In the terminal, change to the folder where you edited the configuration file.

  2. Make sure the configuration file is correct using the command:

    terraform validate
    

    If the configuration is correct, the following message is returned:

    Success! The configuration is valid.
    
  3. Run the command:

    terraform plan
    

    The terminal will display a list of resources with parameters. No changes are made at this step. If the configuration contains errors, Terraform will point them out.

  4. Apply the configuration changes:

    terraform apply
    
  5. Confirm the changes: type yes in the terminal and press Enter.

You can verify the changes in the management console.

Use the putBucketLogging S3 API method. In the request body, send the <BucketLoggingStatus> parameter with an empty value.

Example of the HTTP request body:

<BucketLoggingStatus xmlns="http://doc.s3.amazonaws.com/2006-03-01" />

Was the article helpful?

Yandex project
© 2023 Intertech Services AG
In this article:
  • Enable logging
  • Get the logging settings
  • Get the logs
  • Disable logging