Logging actions with the bucket

To track operations with the bucket, enable logging. Detailed information about requests to the source bucket will be saved in an object in the target bucket. However, Object Storage does not guarantee that the logs are saved in a complete and timely manner.

By default, logging is disabled. After you enable it, Object Storage will save information about actions with the bucket once an hour.

Enable loggingEnable logging

To log requests to the bucket:

1. Use an existing target bucket or create a new one. This is the bucket your logs will be written to.

   How to create a bucket

   The minimum role required to create a bucket is storage.editor. See the role description for details.

   Management console

   Yandex Cloud CLI

   AWS CLI

   Terraform

   API

   1. In the management console, select the folder where you want to create a bucket.
   2. Select Object Storage.
   3. Click Create bucket.
   4. On the bucket creation page:

      1. Enter a name for the bucket according to the naming requirements.

         By default, a bucket with a dot in the name is only available over HTTP. To provide HTTPS support for your bucket, upload your own security certificate to Object Storage.

      2. Limit the maximum bucket size, if required.

         If the value is 0, the maximum size is not limited and is similar to the enabled No limit option.

      3. Select the type of access.

      4. Select the default storage class.

      5. Add labels, if needed:

         1. Click Add label.
         2. Enter a label in key: value format.
         3. Click Enter.

      6. Click Create bucket to complete the operation.

   If you don't have the Yandex Cloud command line interface yet, install and initialize it.

   The folder specified in the CLI profile is used by default. You can specify a different folder using the --folder-name or --folder-id parameter.

   1. View a description of the CLI command to create a bucket:

      yc storage bucket create --help
      

   2. Create a bucket in the default folder:

      yc storage bucket create \
        --name <bucket_name> \
        --default-storage-class <storage_class> \
        --max-size <maximum_bucket_size> \
        --public-read \
        --public-list \
        --public-config-read
      

      Where:

      • --name: Bucket name. For more information, see Bucket naming rules.

        By default, a bucket with a dot in the name is only available over HTTP. To provide HTTPS support for your bucket, upload your own security certificate to Object Storage.

      • --default-storage-class: Storage class. The possible values include:

        • standard: Standard storage
        • cold: Cold storage
        • ice: Ice storage

      • --max_size: Maximum bucket size (bytes) 0: No limit

      • --public-read: Flag to enable public read access to bucket objects.

      • --public-list: Flag to enable public access to view the list of bucket objects.

      • --public-config-read: Flag to enable public read access to the bucket configuration.

      The name parameter is required. Other parameters are optional. By default, the --max-size parameter is set to 0 and public access to the bucket is disabled.

      Result:

      name: <bucket_name>
      folder_id: b1gmit33ngp6********
      anonymous_access_flags:
        read: false
        list: false
      default_storage_class: STANDARD
      versioning: VERSIONING_DISABLED
      max_size: "53687091200"
      acl: {}
      created_at: "2022-12-16T14:05:12.196007Z"
      

   If you do not have the AWS CLI yet, install and configure it.

   In the terminal, run the command below by specifying the bucket name and the Object Storage endpoint:

   aws --endpoint-url=https://storage.yandexcloud.net \
     s3 mb s3://<bucket_name>
   

   Note

   By default, a bucket with a dot in the name is only available over HTTP. To provide HTTPS support for your bucket, upload your own security certificate to Object Storage. For more information, see Bucket naming rules.

   If you don't have Terraform, install it and configure the Yandex Cloud provider.

   1. In the configuration file, describe the parameters of the resources you want to create:

      terraform {
        required_providers {
          yandex = {
            source = "yandex-cloud/yandex"
          }
        }
        required_version = ">= 0.13"
      }
      
      provider "yandex" {
        token     = "<IAM_or_OAuth_token>"
        cloud_id  = "<cloud_ID>"
        folder_id = "<folder_ID>"
        zone      = "ru-central1-a"
      }
      
      resource "yandex_iam_service_account" "sa" {
        name = "<service_account_name>"
      }
      
      // Assigning roles to the service account
      resource "yandex_resourcemanager_folder_iam_member" "sa-editor" {
        folder_id = "<folder_ID>"
        role      = "storage.editor"
        member    = "serviceAccount:${yandex_iam_service_account.sa.id}"
      }
      
      // Creating a static access key
      resource "yandex_iam_service_account_static_access_key" "sa-static-key" {
        service_account_id = yandex_iam_service_account.sa.id
        description        = "static access key for object storage"
      }
      
      // Creating a bucket using the key
      resource "yandex_storage_bucket" "test" {
        access_key = yandex_iam_service_account_static_access_key.sa-static-key.access_key
        secret_key = yandex_iam_service_account_static_access_key.sa-static-key.secret_key
        bucket     = "<bucket_name>"
      }
      

      Where:

      • yandex_iam_service_account: Description of the service account that will create and use a bucket:
        • name: Service account name
      • yandex_storage_bucket: Bucket description:
        • bucket: Bucket name

      For more information about resources you can create using Terraform, see the provider documentation.

   2. Make sure the configuration files are valid.

      1. In the command line, go to the directory where you created the configuration file.
      2. Run a check using this command:

         terraform plan
         

      If the configuration is described correctly, the terminal will display a list of created resources and their parameters. If the configuration contains any errors, Terraform will point them out.

   3. Deploy cloud resources.

      1. If the configuration does not contain any errors, run this command:

         terraform apply
         

      2. Confirm that you want to create the resources.

      All the resources you need will then be created in the specified folder. You can check the new resources and their configuration using the management console.

   To create a bucket, use the create REST API method for the Bucket resource, the BucketService/Create gRPC API call, or the create S3 API method.

1. Enable logging in the source bucket that you want to track.

   AWS CLI

   Terraform

   API

   To enable logging via the AWS CLI:

   1. Create a file with logging settings in JSON format, e.g.:

      {
         "LoggingEnabled": {
            "TargetBucket": "<target_bucket_name>",
            "TargetPrefix": "<key_prefix>"
         }
      }
      

      Where:

      • TargetBucket: Name of the target bucket to write logs to.
      • TargetPrefix: Prefix of the key used for log objects, e.g., logs/.

   2. Enable logging in the bucket:

      aws s3api put-bucket-logging \
        --bucket <source_bucket_name> \
        --endpoint-url https://storage.yandexcloud.net \
        --bucket-logging-status file://<path_to_settings_file>
      

      Where:

      • --bucket: Name of the source bucket to enable action logging for.
      • --bucket-logging-status: Path to the logging settings file.

   With Terraform, you can quickly create a cloud infrastructure in Yandex Cloud and manage it using configuration files. They store the infrastructure description in HashiCorp Configuration Language (HCL). Terraform and its providers are distributed under the Business Source License.

   For more information about the provider resources, see the documentation on the Terraform website or mirror website.

   If you change the configuration files, Terraform automatically determines which part of your configuration is already deployed and what should be added or removed.

   If you don't have Terraform, install it and configure the Yandex Cloud provider.

   To enable logging for a bucket that you wish to monitor:

   1. Open the Terraform configuration file and add a section called logging to the fragment describing the bucket.

      resource "yandex_storage_bucket" "log_bucket" {
        access_key = "<static_key_ID>"
        secret_key = "<private_key>"
        bucket     = "<name_of_bucket_to_store_logs>"
      }
      
      resource "yandex_storage_bucket" "bucket" {
        access_key = "<static_key_ID>"
        secret_key = "<private_key>"
        bucket     = "<source_bucket_name>"
        acl        = "private"
      
        logging {
          target_bucket = yandex_storage_bucket.log_bucket.id
          target_prefix = "log/"
        }
      }
      

      Where:

      • access_key: ID of the static access key.
      • secret_key: Value of the secret access key.
      • target_bucket: Reference to the bucket that will store logs.
      • target_prefix: Prefix of the key used for log objects, e.g., logs/.

      For more information about yandex_storage_bucket resource parameters in Terraform, see the provider documentation.

   1. In the terminal, change to the folder where you edited the configuration file.

   2. Make sure the configuration file is correct using the command:

      terraform validate
      

      If the configuration is correct, the following message is returned:

      Success! The configuration is valid.
      

   3. Run the command:

      terraform plan
      

      The terminal will display a list of resources with parameters. No changes are made at this step. If the configuration contains errors, Terraform will point them out.

   4. Apply the configuration changes:

      terraform apply
      

   5. Confirm the changes: type yes in the terminal and press Enter.

   All the resources you need will then be created in the specified folder. You can check the new resources and their configuration using the management console.

   To enable logging for your bucket, use the putBucketLogging S3 API method.

   Example of the HTTP request body:

   <BucketLoggingStatus xmlns="http://doc.s3.amazonaws.com/2006-03-01">
     <LoggingEnabled>
       <TargetBucket>target bucket name</TargetBucket>
       <TargetPrefix>logs/</TargetPrefix>
     </LoggingEnabled>
   </BucketLoggingStatus>
   

   Where:

   • <TargetBucket>: Target bucket name.
   • <TargetPrefix>: Prefix of the key used for log objects, e.g., logs/. This is an optional parameter.

Get the logging settingsGet the logging settings

To get the name of the target bucket and the prefix of the key for the log object, follow these steps:

HTTP/1.1 200
<?xml version="1.0" encoding="UTF-8"?>
<BucketLoggingStatus>
   <LoggingEnabled>
      <TargetBucket>target bucket name</TargetBucket>
      <TargetPrefix>logs/</TargetPrefix>
   </LoggingEnabled>
</BucketLoggingStatus>

Get the logsGet the logs

To get logs, download the object prefixed with logs/ from the target bucket:

Disable loggingDisable logging

To disable logging, follow these steps:

<BucketLoggingStatus xmlns="http://doc.s3.amazonaws.com/2006-03-01" />