Yandex Cloud
  • Services
  • Solutions
  • Why Yandex Cloud
  • Pricing
  • Documentation
  • Contact us
Get started
Language / Region
© 2022 Yandex.Cloud LLC
Yandex Object Storage
  • Getting started
  • Step-by-step instructions
    • All instructions
    • Buckets
      • Creating a bucket
      • Deleting a bucket
      • Limiting the maximum size of a bucket
      • Allowing public access to a bucket
      • Bucket encryption
      • Managing object lifecycles
      • Managing CORS configurations
      • Managing bucket policies
      • Managing bucket versioning
      • Editing a bucket ACL
      • Enabling logging
    • Objects
      • Uploading an object
      • Getting information about an object
      • Downloading an object
      • Getting a public link to an object
      • Deleting an object
      • Deleting all objects
      • Deleting a partially uploaded object
      • Editing an object ACL
    • Static website hosting
      • Hosting setup
      • Support for your own domain
      • Support for multiple domain names
      • Configuring HTTPS
    • Access management
      • Applying an access policy
  • Practical guidelines
    • Getting object query statistics with S3 Select
    • Getting website traffic statistics with S3 Select
    • Generating a billing report with resource break-down using S3 Select
    • Server-side encryption
    • Integrating an L7 load balancer with the CDN and Object Storage
    • Blue-green and canary deployment
    • Analyzing logs in DataLens
  • Concepts
    • Overview
    • Bucket
    • Objects
    • Bucket versioning
    • Object lifecycles
    • CORS
    • Static website hosting
    • Pre-signed URLs
    • Multipart upload
    • Access control lists (ACLs)
    • Bucket Policy
    • Uploading files via an HTML form
    • Storage class
    • Logging actions with a bucket
    • Backups
    • TLS protocol
    • S3 Select query language
    • Quotas and limits
  • Tools
    • All tools
    • Console clients
      • AWS CLI
      • S3cmd
    • File browsers
      • CyberDuck
      • WinSCP
    • SDK
      • AWS SDK for Java
      • Python SDK (boto)
      • The AWS SDK for .NET
      • AWS SDK for C++
      • AWS SDK for PHP
      • AWS SDK for Go
    • FUSE
      • GeeseFS
      • s3fs
      • goofys
    • FTP
      • Docker container for (S)FTP(S)
  • API reference
    • REST (Amazon S3-compatible)
      • How to use the API
      • Signing requests
      • API reference
        • All services and methods
        • Bucket
          • All methods
          • create
          • getBucketEncryption
          • getMeta
          • listObjects
          • listBuckets
          • deleteBucket
          • deleteBucketEncryption
          • putBucketEncryption
          • putBucketVersioning
          • getBucketVersioning
          • putBucketLogging
          • getBucketLogging
          • listObjectVersions
        • Object
          • All methods
          • upload
          • get
          • copy
          • getObjectMeta
          • delete
          • deleteMultipleObjects
          • options
          • selectObjectContent
        • Multipart upload
          • General multipart upload procedure
          • startUpload
          • uploadPart
          • copyPart
          • listParts
          • abortUpload
          • completeUpload
          • listUploads
        • Static Website Hosting
          • All methods
          • upload
          • get
          • delete
          • Static website response codes
        • CORS
          • All methods
          • XML structure of CORS configuration
          • upload
          • get
          • delete
        • Lifecycles
          • All methods
          • XML structure of lifecycle configuration
          • upload
          • get
          • delete
        • ACL
          • All methods
          • XML structure of ACL configuration
          • objectGetAcl
          • objectPutAcl
          • bucketGetAcl
          • bucketPutAcl
        • Bucket Policy
          • All methods
          • GetBucketPolicy
          • PutBucketPolicy
          • DeleteBucketPolicy
          • Data schema
          • Actions
          • Conditions
        • Common request headers
        • Common response headers
        • Responses
    • gRPC and REST
      • Authentication in the API
      • gRPC
        • Overview
        • BucketService
        • OperationService
      • REST
        • Overview
        • Bucket
          • Overview
          • create
          • delete
          • deleteHTTPSConfig
          • get
          • getHTTPSConfig
          • getStats
          • list
          • setHTTPSConfig
          • update
  • Access management
  • Pricing policy
    • Current pricing policy
    • Archive
      • Before January 1, 2019
      • From January 1 to March 1, 2019
  • Questions and answers
  1. Concepts
  2. Logging actions with a bucket

Logging actions with a bucket

Written by
Yandex.Cloud
  • Prerequisites
  • Format of the key for the log object
    • Prefix of the key
  • Format of the log object
  • Logging specifics
    • Best-effort log delivery
    • It takes a while to enable logging
  • Pricing

In Object Storage, there is an option to log all actions with a bucket. You can record logs, for example, to run an internal security audit or get more granular information about bucket operations.

Logging is disabled by default. After you enable this option, Object Storage will write actions with the bucket to an object once an hour.

To save logs, do the following:

  • Define the source bucket that you want to log actions with.
  • Create a target bucket where you want to save the logs.
  • Enable logging.
  • (optional) Select the prefix of the object key.

Prerequisites

The source and target buckets must be in the same cloud.

Format of the key for the log object

Object Storage uses the following format of the key for the log object:

<prefix>/YYYY-MM-DD-HH-MM-SS-<ID>

Where:

  • prefix/: The prefix of the key for the log object. You can specify your own prefix when enabling logging.
  • YYYY-MM-DD-HH-MM-SS: Date and time of saving the log object in the target bucket (UTC format).
  • ID: A unique record ID that prevents the object from being overwritten.

Prefix of the key

The key prefix lets you distinguish:

  • Data belonging to different buckets, if the logs for multiple source buckets are saved to the same target bucket.
  • Logging actions from other actions with the bucket, if the logs are saved to the source bucket. That's because the logging operation is also considered an action with the bucket in this case.
  • The log objects from other objects, in order to delete logs from time to time. You can set up the lifecycle for the target bucket so that objects with a specific key prefix are deleted automatically.

Format of the log object

Logs are saved to a JSON file. For every action with the bucket, a record is written to the file in the following format:

Field Type Description
bucket String Bucket name.
bytes_received Int64 Size of the request in bytes.
bytes_send Int64 Response size in bytes.
handler String Request method in the REST format.<HTTP method>.<subject>.
http_referer String URL of the request source.
ip String User's IP address.
method String HTTP request method.
object_key String The object's key in URL encoded format.
protocol String Data transfer protocol version.
range String An HTTP header that defines the range of bytes to load from the object.
requester String User ID.
request_args String Arguments of the URL request.
request_id String Request ID.
request_path String Full path of the request.
request_time Int64 Request processing time, in milliseconds.
scheme String Type of data transfer protocol.
Acceptable values:
- http, an application layer protocol.
- https, an application layer protocol with encryption support.
ssl_protocol String Security protocol.
status Int64 HTTP response code.
storage_class String Storage class of the object.
timestamp String Date and time of the operation with the bucket, in the YYYY-MM-DDTHH:MM:MMZ format.
user_agent String Client application (User Agent) that executed the request.
version_id String Version of the object.
vhost String Virtual host of the request.
Acceptable values:
– storage.yandexcloud.net.
– bucket name>.storage.yandexcloud.net.
– website.yandexcloud.net.
– <bucket name>.website.yandexcloud.net.

Example of a record in the log file:

{
  "bucket": "my-bucket-example",
  "bytes_received": 749,
  "bytes_send": 1251,
  "handler": "REST.GET.OBJECT",
  "http_referer": "https://example.com/page",
  "ip": "84.201.121.46",
  "method": "GET",
  "object_key": "path/logs/2020-11-10-14-42-11-123f57b5-1853-4120-8d7a-5bcc1e9e9b4f",
  "protocol": "HTTP/1.1",
  "range": "-",
  "requester": "-",
  "request_args": "X-Amz-Algorithm=AWS4-HMAC-SHA256\u0026X-Amz-Date=20201030T072100Z\u0026X-Amz-SignedHeaders=host\u0026X-Amz-Expires=43200\u0026X-Amz-Credential=ZGB4EY1...\u0026X-Amz-Signature=12f350...",
  "request_id": "1235efdab7445028",
  "request_path": "/my-bucket-example/path/logs/2020-11-10-14-42-11-123f57b5-1853-4120-8d7a-5bcc1e9e9b4f?X-Amz-...",
  "request_time":88,
  "scheme": "https",
  "ssl_protocol": "TLSv1.2",
  "status": 200,
  "storage_class": "STANDARD",
  "timestamp": "2020-11-10T13:21:18Z",
  "user_agent": "docker/19.03.9 go/go1.13.10 git-commit/1d238398e7 kernel/4.4.0-142-generic os/linux arch/amd64 UpstreamClient(Go-http-client/1.1)",
  "version_id": "",
  "vhost": "storage.yandexcloud.net"
}

Logging specifics

There are several points to note about how actions with a bucket are logged in Object Storage.

Best-effort log delivery

Most requests to a bucket are written to the log file (if the bucket was set up correctly to support logging). Most records are written within a few hours after the request is actually processed.

However, Object Storage doesn't guarantee that the logs are saved in a complete and timely manner. It may take several hours to record an action with the bucket in a log file. In some cases, a record might fail to appear in the file.

The log file provides an overview of the nature of traffic in the bucket, but is not intended for logging every request. In the payment documents, you can find several requests that are not saved in the log file.

It takes a while to enable logging

Enabling logging and changing the settings takes about an hour. The first log object is saved to the bucket in a couple of hours after logging is enabled, but sometimes it may take longer to deliver logs.

When you change the target bucket, some logs are still delivered to the previous target bucket, and other logs are delivered to the new target bucket.

All changes to the settings will take effect without additional user actions.

Pricing

The standard Object Storage pricing rules apply to logging.

Was the article helpful?

Language / Region
© 2022 Yandex.Cloud LLC
In this article:
  • Prerequisites
  • Format of the key for the log object
  • Prefix of the key
  • Format of the log object
  • Logging specifics
  • Best-effort log delivery
  • It takes a while to enable logging
  • Pricing