Fluent Bit with a Cloud Logging plugin

Updated November 30, 2022

Fluent Bit is an open-source multi-platform tool for collecting and processing logs. Fluent Bit collects, parses, and filters messages from a variety of inputs and saves them to storage. Once a message is saved, it is forwarded from storage to a router that determines which destination to route it to. Plugins are used to work with various inputs and outputs.

A plugin for Cloud Logging is used to export logs generated in Kubernetes clusters, including at the pod level, to Yandex Cloud Logging.

Deployment instructions
  1. Create a Kubernetes cluster and a node group.

  2. Create a service account to run Fluent Bit.

  3. Assign the service account the logging.writer role.

  4. Create an authorized key for the service account and save it to a file named sa-key.json:

    yc iam key create \
       --service-account-name <service account name> \
       --output sa-key.json
    
  5. Create a log group.

  6. If you want to save logs to Yandex Object Storage:

    1. Create a bucket.

    2. Assign the service account the storage.uploader role.

    3. Create a static access key for the service account and save it in JSON format:

      {
          "access_key": {
          "id": "xxxxxx",
          "created_at": "xxxxxx",
          "key_id": "xxxxxx"
          },
          "secret": "xxxxxx"
      }
      
  7. Set the Fluent Bit settings:

    • Namespace: Select a namespace or create a new one.
    • Application name: Enter an application name.
    • Target log group ID: Specify an ID of the log group to which logs will be transmitted.
    • Service account key for Cloud Logging: Paste the contents of the sa-key.json file.
    • Save logs to Object Storage: Select this option if you want to save logs to Object Storage.
    • Object Storage bucket name: Specify the name of the bucket where logs will be stored. Leave the field blank if you do not want to save logs.
    • Static access key for Object Storage: Paste the static access key in JSON format. Leave the field blank if you do not want to save logs.
  8. Click Install.

Billing type
Free
Type
Kubernetes® Application
Category
Developer tools
Publisher
Yandex Cloud
Use cases
  • Exporting Kubernetes cluster logs to Yandex Cloud Logging.
Technical support

Yandex Cloud technical support responds to requests 24/7. The types of requests available and their response time depend on your pricing plan. You can activate paid support in the management console. Learn more about requesting technical support.

Product composition
Helm chartVersion
Pull-command
Documentation
yandex-cloud/fluent-bit/fluent-bit-cloud-logging1.0-6Open
Docker-imageVersion
Pull-command
yandex-cloud/fluent-bit/cloud-logging2.0.2-1.9.3
Terms
By using this product you agree to the Yandex Cloud Marketplace Terms of Use and the terms and conditions of the following software: Fluent BitCloud Logging Plugin
Billing type
Free
Type
Kubernetes® Application
Category
Developer tools
Publisher
Yandex Cloud