Installing Fluent Bit with a plugin for Cloud Logging

Fluent Bit is an open-source multi-platform tool for collecting and processing logs. Fluent Bit collects, parses, and filters messages from a variety of inputs and saves them to storage. Messages then go to the router, which defines the output to send them to. Plugins are used to work with various inputs and outputs.

The plugin for Cloud Logging enables you to export logs generated in a Managed Service for Kubernetes cluster, including at the pod level, to Yandex Cloud Logging.

Creating a service account

1. If you don't have the Yandex Cloud command line interface yet, install and initialize it.

   The folder specified in the CLI profile is used by default. You can specify a different folder using the --folder-name or --folder-id parameter.

2. Create a service account, which is required for Fluent Bit.

3. Assign it the logging.writer and monitoring.editor roles.

4. Create an authorized key and save it to a file named sa-key.json:

Installing Fluent Bit using Yandex Cloud Marketplace

1. Go to the folder page and select Managed Service for Kubernetes.
2. Click the name of the desired cluster and open the Marketplace tab.
3. Under Applications available for installation, select Fluent Bit with a plugin for Cloud Logging and click Use.
4. Configure the application:
   • Namespace: Select a namespace or create a new one.
   • Application name: Enter an application name.
   • Target log group ID: Specify an ID of the log group to which Fluent Bit logs will be saved.
   • Cluster ID: Specify a cluster ID.
   • Service account key for Cloud Logging: Paste the contents of the sa-key.json file.
5. Click Install.

Installation using a Helm chart

1. Install the Helm package manager.

2. To install a Helm chart with Fluent Bit, run this command:

   export HELM_EXPERIMENTAL_OCI=1 && \
   cat sa-key.json | helm registry login cr.yandex --username 'json_key' --password-stdin && \
   helm pull oci://cr.yandex/yc-marketplace/yandex-cloud/fluent-bit/fluent-bit-cloud-logging \
     --version 1.0-3 \
     --untar && \
   helm install \
     --namespace fluentbit \
     --create-namespace \
     --set loggingGroupId=<log group ID> \
     --set loggingFilter=<cluster ID> \
     --set-file auth.json=sa-key.json \
     fluentbit ./fluent-bit-cloud-logging/
   

   This command also creates a new fluentbit namespace required for Fluent Bit.

See also

• Fluent Bit documentation.
• Cloud Logging documentation.