Filebeat OSS

Filebeat OSS is a plugin that helps you collect and forward logs to the OpenSearch ecosystem. Filebeat is installed in a Managed Service for Kubernetes cluster; it collects cluster and pod logs and forwards them to Managed Service for OpenSearch.

Filebeat version: 7.12.1.

1. Make sure that both your Managed Service for Kubernetes cluster and Managed Service for OpenSearch cluster are in the same cloud network.

2. Enable the compatibility mode to support the Filebeat OSS client in OpenSearch. For this, run the following query:

   wget "https://storage.yandexcloud.net/cloud-certs/CA.pem" && \
   curl --user <username>:<password> --cacert CA.pem \
   --request PUT https://<host URL>:9200/_cluster/settings \
   -H "Content-Type: application/json" -d \
   '{
   "persistent": {
       "compatibility": {
       "override_main_response_version": true
       }
   }
   }'
   

   Where:

   • <username>: OpenSearch username.
   • <password>: OpenSearch password.
   • <host URL>: URL of the OpenSearch host with the DATA role, e.g., rc1a-6khpaeo31lacqo21.mdb.yandexcloud.net.

   Example of a successful response:

   {"acknowledged":true,"persistent":{"compatibility":{"override_main_response_version":"true"}},"transient":{}}
   

3. Configure the application:

   • Namespace: Select a namespace or create a new one.
   • Application name: Enter an application name, such as filebeat-oss.
   • OpenSearch username: Enter the account name that Filebeat OSS will use to connect to the Managed Service for OpenSearch cluster.
   • OpenSearch connection password: Enter the password for your account in the Managed Service for OpenSearch cluster.
   • OpenSearch FQDN: Enter the URL and port for the Managed Service for OpenSearch cluster with the DATA role, e.g., https://rc1a-6khpaeo31lacqo21.mdb.yandexcloud.net. For more information about connecting to a cluster, see the service documentation.

4. Click Install.

5. Make sure that OpenSearch is receiving logs from the Kubernetes cluster:

   1. Connect to the OpenSearch Dashboards web interface.
   2. In the management console, under OpenSearch Plugins, select Index Management.
   3. Go to Indices.
   4. Make sure the list has the Kubernetes cluster logs.

• Collecting and forwarding Kubernetes cluster logs to OpenSearch
• Analyzing logs in OpenSearch

Yandex Cloud technical support responds to requests 24 hours a day, 7 days a week. The types of requests available and their response time depend on your pricing plan. You can activate paid support in the management console. Learn more about requesting technical support.