Collecting, monitoring and analyzing audit logs
An audit log is a record of all events in the system, including access to it and operations performed. By collecting and verifying audit logs, you can monitor compliance with the established security procedures and standards and identify vulnerabilities in your security mechanisms.
Events in audit logs occur on different levels:
- Yandex.Cloud level: Events that occur with Yandex.Cloud resources.
- OS level.
- Application level.
- Network level (Flow Logs).
For more information about Kubernetes events, see "Collecting, monitoring, and analyzing audit logs in Yandex Managed Service for Kubernetes".
The main tool for collecting Yandex.Cloud level logs is Yandex Audit Trails. The service lets you collect audit logs about events happening to Yandex.Cloud resources and upload these logs to Object Storage buckets or Cloud Logging log groups for further analysis or export. See the instructions on how to start collecting logs, as well as the format and events reference.
Learn more about the security recommendations for Object Storage in Object Storage.
To collect metrics, analyze Yandex.Cloud-level events and set up notifications, we recommend using Yandex Monitoring. It helps you track, for example, a sharp increase in the load on Yandex Compute Cloud, the number of Application LoadBalancer requests per second (RPS), or significant changes in event statistics in IAM.
You can also use Yandex Monitoring to monitor the health of the Audit Trails service itself and track security events.
List of important Yandex.Cloud-level events to search for in audit logs:
Solution: Searching for important security events in audit logs
Exporting events to SIEM
Solutions for exporting Yandex.Cloud audit logs are available for the following SIEM systems:
Yandex Managed Service for Elasticsearch (ELK)
To set up export to any SIEM, use utilities such as GeeseFS or s3fs. They let you mount an Object Storage bucket as a VM's local disk. Next, you need to install a SIEM connector on the VM and configure reading JSON files from the bucket.
Yandex Monitoring metrics
You can export metrics to a SIEM system via the API, see the instructions.
Responding to events
Using Cloud Functions, you can configure alerts about Audit Trails events, as well as automatic responses to malicious actions, including removing dangerous rules or revoking access rights.
When using IaaS cloud services and Kubernetes node groups, the customer is responsible for ensuring OS security and collecting OS-level events on their own. Free tools for collecting standard OS-generated events and exporting them to the customer's SIEM system include:
Additional event generation options can be implemented using Auditd for Linux or Sysmon for Windows.
You can collect Linux system metrics (CPU, RAM and disk space usage) with Yandex Unified Agent that is a Yandex Monitoring service component.
You can also export OS events to Cloud Logging using a Fluent Bit plugin.
To describe events to be searched for in audit logs, we recommend using Sigma format, which is supported by popular SIEM systems. The Sigma repository contains a library of events described in this format.
Customers may collect events that occur at the level of applications deployed on Compute Cloud resources on their own. For example, save application logs to files and transfer them to a SIEM system using the tools listed in OS level above.
Currently, VPC network traffic event logs (Flow Logs) can only be collected by customers. You can use Yandex Cloud Marketplace solutions (such as NGFW, IDS/IPS, or network products) or free software for collecting and transmitting events.
To get the exact time of OS- and application-level events, configure clock synchronization by following the instructions.