Yandex Managed Service for Apache Kafka®
A service for creating and managing Apache Kafka® clusters in the Yandex Cloud infrastructure.
Apache Kafka is an open‑source platform for distributed delivery, storage, and processing of real‑time data.
Ease of use
Focus on developing data stream processing applications and don’t waste time maintaining the infrastructure. Managed Service for Apache Kafka is responsible for managing Zookeeper brokers and clusters, configuring clusters, and updating their versions.
Distribute your cluster brokers across different availability zones and set the replication factor to ensure the desired level of fault tolerance. The service analyzes the metrics and status of the cluster and automatically replaces it if one of the nodes fails.
Efficient resource usage
For each topic, you can set the replication factor, log cleanup policy, compression type, and maximum number of messages to make better use of computing, network, and disk resources.
You can add brokers to your cluster with just a click of a button to improve its performance, or change the class of high-availability hosts without stopping them or losing any data.
All connections are encrypted using the TLS protocol. User access to topics is provided using SASL authentication and access control lists (ACL). The service meets local regulatory requirements, GDPR, and ISO standards.
With Apache Kafka, you can deliver data directly to ClickHouse without any additional development. ClickHouse is great at handling queries to large amounts of data. And Yandex DataLens lets you visualize this data and create reports.
We'll take care of most cluster maintenance
Yandex Cloud controls
Select the necessary computing capacity and create a ready-to-use Apache Kafka cluster.
Questions and answers
What are the characteristic features of Apache Kafka?
Apache Kafka is a distributed message broker that can be scaled horizontally for streaming data between applications and their components. Apache Kafka is based on the publication-subscription model and delivers data to and from applications using topics.
What is a topic?
A topic is a way to group message streams into categories. Producers publish a certain category of messages to a topic, while consumers subscribe to this topic and pull messages from it. For each topic, Apache Kafka maintains a partitioned message log.
How are partitions created?
A partition is a sequence of topic messages that are stored in the order they are received. Consumers read messages from the beginning to the end of a partition, so messages that were received earlier are consumed first.
What Apache Kafka versions are available in Yandex Cloud?
Managed Service for Apache Kafka supports versions 2.8, and 3.0.
Apache® and Apache Kafka® are either registered trademarks or trademarks of the Apache Software Foundation in the United States and/or other countries.