Yandex Managed Service for Apache Kafka®

A service for creating and managing Apache Kafka® clusters in the Yandex.Cloud infrastructure.

Apache Kafka is an open‑source platform for distributed delivery, storage, and processing of real‑time data.

Ease of use
Focus on developing data stream processing applications and don’t waste time maintaining the infrastructure. Managed Service for Apache Kafka is responsible for managing Zookeeper brokers and clusters, configuring clusters, and updating their versions.
High availability
Distribute your cluster brokers across different availability zones and set the replication factor to ensure the desired level of fault tolerance. The service analyzes the metrics and status of the cluster and automatically replaces it if one of the nodes fails.
Efficient resource usage
For each topic, you can set the replication factor, log cleanup policy, compression type, and maximum number of messages to make better use of computing, network, and disk resources.
Scalability
You can add brokers to your cluster with just a click of a button to improve its performance, or change the class of high-availability hosts without stopping them or losing any data.
Security
All connections are encrypted using the TLS protocol. User access to topics is provided using SASL authentication and access control lists (ACL). The service meets local regulatory requirements, GDPR, and ISO standards.
Real-time analytics
With Apache Kafka, you can deliver data directly to ClickHouse without any additional development. ClickHouse is great at handling queries to large amounts of data. And Yandex DataLens lets you visualize this data and create reports.

We'll take care of most cluster maintenance

Processes
Managed Service for Apache Kafka
Apache Kafka self‑installation
Data access control
VM deployment
Network configurations
OS and software installation
Broker updates
Data storage and hardware security
Monitoring tools

You control

Yandex.Cloud controls

Getting started

Select the necessary computing capacity and create a ready-to-use Apache Kafka cluster.

Create cluster

Questions and answers

What are the characteristic features of Apache Kafka?

Apache Kafka is a distributed message broker that can be scaled horizontally for streaming data between applications and their components. Apache Kafka is based on the publication-subscription model and delivers data to and from applications using topics.

Apache Kafka is a distributed message broker that can be scaled horizontally for streaming data between applications and their components. Apache Kafka is based on the publication-subscription model and delivers data to and from applications using topics.

What is a topic?

A topic is a way to group message streams into categories. Producers publish a certain category of messages to a topic, while consumers subscribe to this topic and pull messages from it. For each topic, Apache Kafka maintains a partitioned message log.

A topic is a way to group message streams into categories. Producers publish a certain category of messages to a topic, while consumers subscribe to this topic and pull messages from it. For each topic, Apache Kafka maintains a partitioned message log.

How are partitions created?

A partition is a sequence of topic messages that are stored in the order they are received. Consumers read messages from the beginning to the end of a partition, so messages that were received earlier are consumed first.

A partition is a sequence of topic messages that are stored in the order they are received. Consumers read messages from the beginning to the end of a partition, so messages that were received earlier are consumed first.

Get started with Managed Service for Apache Kafka

  1. Apache® and Apache Kafka® are either registered trademarks or trademarks of the Apache Software Foundation in the United States and/or other countries.