A cluster in Managed Service for Apache Kafka® is one or more broker hosts where topics and their partitions are located. Producers and consumers can work with these topics by connecting to cluster hosts.
If you create a cluster with more than one host, three dedicated ZooKeeper hosts will be added to the cluster. For more information, see Relationship between service resources.
To create a cluster, follow these steps:
In the management console, select the folder where you want to create a cluster.
Select Managed Service for Apache Kafka®.
Click Create cluster.
Under Basic parameters:
- Enter a name for the cluster and, if necessary, a description. The cluster name must be unique within the folder.
- Select the environment where you want to create the cluster (you can't change the environment once the cluster is created):
PRODUCTION: For stable versions of your apps.
PRESTABLE: For testing, including the Managed Service for Apache Kafka® service itself. The Prestable environment is first updated with new features, improvements, and bug fixes. However, not every update ensures backward compatibility.
- Select the Apache Kafka® version.
Under Host class, select the platform, host type, and host class.
When you change the host class for the cluster, the characteristics of all existing instances change.
- Choose the type of storage, either a more flexible network type (network-hdd or network-ssd) or faster local SSD storage (local-ssd).
- Select the size of storage to be used for data. The size of the local storage can only be changed in 100 GB increments.
Under Network settings:
Select one or more availability zones where the Apache Kafka® brokers will reside.
Select the network.
Select subnets in each availability zone for this network. To create a new subnet, click Create new subnet next to the desired availability zone.
For a cluster with multiple broker hosts, you need to specify subnets in each availability zone even if you plan to host brokers only in some of them. These subnets are required to host three ZooKeeper hosts — one in each availability zone. For more information, see Relationship between service resources.
To access broker hosts from the internet, select Public access. In this case, you can only connect to them over an SSL connection. For more information, see Connecting to topics in an Apache Kafka® cluster.
You can't request public access after creating a cluster.
Under Hosts, specify the number of broker hosts Apache Kafka® to be located in each of the selected availability zones.
When choosing the number of hosts, keep in mind that:
- The Apache Kafka® cluster hosts will be evenly deployed in the selected availability zones. Decide on the number of zones and hosts per zone based on the required fault tolerance model and cluster load.
- Replication is possible if there are at least two hosts in the cluster.
- Adding more than one host to the cluster automatically adds three ZooKeeper hosts.
If you specify two or more broker hosts, under Host class ZooKeeper, specify the characteristics of the hostsZooKeeper to be located in each of the selected availability zones.
Click Create cluster.
Wait until the cluster is ready: its status on the Managed Service for Apache Kafka® dashboard changes to
Runningand its state to
Alive. This may take some time.
To create a cluster, use the create API method and pass the following in the request:
- In the
folderIdparameter, the ID of the folder where the cluster should be placed.
- The cluster name in the