Networking in Managed Service for Apache Kafka®
When creating a cluster, you can:
- Set the network for cluster hosts.
- Specify the availability zones where the cluster hosts will reside.
- Set subnets in all availability zones.
- Turn on public access to the cluster from outside Yandex Cloud.
If there are ZooKeeper hosts in the cluster, each of the three ZooKeeper hosts will use its dedicated availability zone and the subnet selected in it. For more information, see Resource relationships in the service.
Host name and FQDN
Managed Service for Apache Kafka® generates the name of each cluster host when creating it. This name will be the host's fully qualified domain name (FQDN). The host name and, consequently, FQDN cannot be changed.
To learn how to get a host FQDN, see this guide.
You can use the FQDN to access the host within a single cloud network. For more information, see the Yandex Virtual Private Cloud documentation.
Public access to clusters
All broker hosts in the cluster are available from outside Yandex Cloud if you request public access when creating a cluster. To connect to such a cluster, use the FQDN of one or more cluster's broker hosts.
You cannot request public access after creating a cluster.
When you delete a cluster with public access enabled, all public IP addresses assigned to this cluster are revoked.
Security groups
Security groups follow the All traffic that is not allowed is prohibited principle. To connect to a cluster, security groups must include rules allowing traffic from certain ports, IP addresses, or other security groups.
For example, let's assume a VM in Yandex Cloud is used to access the cluster. In this case, if only the 10.133.0.0/24 subnet is specified in the incoming traffic rules for the security group, while the VM is in the 10.128.0.0/16 subnet, the VM will not be able to connect to the cluster. A VM from the 10.133.0.0/24 subnet will not be able to connect either, in case it tries to access a port that is not specified in the security group rules.
Tip
When connecting to a cluster from within its cloud network, make sure to configure security groups both for the cluster and for the connecting host.
Specifics of working with security groups:
-
Security group settings only affect whether it will be possible to connect to the cluster. They do not affect cluster operation, such as replication of topic sections by broker hosts, connections between brokers and ZooKeeper hosts, and other features.
-
Even if the cluster and the connecting host are in the same security group, there will be no connection unless you set up rules within this group that allow traffic between the host and the cluster.
However, by default, such rules are contained in the security group that is added automatically when creating a cloud network. Those are the
Self
rules that allow unlimited traffic within a group.
For more information, see the Virtual Private Cloud documentation.