Managing Apache Kafka® users
Users in Apache Kafka®:
-
Keep the access permissions of data producers and consumers separate.
A producer or consumer can only access topics that are allowed for their users. You can use the same user for multiple producers or consumers: the former get the rights to write data to certain topics and the latter get the read rights.
-
Manage topics. For more information, see Topics and partitions.
After creating an Apache Kafka® cluster, you can:
- Creating a user
- Changing a user password
- Changing user settings
- Granting user permissions
- Revoking user permissions
- Getting a list of users in a cluster
- Importing users to Terraform
- Deleting a user
Creating a user
Note
Use the CLI, API, or Terraform to create an admin user.
To create a user for a producer or consumer in a cluster:
-
In the management console
, go to the appropriate folder. -
In the list of services, select Managed Service for Kafka.
-
Click the cluster name and go to the Users tab.
-
Click Add.
-
Enter your username and password.
Note
The username may contain Latin letters, numbers, hyphens, and underscores, but must begin with a letter or an underscore.
The password must be between 8 and 128 characters.
-
Grant permissions for relevant topics.
-
Click Add.
If you do not have the Yandex Cloud command line interface yet, install and initialize it.
The folder specified in the CLI profile is used by default. You can specify a different folder using the --folder-name
or --folder-id
parameter.
To create a user:
-
View a description of the CLI create user command:
yc managed-kafka user create --help
-
Create a user with the
producer
role for the producer orconsumer
role for the consumer and grant permissions for relevant topics:yc managed-kafka user create <username> \ --cluster-name <cluster_name> \ --password <password> \ --permission topic=<topic_name>,role=<user_role>
To create an admin user to manage topics in a cluster:
-
See the description of the create user CLI command:
yc managed-kafka user create --help
-
Create a user with the
admin
role applicable to all (*
) cluster topics:yc managed-kafka user create <username> \ --cluster-name <cluster_name> \ --password <password> \ --permission topic=*,role=admin
Note
The username may contain Latin letters, numbers, hyphens, and underscores, but must begin with a letter or an underscore.
The password must be between 8 and 128 characters.
-
Open the current Terraform configuration file with an infrastructure plan.
For more information about how to create this file, see Creating clusters.
-
Add the
yandex_mdb_kafka_user
resource:resource "yandex_mdb_kafka_user" "<username>" { cluster_id = "<cluster_ID>" name = "<username>" password = "<password>" ... }
Note
The username may contain Latin letters, numbers, hyphens, and underscores, but must begin with a letter or an underscore.
The password must be between 8 and 128 characters.
-
Grant permissions for relevant topics.
-
Make sure the settings are correct.
-
Using the command line, navigate to the folder that contains the up-to-date Terraform configuration files with an infrastructure plan.
-
Run the command:
terraform validate
If there are errors in the configuration files, Terraform will point to them.
-
-
Confirm updating the resources.
-
Run the command to view planned changes:
terraform plan
If the resource configuration descriptions are correct, the terminal will display a list of the resources to modify and their parameters. This is a test step. No resources are updated.
-
If you are happy with the planned changes, apply them:
-
Run the command:
terraform apply
-
Confirm the update of resources.
-
Wait for the operation to complete.
-
-
For more information, see the Terraform provider documentation
Time limits
The Terraform provider limits the amount of time for all Managed Service for Apache Kafka® cluster operations to complete to 60 minutes.
Operations exceeding the set timeout are interrupted.
Add the timeouts
block to the cluster description, for example:
resource "yandex_mdb_kafka_cluster" "<cluster_name>" {
...
timeouts {
create = "1h30m" # 1 hour 30 minutes
update = "2h" # 2 hours
delete = "30m" # 30 minutes
}
}
To create a user, use the create REST API method for the User resource or the UserService/Create gRPC API call and provide the following in the request:
- Cluster ID in the
clusterId
parameter. To find out the cluster ID, get a list of clusters in the folder. - User settings in the
userSpec
parameter:- Username in the
name
parameter. - User password in the
password
parameter. - Topic permissions (one or more
permissions
parameters, one for each topic):- Topic name in the
topicName
parameter. To find out the name, get a list of cluster topics. - Topic permissions in the
role
parameter:ACCESS_ROLE_PRODUCER
for the producer orACCESS_ROLE_CONSUMER
for the consumer.
- Topic name in the
- Username in the
To create an admin user to manage topics in a cluster, provide the following values under permission
in the userSpec
parameter:
topicName
:*
role
:ACCESS_ROLE_ADMIN
Note
The username may contain Latin letters, numbers, hyphens, and underscores, but must begin with a letter or an underscore.
The password must be between 8 and 128 characters.
Changing a user password
- In the management console
, go to the appropriate folder. - In the list of services, select Managed Service for Kafka.
- Click the cluster name and go to the Users tab.
- Click
- Set a new password and click Edit.
Note
The password must be between 8 and 128 characters.
If you do not have the Yandex Cloud command line interface yet, install and initialize it.
The folder specified in the CLI profile is used by default. You can specify a different folder using the --folder-name
or --folder-id
parameter.
To change the user password, run the command:
yc managed-kafka user update <username> \
--cluster-name <cluster_name> \
--password <new_password>
Note
The password must be between 8 and 128 characters.
-
Open the current Terraform configuration file with an infrastructure plan.
For more information about how to create this file, see Creating clusters.
-
In the file, find the
yandex_mdb_kafka_user
resource for the required user. -
Change the value of the
password
field:resource "yandex_mdb_kafka_user" "<username>" { ... password = "<password>" ... }
Note
The password must be between 8 and 128 characters.
-
Make sure the settings are correct.
-
Using the command line, navigate to the folder that contains the up-to-date Terraform configuration files with an infrastructure plan.
-
Run the command:
terraform validate
If there are errors in the configuration files, Terraform will point to them.
-
-
Confirm updating the resources.
-
Run the command to view planned changes:
terraform plan
If the resource configuration descriptions are correct, the terminal will display a list of the resources to modify and their parameters. This is a test step. No resources are updated.
-
If you are happy with the planned changes, apply them:
-
Run the command:
terraform apply
-
Confirm the update of resources.
-
Wait for the operation to complete.
-
-
For more information, see the Terraform provider documentation
Time limits
The Terraform provider limits the amount of time for all Managed Service for Apache Kafka® cluster operations to complete to 60 minutes.
Operations exceeding the set timeout are interrupted.
Add the timeouts
block to the cluster description, for example:
resource "yandex_mdb_kafka_cluster" "<cluster_name>" {
...
timeouts {
create = "1h30m" # 1 hour 30 minutes
update = "2h" # 2 hours
delete = "30m" # 30 minutes
}
}
To update a user's password, use the update REST API method for the User resource or the UserService/Update gRPC API call and provide the following in the request:
-
Cluster ID in the
clusterId
parameter. To find out the cluster ID, get a list of clusters in the folder. -
Username in the
userName
parameter. To find out the name, get a list of users in the cluster. -
Name of the
password
setting in theupdateMask
parameter. If this parameter is omitted, the API method resets any user settings that are not explicitly specified in the request to their default values. -
New user password in the
password
parameter.Note
The password must be between 8 and 128 characters.
Warning
This API method overrides all parameters of the object being modified that were not explicitly passed in the request to the default values. To avoid this, list the settings you want to change in the updateMask
parameter (one line separated by commas).
Changing user settings
- In the management console
, go to the appropriate folder. - In the list of services, select Managed Service for Kafka.
- Click the cluster name and go to the Users tab.
- Click
- Grant or revoke permissions for topics, if necessary.
- Click Save.
If you do not have the Yandex Cloud command line interface yet, install and initialize it.
The folder specified in the CLI profile is used by default. You can specify a different folder using the --folder-name
or --folder-id
parameter.
-
Open the current Terraform configuration file with an infrastructure plan.
For more information about how to create this file, see Creating clusters.
-
In the file, find the
yandex_mdb_kafka_user
resource for the required user. -
Edit the
permission
section to grant or revoke topic permissions:resource "yandex_mdb_kafka_user" "<username>" { cluster_id = "<cluster_ID>" name = "<username>" password = "<password>" ... }
-
Make sure the settings are correct.
-
Using the command line, navigate to the folder that contains the up-to-date Terraform configuration files with an infrastructure plan.
-
Run the command:
terraform validate
If there are errors in the configuration files, Terraform will point to them.
-
-
Confirm updating the resources.
-
Run the command to view planned changes:
terraform plan
If the resource configuration descriptions are correct, the terminal will display a list of the resources to modify and their parameters. This is a test step. No resources are updated.
-
If you are happy with the planned changes, apply them:
-
Run the command:
terraform apply
-
Confirm the update of resources.
-
Wait for the operation to complete.
-
-
For more information, see the Terraform provider documentation
Time limits
The Terraform provider limits the amount of time for all Managed Service for Apache Kafka® cluster operations to complete to 60 minutes.
Operations exceeding the set timeout are interrupted.
Add the timeouts
block to the cluster description, for example:
resource "yandex_mdb_kafka_cluster" "<cluster_name>" {
...
timeouts {
create = "1h30m" # 1 hour 30 minutes
update = "2h" # 2 hours
delete = "30m" # 30 minutes
}
}
To update user settings, use the update REST API method for the User resource or the UserService/Update gRPC API call and provide the following in the request:
- Cluster ID in the
clusterId
parameter. To find out the cluster ID, get a list of clusters in the folder. - Username in the
userName
parameter. To find out the name, get a list of users in the cluster. - In the
updateMask
parameter, a list of settings to update (in a single line, comma-separated). If this parameter is omitted, the API method resets any user settings that are not explicitly specified in the request to their default values. - New set of permissions to topics (one or more
permissions
parameters, one for each topic).
Warning
This API method overrides all parameters of the object being modified that were not explicitly passed in the request to the default values. To avoid this, list the settings you want to change in the updateMask
parameter (one line separated by commas).
Granting user permissions
Note
Permissions granted to the user for a topic remain even after the topic is deleted. If, after deleting a topic, you don't revoke the permissions, then, when you create a topic with the same name, the user will have access to it even if you don't explicitly assign them new permissions.
-
In the management console
, go to the appropriate folder. -
In the list of services, select Managed Service for Kafka.
-
Select the cluster.
-
Go to the Users tab.
-
Click
-
Click
If the user does not need permissions to certain topics, you can revoke them.
-
Select the appropriate topic from the drop-down list or enter its name:
-
Specify the following in the Topic field:
*
to allow access to any topics.- Full topic name to allow access to a specific topic.
<prefix>*
to grant access to topics whose names start with the specified prefix. For example, if you have topics namedtopic_a1
,topic_a2
, anda3
, and you set thetopic*
value, access will be granted totopic_a1
andtopic_a2
.
-
Click Add topic.
-
-
Click the
*ACCESS_ROLE_CONSUMER
: Consumers using this user will be granted access to the topic.
*ACCESS_ROLE_PRODUCER
: Producers using this user will be granted access to the topic.
*ACCESS_ROLE_ADMIN
: Only available if access to any topics is selected.
You can select the `ACCESS_ROLE_CONSUMER` and `ACCESS_ROLE_PRODUCER` roles at the same time to make a user suitable for both producers and consumers.
The user also gains access to data schema subjects. The availability of subjects depends on the specified roles and topics. For more information, see [{#T}](../concepts/managed-schema-registry.md#subjects).
- To grant permissions to other topics, repeat the steps.
- (Optional) If you granted permissions for a topic accidentally, revoke them.
If you do not have the Yandex Cloud command line interface yet, install and initialize it.
The folder specified in the CLI profile is used by default. You can specify a different folder using the --folder-name
or --folder-id
parameter.
To grant user permissions:
-
Retrieve a list of cluster topics:
yc managed-kafka topic list --cluster-name <cluster_name>
-
Grant permissions to the required topics by providing the
--permission
parameters:yc managed-kafka user update <username> \ --cluster-name <cluster_name> \ --permission topic=<topic_name>,role=<user_role>
The following
--permission
parameters are available:-
topic
: Name of the topic the permissions are granted for.If the user does not need permissions to certain topics, you can revoke them.
-
role
: User role, such asproducer
,consumer
, oradmin
.The
admin
role is only available with all topics selected (topic=*
).
When you update user permissions, the existing permissions are revoked and replaced with the new ones. This means the command must always include a complete list of permissions to be assigned to the user.
For example, to grant user
test-user
permissions in clusterkafka-cli
to topictopic2
with theproducer
role while keeping existingtopic1
permissions, run the command:yc managed-kafka user update test-user \ --cluster-name kafka-cli \ --permission topic=topic1,role=consumer \ --permission topic=topic2,role=producer
Along with access to the topic, users also gain access to data schema subjects. The availability of subjects depends on the specified roles and topics. For more information, see Managed Schema Registry subjects.
-
-
Open the current Terraform configuration file with an infrastructure plan.
For more information about how to create this file, see Creating clusters.
-
In the file, find the
yandex_mdb_kafka_cluster
resource for the required user. -
Add a
permission
section:resource "yandex_mdb_kafka_user" "<username>" { ... permission { topic_name = "<topic>" role = "<user_role>" } }
Where:
-
topic_name
: Topic name. Specify the following:*
to allow access to any topics.- Full topic name to allow access to a specific topic.
<prefix>*
to grant access to topics whose names start with the specified prefix. For example, if you have topics namedtopic_a1
,topic_a2
, anda3
, and you set thetopic*
value, access will be granted totopic_a1
andtopic_a2
.
-
role
: User role:ACCESS_ROLE_CONSUMER
,ACCESS_ROLE_PRODUCER
orACCESS_ROLE_ADMIN
. TheACCESS_ROLE_ADMIN
is only available with all topics selected (topic_name = "*"
).
Along with access to the topic, users also gain access to data schema subjects. The availability of subjects depends on the specified roles and topics. For more information, see Managed Schema Registry subjects.
If the user does not need permissions to certain topics, you can revoke them.
-
-
Make sure the settings are correct.
-
Using the command line, navigate to the folder that contains the up-to-date Terraform configuration files with an infrastructure plan.
-
Run the command:
terraform validate
If there are errors in the configuration files, Terraform will point to them.
-
-
Confirm updating the resources.
-
Run the command to view planned changes:
terraform plan
If the resource configuration descriptions are correct, the terminal will display a list of the resources to modify and their parameters. This is a test step. No resources are updated.
-
If you are happy with the planned changes, apply them:
-
Run the command:
terraform apply
-
Confirm the update of resources.
-
Wait for the operation to complete.
-
-
For more information, see the Terraform provider documentation
Time limits
The Terraform provider limits the amount of time for all Managed Service for Apache Kafka® cluster operations to complete to 60 minutes.
Operations exceeding the set timeout are interrupted.
Add the timeouts
block to the cluster description, for example:
resource "yandex_mdb_kafka_cluster" "<cluster_name>" {
...
timeouts {
create = "1h30m" # 1 hour 30 minutes
update = "2h" # 2 hours
delete = "30m" # 30 minutes
}
}
To grant user permissions, use the grantPermission REST API method for the User resource or the UserService/GrantPermission gRPC API call and provide the following in the request:
- Cluster ID in the
clusterId
parameter. To find out the cluster ID, get a list of clusters in the folder. - Username in the
userName
parameter. To find out the name, get a list of users in the cluster. - New permission to the topic in the
permission
parameter.
Along with access to the topic, users also gain access to data schema subjects. The availability of subjects depends on the specified roles and topics. For more information, see Managed Schema Registry subjects.
Revoking user permissions
If you revoke the ACCESS_ROLE_ADMIN
role from the admin user in a cluster, you will no longer be able to manage topics. Do not revoke this role without first granting it to another user.
- In the management console
, go to the appropriate folder. - In the list of services, select Managed Service for Kafka.
- Select the cluster.
- Go to the Users tab.
- Click
- Find the appropriate topic in the list of topics.
- Delete the role you no longer need: click the
If you do not have the Yandex Cloud command line interface yet, install and initialize it.
The folder specified in the CLI profile is used by default. You can specify a different folder using the --folder-name
or --folder-id
parameter.
To revoke access permissions to certain topics, provide an updated list of --permission
parameters:
yc managed-kafka user update <username> \
--cluster-name <cluster_name> \
--permission topic=<topic_name>,role=<user_role>
When you update user permissions, the existing permissions are revoked and replaced with the new ones. This means the command must always include a complete list of permissions to be assigned to the user.
The --permission
flag must include at least one topic/role pair, where:
topic
: Topic name.role
: User role, such asproducer
,consumer
, oradmin
.
To revoke all the permissions granted to the user, use the console or delete the user.
-
Open the current Terraform configuration file with an infrastructure plan.
For more information about how to create this file, see Creating clusters.
-
In the file, find the
yandex_mdb_kafka_user
resource for the required user. -
Edit or delete the
permission
section. -
Make sure the settings are correct.
-
Using the command line, navigate to the folder that contains the up-to-date Terraform configuration files with an infrastructure plan.
-
Run the command:
terraform validate
If there are errors in the configuration files, Terraform will point to them.
-
-
Confirm updating the resources.
-
Run the command to view planned changes:
terraform plan
If the resource configuration descriptions are correct, the terminal will display a list of the resources to modify and their parameters. This is a test step. No resources are updated.
-
If you are happy with the planned changes, apply them:
-
Run the command:
terraform apply
-
Confirm the update of resources.
-
Wait for the operation to complete.
-
-
For more information, see the Terraform provider documentation
Time limits
The Terraform provider limits the amount of time for all Managed Service for Apache Kafka® cluster operations to complete to 60 minutes.
Operations exceeding the set timeout are interrupted.
Add the timeouts
block to the cluster description, for example:
resource "yandex_mdb_kafka_cluster" "<cluster_name>" {
...
timeouts {
create = "1h30m" # 1 hour 30 minutes
update = "2h" # 2 hours
delete = "30m" # 30 minutes
}
}
To revoke user permissions, use the revokePermission REST API method for the User resource or the UserService/RevokePermission gRPC API call and provide the following in the request:
- Cluster ID in the
clusterId
parameter. To find out the cluster ID, get a list of clusters in the folder. - Username in the
userName
parameter. To find out the name, get a list of users in the cluster. - The topic permission to be revoked in the
permission
parameter.
Getting a list of users in a cluster
- In the management console
, go to the appropriate folder. - In the list of services, select Managed Service for Kafka.
- Click the cluster name and go to the Users tab.
If you do not have the Yandex Cloud command line interface yet, install and initialize it.
The folder specified in the CLI profile is used by default. You can specify a different folder using the --folder-name
or --folder-id
parameter.
To get a list of users:
-
To get a list of users, run the following command:
yc managed-kafka user list --cluster-name <cluster_name>
-
To get detailed information for a specific user, run this command:
yc managed-kafka user get <username> --cluster-name <cluster_name>
To get a list of users, use the list REST API method for the User resource or the UserService/List gRPC API call and provide the cluster ID in the clusterId
request parameter.
To find out the cluster ID, get a list of clusters in the folder.
Importing users to Terraform
Using import, you can bring the existing cluster users under Terraform management.
-
In the Terraform configuration file, specify the user you want to import:
resource "yandex_mdb_kafka_user" "<username>" {}
-
Run the following command to import the user:
terraform import yandex_mdb_kafka_user.<username> <cluster_ID>:<username>
To learn more about importing users, see the Terraform provider documentation
.
Deleting a user
If you delete the admin user with the ACCESS_ROLE_ADMIN
role in a cluster, you will no longer be able to manage topics. To avoid this, assign this role to another user before deleting the admin user.
- In the management console
, go to the appropriate folder. - In the list of services, select Managed Service for Kafka.
- Click the cluster name and go to the Users tab.
- Click
- In the window that opens, click Delete.
If you do not have the Yandex Cloud command line interface yet, install and initialize it.
The folder specified in the CLI profile is used by default. You can specify a different folder using the --folder-name
or --folder-id
parameter.
To remove a user, run:
yc managed-kafka user delete <username> --cluster-name <cluster_name>
-
Open the current Terraform configuration file with an infrastructure plan.
For more information about how to create this file, see Creating clusters.
-
Delete the
yandex_mdb_kafka_user
resource for the appropriate user. -
Make sure the settings are correct.
-
Using the command line, navigate to the folder that contains the up-to-date Terraform configuration files with an infrastructure plan.
-
Run the command:
terraform validate
If there are errors in the configuration files, Terraform will point to them.
-
-
Confirm updating the resources.
-
Run the command to view planned changes:
terraform plan
If the resource configuration descriptions are correct, the terminal will display a list of the resources to modify and their parameters. This is a test step. No resources are updated.
-
If you are happy with the planned changes, apply them:
-
Run the command:
terraform apply
-
Confirm the update of resources.
-
Wait for the operation to complete.
-
-
For more information, see the Terraform provider documentation
Time limits
The Terraform provider limits the amount of time for all Managed Service for Apache Kafka® cluster operations to complete to 60 minutes.
Operations exceeding the set timeout are interrupted.
Add the timeouts
block to the cluster description, for example:
resource "yandex_mdb_kafka_cluster" "<cluster_name>" {
...
timeouts {
create = "1h30m" # 1 hour 30 minutes
update = "2h" # 2 hours
delete = "30m" # 30 minutes
}
}
To delete a user, use the delete REST API method for the User resource or the UserService/Delete gRPC API call and provide the following in the request:
- Cluster ID in the
clusterId
parameter. To find out the cluster ID, get a list of clusters in the folder. - Username to delete in the
userName
parameter. To find out the name, get a list of users in the cluster.