Apache Kafka® version upgrade
You can upgrade a Managed Service for Apache Kafka® cluster to any supported version.
You can only upgrade to a version that immediately follows the current one, such as version 2.8 to 3.0. Upgrades to higher versions are performed in steps. To upgrade Apache Kafka® from 2.8 to 3.1, for instance, you have to follow the steps: 2.8 → 3.0 → 3.1.
To learn more about updates within one version and host maintenance, see Maintenance in Managed Service for Apache Kafka®.
Before upgrading
Prior to upgrading a cluster, make sure this doesn't affect your applications. See the Apache Kafka® changelog to check which updates might affect your apps.
Upgrading a cluster
Alert
Once upgraded, you cannot roll a Apache Kafka® cluster back to the previous version.
During an upgrade, topics may be unavailable if their replication factor is 1.
- Go to the folder page and select Managed Service for Apache Kafka®.
- Select the desired cluster from the list and click Edit.
- In the Version field, select a new version number.
- Click Save.
If you don't have the Yandex Cloud command line interface yet, install and initialize it.
The folder specified in the CLI profile is used by default. You can specify a different folder using the --folder-name
or --folder-id
parameter.
-
Get a list of your Managed Service for Apache Kafka® clusters:
yc managed-kafka cluster list
-
Get information about a cluster and check the version in the
config.version
parameter:yc managed-kafka cluster get <cluster ID or name>
-
Start the Apache Kafka® upgrade:
yc managed-kafka cluster update <cluster ID or name> \ --version=<new version number>
-
Open the current Terraform configuration file with an infrastructure plan.
For more information about creating this file, see Creating a Apache Kafka® cluster.
-
Add a
version
field to theconfig
section of the desired Managed Service for Apache Kafka® cluster or edit the field value if the field already exists:resource "yandex_mdb_kafka_cluster" "<cluster name>" { ... config { version = "<Apache Kafka® version>" } }
-
Make sure the settings are correct.
-
Using the command line, navigate to the folder that contains the up-to-date Terraform configuration files with an infrastructure plan.
-
Run the command:
terraform validate
If there are errors in the configuration files, Terraform will point to them.
-
-
Confirm the update of resources.
-
Run the command to view planned changes:
terraform plan
If the resource configuration descriptions are correct, the terminal will display a list of the resources to modify and their parameters. This is a test step. No resources are updated.
-
If you are happy with the planned changes, apply them:
-
Run the command:
terraform apply
-
Confirm the update of resources.
-
Wait for the operation to complete.
-
-
For more information, see the Terraform provider documentation.
Warning
The Terraform provider limits the amount of time for all Managed Service for Apache Kafka® cluster operations to complete to 60 minutes.
Operations exceeding the set timeout are interrupted.
Add the timeouts
block to the cluster description, for example:
resource "yandex_mdb_kafka_cluster" "<cluster name>" {
...
timeouts {
create = "1h30m" # 1 hour 30 minutes
update = "2h" # 2 hours
delete = "30m" # 30 minutes
}
}
Use the update API method and pass the following in the request:
-
The cluster ID in the
clusterId
parameter. You can retrieve it with a list of clusters in the folder. -
The Apache Kafka® version number in the
configSpec.version
parameter. -
List of cluster configuration fields to be changed in the
updateMask
parameter.Warning
This API method resets the values of all parameters of the object to update, which aren't passed explicitly in the request. To avoid this, in the
updateMask
parameter, list the settings you want to change (in a single line, separated by commas).
Examples
Let's assume you need to upgrade your cluster from version 2.8 to version 3.0.
-
To get a list of clusters and find out their IDs and names, run the command:
yc managed-kafka cluster list
+----------------------+---------------+---------------------+--------+---------+ | ID | NAME | CREATED AT | HEALTH | STATUS | +----------------------+---------------+---------------------+--------+---------+ | c9q8p8j2gaih8iti42mh | kafka35 | 2021-10-23 12:44:17 | ALIVE | RUNNING | +----------------------+---------------+---------------------+--------+---------+
-
To get information about the cluster
kafka35
, run the command below:yc managed-kafka cluster get kafka35
id: c9q8p8j2gaih8iti42mh ... config: version: "2.8" ...
-
To upgrade the
kafka35
cluster to version 3.0, run the command below:yc managed-kafka cluster update kafka35 --version=3.0