Yandex Cloud
  • Services
  • Solutions
  • Why Yandex Cloud
  • Blog
  • Pricing
  • Documentation
  • Contact us
Get started
Language / Region
Yandex project
© 2023 Yandex.Cloud LLC
Yandex Managed Service for Apache Kafka®
  • Getting started
  • Step-by-step instructions
    • All instructions
    • Information about existing clusters
    • Creating clusters
    • Connecting to a cluster
    • Stopping and starting clusters
    • Upgrading the Apache Kafka® version
    • Changing cluster settings
    • Managing Apache Kafka® hosts
    • Working with topics and partitions
    • Managing Apache Kafka® users
    • Managing connectors
    • Viewing cluster logs
    • Deleting clusters
    • Monitoring the state of clusters and hosts
  • Practical guidelines
  • Concepts
  • Access management
  • Pricing policy
  • API reference
  • Revision history
  • Questions and answers
  1. Step-by-step instructions
  2. Upgrading the Apache Kafka® version

Apache Kafka® version upgrade

Written by
Yandex Cloud
  • Before upgrading
  • Upgrading a cluster
  • Examples

You can upgrade a Managed Service for Apache Kafka® cluster to any supported version.

You can only upgrade to a version that immediately follows the current one, such as version 2.8 to 3.0. Upgrades to higher versions are performed in steps. To upgrade Apache Kafka® from 2.8 to 3.1, for instance, you have to follow the steps: 2.8 → 3.0 → 3.1.

To learn more about updates within one version and host maintenance, see Maintenance in Managed Service for Apache Kafka®.

Before upgrading

Prior to upgrading a cluster, make sure this doesn't affect your applications. See the Apache Kafka® changelog to check which updates might affect your apps.

Upgrading a cluster

Alert

Once upgraded, you cannot roll a Apache Kafka® cluster back to the previous version.

During an upgrade, topics may be unavailable if their replication factor is 1.

Management console
CLI
Terraform
API
  1. Go to the folder page and select Managed Service for Apache Kafka®.
  2. Select the desired cluster from the list and click Edit.
  3. In the Version field, select a new version number.
  4. Click Save.

If you don't have the Yandex Cloud command line interface yet, install and initialize it.

The folder specified in the CLI profile is used by default. You can specify a different folder using the --folder-name or --folder-id parameter.

  1. Get a list of your Managed Service for Apache Kafka® clusters:

    yc managed-kafka cluster list
    
  2. Get information about a cluster and check the version in the config.version parameter:

    yc managed-kafka cluster get <cluster ID or name>
    
  3. Start the Apache Kafka® upgrade:

    yc managed-kafka cluster update <cluster ID or name> \
       --version=<new version number>
    
  1. Open the current Terraform configuration file with an infrastructure plan.

    For more information about creating this file, see Creating a Apache Kafka® cluster.

  2. Add a version field to the config section of the desired Managed Service for Apache Kafka® cluster or edit the field value if the field already exists:

    resource "yandex_mdb_kafka_cluster" "<cluster name>" {
      ...
      config {
        version = "<Apache Kafka® version>"
      }
    }
    
  3. Make sure the settings are correct.

    1. Using the command line, navigate to the folder that contains the up-to-date Terraform configuration files with an infrastructure plan.

    2. Run the command:

      terraform validate
      

      If there are errors in the configuration files, Terraform will point to them.

  4. Confirm the update of resources.

    1. Run the command to view planned changes:

      terraform plan
      

      If the resource configuration descriptions are correct, the terminal will display a list of the resources to modify and their parameters. This is a test step. No resources are updated.

    2. If you are happy with the planned changes, apply them:

      1. Run the command:

        terraform apply
        
      2. Confirm the update of resources.

      3. Wait for the operation to complete.

For more information, see the Terraform provider documentation.

Warning

The Terraform provider limits the amount of time for all Managed Service for Apache Kafka® cluster operations to complete to 60 minutes.

Operations exceeding the set timeout are interrupted.

How do I change these limits?

Add the timeouts block to the cluster description, for example:

resource "yandex_mdb_kafka_cluster" "<cluster name>" {
  ...
  timeouts {
    create = "1h30m" # 1 hour 30 minutes
    update = "2h"    # 2 hours
    delete = "30m"   # 30 minutes
  }
}

Use the update API method and pass the following in the request:

  • The cluster ID in the clusterId parameter. You can retrieve it with a list of clusters in the folder.

  • The Apache Kafka® version number in the configSpec.version parameter.

  • List of cluster configuration fields to be changed in the updateMask parameter.

    Warning

    This API method resets the values of all parameters of the object to update, which aren't passed explicitly in the request. To avoid this, in the updateMask parameter, list the settings you want to change (in a single line, separated by commas).

Examples

Let's assume you need to upgrade your cluster from version 2.8 to version 3.0.

CLI
  1. To get a list of clusters and find out their IDs and names, run the command:

    yc managed-kafka cluster list
    
    +----------------------+---------------+---------------------+--------+---------+
    |          ID          |     NAME      |     CREATED AT      | HEALTH | STATUS  |
    +----------------------+---------------+---------------------+--------+---------+
    | c9q8p8j2gaih8iti42mh |    kafka35    | 2021-10-23 12:44:17 | ALIVE  | RUNNING |
    +----------------------+---------------+---------------------+--------+---------+
    
  2. To get information about the cluster kafka35, run the command below:

    yc managed-kafka cluster get kafka35
    
      id: c9q8p8j2gaih8iti42mh
      ...
      config:
        version: "2.8"
        ...
    
  3. To upgrade the kafka35 cluster to version 3.0, run the command below:

    yc managed-kafka cluster update kafka35 --version=3.0
    

Was the article helpful?

Language / Region
Yandex project
© 2023 Yandex.Cloud LLC
In this article:
  • Before upgrading
  • Upgrading a cluster
  • Examples