Yandex Cloud
  • Services
  • Solutions
  • Why Yandex Cloud
  • Pricing
  • Documentation
  • Contact us
Get started
Language / Region
© 2022 Yandex.Cloud LLC
Yandex Data Proc
  • Practical guidelines
    • Working with jobs
      • Overview
      • Working with Hive jobs
      • Working with MapReduce jobs
      • Working with PySpark jobs
      • Working with Spark jobs
      • Using Apache Hive
      • Running Spark applications
      • Running applications from a remote host
    • Configuring networks for Data Proc clusters
    • Using Yandex Object Storage in Data Proc
  • Step-by-step instructions
    • All instructions
    • Information about existing clusters
    • Creating clusters
    • Connecting to clusters
    • Editing clusters
    • Updating subclusters
    • Managing subclusters
    • Sqoop usage
    • Managing jobs
      • All jobs
      • Spark jobs
      • PySpark jobs
      • Hive jobs
      • MapReduce jobs
    • Deleting clusters
    • Monitoring the state of a cluster and hosts
    • Working with logs
  • Concepts
    • Data Proc overview
    • Host classes
    • Hadoop and component versions
    • Component interfaces and ports
    • Component web interfaces
    • Jobs in Data Proc
    • Autoscaling
    • Decommissioning subclusters and hosts
    • Network in Data Proc
    • Quotas and limits
    • Storage in Data Proc
    • Component properties
    • Logs in Data Proc
    • Initialization scripts
  • Access management
  • Pricing policy
  • API reference
    • Authentication in the API
    • gRPC
      • Overview
      • ClusterService
      • JobService
      • ResourcePresetService
      • SubclusterService
      • OperationService
    • REST
      • Overview
      • Cluster
        • Overview
        • create
        • delete
        • get
        • list
        • listHosts
        • listOperations
        • listUILinks
        • start
        • stop
        • update
      • Job
        • Overview
        • cancel
        • create
        • get
        • list
        • listLog
      • ResourcePreset
        • Overview
        • get
        • list
      • Subcluster
        • Overview
        • create
        • delete
        • get
        • list
        • update
  • Releases
    • Images
  • Questions and answers
  1. Step-by-step instructions
  2. Editing clusters

Updating clusters

Written by
Yandex Cloud

    After creating a cluster, you can modify its basic and advanced settings.

    Note

    You can disable sending cluster logs to Yandex Cloud Logging. For more information, see Working with logs.

    Management console
    CLI
    Terraform
    1. Go to the folder page and select Data Proc.

    2. Select the cluster and click Edit cluster in the top panel.

    3. To edit the log group that cluster logs are sent to, select a new log group from the list. If necessary, create a new log group.

      To enable this functionality, assign the cluster service account the logging.writer role. For more information, see the Yandex Cloud Logging documentation.

    4. Change additional cluster settings:

      Deletion protection: Manages cluster protection from accidental deletion by a user.

      Enabled protection will not prevent a manual connection to a cluster to delete data.

    5. Click Save changes.

    If you don't have the Yandex Cloud command line interface yet, install and initialize it.

    The folder specified in the CLI profile is used by default. You can specify a different folder using the --folder-name or --folder-id parameter.

    1. View a description of the CLI's update cluster command:

      yc dataproc cluster update --help
      
    2. To edit the log group that cluster logs are sent to, pass the log group ID in the --log-group-id parameter:

      yc dataproc cluster update <cluster ID or name> \
        --log-group-id=<log group ID>
      

      You can request the log group ID with a list of log groups in the folder.

    3. To protect a cluster from accidental deletion by a user of your cloud, set --deletion-protection to true:

      yc dataproc cluster update <cluster ID or name> \
        --deletion-protection=<cluster deletion protection: true or false>
      

      Enabled protection will not prevent a manual connection to a cluster to delete data.

    You can find out the cluster ID and name in a list of clusters in the folder.

    1. Open the current Terraform configuration file with an infrastructure plan.

      For more information about creating this file, see Creating a Data Proc cluster.

    2. To activate cluster deletion protection and access to Data Proc component web interfaces, update the values in the appropriate fields of the Data Proc cluster description:

      resource "yandex_dataproc_cluster" "< cluster name>" {
        ...
        deletion_protection = true
        ui_proxy            = true
        ...
      }
      
    3. Make sure the settings are correct.

      1. Using the command line, navigate to the folder that contains the up-to-date Terraform configuration files with an infrastructure plan.

      2. Run the command:

        terraform validate
        

        If there are errors in the configuration files, Terraform will point to them.

    4. Confirm the update of resources.

      1. Run the command to view planned changes:

        terraform plan
        

        If the resource configuration descriptions are correct, the terminal will display a list of the resources to modify and their parameters. This is a test step. No resources are updated.

      2. If you are happy with the planned changes, apply them:

        1. Run the command:

          terraform apply
          
        2. Confirm the update of resources.

        3. Wait for the operation to complete.

    For more information, see the Terraform provider documentation.

    Was the article helpful?

    Language / Region
    © 2022 Yandex.Cloud LLC