Yandex.Cloud
  • Services
  • Why Yandex.Cloud
  • Solutions
  • Pricing
  • Documentation
  • Contact us
Get started
Yandex Data Proc
  • Use cases
    • Configuring networks for Data Proc clusters
    • Using Apache Hive
    • Running Spark applications
    • Running applications from a remote host
    • Copying files from Yandex Object Storage
  • Step-by-step instructions
    • All instructions
    • Creating clusters
    • Connecting to clusters
    • Updating subclusters
    • Managing subclusters
    • Deleting clusters
  • Concepts
    • Data Proc overview
    • Host classes
    • Hadoop and component versions
    • Component interfaces and ports
    • Component web interfaces
    • Auto scaling
    • Decommissioning subclusters and hosts
    • Network in Data Proc
    • Quotas and limits
  • Access management
  • Pricing policy
  • API reference
    • Authentication in the API
    • gRPC
      • Overview
      • ClusterService
      • JobService
      • ResourcePresetService
      • SubclusterService
      • OperationService
    • REST
      • Overview
      • Cluster
        • Overview
        • create
        • delete
        • get
        • list
        • listHosts
        • listOperations
        • listUILinks
        • start
        • stop
        • update
      • Job
        • Overview
        • create
        • get
        • list
        • listLog
      • ResourcePreset
        • Overview
        • get
        • list
      • Subcluster
        • Overview
        • create
        • delete
        • get
        • list
        • update
  • Questions and answers
  1. Step-by-step instructions
  2. Managing subclusters

Managing Data Proc subclusters

  • Getting a list of subclusters
  • Adding a subcluster
  • Deleting a subcluster

In addition to updating the settings of a particular subcluster, you can create new and delete existing subclusters.

Warning

Each cluster may have only 1 MASTERNODE subcluster, which is why you can't create or delete such subclusters. You can't delete DATANODE subclusters, either.

Getting a list of subclusters

Management console
  1. Go to the folder page and select Data Proc.

  2. Click on the name of the desired cluster and then select the Subclusters tab.

Adding a subcluster

The number of hosts in Data Proc clusters is limited quotas by the number of vCPUs and the amount of RAM that VMs in your cloud can use. To view the resources available, go to the Quotas section and find Compute Cloud.

To add a subcluster:

Management console
  1. In management console, select the required folder.

  2. Select Data Proc and the desired cluster.

  3. Go to Subclusters.

  4. Click Add subcluster.

  5. Select the subcluster type. To do this, decide what services should be deployed on the hosts:

    • In COMPUTENODE subclusters, you can deploy:
      • YARN NodeManager
      • Spark libraries
    • In DATANODE subclusters, you can deploy:
      • HDFS Datanode
      • YARN NodeManager
      • HBase RegionServer
      • Spark libraries
  6. Select the other subcluster settings:

    • The number of hosts.
    • The host class is the platform and computing resources available to the host.
    • Storage size and type.
    • The subnet of the network where the cluster is located.
  7. Click Add subcluster.

Data Proc runs the create subcluster operation.

Deleting a subcluster

Warning

You can't delete DATANODE subclusters.

Management console

To delete a subcluster:

  1. In management console, select the required folder.
  2. Select Data Proc and the desired cluster.
  3. Go to Subclusters.
  4. Optionally specify the decommissioning timeout.
  5. Click for the appropriate subcluster and select Delete.
  6. Confirm the deletion.

Data Proc runs the delete subcluster operation.

In this article:
  • Getting a list of subclusters
  • Adding a subcluster
  • Deleting a subcluster
Language / Region
Careers
Privacy policy
Terms of use
Brandbook
© 2021 Yandex.Cloud LLC