Managing ClickHouse® cluster hosts
You can perform the following actions on ClickHouse® hosts:
- Get a list of cluster hosts.
- Add a host.
- Update ClickHouse® host settings.
- Restart a host.
- Remove a host.
For information about moving ClickHouse® hosts to a different availability zone, see this guide.
Warning
If you have created a cluster without ClickHouse® Keeper support, then before adding new hosts to any of the shards, enable fault tolerance using hosts ZooKeeper.
Getting a list of cluster hosts
- Go to the folder page
and select Managed Service for ClickHouse. - Click the cluster name and select the Hosts tab.
If you do not have the Yandex Cloud command line interface yet, install and initialize it.
The folder specified in the CLI profile is used by default. You can specify a different folder using the --folder-name
or --folder-id
parameter.
To get a list of cluster hosts, run the command:
yc managed-clickhouse host list \
--cluster-name=<cluster_name>
+----------------------------+--------------+---------+--------+---------------+
| NAME | CLUSTER ID | ROLE | HEALTH | ZONE ID |
+----------------------------+--------------+---------+--------+---------------+
| rc1b...mdb.yandexcloud.net | c9qp71dk1... | MASTER | ALIVE | ru-central1-b |
| rc1a...mdb.yandexcloud.net | c9qp71dk1... | REPLICA | ALIVE | ru-central1-a |
+----------------------------+--------------+---------+--------+---------------+
You can request the cluster name with a list of clusters in the folder.
To get a list of cluster hosts, use the listHosts REST API method for the Cluster resource or the ClusterService/ListHosts gRPC API call and provide the cluster ID in the clusterId
request parameter.
To find out the cluster ID, get a list of clusters in the folder.
Adding a host
The number of hosts in Managed Service for ClickHouse® clusters is limited by the CPU and RAM quotas available to DB clusters in your cloud. To check the resources in use, open the Quotas
-
Go to the folder page
and select Managed Service for ClickHouse. -
Click the cluster name and go to the Hosts tab.
-
Click
Add host. -
Specify the host parameters:
- Availability zone.
- Subnet (if the required subnet is not on the list, create it).
- Select Public access if the host must be accessible from outside Yandex Cloud.
- Shard name.
- Select the Copy data schema option to copy the schema from a random replica to the new host.
If you do not have the Yandex Cloud command line interface yet, install and initialize it.
The folder specified in the CLI profile is used by default. You can specify a different folder using the --folder-name
or --folder-id
parameter.
To add a host to a cluster:
-
Request a list of cluster subnets to select one for the new host:
yc vpc subnet list
Result:
+-----------+-----------+------------+---------------+------------------+ | ID | NAME | NETWORK ID | ZONE | RANGE | +-----------+-----------+------------+---------------+------------------+ | b0cl69... | default-d | enp6rq7... | ru-central1-d | [172.16.0.0/20] | | e2lkj9... | default-b | enp6rq7... | ru-central1-b | [10.10.0.0/16] | | e9b0ph... | a-2 | enp6rq7... | ru-central1-a | [172.16.32.0/20] | | e9b9v2... | default-a | enp6rq7... | ru-central1-a | [172.16.16.0/20] | +-----------+-----------+------------+---------------+------------------+
If the required subnet is not in the list, create it.
-
View a description of the CLI command for adding a host:
yc managed-clickhouse host add --help
-
Run the add host command:
yc managed-clickhouse host add \ --cluster-name=<cluster_name> \ --host zone-id=<availability_zone>,` `subnet-id=<subnet_ID>,` `assign-public-ip=<public_access_to_host>,` `shard-name=<shard_name>,` `type=clickhouse
Where
assign-public-ip
indicates whether the host is reachable from the internet over a public IP,true
orfalse
.To copy the data schema from a random replica to the new host, set the
--copy-schema
optional parameter.Managed Service for ClickHouse® will run the add host operation.
The subnet ID should be specified if the availability zone contains multiple subnets; otherwise, Managed Service for ClickHouse® will automatically select a single subnet. You can request the cluster name with a list of clusters in the folder.
-
Open the current Terraform configuration file with an infrastructure plan.
For more information about how to create this file, see Creating clusters.
-
Add the
host
block to the Managed Service for ClickHouse® cluster description.resource "yandex_mdb_clickhouse_cluster" "<cluster_name>" { ... host { type = "CLICKHOUSE" zone = "<availability_zone>" subnet_id = "<subnet_ID>" assign_public_ip = <public_access_to_host> } }
Where
assign_public_ip
indicates whether the host is reachable from the internet over a public IP,true
orfalse
. -
Make sure the settings are correct.
-
Using the command line, navigate to the folder that contains the up-to-date Terraform configuration files with an infrastructure plan.
-
Run the command:
terraform validate
If there are errors in the configuration files, Terraform will point to them.
-
-
Confirm updating the resources.
-
Run the command to view planned changes:
terraform plan
If the resource configuration descriptions are correct, the terminal will display a list of the resources to modify and their parameters. This is a test step. No resources are updated.
-
If you are happy with the planned changes, apply them:
-
Run the command:
terraform apply
-
Confirm the update of resources.
-
Wait for the operation to complete.
-
-
For more information, see the Terraform provider documentation
Time limits
A Terraform provider sets the timeout for Managed Service for ClickHouse® cluster operations:
- Creating a cluster, including by restoring one from a backup: 60 minutes.
- Editing a cluster: 90 minutes.
- Deleting a cluster: 30 minutes.
Operations exceeding the set timeout are interrupted.
Add the timeouts
block to the cluster description, for example:
resource "yandex_mdb_clickhouse_cluster" "<cluster_name>" {
...
timeouts {
create = "1h30m" # 1 hour 30 minutes
update = "2h" # 2 hours
delete = "30m" # 30 minutes
}
}
To add a host, use the addHosts REST API method for the Cluster resource or the ClusterService/AddHosts gRPC API call and provide the following in the request:
- Cluster ID in the
clusterId
parameter. To find out the cluster ID, get a list of clusters in the folder. - New host settings in one or more
hostSpecs
parameters.
To copy the data schema from a random replica to the new host, include the copySchema
parameter set to true
in the request.
Warning
If you cannot connect to the host you added, check that the cluster security group is configured correctly for the host's subnet.
Use the copy data schema option only if the schema is the same on all replica hosts of the cluster.
Changing a host
You can modify public access settings for every host in a Managed Service for ClickHouse® cluster.
To change the parameters of the cluster host:
- Go to the folder page
and select Managed Service for ClickHouse. - Click the cluster name and open the Hosts tab.
- Click
in the required host row and select Edit. - Enable Public access if the host must be accessible from outside Yandex Cloud.
- Click Save.
If you do not have the Yandex Cloud command line interface yet, install and initialize it.
The folder specified in the CLI profile is used by default. You can specify a different folder using the --folder-name
or --folder-id
parameter.
To change the parameters of the host, run the command:
yc managed-clickhouse host update <host_name> \
--cluster-name=<cluster_name> \
--assign-public-ip=<public_access_to_host>
Where assign-public-ip
indicates whether the host is reachable from the internet over a public IP, true
or false
.
You can request the host name with a list of cluster hosts, and the cluster name, with a list of clusters in the folder.
-
Open the current Terraform configuration file with an infrastructure plan.
For more information about how to create this file, see Creating clusters.
-
In the host, use the
host
section to add or edit theassign_public_ip
parameter.resource "yandex_mdb_clickhouse_cluster" "<cluster_name>" { ... host { ... assign_public_ip = <public_access_to_host> } ... }
Where
assign_public_ip
indicates whether the host is reachable from the internet over a public IP,true
orfalse
. -
Make sure the settings are correct.
-
Using the command line, navigate to the folder that contains the up-to-date Terraform configuration files with an infrastructure plan.
-
Run the command:
terraform validate
If there are errors in the configuration files, Terraform will point to them.
-
-
Confirm updating the resources.
-
Run the command to view planned changes:
terraform plan
If the resource configuration descriptions are correct, the terminal will display a list of the resources to modify and their parameters. This is a test step. No resources are updated.
-
If you are happy with the planned changes, apply them:
-
Run the command:
terraform apply
-
Confirm the update of resources.
-
Wait for the operation to complete.
-
-
For more information, see the Terraform provider documentation
To update host parameters, use the updateHosts REST API method for the Cluster resource or the ClusterService/UpdateHosts gRPC API call and provide the following in the request:
- In the
clusterId
parameter, the ID of the cluster where you want to change the host. To find out the cluster ID, get a list of clusters in the folder. - In the
updateHostSpecs.hostName
parameter, the name of the host you want to change. To find out the name, get a list of hosts in the cluster. - Host public access settings as
updateHostSpecs.assignPublicIp
. - A list of cluster configuration fields to modify (
assignPublicIp
in this case) asupdateMask
.
Warning
This API method overrides all parameters of the object being modified that were not explicitly passed in the request to the default values. To avoid this, list the settings you want to change in the updateMask
parameter (one line separated by commas).
Warning
If you cannot connect to the host after you changed it, check that the cluster security group is configured correctly for the host's subnet.
Restarting a host
You may need to restart hosts to promptly address such issues as the following:
- Resource overrun
- Memory leak
- Deadlock between requests
- Unresponsive ClickHouse® operations and internal processes
To restart a host:
- Go to the folder page
and select Managed Service for ClickHouse. - Click the cluster name and go to the Hosts tab.
- In the required host row, click
and select Restart. - Confirm the host restart.
If you do not have the Yandex Cloud command line interface yet, install and initialize it.
The folder specified in the CLI profile is used by default. You can specify a different folder using the --folder-name
or --folder-id
parameter.
To restart a host, run this command:
yc managed-clickhouse host restart <host_name> \
--cluster-name=<cluster_name>
You can request the host name with a list of cluster hosts, and the cluster name, with a list of clusters in the folder.
To restart a host, use the restartHosts REST API method for the Cluster resource or the ClusterService/RestartHosts gRPC API call and provide the following in your request:
- In the
clusterId
parameter, the ID of the cluster where you want to change the host. To find out the cluster ID, get a list of clusters in the folder. - In the
hostNames
parameter, the name of the host you want to restart. To find out the host name, get a list of hosts in the cluster.
Removing a host
You can remove a host from a ClickHouse® cluster if it contains three or more hosts.
Note
A cluster created with ClickHouse® Keeper replication support must include three or more hosts.
- Go to the folder page
and select Managed Service for ClickHouse. - Click the cluster name and open the Hosts tab.
- Click
in the required host row and select Delete.
If you do not have the Yandex Cloud command line interface yet, install and initialize it.
The folder specified in the CLI profile is used by default. You can specify a different folder using the --folder-name
or --folder-id
parameter.
To remove a host from the cluster, run:
yc managed-clickhouse host delete <host_name> \
--cluster-name=<cluster_name>
You can request the host name with a list of cluster hosts, and the cluster name, with a list of clusters in the folder.
-
Open the current Terraform configuration file with an infrastructure plan.
For more information about how to create this file, see Creating clusters.
-
In the Managed Service for ClickHouse® cluster description, remove the
CLICKHOUSE
typehost
block. -
Make sure the settings are correct.
-
Using the command line, navigate to the folder that contains the up-to-date Terraform configuration files with an infrastructure plan.
-
Run the command:
terraform validate
If there are errors in the configuration files, Terraform will point to them.
-
-
Type
yes
and press Enter.-
Run the command to view planned changes:
terraform plan
If the resource configuration descriptions are correct, the terminal will display a list of the resources to modify and their parameters. This is a test step. No resources are updated.
-
If you are happy with the planned changes, apply them:
-
Run the command:
terraform apply
-
Confirm the update of resources.
-
Wait for the operation to complete.
-
-
For more information, see the Terraform provider documentation
Time limits
A Terraform provider sets the timeout for Managed Service for ClickHouse® cluster operations:
- Creating a cluster, including by restoring one from a backup: 60 minutes.
- Editing a cluster: 90 minutes.
- Deleting a cluster: 30 minutes.
Operations exceeding the set timeout are interrupted.
Add the timeouts
block to the cluster description, for example:
resource "yandex_mdb_clickhouse_cluster" "<cluster_name>" {
...
timeouts {
create = "1h30m" # 1 hour 30 minutes
update = "2h" # 2 hours
delete = "30m" # 30 minutes
}
}
To delete a host, use the deleteHosts REST API method for the Cluster resource or the ClusterService/DeleteHosts gRPC API call and provide the following in the request:
- Cluster ID in the
clusterId
parameter. To find out the cluster ID, get a list of clusters in the folder. - Name(s) of the host(s) to delete in the
hostNames
parameter.
ClickHouse® is a registered trademark of ClickHouse, Inc