Managing OpenSearch host groups
In Managed Service for OpenSearch clusters, you cannot add, update, or delete individual hosts. Instead, you can manage host groups:
- Getting a list of host groups in a cluster
- Adding a host group to a cluster
- Updating a host group configuration
- Deleting a host group
You can also get a list of cluster hosts.
To learn how to migrate Managed Service for OpenSearch cluster host groups to a different availability zone, read this guide.
Getting a list of host groups in a cluster
- In the management console
, go to the folder page and select Managed Service for OpenSearch. - Click the name of the cluster you need and select the
Node groups tab.
If you do not have the Yandex Cloud command line interface yet, install and initialize it.
The folder specified in the CLI profile is used by default. You can specify a different folder using the --folder-name
or --folder-id
parameter.
To get a list of host groups in a cluster, request information about the OpenSearch cluster:
yc managed-opensearch cluster get <cluster_name_or_ID>
You can find the list of host groups in the config.opensearch.node_groups
and config.dashboards.node_groups
parameters.
You can request the cluster name and ID with a list of clusters in the folder.
To get a list of host groups in a cluster, use the get REST API method for the Cluster resource or the ClusterService/Get gRPC API call and provide the cluster ID in the clusterId
request parameter.
You can get the cluster ID with a list of clusters in the folder.
Adding a host group to a cluster
The following restrictions apply when adding host groups:
- A Managed Service for OpenSearch cluster may contain only one
Dashboards
host group. - If you are adding a group of
OpenSearch
hosts and assign theMANAGER
role to the hosts, the minimum number of hosts with such a role will be three.
To add a host group to a cluster:
-
In the management console
, go to the folder page and select Managed Service for OpenSearch. -
Click the name of the cluster you need and select the
Node groups tab. -
Click
Add node group. -
Specify the group parameters:
-
Group type:
OpenSearch
orDashboards
. -
Name which must be unique within the cluster.
-
For the
OpenSearch
host group, select a host role. -
Platform, host type, and host class.
The host class defines the technical characteristics of virtual machines that OpenSearch nodes are deployed on. All available options are listed under Host classes.
-
Disk type and data storage size.
The selected type determines the increments in which you can change your disk size:
- Network HDD and SSD storage: In 1 GB increments.
- Local SSD storage:
- For Intel Cascade Lake: In increments of 100 GB.
- For Intel Ice Lake: In 368 GB increments.
- Non-replicated SSD storage: In increments of 93 GB.
-
Host distribution across availability zones and subnets.
-
Number of hosts to create.
-
Enable Public access if you want to allow connecting to hosts over the internet.
-
-
Click Create node group.
Warning
Once you have added a host group, you will not be able to change the Public access option. You can only change other configuration settings using the API. However, you can also create a new host group with a different configuration, if required.
If you do not have the Yandex Cloud command line interface yet, install and initialize it.
The folder specified in the CLI profile is used by default. You can specify a different folder using the --folder-name
or --folder-id
parameter.
To add a host group to a cluster, run the command:
yc managed-opensearch node-group add --cluster-name <cluster_name> \
--opensearch-node-group name=<name_of_OpenSearch_host_group>,`
`resource-preset-id=<host_class>,`
`disk-size=<disk_size_in_bytes>,`
`disk-type-id=<disk_type>,`
`hosts-count=<number_of_hosts_per_group>,`
`zone-ids=<availability_zones>,`
`subnet-names=<subnet_names>,`
`assign-public-ip=<assign_public_IP:_true_or_false>,`
`roles=<host_roles> \
--dashboards-node-group name=<name_of_Dashboards_host_group>,`
`resource-preset-id=<host_class>,`
`disk-size=<disk_size_in_bytes>,`
`disk-type-id=<disk_type>,`
`hosts-count=<number_of_hosts_per_group>,`
`zone-ids=<availability_zones>,`
`subnet-names=<subnet_names>,`
`assign-public-ip=<assign-public-ip=<assign_public_IP:_true_or_false>
Specify the required parameters in the command depending on what type of host group you want to add:
-
--opensearch-node-group
:OpenSearch
host group configuration, where:-
--resource-preset-id
: Host class that defines the configuration of virtual machines the OpenSearch nodes will be deployed on. All available options are listed under Host classes. -
disk-size
: Disk size in bytes. Minimum and maximum values depend on the selected host class. -
disk-type-id
: Disk type. -
roles
: Host roles. The possible values include:data
: Assigns theDATA
role only.manager
: Assigns theMANAGER
role only.data+manager
ormanager+data
: Assigns both roles.
Tip
We do not recommend enabling public access to hosts with the
MANAGER
role, as this is not secure.
-
-
--dashboards-node-group
:Dashboards
host group configuration. It is configured in the same way as theOpenSearch
host group, except for the host roles. You do not need to configure any roles for theDashboards
group.
-
Open the current Terraform configuration file with an infrastructure plan.
For more information about creating this file, see Creating an OpenSearch cluster.
For a complete list of available Managed Service for OpenSearch cluster configuration fields, see the Terraform provider documentation
. -
To add а host group of the
OpenSearch
type, add thenode_groups
section toopensearch
:resource "yandex_mdb_opensearch_cluster" "<cluster_name>" { ... config { opensearch { ... node_groups { name = "<virtual_host_group_name>" assign_public_ip = <public_access> hosts_count = <number_of_hosts> zone_ids = ["<list_of_availability_zones>"] subnet_ids = ["<list_of_subnet_IDs>"] roles = ["<list_of_roles>"] resources { resource_preset_id = "<host_class>" disk_size = <storage_size_in_bytes> disk_type_id = "<disk_type>" } } } ... } }
Where:
assign_public_ip
: Public access to the host,true
orfalse
.roles
:DATA
andMANAGER
host roles.
-
To add а host group of the
Dashboards
type, add thedashboards
section toconfig
:resource "yandex_mdb_opensearch_cluster" "<cluster_name>" { ... config { ... dashboards { node_groups { name = "<virtual_host_group_name>" assign_public_ip = <public_access> hosts_count = <number_of_hosts> zone_ids = ["<list_of_availability_zones>"] subnet_ids = ["<list_of_subnet_IDs>"] resources { resource_preset_id = "<host_class>" disk_size = <storage_size_in_bytes> disk_type_id = "<disk_type>" } } } } }
Where
assign_public_ip
is public access to the host,true
orfalse
.A cluster may contain only one
Dashboards
host group. -
Make sure the settings are correct.
-
Using the command line, navigate to the folder that contains the up-to-date Terraform configuration files with an infrastructure plan.
-
Run the command:
terraform validate
If there are errors in the configuration files, Terraform will point to them.
-
-
Confirm updating the resources.
-
Run the command to view planned changes:
terraform plan
If the resource configuration descriptions are correct, the terminal will display a list of the resources to modify and their parameters. This is a test step. No resources are updated.
-
If you are happy with the planned changes, apply them:
-
Run the command:
terraform apply
-
Confirm the update of resources.
-
Wait for the operation to complete.
-
Timeouts
The Terraform provider sets the following timeouts for Managed Service for OpenSearch cluster operations:
- Creating a cluster, including restoring from a backup: 30 minutes.
- Editing a cluster: 60 minutes.
- Deleting a cluster: 15 minutes.
Operations exceeding the set timeout are interrupted.
How do I change these limits?Add the
timeouts
block to the cluster description, for example:resource "yandex_mdb_opensearch_cluster" "<cluster_name>" { ... timeouts { create = "1h30m" # 1 hour 30 minutes update = "2h" # 2 hours delete = "30m" # 30 minutes } }
-
To add a group of OpenSearch
hosts, use the addOpenSearchNodeGroup REST API method for the Cluster resource or the ClusterService/AddOpenSearchNodeGroup gRPC API call.
To add a group of Dashboards
hosts, use the addDashboardsNodeGroup REST API method for the Cluster resource or the ClusterService/AddDashboardsNodeGroup gRPC API call.
Provide the group configuration under nodeGroupSpec
in the request:
-
Host group name in the
name
parameter. -
Host class in the
resources.resourcePresetId
parameter. -
Disk type in the
resources.diskTypeId
parameter. -
Data storage size in the
resources.diskSize
parameter. -
Number of group hosts in the
hostsCount
parameter. -
List of availability zones in the
zoneIds
parameter. -
List of subnets in the
subnetIds
parameter. -
Public access settings in the
assignPublicIp
parameter. -
List of host roles in the
roles
parameter (forOpenSearch
host groups only).
Updating a host group configuration
If you do not have the Yandex Cloud command line interface yet, install and initialize it.
The folder specified in the CLI profile is used by default. You can specify a different folder using the --folder-name
or --folder-id
parameter.
To change the configuration of a host group, run the command:
yc managed-opensearch node-group update --cluster-name <cluster_name> \
--node-group-name <host_group_name> \
--resource-preset-id <host_class> \
--disk-size <disk_size_in_bytes> \
--hosts-count <number_of_hosts_per_group> \
--roles <host_roles>
Specify the required parameters in the command depending on what type of host group configuration you need:
-
--node-group-name
: Name of the host group you want to update. -
--resource-preset-id
: New host class that defines the configuration of virtual machines the OpenSearch nodes will be deployed on. All available options are listed under Host classes. -
--disk-size
: New disk size in bytes. Minimum and maximum values depend on the selected host class. -
--hosts-count
: New number of hosts in the group. -
--roles
: New host roles. The possible values include:data
: Assigns theDATA
role only.manager
: Assigns theMANAGER
role only.data+manager
ormanager+data
: Assigns both roles.
-
Open the current Terraform configuration file with an infrastructure plan.
For more information about creating this file, see Creating an OpenSearch cluster.
For a complete list of available Managed Service for OpenSearch cluster configuration fields, see the Terraform provider documentation
. -
To update the configuration of аn
OpenSearch
host group, edit the parameters of the requirednode_groups
section inopensearch
:resource "yandex_mdb_opensearch_cluster" "<cluster_name>" { ... opensearch { ... node_groups { name = "<virtual_host_group_name>" assign_public_ip = <public_access> hosts_count = <number_of_hosts> roles = ["<list_of_roles>"] resources { resource_preset_id = "<host_class>" disk_size = <storage_size_in_bytes> disk_type_id = "<disk_type>" } } ... } }
Where:
assign_public_ip
: Public access to the host,true
orfalse
.roles
:DATA
andMANAGER
host roles.
-
To update the configuration of а
Dashboards
host group, edit thedashboards
section parameters:resource "yandex_mdb_opensearch_cluster" "<cluster_name>" { ... dashboards { node_groups { name = "<virtual_host_group_name>" assign_public_ip = <public_access> hosts_count = <number_of_hosts> resources { resource_preset_id = "<host_class>" disk_size = <storage_size_in_bytes> disk_type_id = "<disk_type>" } } } }
Where
assign_public_ip
is public access to the host,true
orfalse
. -
Make sure the settings are correct.
-
Using the command line, navigate to the folder that contains the up-to-date Terraform configuration files with an infrastructure plan.
-
Run the command:
terraform validate
If there are errors in the configuration files, Terraform will point to them.
-
-
Confirm updating the resources.
-
Run the command to view planned changes:
terraform plan
If the resource configuration descriptions are correct, the terminal will display a list of the resources to modify and their parameters. This is a test step. No resources are updated.
-
If you are happy with the planned changes, apply them:
-
Run the command:
terraform apply
-
Confirm the update of resources.
-
Wait for the operation to complete.
-
Time limits
A Terraform provider sets the timeout for Managed Service for Elasticsearch cluster operations:
- Creating a cluster, including by restoring one from a backup: 30 minutes.
- Editing a cluster: 60 minutes.
- Deleting a cluster: 15 minutes.
Operations exceeding the set timeout are interrupted.
How do I change these limits?Add the
timeouts
block to the cluster description, for example:resource "yandex_mdb_elasticsearch_cluster" "<cluster_name>" { ... timeouts { create = "1h30m" # 1 hour 30 minutes update = "2h" # 2 hours delete = "30m" # 30 minutes } }
-
To update the OpenSearch
host group configuration, use the updateOpenSearchNodeGroup REST API method for the Cluster resource or the ClusterService/UpdateOpenSearchNodeGroup gRPC API call.
To update the Dashboards
host group configuration, use the updateDashboardsNodeGroup REST API method for the Cluster resource or the ClusterService/UpdateDashboardsNodeGroup gRPC API call.
Provide a new configuration under nodeGroupSpec
in the request:
- Host class in the
resources.resourcePresetId
parameter. - Disk type in the
resources.diskTypeId
parameter. - Data storage size in the
resources.diskSize
parameter. - Number of group hosts in the
hostsCount
parameter. - List of host roles in the
roles
parameter (forOpenSearch
host groups only).
Warning
This API method overrides all parameters of the object being modified that were not explicitly passed in the request to the default values. To avoid this, list the settings you want to change in the updateMask
parameter (one line separated by commas).
Deleting a host group
When deleting a host group, the following limitation applies: you cannot delete the only host group with the DATA
role.
- In the management console
, go to the folder page and select Managed Service for OpenSearch. - Click the name of the cluster you need and select the
Node groups tab. - Click
in the line with the appropriate group and select Delete.
If you do not have the Yandex Cloud command line interface yet, install and initialize it.
The folder specified in the CLI profile is used by default. You can specify a different folder using the --folder-name
or --folder-id
parameter.
To delete a host group, run the command:
yc managed-opensearch node-group delete --cluster-name <cluster_name> \
--node-group-name <host_group_name>
In the command, specify the host group you want to delete.
To remove a host group from a cluster:
-
Open the current Terraform configuration file with an infrastructure plan.
For more information about creating this file, see Creating an OpenSearch cluster.
For a complete list of available Managed Service for OpenSearch cluster configuration fields, see the Terraform provider documentation
. -
To remove аn
OpenSearch
host group, remove its correspondingnode_groups
section fromopensearch
. -
To remove а
Dashboards
host group, remove the correspondingdashboards
section. -
Make sure the settings are correct.
-
Using the command line, navigate to the folder that contains the up-to-date Terraform configuration files with an infrastructure plan.
-
Run the command:
terraform validate
If there are errors in the configuration files, Terraform will point to them.
-
-
Confirm the deletion of resources.
-
Run the command to view planned changes:
terraform plan
If the resource configuration descriptions are correct, the terminal will display a list of the resources to modify and their parameters. This is a test step. No resources are updated.
-
If you are happy with the planned changes, apply them:
-
Run the command:
terraform apply
-
Confirm the update of resources.
-
Wait for the operation to complete.
-
Time limits
A Terraform provider sets the timeout for Managed Service for Elasticsearch cluster operations:
- Creating a cluster, including by restoring one from a backup: 30 minutes.
- Editing a cluster: 60 minutes.
- Deleting a cluster: 15 minutes.
Operations exceeding the set timeout are interrupted.
How do I change these limits?Add the
timeouts
block to the cluster description, for example:resource "yandex_mdb_elasticsearch_cluster" "<cluster_name>" { ... timeouts { create = "1h30m" # 1 hour 30 minutes update = "2h" # 2 hours delete = "30m" # 30 minutes } }
-
To delete a group of OpenSearch
hosts, use the deleteOpenSearchNodeGroup REST API method for the Cluster resource or the ClusterService/DeleteOpenSearchNodeGroup gRPC API call.
To delete a group of Dashboards
hosts, use the deleteDashboardsNodeGroup REST API method for the Cluster resource or the ClusterService/DeleteDashboardsNodeGroup gRPC API call.
Provide the following in the request:
-
Cluster ID in the
clusterID
parameter.You can get the cluster ID with a list of clusters in the folder.
-
Name of the host group to delete, in the
name
parameter.
Getting a list of cluster hosts
- Go to the folder page
and select Managed Service for OpenSearch. - Click the cluster name and select the
Hosts tab.
To get a list of cluster hosts, use the listHosts REST API method for the Cluster resource or the ClusterService/ListHosts gRPC API call and provide the cluster ID in the clusterId
request parameter.
To find out the cluster ID, get a list of clusters in the folder.