Creating ClickHouse clusters

If you don't have the Yandex.Cloud command line interface yet, install it.

The folder specified in the CLI profile is used by default. You can specify a different folder using the --folder-name or --folder-id parameter.

To create a cluster:

1. Check whether the folder has any subnets for the cluster hosts:

   $ yc vpc subnet list
   

   If there are no subnets in the folder, create the necessary subnets in VPC.

2. View a description of the CLI's create cluster command:

   $ yc managed-clickhouse cluster create --help
   

3. Specify the cluster parameters in the create command (the example shows only mandatory flags):

   $ yc managed-clickhouse cluster create \
      --cluster-name <cluster name> \
      --environment <prestable or production> \
      --network-name <network name> \
      --host type=<clickhouse or zookeeper>,zone-id=<availability zone>,subnet-id=<subnet ID> \
      --resource-preset <host class> \
      --clickhouse-disk-type <network-hdd | network-ssd | local-ssd> \
      --clickhouse-disk-size <storage size in GB> \
      --user name=<username>,password=<user password> \
      --database name=<DB name>
   

   The subnet ID subnet-id should be specified if the selected availability zone contains two or more subnets.

With Terraform, you can quickly create a cloud infrastructure in Yandex.Cloud. The infrastructure components are identified through configuration files that specify the required cloud resources and their parameters.

If you don't have Terraform yet, install it and configure the provider.

To create a cluster:

1. In the configuration file, describe the parameters of resources that you want to create:

   • Database cluster: Description of the cluster and its hosts.
   • Network: Description of the cloud network where the cluster will be located. If you already have a suitable network, you don't need to describe it again.
   • Subnets: Description of the subnets to connect the cluster hosts to. If you already have suitable subnets, you don't need to describe them again.

   Sample configuration file structure:

   resource "yandex_mdb_clickhouse_cluster" "<cluster name>" {
     name        = "<cluster name>"
     environment = "<environment>"
     network_id  = "<network ID>"
   
     clickhouse {
       resources {
         resource_preset_id = "<host class>"
         disk_type_id       = "<storage type>"
         disk_size          = "<storage size, GB>"
       }
     }
   
     database {
       name = "<DB name>"
     }
   
     user {
       name     = "<DB username>"
       password = "<password>"
       permission {
         database_name = "<name of the DB where the user is created>"
       }
     }
   
     host {
       type      = "CLICKHOUSE"
       zone      = "<availability zone>"
       subnet_id = "<subnet ID>"
     }
   }
   
   resource "yandex_vpc_network" "<network name>" {}
   
   resource "yandex_vpc_subnet" "<subnet name>" {
     zone           = "<availability zone>"
     network_id     = "<network ID>"
     v4_cidr_blocks = ["<range>"]
   }
   

   For more information about resources that you can create using Terraform, see the provider's documentation.

2. Make sure that the configuration files are correct.

   1. In the command line, go to the folder where you created the configuration file.

   2. Run the check using the command:

      terraform plan
      

   If the configuration is described correctly, the terminal will display a list of created resources and their parameters. If there are errors in the configuration, Terraform will point them out. This is a test step. No resources are created.

3. Create a cluster.

   1. If there are no errors in the configuration, run the command:

      terraform apply
      

   2. Confirm the creation of resources.

   After this, all the necessary resources will be created in the specified folder and the IP addresses of the VMs will be displayed in the terminal. You can check resource availability and their settings in консоли управления.

Creating a single-host cluster

Let's say we need to create a ClickHouse cluster and a network for it with the following characteristics:

• Named mych.
• In the PRESTABLE environment.
• In the cloud with ID b1gq90dgh25иuebiu75o.
• In a folder named myfolder.
• In a new network named mynet.
• With a single s2.micro class host in a new subnet named mysubnet and in the ru-central1-c availability zone. The mysubnet subnet will have the 10.5.0.0/24 range.
• With 32 GB of fast network storage.
• With the database name my_db.
• With the username user1 and password user1user1.

The configuration file for the cluster looks like this:

provider "yandex" {
  token = "<OAuth or static key of service account>"
  cloud_id  = "b1gq90dgh25иuebiu75o"
  folder_id = "${data.yandex_resourcemanager_folder.myfolder.id}"
  zone      = "ru-central1-c"
}

resource "yandex_mdb_clickhouse_cluster" "mych" {
  name        = "mych"
  environment = "PRESTABLE"
  network_id  = "${yandex_vpc_network.mynet.id}"

  clickhouse {
    resources {
      resource_preset_id = "s2.micro"
      disk_type_id       = "network-ssd"
      disk_size          = 32
    }
  }

  database {
    name = "my_db"
  }

  user {
    name     = "user1"
    password = "user1user1"
    permission {
      database_name = "my_db"
    }
  }

  host {
    type      = "CLICKHOUSE"
    zone      = "ru-central1-c"
    subnet_id = "${yandex_vpc_subnet.mysubnet.id}"
  }
}

resource "yandex_vpc_network" "mynet" {}

resource "yandex_vpc_subnet" "mysubnet" {
  zone           = "ru-central1-c"
  network_id     = "${yandex_vpc_network.mynet.id}"
  v4_cidr_blocks = ["10.5.0.0/24"]
}