Managed Service for ClickHouse API, gRPC: ClusterService
- Calls ClusterService
- Get
- GetClusterRequest
- Cluster
- Monitoring
- ClusterConfig
- Clickhouse
- Zookeeper
- Access
- CloudStorage
- ClickhouseConfigSet
- ClickhouseConfig
- MergeTree
- Kafka
- KafkaTopic
- Rabbitmq
- Compression
- ExternalDictionary
- HttpSource
- MysqlSource
- Replica
- ClickhouseSource
- MongodbSource
- PostgresqlSource
- Structure
- Attribute
- Id
- Key
- Layout
- Range
- GraphiteRollup
- Pattern
- Retention
- Resources
- MaintenanceWindow
- AnytimeMaintenanceWindow
- WeeklyMaintenanceWindow
- MaintenanceOperation
- List
- ListClustersRequest
- ListClustersResponse
- Cluster
- Monitoring
- ClusterConfig
- Clickhouse
- Zookeeper
- Access
- CloudStorage
- ClickhouseConfigSet
- ClickhouseConfig
- MergeTree
- Kafka
- KafkaTopic
- Rabbitmq
- Compression
- ExternalDictionary
- HttpSource
- MysqlSource
- Replica
- ClickhouseSource
- MongodbSource
- PostgresqlSource
- Structure
- Attribute
- Id
- Key
- Layout
- Range
- GraphiteRollup
- Pattern
- Retention
- Resources
- MaintenanceWindow
- AnytimeMaintenanceWindow
- WeeklyMaintenanceWindow
- MaintenanceOperation
- Create
- CreateClusterRequest
- ConfigSpec
- Clickhouse
- Zookeeper
- Access
- CloudStorage
- ClickhouseConfig
- MergeTree
- Kafka
- KafkaTopic
- Rabbitmq
- Compression
- ExternalDictionary
- HttpSource
- MysqlSource
- Replica
- ClickhouseSource
- MongodbSource
- PostgresqlSource
- Structure
- Attribute
- Id
- Key
- Layout
- Range
- GraphiteRollup
- Pattern
- Retention
- Resources
- DatabaseSpec
- UserSpec
- Permission
- UserSettings
- UserQuota
- HostSpec
- MaintenanceWindow
- AnytimeMaintenanceWindow
- WeeklyMaintenanceWindow
- Operation
- CreateClusterMetadata
- Cluster
- Monitoring
- ClusterConfig
- Clickhouse
- Zookeeper
- ClickhouseConfigSet
- ClickhouseConfig
- MergeTree
- Kafka
- KafkaTopic
- Rabbitmq
- Compression
- ExternalDictionary
- HttpSource
- MysqlSource
- Replica
- ClickhouseSource
- MongodbSource
- PostgresqlSource
- Structure
- Attribute
- Id
- Key
- Layout
- Range
- GraphiteRollup
- Pattern
- Retention
- MaintenanceOperation
- Update
- UpdateClusterRequest
- ConfigSpec
- Clickhouse
- Zookeeper
- Access
- CloudStorage
- ClickhouseConfig
- MergeTree
- Kafka
- KafkaTopic
- Rabbitmq
- Compression
- ExternalDictionary
- HttpSource
- MysqlSource
- Replica
- ClickhouseSource
- MongodbSource
- PostgresqlSource
- Structure
- Attribute
- Id
- Key
- Layout
- Range
- GraphiteRollup
- Pattern
- Retention
- Resources
- MaintenanceWindow
- AnytimeMaintenanceWindow
- WeeklyMaintenanceWindow
- Operation
- UpdateClusterMetadata
- Cluster
- Monitoring
- ClusterConfig
- Clickhouse
- Zookeeper
- ClickhouseConfigSet
- ClickhouseConfig
- MergeTree
- Kafka
- KafkaTopic
- Rabbitmq
- Compression
- ExternalDictionary
- HttpSource
- MysqlSource
- Replica
- ClickhouseSource
- MongodbSource
- PostgresqlSource
- Structure
- Attribute
- Id
- Key
- Layout
- Range
- GraphiteRollup
- Pattern
- Retention
- MaintenanceOperation
- Delete
- Start
- StartClusterRequest
- Operation
- StartClusterMetadata
- Cluster
- Monitoring
- ClusterConfig
- Clickhouse
- Zookeeper
- Access
- CloudStorage
- ClickhouseConfigSet
- ClickhouseConfig
- MergeTree
- Kafka
- KafkaTopic
- Rabbitmq
- Compression
- ExternalDictionary
- HttpSource
- MysqlSource
- Replica
- ClickhouseSource
- MongodbSource
- PostgresqlSource
- Structure
- Attribute
- Id
- Key
- Layout
- Range
- GraphiteRollup
- Pattern
- Retention
- Resources
- MaintenanceWindow
- AnytimeMaintenanceWindow
- WeeklyMaintenanceWindow
- MaintenanceOperation
- Stop
- StopClusterRequest
- Operation
- StopClusterMetadata
- Cluster
- Monitoring
- ClusterConfig
- Clickhouse
- Zookeeper
- Access
- CloudStorage
- ClickhouseConfigSet
- ClickhouseConfig
- MergeTree
- Kafka
- KafkaTopic
- Rabbitmq
- Compression
- ExternalDictionary
- HttpSource
- MysqlSource
- Replica
- ClickhouseSource
- MongodbSource
- PostgresqlSource
- Structure
- Attribute
- Id
- Key
- Layout
- Range
- GraphiteRollup
- Pattern
- Retention
- Resources
- MaintenanceWindow
- AnytimeMaintenanceWindow
- WeeklyMaintenanceWindow
- MaintenanceOperation
- Move
- MoveClusterRequest
- Operation
- MoveClusterMetadata
- Cluster
- Monitoring
- ClusterConfig
- Clickhouse
- Zookeeper
- Access
- CloudStorage
- ClickhouseConfigSet
- ClickhouseConfig
- MergeTree
- Kafka
- KafkaTopic
- Rabbitmq
- Compression
- ExternalDictionary
- HttpSource
- MysqlSource
- Replica
- ClickhouseSource
- MongodbSource
- PostgresqlSource
- Structure
- Attribute
- Id
- Key
- Layout
- Range
- GraphiteRollup
- Pattern
- Retention
- Resources
- MaintenanceWindow
- AnytimeMaintenanceWindow
- WeeklyMaintenanceWindow
- MaintenanceOperation
- AddZookeeper
- AddClusterZookeeperRequest
- Resources
- HostSpec
- Operation
- AddClusterZookeeperMetadata
- Cluster
- Monitoring
- ClusterConfig
- Clickhouse
- Zookeeper
- Access
- CloudStorage
- ClickhouseConfigSet
- ClickhouseConfig
- MergeTree
- Kafka
- KafkaTopic
- Rabbitmq
- Compression
- ExternalDictionary
- HttpSource
- MysqlSource
- Replica
- ClickhouseSource
- MongodbSource
- PostgresqlSource
- Structure
- Attribute
- Id
- Key
- Layout
- Range
- GraphiteRollup
- Pattern
- Retention
- MaintenanceWindow
- AnytimeMaintenanceWindow
- WeeklyMaintenanceWindow
- MaintenanceOperation
- Backup
- BackupClusterRequest
- Operation
- BackupClusterMetadata
- Cluster
- Monitoring
- ClusterConfig
- Clickhouse
- Zookeeper
- Access
- CloudStorage
- ClickhouseConfigSet
- ClickhouseConfig
- MergeTree
- Kafka
- KafkaTopic
- Rabbitmq
- Compression
- ExternalDictionary
- HttpSource
- MysqlSource
- Replica
- ClickhouseSource
- MongodbSource
- PostgresqlSource
- Structure
- Attribute
- Id
- Key
- Layout
- Range
- GraphiteRollup
- Pattern
- Retention
- Resources
- MaintenanceWindow
- AnytimeMaintenanceWindow
- WeeklyMaintenanceWindow
- MaintenanceOperation
- Restore
- RestoreClusterRequest
- ConfigSpec
- Clickhouse
- Zookeeper
- Access
- CloudStorage
- ClickhouseConfig
- MergeTree
- Kafka
- KafkaTopic
- Rabbitmq
- Compression
- ExternalDictionary
- HttpSource
- MysqlSource
- Replica
- ClickhouseSource
- MongodbSource
- PostgresqlSource
- Structure
- Attribute
- Id
- Key
- Layout
- Range
- GraphiteRollup
- Pattern
- Retention
- Resources
- HostSpec
- Operation
- RestoreClusterMetadata
- Cluster
- Monitoring
- ClusterConfig
- Clickhouse
- Zookeeper
- ClickhouseConfigSet
- ClickhouseConfig
- MergeTree
- Kafka
- KafkaTopic
- Rabbitmq
- Compression
- ExternalDictionary
- HttpSource
- MysqlSource
- Replica
- ClickhouseSource
- MongodbSource
- PostgresqlSource
- Structure
- Attribute
- Id
- Key
- Layout
- Range
- GraphiteRollup
- Pattern
- Retention
- MaintenanceWindow
- AnytimeMaintenanceWindow
- WeeklyMaintenanceWindow
- MaintenanceOperation
- RescheduleMaintenance
- RescheduleMaintenanceRequest
- Operation
- RescheduleMaintenanceMetadata
- Cluster
- Monitoring
- ClusterConfig
- Clickhouse
- Zookeeper
- Access
- CloudStorage
- ClickhouseConfigSet
- ClickhouseConfig
- MergeTree
- Kafka
- KafkaTopic
- Rabbitmq
- Compression
- ExternalDictionary
- HttpSource
- MysqlSource
- Replica
- ClickhouseSource
- MongodbSource
- PostgresqlSource
- Structure
- Attribute
- Id
- Key
- Layout
- Range
- GraphiteRollup
- Pattern
- Retention
- Resources
- MaintenanceWindow
- AnytimeMaintenanceWindow
- WeeklyMaintenanceWindow
- MaintenanceOperation
- ListLogs
- StreamLogs
- ListOperations
- ListBackups
- ListHosts
- AddHosts
- UpdateHosts
- DeleteHosts
- RestartHosts
- GetShard
- ListShards
- ListClusterShardsRequest
- ListClusterShardsResponse
- Shard
- ShardConfig
- Clickhouse
- ClickhouseConfigSet
- ClickhouseConfig
- MergeTree
- Kafka
- KafkaTopic
- Rabbitmq
- Compression
- ExternalDictionary
- HttpSource
- MysqlSource
- Replica
- ClickhouseSource
- MongodbSource
- PostgresqlSource
- Structure
- Attribute
- Id
- Key
- Layout
- Range
- GraphiteRollup
- Pattern
- Retention
- Resources
- AddShard
- AddClusterShardRequest
- ShardConfigSpec
- Clickhouse
- ClickhouseConfig
- MergeTree
- Kafka
- KafkaTopic
- Rabbitmq
- Compression
- ExternalDictionary
- HttpSource
- MysqlSource
- Replica
- ClickhouseSource
- MongodbSource
- PostgresqlSource
- Structure
- Attribute
- Id
- Key
- Layout
- Range
- GraphiteRollup
- Pattern
- Retention
- Resources
- HostSpec
- Operation
- AddClusterShardMetadata
- Shard
- ShardConfig
- Clickhouse
- ClickhouseConfigSet
- ClickhouseConfig
- MergeTree
- Kafka
- KafkaTopic
- Rabbitmq
- Compression
- ExternalDictionary
- HttpSource
- MysqlSource
- Replica
- ClickhouseSource
- MongodbSource
- PostgresqlSource
- Structure
- Attribute
- Id
- Key
- Layout
- Range
- GraphiteRollup
- Pattern
- Retention
- UpdateShard
- UpdateClusterShardRequest
- ShardConfigSpec
- Clickhouse
- ClickhouseConfig
- MergeTree
- Kafka
- KafkaTopic
- Rabbitmq
- Compression
- ExternalDictionary
- HttpSource
- MysqlSource
- Replica
- ClickhouseSource
- MongodbSource
- PostgresqlSource
- Structure
- Attribute
- Id
- Key
- Layout
- Range
- GraphiteRollup
- Pattern
- Retention
- Resources
- Operation
- UpdateClusterShardMetadata
- Shard
- ShardConfig
- Clickhouse
- ClickhouseConfigSet
- ClickhouseConfig
- MergeTree
- Kafka
- KafkaTopic
- Rabbitmq
- Compression
- ExternalDictionary
- HttpSource
- MysqlSource
- Replica
- ClickhouseSource
- MongodbSource
- PostgresqlSource
- Structure
- Attribute
- Id
- Key
- Layout
- Range
- GraphiteRollup
- Pattern
- Retention
- DeleteShard
- GetShardGroup
- ListShardGroups
- CreateShardGroup
- UpdateShardGroup
- DeleteShardGroup
- ListExternalDictionaries
- CreateExternalDictionary
- CreateClusterExternalDictionaryRequest
- Operation
- CreateClusterExternalDictionaryMetadata
- Cluster
- Monitoring
- ClusterConfig
- Clickhouse
- Zookeeper
- Access
- CloudStorage
- ClickhouseConfigSet
- ClickhouseConfig
- MergeTree
- Kafka
- KafkaTopic
- Rabbitmq
- Compression
- ExternalDictionary
- HttpSource
- MysqlSource
- Replica
- ClickhouseSource
- MongodbSource
- PostgresqlSource
- Structure
- Attribute
- Id
- Key
- Layout
- Range
- GraphiteRollup
- Pattern
- Retention
- Resources
- MaintenanceWindow
- AnytimeMaintenanceWindow
- WeeklyMaintenanceWindow
- MaintenanceOperation
- UpdateExternalDictionary
- UpdateClusterExternalDictionaryRequest
- Operation
- UpdateClusterExternalDictionaryMetadata
- Cluster
- Monitoring
- ClusterConfig
- Clickhouse
- Zookeeper
- Access
- CloudStorage
- ClickhouseConfigSet
- ClickhouseConfig
- MergeTree
- Kafka
- KafkaTopic
- Rabbitmq
- Compression
- ExternalDictionary
- HttpSource
- MysqlSource
- Replica
- ClickhouseSource
- MongodbSource
- PostgresqlSource
- Structure
- Attribute
- Id
- Key
- Layout
- Range
- GraphiteRollup
- Pattern
- Retention
- Resources
- MaintenanceWindow
- AnytimeMaintenanceWindow
- WeeklyMaintenanceWindow
- MaintenanceOperation
- DeleteExternalDictionary
- DeleteClusterExternalDictionaryRequest
- Operation
- DeleteClusterExternalDictionaryMetadata
- Cluster
- Monitoring
- ClusterConfig
- Clickhouse
- Zookeeper
- Access
- CloudStorage
- ClickhouseConfigSet
- ClickhouseConfig
- MergeTree
- Kafka
- KafkaTopic
- Rabbitmq
- Compression
- ExternalDictionary
- HttpSource
- MysqlSource
- Replica
- ClickhouseSource
- MongodbSource
- PostgresqlSource
- Structure
- Attribute
- Id
- Key
- Layout
- Range
- GraphiteRollup
- Pattern
- Retention
- Resources
- MaintenanceWindow
- AnytimeMaintenanceWindow
- WeeklyMaintenanceWindow
- MaintenanceOperation
A set of methods for managing ClickHouse clusters.
Call | Description |
---|---|
Get | Returns the specified ClickHouse cluster. |
List | Retrieves a list of ClickHouse clusters that belong to the specified folder. |
Create | Creates a ClickHouse cluster in the specified folder. |
Update | Updates the specified ClickHouse cluster. |
Delete | Deletes the specified ClickHouse cluster. |
Start | Starts the specified ClickHouse cluster. |
Stop | Stops the specified ClickHouse cluster. |
Move | Moves a ClickHouse cluster to the specified folder. |
AddZookeeper | Adds a ZooKeeper subcluster to the specified ClickHouse cluster. |
Backup | Creates a backup for the specified ClickHouse cluster. |
Restore | Creates a new ClickHouse cluster using the specified backup. |
RescheduleMaintenance | Reschedules planned maintenance operation. |
ListLogs | Retrieves logs for the specified ClickHouse cluster. |
StreamLogs | Same as ListLogs but using server-side streaming. |
ListOperations | Retrieves the list of Operation resources for the specified cluster. |
ListBackups | Retrieves the list of available backups for the specified ClickHouse cluster. |
ListHosts | Retrieves a list of hosts for the specified cluster. |
AddHosts | Creates new hosts for a cluster. |
UpdateHosts | Updates the specified hosts. |
DeleteHosts | Deletes the specified hosts for a cluster. |
RestartHosts | |
GetShard | Returns the specified shard. |
ListShards | Retrieves a list of shards that belong to the specified cluster. |
AddShard | Creates a new shard in the specified cluster. |
UpdateShard | Modifies the specified shard. |
DeleteShard | Deletes the specified shard. |
GetShardGroup | Returns the specified shard group. |
ListShardGroups | Retrieves a list of shard groups that belong to specified cluster. |
CreateShardGroup | Creates a new shard group in the specified cluster. |
UpdateShardGroup | Updates the specified shard group. |
DeleteShardGroup | Deletes the specified shard group. |
ListExternalDictionaries | Retrieves a list of external dictionaries that belong to specified cluster. |
CreateExternalDictionary | Creates an external dictionary for the specified ClickHouse cluster. |
UpdateExternalDictionary | Updates an external dictionary for the specified ClickHouse cluster. |
DeleteExternalDictionary | Deletes the specified external dictionary. |
Calls ClusterService
Get
Returns the specified ClickHouse cluster.
To get the list of available ClickHouse clusters, make a List request.
rpc Get (GetClusterRequest) returns (Cluster)
GetClusterRequest
Field | Description |
---|---|
cluster_id | string Required. ID of the ClickHouse Cluster resource to return. To get the cluster ID, use a ClusterService.List request. The maximum string length in characters is 50. |
Cluster
Field | Description |
---|---|
id | string ID of the ClickHouse cluster. This ID is assigned by MDB at creation time. |
folder_id | string ID of the folder that the ClickHouse cluster belongs to. |
created_at | google.protobuf.Timestamp Creation timestamp in RFC3339 |
name | string Name of the ClickHouse cluster. The name is unique within the folder. 1-63 characters long. |
description | string Description of the ClickHouse cluster. 0-256 characters long. |
labels | map<string,string> Custom labels for the ClickHouse cluster as key:value pairs. Maximum 64 per resource. |
environment | enum Environment Deployment environment of the ClickHouse cluster.
|
monitoring[] | Monitoring Description of monitoring systems relevant to the ClickHouse cluster. |
config | ClusterConfig Configuration of the ClickHouse cluster. |
network_id | string ID of the network that the cluster belongs to. |
health | enum Health Aggregated cluster health.
|
status | enum Status Current state of the cluster.
|
service_account_id | string ID of the service account used for access to Object Storage. |
maintenance_window | MaintenanceWindow Maintenance window for the cluster. |
planned_operation | MaintenanceOperation Planned maintenance operation to be started for the cluster within the nearest maintenance_window . |
security_group_ids[] | string User security groups |
deletion_protection | bool Deletion Protection inhibits deletion of the cluster |
Monitoring
Field | Description |
---|---|
name | string Name of the monitoring system. |
description | string Description of the monitoring system. |
link | string Link to the monitoring system charts for the ClickHouse cluster. |
ClusterConfig
Field | Description |
---|---|
version | string Version of the ClickHouse server software. |
clickhouse | Clickhouse Configuration and resource allocation for ClickHouse hosts. |
zookeeper | Zookeeper Configuration and resource allocation for ZooKeeper hosts. |
backup_window_start | google.type.TimeOfDay Time to start the daily backup, in the UTC timezone. |
access | Access Access policy for external services. |
cloud_storage | CloudStorage |
sql_database_management | google.protobuf.BoolValue Whether database management through SQL commands is enabled. |
sql_user_management | google.protobuf.BoolValue Whether user management through SQL commands is enabled. |
embedded_keeper | google.protobuf.BoolValue Whether cluster should use embedded Keeper instead of Zookeeper. |
Clickhouse
Field | Description |
---|---|
config | config.ClickhouseConfigSet Configuration settings of a ClickHouse server. |
resources | Resources Resources allocated to ClickHouse hosts. |
Zookeeper
Field | Description |
---|---|
resources | Resources Resources allocated to ZooKeeper hosts. |
Access
Field | Description |
---|---|
data_lens | bool Allow to export data from the cluster to DataLens. |
web_sql | bool Allow SQL queries to the cluster databases from the management console. See SQL queries in the management console for more details. |
metrika | bool Allow to import data from Yandex Metrica and AppMetrica to the cluster. See AppMetrica documentation |
serverless | bool Allow access to cluster for Serverless. |
data_transfer | bool Allow access for DataTransfer |
yandex_query | bool Allow access for Query |
CloudStorage
Field | Description |
---|---|
enabled | bool Whether to use Object Storage for storing ClickHouse data. |
move_factor | google.protobuf.DoubleValue Acceptable values are 0 to 1, inclusive. |
data_cache_enabled | google.protobuf.BoolValue |
data_cache_max_size | google.protobuf.Int64Value |
prefer_not_to_merge | google.protobuf.BoolValue |
ClickhouseConfigSet
Field | Description |
---|---|
effective_config | ClickhouseConfig Required. Effective settings for a ClickHouse cluster (a combination of settings defined in user_config and default_config ). |
user_config | ClickhouseConfig User-defined settings for a ClickHouse cluster. |
default_config | ClickhouseConfig Default configuration for a ClickHouse cluster. |
ClickhouseConfig
Field | Description |
---|---|
log_level | enum LogLevel Logging level for the ClickHouse cluster. Possible values: TRACE, DEBUG, INFORMATION, WARNING, ERROR. |
merge_tree | MergeTree Settings for the MergeTree engine. See description in ClickHouse documentation |
compression[] | Compression Compression settings for the ClickHouse cluster. See in-depth description in ClickHouse documentation |
dictionaries[] | ExternalDictionary Configuration of external dictionaries to be used by the ClickHouse cluster. See in-depth description in ClickHouse documentation |
graphite_rollup[] | GraphiteRollup Settings for thinning Graphite data. See in-depth description in ClickHouse documentation |
kafka | Kafka |
kafka_topics[] | KafkaTopic |
rabbitmq | Rabbitmq |
max_connections | google.protobuf.Int64Value Maximum number of inbound connections. The minimum value is 10. |
max_concurrent_queries | google.protobuf.Int64Value Maximum number of simultaneously processed requests. The minimum value is 10. |
keep_alive_timeout | google.protobuf.Int64Value Number of milliseconds that ClickHouse waits for incoming requests before closing the connection. |
uncompressed_cache_size | google.protobuf.Int64Value Cache size (in bytes) for uncompressed data used by MergeTree tables. |
mark_cache_size | google.protobuf.Int64Value Approximate size (in bytes) of the cache of "marks" used by MergeTree tables. Value must be greater than 0. |
max_table_size_to_drop | google.protobuf.Int64Value Maximum size of the table that can be deleted using a DROP query. |
max_partition_size_to_drop | google.protobuf.Int64Value Maximum size of the partition that can be deleted using a DROP query. |
builtin_dictionaries_reload_interval | google.protobuf.Int64Value The setting is deprecated and has no effect. |
timezone | string The server's time zone to be used in DateTime fields conversions. Specified as an IANA identifier. |
geobase_enabled | google.protobuf.BoolValue Enable or disable geobase. |
geobase_uri | string Address of the archive with the user geobase in Object Storage. |
query_log_retention_size | google.protobuf.Int64Value The maximum size that query_log can grow to before old data will be removed. If set to 0, automatic removal of query_log data based on size is disabled. |
query_log_retention_time | google.protobuf.Int64Value The maximum time that query_log records will be retained before removal. If set to 0, automatic removal of query_log data based on time is disabled. |
query_thread_log_enabled | google.protobuf.BoolValue Whether query_thread_log system table is enabled. |
query_thread_log_retention_size | google.protobuf.Int64Value The maximum size that query_thread_log can grow to before old data will be removed. If set to 0, automatic removal of query_thread_log data based on size is disabled. |
query_thread_log_retention_time | google.protobuf.Int64Value The maximum time that query_thread_log records will be retained before removal. If set to 0, automatic removal of query_thread_log data based on time is disabled. |
part_log_retention_size | google.protobuf.Int64Value The maximum size that part_log can grow to before old data will be removed. If set to 0, automatic removal of part_log data based on size is disabled. |
part_log_retention_time | google.protobuf.Int64Value The maximum time that part_log records will be retained before removal. If set to 0, automatic removal of part_log data based on time is disabled. |
metric_log_enabled | google.protobuf.BoolValue Whether metric_log system table is enabled. |
metric_log_retention_size | google.protobuf.Int64Value The maximum size that metric_log can grow to before old data will be removed. If set to 0, automatic removal of metric_log data based on size is disabled. |
metric_log_retention_time | google.protobuf.Int64Value The maximum time that metric_log records will be retained before removal. If set to 0, automatic removal of metric_log data based on time is disabled. |
trace_log_enabled | google.protobuf.BoolValue Whether trace_log system table is enabled. |
trace_log_retention_size | google.protobuf.Int64Value The maximum size that trace_log can grow to before old data will be removed. If set to 0, automatic removal of trace_log data based on size is disabled. |
trace_log_retention_time | google.protobuf.Int64Value The maximum time that trace_log records will be retained before removal. If set to 0, automatic removal of trace_log data based on time is disabled. |
text_log_enabled | google.protobuf.BoolValue Whether text_log system table is enabled. |
text_log_retention_size | google.protobuf.Int64Value The maximum size that text_log can grow to before old data will be removed. If set to 0, automatic removal of text_log data based on size is disabled. |
text_log_retention_time | google.protobuf.Int64Value The maximum time that text_log records will be retained before removal. If set to 0, automatic removal of text_log data based on time is disabled. |
text_log_level | enum LogLevel Logging level for text_log system table. Possible values: TRACE, DEBUG, INFORMATION, WARNING, ERROR. |
opentelemetry_span_log_enabled | google.protobuf.BoolValue Enable or disable opentelemetry_span_log system table. Default value: false. |
opentelemetry_span_log_retention_size | google.protobuf.Int64Value The maximum size that opentelemetry_span_log can grow to before old data will be removed. If set to 0 (default), automatic removal of opentelemetry_span_log data based on size is disabled. The minimum value is 0. |
opentelemetry_span_log_retention_time | google.protobuf.Int64Value The maximum time that opentelemetry_span_log records will be retained before removal. If set to 0, automatic removal of opentelemetry_span_log data based on time is disabled. The minimum value is 0. |
query_views_log_enabled | google.protobuf.BoolValue Enable or disable query_views_log system table. Default value: false. |
query_views_log_retention_size | google.protobuf.Int64Value The maximum size that query_views_log can grow to before old data will be removed. If set to 0 (default), automatic removal of query_views_log data based on size is disabled. The minimum value is 0. |
query_views_log_retention_time | google.protobuf.Int64Value The maximum time that query_views_log records will be retained before removal. If set to 0, automatic removal of query_views_log data based on time is disabled. The minimum value is 0. |
asynchronous_metric_log_enabled | google.protobuf.BoolValue Enable or disable asynchronous_metric_log system table. Default value: false. |
asynchronous_metric_log_retention_size | google.protobuf.Int64Value The maximum size that asynchronous_metric_log can grow to before old data will be removed. If set to 0 (default), automatic removal of asynchronous_metric_log data based on size is disabled. The minimum value is 0. |
asynchronous_metric_log_retention_time | google.protobuf.Int64Value The maximum time that asynchronous_metric_log records will be retained before removal. If set to 0, automatic removal of asynchronous_metric_log data based on time is disabled. The minimum value is 0. |
session_log_enabled | google.protobuf.BoolValue Enable or disable session_log system table. Default value: false. |
session_log_retention_size | google.protobuf.Int64Value The maximum size that session_log can grow to before old data will be removed. If set to 0 (default), automatic removal of session_log data based on size is disabled. The minimum value is 0. |
session_log_retention_time | google.protobuf.Int64Value The maximum time that session_log records will be retained before removal. If set to 0, automatic removal of session_log data based on time is disabled. The minimum value is 0. |
zookeeper_log_enabled | google.protobuf.BoolValue Enable or disable zookeeper_log system table. Default value: false. |
zookeeper_log_retention_size | google.protobuf.Int64Value The maximum size that zookeeper_log can grow to before old data will be removed. If set to 0 (default), automatic removal of zookeeper_log data based on size is disabled. The minimum value is 0. |
zookeeper_log_retention_time | google.protobuf.Int64Value The maximum time that zookeeper_log records will be retained before removal. If set to 0, automatic removal of zookeeper_log data based on time is disabled. The minimum value is 0. |
asynchronous_insert_log_enabled | google.protobuf.BoolValue Enable or disable asynchronous_insert_log system table. Default value: false. Minimal required ClickHouse version: 22.10. |
asynchronous_insert_log_retention_size | google.protobuf.Int64Value The maximum size that asynchronous_insert_log can grow to before old data will be removed. If set to 0 (default), automatic removal of asynchronous_insert_log data based on size is disabled. The minimum value is 0. |
asynchronous_insert_log_retention_time | google.protobuf.Int64Value The maximum time that asynchronous_insert_log records will be retained before removal. If set to 0, automatic removal of asynchronous_insert_log data based on time is disabled. The minimum value is 0. |
background_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_merges_mutations_concurrency_ratio | google.protobuf.Int64Value Sets a ratio between the number of threads and the number of background merges and mutations that can be executed concurrently. For example, if the ratio equals to 2 and background_pool_size is set to 16 then ClickHouse can execute 32 background merges concurrently. This is possible, because background operations could be suspended and postponed. This is needed to give small merges more execution priority. You can only increase this ratio at runtime. To lower it you have to restart the server. The same as for background_pool_size setting background_merges_mutations_concurrency_ratio could be applied from the default profile for backward compatibility. Default: 2 See in-depth description in ClickHouse documentation |
background_schedule_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_fetches_pool_size | google.protobuf.Int64Value Sets the number of threads performing background fetches for tables with ReplicatedMergeTree engines. Default value: 8. More info see in ClickHouse documentation |
background_move_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_distributed_schedule_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_buffer_flush_schedule_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_message_broker_schedule_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_common_pool_size | google.protobuf.Int64Value The maximum number of threads that will be used for performing a variety of operations (mostly garbage collection) for *MergeTree-engine tables in a background. Default: 8 See in-depth description in ClickHouse documentation |
default_database | google.protobuf.StringValue The default database. To get a list of cluster databases, see Yandex Managed ClickHouse documentation. |
total_memory_profiler_step | google.protobuf.Int64Value Sets the memory size (in bytes) for a stack trace at every peak allocation step. Default value: 4194304. More info see in ClickHouse documentation |
total_memory_tracker_sample_probability | google.protobuf.DoubleValue |
MergeTree
Field | Description |
---|---|
replicated_deduplication_window | google.protobuf.Int64Value Number of blocks of hashes to keep in ZooKeeper. |
replicated_deduplication_window_seconds | google.protobuf.Int64Value Period of time to keep blocks of hashes for. |
parts_to_delay_insert | google.protobuf.Int64Value If table contains at least that many active parts in single partition, artificially slow down insert into table. |
parts_to_throw_insert | google.protobuf.Int64Value If more than this number active parts in single partition, throw 'Too many parts ...' exception. |
inactive_parts_to_delay_insert | google.protobuf.Int64Value |
inactive_parts_to_throw_insert | google.protobuf.Int64Value |
max_replicated_merges_in_queue | google.protobuf.Int64Value How many tasks of merging and mutating parts are allowed simultaneously in ReplicatedMergeTree queue. |
number_of_free_entries_in_pool_to_lower_max_size_of_merge | google.protobuf.Int64Value If there is less than specified number of free entries in background pool (or replicated queue), start to lower maximum size of merge to process. |
max_bytes_to_merge_at_min_space_in_pool | google.protobuf.Int64Value Maximum in total size of parts to merge, when there are minimum free threads in background pool (or entries in replication queue). |
max_bytes_to_merge_at_max_space_in_pool | google.protobuf.Int64Value |
min_bytes_for_wide_part | google.protobuf.Int64Value Minimum number of bytes in a data part that can be stored in Wide format. More info see in ClickHouse documentation |
min_rows_for_wide_part | google.protobuf.Int64Value Minimum number of rows in a data part that can be stored in Wide format. More info see in ClickHouse documentation |
ttl_only_drop_parts | google.protobuf.BoolValue Enables or disables complete dropping of data parts where all rows are expired in MergeTree tables. More info see in ClickHouse documentation |
allow_remote_fs_zero_copy_replication | google.protobuf.BoolValue |
merge_with_ttl_timeout | google.protobuf.Int64Value |
merge_with_recompression_ttl_timeout | google.protobuf.Int64Value |
max_parts_in_total | google.protobuf.Int64Value |
max_number_of_merges_with_ttl_in_pool | google.protobuf.Int64Value |
cleanup_delay_period | google.protobuf.Int64Value |
number_of_free_entries_in_pool_to_execute_mutation | google.protobuf.Int64Value |
max_avg_part_size_for_too_many_parts | google.protobuf.Int64Value The 'too many parts' check according to 'parts_to_delay_insert' and 'parts_to_throw_insert' will be active only if the average part size (in the relevant partition) is not larger than the specified threshold. If it is larger than the specified threshold, the INSERTs will be neither delayed or rejected. This allows to have hundreds of terabytes in a single table on a single server if the parts are successfully merged to larger parts. This does not affect the thresholds on inactive parts or total parts. Default: 1 GiB Min version: 22.10 See in-depth description in ClickHouse GitHub |
min_age_to_force_merge_seconds | google.protobuf.Int64Value Merge parts if every part in the range is older than the value of min_age_to_force_merge_seconds. Default: 0 - disabled Min_version: 22.10 See in-depth description in ClickHouse documentation |
min_age_to_force_merge_on_partition_only | google.protobuf.BoolValue Whether min_age_to_force_merge_seconds should be applied only on the entire partition and not on subset. Default: false Min_version: 22.11 See in-depth description in ClickHouse documentation |
merge_selecting_sleep_ms | google.protobuf.Int64Value Sleep time for merge selecting when no part is selected. A lower setting triggers selecting tasks in background_schedule_pool frequently, which results in a large number of requests to ClickHouse Keeper in large-scale clusters. Default: 5000 Min_version: 21.10 See in-depth description in ClickHouse documentation |
Kafka
Field | Description |
---|---|
security_protocol | enum SecurityProtocol |
sasl_mechanism | enum SaslMechanism |
sasl_username | string |
sasl_password | string |
enable_ssl_certificate_verification | google.protobuf.BoolValue |
max_poll_interval_ms | google.protobuf.Int64Value The minimum value is 0. |
session_timeout_ms | google.protobuf.Int64Value The minimum value is 0. |
KafkaTopic
Field | Description |
---|---|
name | string Required. |
settings | Kafka Required. |
Rabbitmq
Field | Description |
---|---|
username | string RabbitMQ |
password | string RabbitMQ |
vhost | string RabbitMQ |
Compression
Field | Description |
---|---|
method | enum Method Compression method to use for the specified combination of min_part_size and min_part_size_ratio . |
min_part_size | int64 Minimum size of a part of a table. The minimum value is 1. |
min_part_size_ratio | double Minimum ratio of a part relative to the size of all the data in the table. |
level | google.protobuf.Int64Value The minimum value is 0. |
ExternalDictionary
Field | Description |
---|---|
name | string Required. Name of the external dictionary. |
structure | Structure Required. Set of attributes for the external dictionary. For in-depth description, see ClickHouse documentation |
layout | Layout Required. Layout for storing the dictionary in memory. For in-depth description, see ClickHouse documentation |
lifetime | oneof: fixed_lifetime or lifetime_range Setting for the period of time between dictionary updates. For details, see ClickHouse documentation |
fixed_lifetime | int64 Fixed interval between dictionary updates. |
lifetime_range | Range Range of intervals between dictionary updates for ClickHouse to choose from. |
source | oneof: http_source , mysql_source , clickhouse_source , mongodb_source or postgresql_source Description of the source for the external dictionary. |
http_source | HttpSource HTTP source for the dictionary. |
mysql_source | MysqlSource MySQL source for the dictionary. |
clickhouse_source | ClickhouseSource ClickHouse source for the dictionary. |
mongodb_source | MongodbSource MongoDB source for the dictionary. |
postgresql_source | PostgresqlSource PostgreSQL source for the dictionary. |
HttpSource
Field | Description |
---|---|
url | string Required. URL of the source dictionary available over HTTP. |
format | string Required. The data format. Valid values are all formats supported by ClickHouse SQL dialect. |
MysqlSource
Field | Description |
---|---|
db | string Required. Name of the MySQL database to connect to. |
table | string Required. Name of the database table to use as a ClickHouse dictionary. |
port | int64 Default port to use when connecting to a replica of the dictionary source. Acceptable values are 0 to 65535, inclusive. |
user | string Name of the default user for replicas of the dictionary source. |
password | string Password of the default user for replicas of the dictionary source. |
replicas[] | Replica List of MySQL replicas of the database used as dictionary source. The number of elements must be greater than 0. |
where | string Selection criteria for the data in the specified MySQL table. |
invalidate_query | string Query for checking the dictionary status, to pull only updated data. For more details, see ClickHouse documentation on dictionaries |
Replica
Field | Description |
---|---|
host | string Required. MySQL host of the replica. The maximum string length in characters is 253. |
priority | int64 Required. The priority of the replica that ClickHouse takes into account when connecting. Replica with the highest priority should have this field set to the lowest number. Value must be greater than 0. |
port | int64 Port to use when connecting to the replica. If a port is not specified for a replica, ClickHouse uses the port specified for the source. Acceptable values are 0 to 65535, inclusive. |
user | string Name of the MySQL database user. |
password | string Password of the MySQL database user. |
ClickhouseSource
Field | Description |
---|---|
db | string Required. Name of the ClickHouse database. |
table | string Required. Name of the table in the specified database to be used as the dictionary source. |
host | string Required. ClickHouse host of the specified database. The maximum string length in characters is 253. |
port | int64 Port to use when connecting to the host. Acceptable values are 0 to 65535, inclusive. |
user | string Required. Name of the ClickHouse database user. |
password | string Password of the ClickHouse database user. |
where | string Selection criteria for the data in the specified ClickHouse table. |
MongodbSource
Field | Description |
---|---|
db | string Required. Name of the MongoDB database. |
collection | string Required. Name of the collection in the specified database to be used as the dictionary source. |
host | string Required. MongoDB host of the specified database. The maximum string length in characters is 253. |
port | int64 Port to use when connecting to the host. Acceptable values are 0 to 65535, inclusive. |
user | string Required. Name of the MongoDB database user. |
password | string Password of the MongoDB database user. |
options | string |
PostgresqlSource
Field | Description |
---|---|
db | string Required. Name of the PostrgreSQL database. |
table | string Required. Name of the table in the specified database to be used as the dictionary source. |
hosts[] | string Name of the PostrgreSQL host The number of elements must be greater than 0. |
port | int64 Port to use when connecting to the host. Acceptable values are 0 to 65535, inclusive. |
user | string Required. Name of the PostrgreSQL database user. |
password | string Password of the PostrgreSQL database user. |
invalidate_query | string Query for checking the dictionary status, to pull only updated data. For more details, see ClickHouse documentation on dictionaries |
ssl_mode | enum SslMode Mode of SSL TCP/IP connection to the PostgreSQL host. For more details, see PostgreSQL documentation
|
Structure
Field | Description |
---|---|
id | Id Single numeric key column for the dictionary. |
key | Key Composite key for the dictionary, containing of one or more key columns. For details, see ClickHouse documentation |
range_min | Attribute Field holding the beginning of the range for dictionaries with RANGE_HASHED layout. For details, see ClickHouse documentation |
range_max | Attribute Field holding the end of the range for dictionaries with RANGE_HASHED layout. For details, see ClickHouse documentation |
attributes[] | Attribute Description of the fields available for database queries. For details, see ClickHouse documentation |
Attribute
Field | Description |
---|---|
name | string Required. Name of the column. |
type | string Required. Type of the column. |
null_value | string Default value for an element without data (for example, an empty string). |
expression | string Expression, describing the attribute, if applicable. |
hierarchical | bool Indication of hierarchy support. Default value: false . |
injective | bool Indication of injective mapping "id -> attribute". Default value: false . |
Id
Field | Description |
---|---|
name | string Required. Name of the numeric key. |
Key
Field | Description |
---|---|
attributes[] | Attribute Attributes of a complex key. The number of elements must be greater than 0. |
Layout
Field | Description |
---|---|
type | enum Type Required. Layout type for an external dictionary.
|
size_in_cells | int64 Number of cells in the cache. Rounded up to a power of two. Applicable only for CACHE and COMPLEX_KEY_CACHE layout types. |
Range
Field | Description |
---|---|
min | int64 Minimum dictionary lifetime. |
max | int64 Maximum dictionary lifetime. |
GraphiteRollup
Field | Description |
---|---|
name | string Required. Name for the specified combination of settings for Graphite rollup. |
patterns[] | Pattern Pattern to use for the rollup. The number of elements must be greater than 0. |
Pattern
Field | Description |
---|---|
regexp | string Pattern for metric names. |
function | string Required. Name of the aggregating function to apply to data of the age specified in retention . |
retention[] | Retention Age of data to use for thinning. The number of elements must be greater than 0. |
Retention
Field | Description |
---|---|
age | int64 Minimum age of the data in seconds. The minimum value is 0. |
precision | int64 Precision of determining the age of the data, in seconds. Value must be greater than 0. |
Resources
Field | Description |
---|---|
resource_preset_id | string ID of the preset for computational resources available to a host (CPU, memory etc.). All available presets are listed in the documentation |
disk_size | int64 Volume of the storage available to a host, in bytes. |
disk_type_id | string Type of the storage environment for the host. Possible values:
|
MaintenanceWindow
Field | Description |
---|---|
policy | oneof: anytime or weekly_maintenance_window The maintenance policy in effect. |
anytime | AnytimeMaintenanceWindow Maintenance operation can be scheduled anytime. |
weekly_maintenance_window | WeeklyMaintenanceWindow Maintenance operation can be scheduled on a weekly basis. |
AnytimeMaintenanceWindow
Empty.
WeeklyMaintenanceWindow
Field | Description |
---|---|
day | enum WeekDay Day of the week (in DDD format). |
hour | int64 Hour of the day in UTC (in HH format). Acceptable values are 1 to 24, inclusive. |
MaintenanceOperation
Field | Description |
---|---|
info | string Information about this maintenance operation. The maximum string length in characters is 256. |
delayed_until | google.protobuf.Timestamp Time until which this maintenance operation is delayed. |
List
Retrieves a list of ClickHouse clusters that belong to the specified folder.
rpc List (ListClustersRequest) returns (ListClustersResponse)
ListClustersRequest
Field | Description |
---|---|
folder_id | string Required. ID of the folder to list ClickHouse clusters in. To get the folder ID, use a yandex.cloud.resourcemanager.v1.FolderService.List request. The maximum string length in characters is 50. |
page_size | int64 The maximum number of results per page to return. If the number of available results is larger than page_size , the service returns a ListClustersResponse.next_page_token that can be used to get the next page of results in subsequent list requests. The maximum value is 1000. |
page_token | string Page token. To get the next page of results, set page_token to the ListClustersResponse.next_page_token returned by the previous list request. The maximum string length in characters is 100. |
filter | string A filter expression that filters resources listed in the response. The expression must specify:
|
ListClustersResponse
Field | Description |
---|---|
clusters[] | Cluster List of ClickHouse Cluster resources. |
next_page_token | string This token allows you to get the next page of results for list requests. If the number of results is larger than ListClustersRequest.page_size, use the next_page_token as the value for the ListClustersRequest.page_token parameter in the next list request. Each subsequent list request will have its own next_page_token to continue paging through the results. |
Cluster
Field | Description |
---|---|
id | string ID of the ClickHouse cluster. This ID is assigned by MDB at creation time. |
folder_id | string ID of the folder that the ClickHouse cluster belongs to. |
created_at | google.protobuf.Timestamp Creation timestamp in RFC3339 |
name | string Name of the ClickHouse cluster. The name is unique within the folder. 1-63 characters long. |
description | string Description of the ClickHouse cluster. 0-256 characters long. |
labels | map<string,string> Custom labels for the ClickHouse cluster as key:value pairs. Maximum 64 per resource. |
environment | enum Environment Deployment environment of the ClickHouse cluster.
|
monitoring[] | Monitoring Description of monitoring systems relevant to the ClickHouse cluster. |
config | ClusterConfig Configuration of the ClickHouse cluster. |
network_id | string ID of the network that the cluster belongs to. |
health | enum Health Aggregated cluster health.
|
status | enum Status Current state of the cluster.
|
service_account_id | string ID of the service account used for access to Object Storage. |
maintenance_window | MaintenanceWindow Maintenance window for the cluster. |
planned_operation | MaintenanceOperation Planned maintenance operation to be started for the cluster within the nearest maintenance_window . |
security_group_ids[] | string User security groups |
deletion_protection | bool Deletion Protection inhibits deletion of the cluster |
Monitoring
Field | Description |
---|---|
name | string Name of the monitoring system. |
description | string Description of the monitoring system. |
link | string Link to the monitoring system charts for the ClickHouse cluster. |
ClusterConfig
Field | Description |
---|---|
version | string Version of the ClickHouse server software. |
clickhouse | Clickhouse Configuration and resource allocation for ClickHouse hosts. |
zookeeper | Zookeeper Configuration and resource allocation for ZooKeeper hosts. |
backup_window_start | google.type.TimeOfDay Time to start the daily backup, in the UTC timezone. |
access | Access Access policy for external services. |
cloud_storage | CloudStorage |
sql_database_management | google.protobuf.BoolValue Whether database management through SQL commands is enabled. |
sql_user_management | google.protobuf.BoolValue Whether user management through SQL commands is enabled. |
embedded_keeper | google.protobuf.BoolValue Whether cluster should use embedded Keeper instead of Zookeeper. |
Clickhouse
Field | Description |
---|---|
config | config.ClickhouseConfigSet Configuration settings of a ClickHouse server. |
resources | Resources Resources allocated to ClickHouse hosts. |
Zookeeper
Field | Description |
---|---|
resources | Resources Resources allocated to ZooKeeper hosts. |
Access
Field | Description |
---|---|
data_lens | bool Allow to export data from the cluster to DataLens. |
web_sql | bool Allow SQL queries to the cluster databases from the management console. See SQL queries in the management console for more details. |
metrika | bool Allow to import data from Yandex Metrica and AppMetrica to the cluster. See AppMetrica documentation |
serverless | bool Allow access to cluster for Serverless. |
data_transfer | bool Allow access for DataTransfer |
yandex_query | bool Allow access for Query |
CloudStorage
Field | Description |
---|---|
enabled | bool Whether to use Object Storage for storing ClickHouse data. |
move_factor | google.protobuf.DoubleValue Acceptable values are 0 to 1, inclusive. |
data_cache_enabled | google.protobuf.BoolValue |
data_cache_max_size | google.protobuf.Int64Value |
prefer_not_to_merge | google.protobuf.BoolValue |
ClickhouseConfigSet
Field | Description |
---|---|
effective_config | ClickhouseConfig Required. Effective settings for a ClickHouse cluster (a combination of settings defined in user_config and default_config ). |
user_config | ClickhouseConfig User-defined settings for a ClickHouse cluster. |
default_config | ClickhouseConfig Default configuration for a ClickHouse cluster. |
ClickhouseConfig
Field | Description |
---|---|
log_level | enum LogLevel Logging level for the ClickHouse cluster. Possible values: TRACE, DEBUG, INFORMATION, WARNING, ERROR. |
merge_tree | MergeTree Settings for the MergeTree engine. See description in ClickHouse documentation |
compression[] | Compression Compression settings for the ClickHouse cluster. See in-depth description in ClickHouse documentation |
dictionaries[] | ExternalDictionary Configuration of external dictionaries to be used by the ClickHouse cluster. See in-depth description in ClickHouse documentation |
graphite_rollup[] | GraphiteRollup Settings for thinning Graphite data. See in-depth description in ClickHouse documentation |
kafka | Kafka |
kafka_topics[] | KafkaTopic |
rabbitmq | Rabbitmq |
max_connections | google.protobuf.Int64Value Maximum number of inbound connections. The minimum value is 10. |
max_concurrent_queries | google.protobuf.Int64Value Maximum number of simultaneously processed requests. The minimum value is 10. |
keep_alive_timeout | google.protobuf.Int64Value Number of milliseconds that ClickHouse waits for incoming requests before closing the connection. |
uncompressed_cache_size | google.protobuf.Int64Value Cache size (in bytes) for uncompressed data used by MergeTree tables. |
mark_cache_size | google.protobuf.Int64Value Approximate size (in bytes) of the cache of "marks" used by MergeTree tables. Value must be greater than 0. |
max_table_size_to_drop | google.protobuf.Int64Value Maximum size of the table that can be deleted using a DROP query. |
max_partition_size_to_drop | google.protobuf.Int64Value Maximum size of the partition that can be deleted using a DROP query. |
builtin_dictionaries_reload_interval | google.protobuf.Int64Value The setting is deprecated and has no effect. |
timezone | string The server's time zone to be used in DateTime fields conversions. Specified as an IANA identifier. |
geobase_enabled | google.protobuf.BoolValue Enable or disable geobase. |
geobase_uri | string Address of the archive with the user geobase in Object Storage. |
query_log_retention_size | google.protobuf.Int64Value The maximum size that query_log can grow to before old data will be removed. If set to 0, automatic removal of query_log data based on size is disabled. |
query_log_retention_time | google.protobuf.Int64Value The maximum time that query_log records will be retained before removal. If set to 0, automatic removal of query_log data based on time is disabled. |
query_thread_log_enabled | google.protobuf.BoolValue Whether query_thread_log system table is enabled. |
query_thread_log_retention_size | google.protobuf.Int64Value The maximum size that query_thread_log can grow to before old data will be removed. If set to 0, automatic removal of query_thread_log data based on size is disabled. |
query_thread_log_retention_time | google.protobuf.Int64Value The maximum time that query_thread_log records will be retained before removal. If set to 0, automatic removal of query_thread_log data based on time is disabled. |
part_log_retention_size | google.protobuf.Int64Value The maximum size that part_log can grow to before old data will be removed. If set to 0, automatic removal of part_log data based on size is disabled. |
part_log_retention_time | google.protobuf.Int64Value The maximum time that part_log records will be retained before removal. If set to 0, automatic removal of part_log data based on time is disabled. |
metric_log_enabled | google.protobuf.BoolValue Whether metric_log system table is enabled. |
metric_log_retention_size | google.protobuf.Int64Value The maximum size that metric_log can grow to before old data will be removed. If set to 0, automatic removal of metric_log data based on size is disabled. |
metric_log_retention_time | google.protobuf.Int64Value The maximum time that metric_log records will be retained before removal. If set to 0, automatic removal of metric_log data based on time is disabled. |
trace_log_enabled | google.protobuf.BoolValue Whether trace_log system table is enabled. |
trace_log_retention_size | google.protobuf.Int64Value The maximum size that trace_log can grow to before old data will be removed. If set to 0, automatic removal of trace_log data based on size is disabled. |
trace_log_retention_time | google.protobuf.Int64Value The maximum time that trace_log records will be retained before removal. If set to 0, automatic removal of trace_log data based on time is disabled. |
text_log_enabled | google.protobuf.BoolValue Whether text_log system table is enabled. |
text_log_retention_size | google.protobuf.Int64Value The maximum size that text_log can grow to before old data will be removed. If set to 0, automatic removal of text_log data based on size is disabled. |
text_log_retention_time | google.protobuf.Int64Value The maximum time that text_log records will be retained before removal. If set to 0, automatic removal of text_log data based on time is disabled. |
text_log_level | enum LogLevel Logging level for text_log system table. Possible values: TRACE, DEBUG, INFORMATION, WARNING, ERROR. |
opentelemetry_span_log_enabled | google.protobuf.BoolValue Enable or disable opentelemetry_span_log system table. Default value: false. |
opentelemetry_span_log_retention_size | google.protobuf.Int64Value The maximum size that opentelemetry_span_log can grow to before old data will be removed. If set to 0 (default), automatic removal of opentelemetry_span_log data based on size is disabled. The minimum value is 0. |
opentelemetry_span_log_retention_time | google.protobuf.Int64Value The maximum time that opentelemetry_span_log records will be retained before removal. If set to 0, automatic removal of opentelemetry_span_log data based on time is disabled. The minimum value is 0. |
query_views_log_enabled | google.protobuf.BoolValue Enable or disable query_views_log system table. Default value: false. |
query_views_log_retention_size | google.protobuf.Int64Value The maximum size that query_views_log can grow to before old data will be removed. If set to 0 (default), automatic removal of query_views_log data based on size is disabled. The minimum value is 0. |
query_views_log_retention_time | google.protobuf.Int64Value The maximum time that query_views_log records will be retained before removal. If set to 0, automatic removal of query_views_log data based on time is disabled. The minimum value is 0. |
asynchronous_metric_log_enabled | google.protobuf.BoolValue Enable or disable asynchronous_metric_log system table. Default value: false. |
asynchronous_metric_log_retention_size | google.protobuf.Int64Value The maximum size that asynchronous_metric_log can grow to before old data will be removed. If set to 0 (default), automatic removal of asynchronous_metric_log data based on size is disabled. The minimum value is 0. |
asynchronous_metric_log_retention_time | google.protobuf.Int64Value The maximum time that asynchronous_metric_log records will be retained before removal. If set to 0, automatic removal of asynchronous_metric_log data based on time is disabled. The minimum value is 0. |
session_log_enabled | google.protobuf.BoolValue Enable or disable session_log system table. Default value: false. |
session_log_retention_size | google.protobuf.Int64Value The maximum size that session_log can grow to before old data will be removed. If set to 0 (default), automatic removal of session_log data based on size is disabled. The minimum value is 0. |
session_log_retention_time | google.protobuf.Int64Value The maximum time that session_log records will be retained before removal. If set to 0, automatic removal of session_log data based on time is disabled. The minimum value is 0. |
zookeeper_log_enabled | google.protobuf.BoolValue Enable or disable zookeeper_log system table. Default value: false. |
zookeeper_log_retention_size | google.protobuf.Int64Value The maximum size that zookeeper_log can grow to before old data will be removed. If set to 0 (default), automatic removal of zookeeper_log data based on size is disabled. The minimum value is 0. |
zookeeper_log_retention_time | google.protobuf.Int64Value The maximum time that zookeeper_log records will be retained before removal. If set to 0, automatic removal of zookeeper_log data based on time is disabled. The minimum value is 0. |
asynchronous_insert_log_enabled | google.protobuf.BoolValue Enable or disable asynchronous_insert_log system table. Default value: false. Minimal required ClickHouse version: 22.10. |
asynchronous_insert_log_retention_size | google.protobuf.Int64Value The maximum size that asynchronous_insert_log can grow to before old data will be removed. If set to 0 (default), automatic removal of asynchronous_insert_log data based on size is disabled. The minimum value is 0. |
asynchronous_insert_log_retention_time | google.protobuf.Int64Value The maximum time that asynchronous_insert_log records will be retained before removal. If set to 0, automatic removal of asynchronous_insert_log data based on time is disabled. The minimum value is 0. |
background_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_merges_mutations_concurrency_ratio | google.protobuf.Int64Value Sets a ratio between the number of threads and the number of background merges and mutations that can be executed concurrently. For example, if the ratio equals to 2 and background_pool_size is set to 16 then ClickHouse can execute 32 background merges concurrently. This is possible, because background operations could be suspended and postponed. This is needed to give small merges more execution priority. You can only increase this ratio at runtime. To lower it you have to restart the server. The same as for background_pool_size setting background_merges_mutations_concurrency_ratio could be applied from the default profile for backward compatibility. Default: 2 See in-depth description in ClickHouse documentation |
background_schedule_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_fetches_pool_size | google.protobuf.Int64Value Sets the number of threads performing background fetches for tables with ReplicatedMergeTree engines. Default value: 8. More info see in ClickHouse documentation |
background_move_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_distributed_schedule_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_buffer_flush_schedule_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_message_broker_schedule_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_common_pool_size | google.protobuf.Int64Value The maximum number of threads that will be used for performing a variety of operations (mostly garbage collection) for *MergeTree-engine tables in a background. Default: 8 See in-depth description in ClickHouse documentation |
default_database | google.protobuf.StringValue The default database. To get a list of cluster databases, see Yandex Managed ClickHouse documentation. |
total_memory_profiler_step | google.protobuf.Int64Value Sets the memory size (in bytes) for a stack trace at every peak allocation step. Default value: 4194304. More info see in ClickHouse documentation |
total_memory_tracker_sample_probability | google.protobuf.DoubleValue |
MergeTree
Field | Description |
---|---|
replicated_deduplication_window | google.protobuf.Int64Value Number of blocks of hashes to keep in ZooKeeper. |
replicated_deduplication_window_seconds | google.protobuf.Int64Value Period of time to keep blocks of hashes for. |
parts_to_delay_insert | google.protobuf.Int64Value If table contains at least that many active parts in single partition, artificially slow down insert into table. |
parts_to_throw_insert | google.protobuf.Int64Value If more than this number active parts in single partition, throw 'Too many parts ...' exception. |
inactive_parts_to_delay_insert | google.protobuf.Int64Value |
inactive_parts_to_throw_insert | google.protobuf.Int64Value |
max_replicated_merges_in_queue | google.protobuf.Int64Value How many tasks of merging and mutating parts are allowed simultaneously in ReplicatedMergeTree queue. |
number_of_free_entries_in_pool_to_lower_max_size_of_merge | google.protobuf.Int64Value If there is less than specified number of free entries in background pool (or replicated queue), start to lower maximum size of merge to process. |
max_bytes_to_merge_at_min_space_in_pool | google.protobuf.Int64Value Maximum in total size of parts to merge, when there are minimum free threads in background pool (or entries in replication queue). |
max_bytes_to_merge_at_max_space_in_pool | google.protobuf.Int64Value |
min_bytes_for_wide_part | google.protobuf.Int64Value Minimum number of bytes in a data part that can be stored in Wide format. More info see in ClickHouse documentation |
min_rows_for_wide_part | google.protobuf.Int64Value Minimum number of rows in a data part that can be stored in Wide format. More info see in ClickHouse documentation |
ttl_only_drop_parts | google.protobuf.BoolValue Enables or disables complete dropping of data parts where all rows are expired in MergeTree tables. More info see in ClickHouse documentation |
allow_remote_fs_zero_copy_replication | google.protobuf.BoolValue |
merge_with_ttl_timeout | google.protobuf.Int64Value |
merge_with_recompression_ttl_timeout | google.protobuf.Int64Value |
max_parts_in_total | google.protobuf.Int64Value |
max_number_of_merges_with_ttl_in_pool | google.protobuf.Int64Value |
cleanup_delay_period | google.protobuf.Int64Value |
number_of_free_entries_in_pool_to_execute_mutation | google.protobuf.Int64Value |
max_avg_part_size_for_too_many_parts | google.protobuf.Int64Value The 'too many parts' check according to 'parts_to_delay_insert' and 'parts_to_throw_insert' will be active only if the average part size (in the relevant partition) is not larger than the specified threshold. If it is larger than the specified threshold, the INSERTs will be neither delayed or rejected. This allows to have hundreds of terabytes in a single table on a single server if the parts are successfully merged to larger parts. This does not affect the thresholds on inactive parts or total parts. Default: 1 GiB Min version: 22.10 See in-depth description in ClickHouse GitHub |
min_age_to_force_merge_seconds | google.protobuf.Int64Value Merge parts if every part in the range is older than the value of min_age_to_force_merge_seconds. Default: 0 - disabled Min_version: 22.10 See in-depth description in ClickHouse documentation |
min_age_to_force_merge_on_partition_only | google.protobuf.BoolValue Whether min_age_to_force_merge_seconds should be applied only on the entire partition and not on subset. Default: false Min_version: 22.11 See in-depth description in ClickHouse documentation |
merge_selecting_sleep_ms | google.protobuf.Int64Value Sleep time for merge selecting when no part is selected. A lower setting triggers selecting tasks in background_schedule_pool frequently, which results in a large number of requests to ClickHouse Keeper in large-scale clusters. Default: 5000 Min_version: 21.10 See in-depth description in ClickHouse documentation |
Kafka
Field | Description |
---|---|
security_protocol | enum SecurityProtocol |
sasl_mechanism | enum SaslMechanism |
sasl_username | string |
sasl_password | string |
enable_ssl_certificate_verification | google.protobuf.BoolValue |
max_poll_interval_ms | google.protobuf.Int64Value The minimum value is 0. |
session_timeout_ms | google.protobuf.Int64Value The minimum value is 0. |
KafkaTopic
Field | Description |
---|---|
name | string Required. |
settings | Kafka Required. |
Rabbitmq
Field | Description |
---|---|
username | string RabbitMQ |
password | string RabbitMQ |
vhost | string RabbitMQ |
Compression
Field | Description |
---|---|
method | enum Method Compression method to use for the specified combination of min_part_size and min_part_size_ratio . |
min_part_size | int64 Minimum size of a part of a table. The minimum value is 1. |
min_part_size_ratio | double Minimum ratio of a part relative to the size of all the data in the table. |
level | google.protobuf.Int64Value The minimum value is 0. |
ExternalDictionary
Field | Description |
---|---|
name | string Required. Name of the external dictionary. |
structure | Structure Required. Set of attributes for the external dictionary. For in-depth description, see ClickHouse documentation |
layout | Layout Required. Layout for storing the dictionary in memory. For in-depth description, see ClickHouse documentation |
lifetime | oneof: fixed_lifetime or lifetime_range Setting for the period of time between dictionary updates. For details, see ClickHouse documentation |
fixed_lifetime | int64 Fixed interval between dictionary updates. |
lifetime_range | Range Range of intervals between dictionary updates for ClickHouse to choose from. |
source | oneof: http_source , mysql_source , clickhouse_source , mongodb_source or postgresql_source Description of the source for the external dictionary. |
http_source | HttpSource HTTP source for the dictionary. |
mysql_source | MysqlSource MySQL source for the dictionary. |
clickhouse_source | ClickhouseSource ClickHouse source for the dictionary. |
mongodb_source | MongodbSource MongoDB source for the dictionary. |
postgresql_source | PostgresqlSource PostgreSQL source for the dictionary. |
HttpSource
Field | Description |
---|---|
url | string Required. URL of the source dictionary available over HTTP. |
format | string Required. The data format. Valid values are all formats supported by ClickHouse SQL dialect. |
MysqlSource
Field | Description |
---|---|
db | string Required. Name of the MySQL database to connect to. |
table | string Required. Name of the database table to use as a ClickHouse dictionary. |
port | int64 Default port to use when connecting to a replica of the dictionary source. Acceptable values are 0 to 65535, inclusive. |
user | string Name of the default user for replicas of the dictionary source. |
password | string Password of the default user for replicas of the dictionary source. |
replicas[] | Replica List of MySQL replicas of the database used as dictionary source. The number of elements must be greater than 0. |
where | string Selection criteria for the data in the specified MySQL table. |
invalidate_query | string Query for checking the dictionary status, to pull only updated data. For more details, see ClickHouse documentation on dictionaries |
Replica
Field | Description |
---|---|
host | string Required. MySQL host of the replica. The maximum string length in characters is 253. |
priority | int64 Required. The priority of the replica that ClickHouse takes into account when connecting. Replica with the highest priority should have this field set to the lowest number. Value must be greater than 0. |
port | int64 Port to use when connecting to the replica. If a port is not specified for a replica, ClickHouse uses the port specified for the source. Acceptable values are 0 to 65535, inclusive. |
user | string Name of the MySQL database user. |
password | string Password of the MySQL database user. |
ClickhouseSource
Field | Description |
---|---|
db | string Required. Name of the ClickHouse database. |
table | string Required. Name of the table in the specified database to be used as the dictionary source. |
host | string Required. ClickHouse host of the specified database. The maximum string length in characters is 253. |
port | int64 Port to use when connecting to the host. Acceptable values are 0 to 65535, inclusive. |
user | string Required. Name of the ClickHouse database user. |
password | string Password of the ClickHouse database user. |
where | string Selection criteria for the data in the specified ClickHouse table. |
MongodbSource
Field | Description |
---|---|
db | string Required. Name of the MongoDB database. |
collection | string Required. Name of the collection in the specified database to be used as the dictionary source. |
host | string Required. MongoDB host of the specified database. The maximum string length in characters is 253. |
port | int64 Port to use when connecting to the host. Acceptable values are 0 to 65535, inclusive. |
user | string Required. Name of the MongoDB database user. |
password | string Password of the MongoDB database user. |
options | string |
PostgresqlSource
Field | Description |
---|---|
db | string Required. Name of the PostrgreSQL database. |
table | string Required. Name of the table in the specified database to be used as the dictionary source. |
hosts[] | string Name of the PostrgreSQL host The number of elements must be greater than 0. |
port | int64 Port to use when connecting to the host. Acceptable values are 0 to 65535, inclusive. |
user | string Required. Name of the PostrgreSQL database user. |
password | string Password of the PostrgreSQL database user. |
invalidate_query | string Query for checking the dictionary status, to pull only updated data. For more details, see ClickHouse documentation on dictionaries |
ssl_mode | enum SslMode Mode of SSL TCP/IP connection to the PostgreSQL host. For more details, see PostgreSQL documentation
|
Structure
Field | Description |
---|---|
id | Id Single numeric key column for the dictionary. |
key | Key Composite key for the dictionary, containing of one or more key columns. For details, see ClickHouse documentation |
range_min | Attribute Field holding the beginning of the range for dictionaries with RANGE_HASHED layout. For details, see ClickHouse documentation |
range_max | Attribute Field holding the end of the range for dictionaries with RANGE_HASHED layout. For details, see ClickHouse documentation |
attributes[] | Attribute Description of the fields available for database queries. For details, see ClickHouse documentation |
Attribute
Field | Description |
---|---|
name | string Required. Name of the column. |
type | string Required. Type of the column. |
null_value | string Default value for an element without data (for example, an empty string). |
expression | string Expression, describing the attribute, if applicable. |
hierarchical | bool Indication of hierarchy support. Default value: false . |
injective | bool Indication of injective mapping "id -> attribute". Default value: false . |
Id
Field | Description |
---|---|
name | string Required. Name of the numeric key. |
Key
Field | Description |
---|---|
attributes[] | Attribute Attributes of a complex key. The number of elements must be greater than 0. |
Layout
Field | Description |
---|---|
type | enum Type Required. Layout type for an external dictionary.
|
size_in_cells | int64 Number of cells in the cache. Rounded up to a power of two. Applicable only for CACHE and COMPLEX_KEY_CACHE layout types. |
Range
Field | Description |
---|---|
min | int64 Minimum dictionary lifetime. |
max | int64 Maximum dictionary lifetime. |
GraphiteRollup
Field | Description |
---|---|
name | string Required. Name for the specified combination of settings for Graphite rollup. |
patterns[] | Pattern Pattern to use for the rollup. The number of elements must be greater than 0. |
Pattern
Field | Description |
---|---|
regexp | string Pattern for metric names. |
function | string Required. Name of the aggregating function to apply to data of the age specified in retention . |
retention[] | Retention Age of data to use for thinning. The number of elements must be greater than 0. |
Retention
Field | Description |
---|---|
age | int64 Minimum age of the data in seconds. The minimum value is 0. |
precision | int64 Precision of determining the age of the data, in seconds. Value must be greater than 0. |
Resources
Field | Description |
---|---|
resource_preset_id | string ID of the preset for computational resources available to a host (CPU, memory etc.). All available presets are listed in the documentation |
disk_size | int64 Volume of the storage available to a host, in bytes. |
disk_type_id | string Type of the storage environment for the host. Possible values:
|
MaintenanceWindow
Field | Description |
---|---|
policy | oneof: anytime or weekly_maintenance_window The maintenance policy in effect. |
anytime | AnytimeMaintenanceWindow Maintenance operation can be scheduled anytime. |
weekly_maintenance_window | WeeklyMaintenanceWindow Maintenance operation can be scheduled on a weekly basis. |
AnytimeMaintenanceWindow
Empty.
WeeklyMaintenanceWindow
Field | Description |
---|---|
day | enum WeekDay Day of the week (in DDD format). |
hour | int64 Hour of the day in UTC (in HH format). Acceptable values are 1 to 24, inclusive. |
MaintenanceOperation
Field | Description |
---|---|
info | string Information about this maintenance operation. The maximum string length in characters is 256. |
delayed_until | google.protobuf.Timestamp Time until which this maintenance operation is delayed. |
Create
Creates a ClickHouse cluster in the specified folder.
rpc Create (CreateClusterRequest) returns (operation.Operation)
Metadata and response of Operation:
Operation.metadata:CreateClusterMetadata
Operation.response:Cluster
CreateClusterRequest
Field | Description |
---|---|
folder_id | string Required. ID of the folder to create the ClickHouse cluster in. The maximum string length in characters is 50. |
name | string Required. Name of the ClickHouse cluster. The name must be unique within the folder. The maximum string length in characters is 63. Value must match the regular expression [a-zA-Z0-9_-]* . |
description | string Description of the ClickHouse cluster. The maximum string length in characters is 256. |
labels | map<string,string> Custom labels for the ClickHouse cluster as key:value pairs. Maximum 64 per resource. For example, "project": "mvp" or "source": "dictionary". No more than 64 per resource. The maximum string length in characters for each value is 63. Each value must match the regular expression [-_0-9a-z]* . The maximum string length in characters for each key is 63. Each key must match the regular expression [a-z][-_0-9a-z]* . |
environment | Cluster.Environment Required. Deployment environment of the ClickHouse cluster. |
config_spec | ConfigSpec Required. Configuration and resources for hosts that should be created for the ClickHouse cluster. |
database_specs[] | DatabaseSpec Descriptions of databases to be created in the ClickHouse cluster. The number of elements must be greater than 0. |
user_specs[] | UserSpec Descriptions of database users to be created in the ClickHouse cluster. The number of elements must be greater than 0. |
host_specs[] | HostSpec Individual configurations for hosts that should be created for the ClickHouse cluster. The number of elements must be greater than 0. |
network_id | string Required. ID of the network to create the cluster in. The maximum string length in characters is 50. |
shard_name | string Name of the first shard in cluster. If not set, defaults to the value 'shard1'. The maximum string length in characters is 63. Value must match the regular expression [a-zA-Z0-9_-]* . |
service_account_id | string ID of the service account used for access to Object Storage. |
security_group_ids[] | string User security groups |
deletion_protection | bool Deletion Protection inhibits deletion of the cluster |
maintenance_window | MaintenanceWindow Window of maintenance operations. |
ConfigSpec
Field | Description |
---|---|
version | string Version of the ClickHouse server software. |
clickhouse | Clickhouse Configuration and resources for a ClickHouse server. |
zookeeper | Zookeeper Configuration and resources for a ZooKeeper server. |
backup_window_start | google.type.TimeOfDay Time to start the daily backup, in the UTC timezone. |
access | Access Access policy for external services. If you want a specific service to access the ClickHouse cluster, then set the necessary values in this policy. |
cloud_storage | CloudStorage |
sql_database_management | google.protobuf.BoolValue Whether database management through SQL commands is enabled. |
sql_user_management | google.protobuf.BoolValue Whether user management through SQL commands is enabled. |
admin_password | string Password for user 'admin' that has SQL user management access. |
embedded_keeper | google.protobuf.BoolValue Whether cluster should use embedded Keeper instead of Zookeeper |
Clickhouse
Field | Description |
---|---|
config | config.ClickhouseConfig Configuration for a ClickHouse server. |
resources | Resources Resources allocated to ClickHouse hosts. |
Zookeeper
Field | Description |
---|---|
resources | Resources Resources allocated to ZooKeeper hosts. If not set, minimal available resources will be used. All available resource presets can be retrieved with a ResourcePresetService.List request. |
Access
Field | Description |
---|---|
data_lens | bool Allow to export data from the cluster to DataLens. |
web_sql | bool Allow SQL queries to the cluster databases from the management console. See SQL queries in the management console for more details. |
metrika | bool Allow to import data from Yandex Metrica and AppMetrica to the cluster. See AppMetrica documentation |
serverless | bool Allow access to cluster for Serverless. |
data_transfer | bool Allow access for DataTransfer |
yandex_query | bool Allow access for Query |
CloudStorage
Field | Description |
---|---|
enabled | bool Whether to use Object Storage for storing ClickHouse data. |
move_factor | google.protobuf.DoubleValue Acceptable values are 0 to 1, inclusive. |
data_cache_enabled | google.protobuf.BoolValue |
data_cache_max_size | google.protobuf.Int64Value |
prefer_not_to_merge | google.protobuf.BoolValue |
ClickhouseConfig
Field | Description |
---|---|
log_level | enum LogLevel Logging level for the ClickHouse cluster. Possible values: TRACE, DEBUG, INFORMATION, WARNING, ERROR. |
merge_tree | MergeTree Settings for the MergeTree engine. See description in ClickHouse documentation |
compression[] | Compression Compression settings for the ClickHouse cluster. See in-depth description in ClickHouse documentation |
dictionaries[] | ExternalDictionary Configuration of external dictionaries to be used by the ClickHouse cluster. See in-depth description in ClickHouse documentation |
graphite_rollup[] | GraphiteRollup Settings for thinning Graphite data. See in-depth description in ClickHouse documentation |
kafka | Kafka |
kafka_topics[] | KafkaTopic |
rabbitmq | Rabbitmq |
max_connections | google.protobuf.Int64Value Maximum number of inbound connections. The minimum value is 10. |
max_concurrent_queries | google.protobuf.Int64Value Maximum number of simultaneously processed requests. The minimum value is 10. |
keep_alive_timeout | google.protobuf.Int64Value Number of milliseconds that ClickHouse waits for incoming requests before closing the connection. |
uncompressed_cache_size | google.protobuf.Int64Value Cache size (in bytes) for uncompressed data used by MergeTree tables. |
mark_cache_size | google.protobuf.Int64Value Approximate size (in bytes) of the cache of "marks" used by MergeTree tables. Value must be greater than 0. |
max_table_size_to_drop | google.protobuf.Int64Value Maximum size of the table that can be deleted using a DROP query. |
max_partition_size_to_drop | google.protobuf.Int64Value Maximum size of the partition that can be deleted using a DROP query. |
builtin_dictionaries_reload_interval | google.protobuf.Int64Value The setting is deprecated and has no effect. |
timezone | string The server's time zone to be used in DateTime fields conversions. Specified as an IANA identifier. |
geobase_enabled | google.protobuf.BoolValue Enable or disable geobase. |
geobase_uri | string Address of the archive with the user geobase in Object Storage. |
query_log_retention_size | google.protobuf.Int64Value The maximum size that query_log can grow to before old data will be removed. If set to 0, automatic removal of query_log data based on size is disabled. |
query_log_retention_time | google.protobuf.Int64Value The maximum time that query_log records will be retained before removal. If set to 0, automatic removal of query_log data based on time is disabled. |
query_thread_log_enabled | google.protobuf.BoolValue Whether query_thread_log system table is enabled. |
query_thread_log_retention_size | google.protobuf.Int64Value The maximum size that query_thread_log can grow to before old data will be removed. If set to 0, automatic removal of query_thread_log data based on size is disabled. |
query_thread_log_retention_time | google.protobuf.Int64Value The maximum time that query_thread_log records will be retained before removal. If set to 0, automatic removal of query_thread_log data based on time is disabled. |
part_log_retention_size | google.protobuf.Int64Value The maximum size that part_log can grow to before old data will be removed. If set to 0, automatic removal of part_log data based on size is disabled. |
part_log_retention_time | google.protobuf.Int64Value The maximum time that part_log records will be retained before removal. If set to 0, automatic removal of part_log data based on time is disabled. |
metric_log_enabled | google.protobuf.BoolValue Whether metric_log system table is enabled. |
metric_log_retention_size | google.protobuf.Int64Value The maximum size that metric_log can grow to before old data will be removed. If set to 0, automatic removal of metric_log data based on size is disabled. |
metric_log_retention_time | google.protobuf.Int64Value The maximum time that metric_log records will be retained before removal. If set to 0, automatic removal of metric_log data based on time is disabled. |
trace_log_enabled | google.protobuf.BoolValue Whether trace_log system table is enabled. |
trace_log_retention_size | google.protobuf.Int64Value The maximum size that trace_log can grow to before old data will be removed. If set to 0, automatic removal of trace_log data based on size is disabled. |
trace_log_retention_time | google.protobuf.Int64Value The maximum time that trace_log records will be retained before removal. If set to 0, automatic removal of trace_log data based on time is disabled. |
text_log_enabled | google.protobuf.BoolValue Whether text_log system table is enabled. |
text_log_retention_size | google.protobuf.Int64Value The maximum size that text_log can grow to before old data will be removed. If set to 0, automatic removal of text_log data based on size is disabled. |
text_log_retention_time | google.protobuf.Int64Value The maximum time that text_log records will be retained before removal. If set to 0, automatic removal of text_log data based on time is disabled. |
text_log_level | enum LogLevel Logging level for text_log system table. Possible values: TRACE, DEBUG, INFORMATION, WARNING, ERROR. |
opentelemetry_span_log_enabled | google.protobuf.BoolValue Enable or disable opentelemetry_span_log system table. Default value: false. |
opentelemetry_span_log_retention_size | google.protobuf.Int64Value The maximum size that opentelemetry_span_log can grow to before old data will be removed. If set to 0 (default), automatic removal of opentelemetry_span_log data based on size is disabled. The minimum value is 0. |
opentelemetry_span_log_retention_time | google.protobuf.Int64Value The maximum time that opentelemetry_span_log records will be retained before removal. If set to 0, automatic removal of opentelemetry_span_log data based on time is disabled. The minimum value is 0. |
query_views_log_enabled | google.protobuf.BoolValue Enable or disable query_views_log system table. Default value: false. |
query_views_log_retention_size | google.protobuf.Int64Value The maximum size that query_views_log can grow to before old data will be removed. If set to 0 (default), automatic removal of query_views_log data based on size is disabled. The minimum value is 0. |
query_views_log_retention_time | google.protobuf.Int64Value The maximum time that query_views_log records will be retained before removal. If set to 0, automatic removal of query_views_log data based on time is disabled. The minimum value is 0. |
asynchronous_metric_log_enabled | google.protobuf.BoolValue Enable or disable asynchronous_metric_log system table. Default value: false. |
asynchronous_metric_log_retention_size | google.protobuf.Int64Value The maximum size that asynchronous_metric_log can grow to before old data will be removed. If set to 0 (default), automatic removal of asynchronous_metric_log data based on size is disabled. The minimum value is 0. |
asynchronous_metric_log_retention_time | google.protobuf.Int64Value The maximum time that asynchronous_metric_log records will be retained before removal. If set to 0, automatic removal of asynchronous_metric_log data based on time is disabled. The minimum value is 0. |
session_log_enabled | google.protobuf.BoolValue Enable or disable session_log system table. Default value: false. |
session_log_retention_size | google.protobuf.Int64Value The maximum size that session_log can grow to before old data will be removed. If set to 0 (default), automatic removal of session_log data based on size is disabled. The minimum value is 0. |
session_log_retention_time | google.protobuf.Int64Value The maximum time that session_log records will be retained before removal. If set to 0, automatic removal of session_log data based on time is disabled. The minimum value is 0. |
zookeeper_log_enabled | google.protobuf.BoolValue Enable or disable zookeeper_log system table. Default value: false. |
zookeeper_log_retention_size | google.protobuf.Int64Value The maximum size that zookeeper_log can grow to before old data will be removed. If set to 0 (default), automatic removal of zookeeper_log data based on size is disabled. The minimum value is 0. |
zookeeper_log_retention_time | google.protobuf.Int64Value The maximum time that zookeeper_log records will be retained before removal. If set to 0, automatic removal of zookeeper_log data based on time is disabled. The minimum value is 0. |
asynchronous_insert_log_enabled | google.protobuf.BoolValue Enable or disable asynchronous_insert_log system table. Default value: false. Minimal required ClickHouse version: 22.10. |
asynchronous_insert_log_retention_size | google.protobuf.Int64Value The maximum size that asynchronous_insert_log can grow to before old data will be removed. If set to 0 (default), automatic removal of asynchronous_insert_log data based on size is disabled. The minimum value is 0. |
asynchronous_insert_log_retention_time | google.protobuf.Int64Value The maximum time that asynchronous_insert_log records will be retained before removal. If set to 0, automatic removal of asynchronous_insert_log data based on time is disabled. The minimum value is 0. |
background_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_merges_mutations_concurrency_ratio | google.protobuf.Int64Value Sets a ratio between the number of threads and the number of background merges and mutations that can be executed concurrently. For example, if the ratio equals to 2 and background_pool_size is set to 16 then ClickHouse can execute 32 background merges concurrently. This is possible, because background operations could be suspended and postponed. This is needed to give small merges more execution priority. You can only increase this ratio at runtime. To lower it you have to restart the server. The same as for background_pool_size setting background_merges_mutations_concurrency_ratio could be applied from the default profile for backward compatibility. Default: 2 See in-depth description in ClickHouse documentation |
background_schedule_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_fetches_pool_size | google.protobuf.Int64Value Sets the number of threads performing background fetches for tables with ReplicatedMergeTree engines. Default value: 8. More info see in ClickHouse documentation |
background_move_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_distributed_schedule_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_buffer_flush_schedule_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_message_broker_schedule_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_common_pool_size | google.protobuf.Int64Value The maximum number of threads that will be used for performing a variety of operations (mostly garbage collection) for *MergeTree-engine tables in a background. Default: 8 See in-depth description in ClickHouse documentation |
default_database | google.protobuf.StringValue The default database. To get a list of cluster databases, see Yandex Managed ClickHouse documentation. |
total_memory_profiler_step | google.protobuf.Int64Value Sets the memory size (in bytes) for a stack trace at every peak allocation step. Default value: 4194304. More info see in ClickHouse documentation |
total_memory_tracker_sample_probability | google.protobuf.DoubleValue |
MergeTree
Field | Description |
---|---|
replicated_deduplication_window | google.protobuf.Int64Value Number of blocks of hashes to keep in ZooKeeper. |
replicated_deduplication_window_seconds | google.protobuf.Int64Value Period of time to keep blocks of hashes for. |
parts_to_delay_insert | google.protobuf.Int64Value If table contains at least that many active parts in single partition, artificially slow down insert into table. |
parts_to_throw_insert | google.protobuf.Int64Value If more than this number active parts in single partition, throw 'Too many parts ...' exception. |
inactive_parts_to_delay_insert | google.protobuf.Int64Value |
inactive_parts_to_throw_insert | google.protobuf.Int64Value |
max_replicated_merges_in_queue | google.protobuf.Int64Value How many tasks of merging and mutating parts are allowed simultaneously in ReplicatedMergeTree queue. |
number_of_free_entries_in_pool_to_lower_max_size_of_merge | google.protobuf.Int64Value If there is less than specified number of free entries in background pool (or replicated queue), start to lower maximum size of merge to process. |
max_bytes_to_merge_at_min_space_in_pool | google.protobuf.Int64Value Maximum in total size of parts to merge, when there are minimum free threads in background pool (or entries in replication queue). |
max_bytes_to_merge_at_max_space_in_pool | google.protobuf.Int64Value |
min_bytes_for_wide_part | google.protobuf.Int64Value Minimum number of bytes in a data part that can be stored in Wide format. More info see in ClickHouse documentation |
min_rows_for_wide_part | google.protobuf.Int64Value Minimum number of rows in a data part that can be stored in Wide format. More info see in ClickHouse documentation |
ttl_only_drop_parts | google.protobuf.BoolValue Enables or disables complete dropping of data parts where all rows are expired in MergeTree tables. More info see in ClickHouse documentation |
allow_remote_fs_zero_copy_replication | google.protobuf.BoolValue |
merge_with_ttl_timeout | google.protobuf.Int64Value |
merge_with_recompression_ttl_timeout | google.protobuf.Int64Value |
max_parts_in_total | google.protobuf.Int64Value |
max_number_of_merges_with_ttl_in_pool | google.protobuf.Int64Value |
cleanup_delay_period | google.protobuf.Int64Value |
number_of_free_entries_in_pool_to_execute_mutation | google.protobuf.Int64Value |
max_avg_part_size_for_too_many_parts | google.protobuf.Int64Value The 'too many parts' check according to 'parts_to_delay_insert' and 'parts_to_throw_insert' will be active only if the average part size (in the relevant partition) is not larger than the specified threshold. If it is larger than the specified threshold, the INSERTs will be neither delayed or rejected. This allows to have hundreds of terabytes in a single table on a single server if the parts are successfully merged to larger parts. This does not affect the thresholds on inactive parts or total parts. Default: 1 GiB Min version: 22.10 See in-depth description in ClickHouse GitHub |
min_age_to_force_merge_seconds | google.protobuf.Int64Value Merge parts if every part in the range is older than the value of min_age_to_force_merge_seconds. Default: 0 - disabled Min_version: 22.10 See in-depth description in ClickHouse documentation |
min_age_to_force_merge_on_partition_only | google.protobuf.BoolValue Whether min_age_to_force_merge_seconds should be applied only on the entire partition and not on subset. Default: false Min_version: 22.11 See in-depth description in ClickHouse documentation |
merge_selecting_sleep_ms | google.protobuf.Int64Value Sleep time for merge selecting when no part is selected. A lower setting triggers selecting tasks in background_schedule_pool frequently, which results in a large number of requests to ClickHouse Keeper in large-scale clusters. Default: 5000 Min_version: 21.10 See in-depth description in ClickHouse documentation |
Kafka
Field | Description |
---|---|
security_protocol | enum SecurityProtocol |
sasl_mechanism | enum SaslMechanism |
sasl_username | string |
sasl_password | string |
enable_ssl_certificate_verification | google.protobuf.BoolValue |
max_poll_interval_ms | google.protobuf.Int64Value The minimum value is 0. |
session_timeout_ms | google.protobuf.Int64Value The minimum value is 0. |
KafkaTopic
Field | Description |
---|---|
name | string Required. |
settings | Kafka Required. |
Rabbitmq
Field | Description |
---|---|
username | string RabbitMQ |
password | string RabbitMQ |
vhost | string RabbitMQ |
Compression
Field | Description |
---|---|
method | enum Method Compression method to use for the specified combination of min_part_size and min_part_size_ratio . |
min_part_size | int64 Minimum size of a part of a table. The minimum value is 1. |
min_part_size_ratio | double Minimum ratio of a part relative to the size of all the data in the table. |
level | google.protobuf.Int64Value The minimum value is 0. |
ExternalDictionary
Field | Description |
---|---|
name | string Required. Name of the external dictionary. |
structure | Structure Required. Set of attributes for the external dictionary. For in-depth description, see ClickHouse documentation |
layout | Layout Required. Layout for storing the dictionary in memory. For in-depth description, see ClickHouse documentation |
lifetime | oneof: fixed_lifetime or lifetime_range Setting for the period of time between dictionary updates. For details, see ClickHouse documentation |
fixed_lifetime | int64 Fixed interval between dictionary updates. |
lifetime_range | Range Range of intervals between dictionary updates for ClickHouse to choose from. |
source | oneof: http_source , mysql_source , clickhouse_source , mongodb_source or postgresql_source Description of the source for the external dictionary. |
http_source | HttpSource HTTP source for the dictionary. |
mysql_source | MysqlSource MySQL source for the dictionary. |
clickhouse_source | ClickhouseSource ClickHouse source for the dictionary. |
mongodb_source | MongodbSource MongoDB source for the dictionary. |
postgresql_source | PostgresqlSource PostgreSQL source for the dictionary. |
HttpSource
Field | Description |
---|---|
url | string Required. URL of the source dictionary available over HTTP. |
format | string Required. The data format. Valid values are all formats supported by ClickHouse SQL dialect. |
MysqlSource
Field | Description |
---|---|
db | string Required. Name of the MySQL database to connect to. |
table | string Required. Name of the database table to use as a ClickHouse dictionary. |
port | int64 Default port to use when connecting to a replica of the dictionary source. Acceptable values are 0 to 65535, inclusive. |
user | string Name of the default user for replicas of the dictionary source. |
password | string Password of the default user for replicas of the dictionary source. |
replicas[] | Replica List of MySQL replicas of the database used as dictionary source. The number of elements must be greater than 0. |
where | string Selection criteria for the data in the specified MySQL table. |
invalidate_query | string Query for checking the dictionary status, to pull only updated data. For more details, see ClickHouse documentation on dictionaries |
Replica
Field | Description |
---|---|
host | string Required. MySQL host of the replica. The maximum string length in characters is 253. |
priority | int64 Required. The priority of the replica that ClickHouse takes into account when connecting. Replica with the highest priority should have this field set to the lowest number. Value must be greater than 0. |
port | int64 Port to use when connecting to the replica. If a port is not specified for a replica, ClickHouse uses the port specified for the source. Acceptable values are 0 to 65535, inclusive. |
user | string Name of the MySQL database user. |
password | string Password of the MySQL database user. |
ClickhouseSource
Field | Description |
---|---|
db | string Required. Name of the ClickHouse database. |
table | string Required. Name of the table in the specified database to be used as the dictionary source. |
host | string Required. ClickHouse host of the specified database. The maximum string length in characters is 253. |
port | int64 Port to use when connecting to the host. Acceptable values are 0 to 65535, inclusive. |
user | string Required. Name of the ClickHouse database user. |
password | string Password of the ClickHouse database user. |
where | string Selection criteria for the data in the specified ClickHouse table. |
MongodbSource
Field | Description |
---|---|
db | string Required. Name of the MongoDB database. |
collection | string Required. Name of the collection in the specified database to be used as the dictionary source. |
host | string Required. MongoDB host of the specified database. The maximum string length in characters is 253. |
port | int64 Port to use when connecting to the host. Acceptable values are 0 to 65535, inclusive. |
user | string Required. Name of the MongoDB database user. |
password | string Password of the MongoDB database user. |
options | string |
PostgresqlSource
Field | Description |
---|---|
db | string Required. Name of the PostrgreSQL database. |
table | string Required. Name of the table in the specified database to be used as the dictionary source. |
hosts[] | string Name of the PostrgreSQL host The number of elements must be greater than 0. |
port | int64 Port to use when connecting to the host. Acceptable values are 0 to 65535, inclusive. |
user | string Required. Name of the PostrgreSQL database user. |
password | string Password of the PostrgreSQL database user. |
invalidate_query | string Query for checking the dictionary status, to pull only updated data. For more details, see ClickHouse documentation on dictionaries |
ssl_mode | enum SslMode Mode of SSL TCP/IP connection to the PostgreSQL host. For more details, see PostgreSQL documentation
|
Structure
Field | Description |
---|---|
id | Id Single numeric key column for the dictionary. |
key | Key Composite key for the dictionary, containing of one or more key columns. For details, see ClickHouse documentation |
range_min | Attribute Field holding the beginning of the range for dictionaries with RANGE_HASHED layout. For details, see ClickHouse documentation |
range_max | Attribute Field holding the end of the range for dictionaries with RANGE_HASHED layout. For details, see ClickHouse documentation |
attributes[] | Attribute Description of the fields available for database queries. For details, see ClickHouse documentation |
Attribute
Field | Description |
---|---|
name | string Required. Name of the column. |
type | string Required. Type of the column. |
null_value | string Default value for an element without data (for example, an empty string). |
expression | string Expression, describing the attribute, if applicable. |
hierarchical | bool Indication of hierarchy support. Default value: false . |
injective | bool Indication of injective mapping "id -> attribute". Default value: false . |
Id
Field | Description |
---|---|
name | string Required. Name of the numeric key. |
Key
Field | Description |
---|---|
attributes[] | Attribute Attributes of a complex key. The number of elements must be greater than 0. |
Layout
Field | Description |
---|---|
type | enum Type Required. Layout type for an external dictionary.
|
size_in_cells | int64 Number of cells in the cache. Rounded up to a power of two. Applicable only for CACHE and COMPLEX_KEY_CACHE layout types. |
Range
Field | Description |
---|---|
min | int64 Minimum dictionary lifetime. |
max | int64 Maximum dictionary lifetime. |
GraphiteRollup
Field | Description |
---|---|
name | string Required. Name for the specified combination of settings for Graphite rollup. |
patterns[] | Pattern Pattern to use for the rollup. The number of elements must be greater than 0. |
Pattern
Field | Description |
---|---|
regexp | string Pattern for metric names. |
function | string Required. Name of the aggregating function to apply to data of the age specified in retention . |
retention[] | Retention Age of data to use for thinning. The number of elements must be greater than 0. |
Retention
Field | Description |
---|---|
age | int64 Minimum age of the data in seconds. The minimum value is 0. |
precision | int64 Precision of determining the age of the data, in seconds. Value must be greater than 0. |
Resources
Field | Description |
---|---|
resource_preset_id | string ID of the preset for computational resources available to a host (CPU, memory etc.). All available presets are listed in the documentation |
disk_size | int64 Volume of the storage available to a host, in bytes. |
disk_type_id | string Type of the storage environment for the host. Possible values:
|
DatabaseSpec
Field | Description |
---|---|
name | string Required. Name of the ClickHouse database. 1-63 characters long. The maximum string length in characters is 63. Value must match the regular expression [a-zA-Z0-9_-]* . |
UserSpec
Field | Description |
---|---|
name | string Required. Name of the ClickHouse user. The maximum string length in characters is 63. Value must match the regular expression [a-zA-Z0-9_][a-zA-Z0-9_-]* . |
password | string Required. Password of the ClickHouse user. The string length in characters must be 8-128. |
permissions[] | Permission Set of permissions to grant to the user. If not set, it's granted permissions to access all databases. |
settings | UserSettings |
quotas[] | UserQuota Set of quotas assigned to the user. |
Permission
Field | Description |
---|---|
database_name | string Name of the database that the permission grants access to. |
UserSettings
Field | Description |
---|---|
readonly | google.protobuf.Int64Value Restricts permissions for non-DDL queries. To restrict permissions for DDL queries, use allow_ddl instead.
See in-depth description in ClickHouse documentation |
allow_ddl | google.protobuf.BoolValue Determines whether DDL queries are allowed (e.g., CREATE, ALTER, RENAME, etc). Default value: true. See in-depth description in ClickHouse documentation |
allow_introspection_functions | google.protobuf.BoolValue Enables introspections functions More info see in ClickHouse documentation |
connect_timeout | google.protobuf.Int64Value Connection timeout in milliseconds. Value must be greater than 0 (default: 10000, 10 seconds). Value must be greater than 0. |
connect_timeout_with_failover | google.protobuf.Int64Value The timeout in milliseconds for connecting to a remote server for a Distributed table engine. Applies only if the cluster uses sharding and replication. If unsuccessful, several attempts are made to connect to various replicas. Default value: 50. More info see in ClickHouse documentation |
receive_timeout | google.protobuf.Int64Value Receive timeout in milliseconds. Value must be greater than 0 (default: 300000, 300 seconds or 5 minutes). Value must be greater than 0. |
send_timeout | google.protobuf.Int64Value Send timeout in milliseconds. Value must be greater than 0 (default: 300000, 300 seconds or 5 minutes). Value must be greater than 0. |
timeout_before_checking_execution_speed | google.protobuf.Int64Value Timeout (in seconds) between checks of execution speed. It is checked that execution speed is not less that specified in min_execution_speed parameter. Default value: 10. |
insert_quorum | google.protobuf.Int64Value Enables or disables write quorum for ClickHouse cluster. If the value is less than 2, then write quorum is disabled, otherwise it is enabled. When used, write quorum guarantees that ClickHouse has written data to the quorum of insert_quorum replicas with no errors until the insert_quorum_timeout expires. All replicas in the quorum are in the consistent state, meaning that they contain linearized data from the previous INSERT queries. Employ write quorum, if you need the guarantees that the written data would not be lost in case of one or more replicas failure. You can use select_sequential_consistency setting to read the data written with write quorum. See in-depth description in ClickHouse documentation |
insert_quorum_timeout | google.protobuf.Int64Value Quorum write timeout in milliseconds. If the write quorum is enabled in the cluster, this timeout expires and some data is not written to the insert_quorum replicas, then ClickHouse will abort the execution of INSERT query and return an error. In this case, the client must send the query again to write the data block into the same or another replica. Minimum value: 1000, 1 second (default: 60000, 1 minute). The minimum value is 1000. |
insert_quorum_parallel | google.protobuf.BoolValue See in-depth description in ClickHouse documentation |
insert_null_as_default | google.protobuf.BoolValue Enables the insertion of default values instead of NULL into columns with not nullable data type. Default value: true. More info see in ClickHouse documentation |
select_sequential_consistency | google.protobuf.BoolValue Determines the behavior of SELECT queries from the replicated table: if enabled, ClickHouse will terminate a query with error message in case the replica does not have a chunk written with the quorum and will not read the parts that have not yet been written with the quorum. Default value: false (sequential consistency is disabled). |
deduplicate_blocks_in_dependent_materialized_views | google.protobuf.BoolValue See in-depth description in ClickHouse documentation |
replication_alter_partitions_sync | google.protobuf.Int64Value Wait mode for asynchronous actions in ALTER queries on replicated tables:
See in-depth description in ClickHouse documentation |
max_replica_delay_for_distributed_queries | google.protobuf.Int64Value Max replica delay in milliseconds. If a replica lags more than the set value, this replica is not used and becomes a stale one. Minimum value: 1000, 1 second (default: 300000, 300 seconds or 5 minutes). See in-depth description in ClickHouse documentation |
fallback_to_stale_replicas_for_distributed_queries | google.protobuf.BoolValue Enables or disables query forcing to a stale replica in case the actual data is unavailable. If enabled, ClickHouse will choose the most up-to-date replica and force the query to use the data in this replica. This setting can be used when doing SELECT query from a distributed table that points to replicated tables. Default value: true (query forcing is enabled). See in-depth description in ClickHouse documentation |
distributed_product_mode | enum DistributedProductMode Determine the behavior of distributed subqueries. See in-depth description in ClickHouse documentation
|
distributed_aggregation_memory_efficient | google.protobuf.BoolValue Enables of disables memory saving mode when doing distributed aggregation. When ClickHouse works with a distributed query, external aggregation is done on remote servers. Enable this setting to achieve a smaller memory footprint on the server that sourced such a distributed query. Default value: false (memory saving mode is disabled). See in-depth description in ClickHouse documentation |
distributed_ddl_task_timeout | google.protobuf.Int64Value Timeout for DDL queries, in milliseconds. |
skip_unavailable_shards | google.protobuf.BoolValue Enables or disables silent skipping of unavailable shards. A shard is considered unavailable if all its replicas are also unavailable. Default value: false (silent skipping is disabled). See in-depth description in ClickHouse documentation |
compile_expressions | google.protobuf.BoolValue Enables or disables expression compilation. If you execute a lot of queries that contain identical expressions, then enable this setting. As a result, such queries may be executed faster due to use of compiled expressions. Use this setting in combination with min_count_to_compile_expression setting. Default value: false (expression compilation is disabled). |
min_count_to_compile_expression | google.protobuf.Int64Value How many identical expressions ClickHouse has to encounter before they are compiled. Minimum value: 0 (default: 3). For the 0 value compilation is synchronous: a query waits for expression compilation process to complete prior to continuing execution. It is recommended to set this value only for testing purposes. For all other values, compilation is asynchronous: the compilation process executes in a separate thread. When a compiled expression is ready, it will be used by ClickHouse for eligible queries, including the ones that are currently running. The minimum value is 0. |
max_block_size | google.protobuf.Int64Value The maximum block size for reading. Data in ClickHouse is organized and processed by blocks (block is a set of columns' parts). The internal processing cycles for a single block are efficient enough, but there are noticeable expenditures on each block. This setting is a recommendation for size of block (in a count of rows) that should be loaded from tables. Value must be greater than 0 (default: 65536). See in-depth description in ClickHouse documentation |
min_insert_block_size_rows | google.protobuf.Int64Value Limits the minimum number of rows in a block to be inserted in a table by INSERT query. Blocks that are smaller than the specified value, will be squashed together into the bigger blocks. Minimal value: 0, block squashing is disabled (default: 1048576). The minimum value is 0. |
min_insert_block_size_bytes | google.protobuf.Int64Value Limits the minimum number of bytes in a block to be inserted in a table by INSERT query. Blocks that are smaller than the specified value, will be squashed together into the bigger blocks. Minimal value: 0, block squashing is disabled (default: 268435456, 256 MB). The minimum value is 0. |
max_insert_block_size | google.protobuf.Int64Value Allows to form blocks of the specified size (in bytes) when inserting data in a table. This setting has effect only if server is creating such blocks by itself. Value must be greater than 0 (default: 1048576). See in-depth description in ClickHouse documentation |
min_bytes_to_use_direct_io | google.protobuf.Int64Value Limits the minimum number of bytes to enable unbuffered direct reads from disk (Direct I/O). By default, ClickHouse does not read data directly from disk, but relies on the filesystem and its cache instead. Such reading strategy is effective when the data volume is small. If the amount of the data to read is huge, it is more effective to read directly from the disk, bypassing the filesystem cache. If the total amount of the data to read is greater than the value of this setting, then ClickHouse will fetch this data directly from the disk. Minimal value and default value: 0, Direct I/O is disabled. The minimum value is 0. |
use_uncompressed_cache | google.protobuf.BoolValue Determines whether to use the cache of uncompressed blocks, or not. Using this cache can significantly reduce latency and increase the throughput when a huge amount of small queries is to be processed. Enable this setting for the users who instantiates small queries frequently. This setting has effect only for tables of the MergeTree family. Default value: false (uncompressed cache is disabled). See in-depth description in ClickHouse documentation |
merge_tree_max_rows_to_use_cache | google.protobuf.Int64Value Limits the maximum size in rows of the request that can use the cache of uncompressed data. The cache is not used for requests larger than the specified value. Use this setting in combination with use_uncompressed_cache setting. Value must be greater than 0 (default: 128x8192). Value must be greater than 0. |
merge_tree_max_bytes_to_use_cache | google.protobuf.Int64Value Limits the maximum size in bytes of the request that can use the cache of uncompressed data. The cache is not used for requests larger than the specified value. Use this setting in combination with use_uncompressed_cache setting. Value must be greater than 0 (default: 192x10x1024x1024). Value must be greater than 0. |
merge_tree_min_rows_for_concurrent_read | google.protobuf.Int64Value Limits the minimum number of rows to be read from a file to enable concurrent read. If the number of rows to be read exceeds this value, then ClickHouse will try to use a few threads to read from a file concurrently. This setting has effect only for tables of the MergeTree family. Value must be greater than 0 (default: 20x8192). Value must be greater than 0. |
merge_tree_min_bytes_for_concurrent_read | google.protobuf.Int64Value Limits the number of bytes to be read from a file to enable concurrent read. If the number of bytes to be read exceeds this value, then ClickHouse will try to use a few threads to read from a file concurrently. This setting has effect only for tables of the MergeTree family. Value must be greater than 0 (default: 24x10x1024x1024). Value must be greater than 0. |
max_bytes_before_external_group_by | google.protobuf.Int64Value Sets the threshold of RAM consumption (in bytes) after that the temporary data, collected during the GROUP BY operation, should be flushed to disk to limit the RAM comsumption. By default, aggregation is done by employing hash table that resides in RAM. A query can result in aggregation of huge data volumes that can lead to memory exhaustion and abortion of the query (see the max_memory_usage setting). For such queries, you can use this setting to force ClickHouse to do flushing and complete aggregation successfully. Minimal value and default value: 0, GROUP BY in the external memory is disabled. When using aggregation in external memory, it is recommended to set the value of this setting twice as low as the max_memory_usage setting value (by default, the maximum memory usage is limited to ten gigabytes). See in-depth description in ClickHouse documentation See also: the distributed_aggregation_memory_efficient setting. |
max_bytes_before_external_sort | google.protobuf.Int64Value This setting is equivalent of the max_bytes_before_external_group_by setting, except for it is for sort operation (ORDER BY), not aggregation. |
group_by_two_level_threshold | google.protobuf.Int64Value Sets the threshold of the number of keys, after that the two-level aggregation should be used. Minimal value: 0, threshold is not set (default: 10000). |
group_by_two_level_threshold_bytes | google.protobuf.Int64Value Sets the threshold of the number of bytes, after that the two-level aggregation should be used. Minimal value: 0, threshold is not set (default: 100000000). |
priority | google.protobuf.Int64Value Sets the priority of a query.
This setting should be set up for each query individually. If ClickHouse is working with the high-priority queries, and a low-priority query enters, then the low-priority query is paused until higher-priority queries are completed. Minimal value and default value: 0, priority is not used. The minimum value is 0. |
max_threads | google.protobuf.Int64Value Limits the maximum number of threads to process the request (setting does not take threads that read data from remote servers into account). This setting applies to threads that perform the same stages of the query processing pipeline in parallel. Minimal value and default value: 0 (the thread number is calculated automatically based on the number of physical CPU cores, no HyperThreading cores are taken into account). See in-depth description in ClickHouse documentation |
max_memory_usage | google.protobuf.Int64Value Limits the maximum memory usage (in bytes) for processing of a single user's query on a single server. This setting does not take server's free RAM amount or total RAM amount into account. This limitation is enforced for any user's single query on a single server. Minimal value: 0, no limitation is set. Value that is set in the ClickHouse default config file: 10737418240 (10 GB). If you use max_bytes_before_external_group_by or max_bytes_before_external_sort setting, then it is recommended to set their values twice as low as max_memory_usage setting value. See in-depth description in ClickHouse documentation |
max_memory_usage_for_user | google.protobuf.Int64Value Limits the maximum memory usage (in bytes) for processing of user's queries on a single server. This setting does not take server's free RAM amount or total RAM amount into account. This limitation is enforced for all queries that belong to one user and run simultaneously on a single server. Minimal value and default value: 0, no limitation is set. The minimum value is 0. |
max_network_bandwidth | google.protobuf.Int64Value The maximum speed of data exchange over the network in bytes per second for a query. Minimal value and default value: 0, no limitation is set. |
max_network_bandwidth_for_user | google.protobuf.Int64Value The maximum speed of data exchange over the network in bytes per second for all concurrently running user queries. Minimal value and default value: 0, no limitation is set. |
max_partitions_per_insert_block | google.protobuf.Int64Value See in-depth description in ClickHouse documentation |
max_concurrent_queries_for_user | google.protobuf.Int64Value The maximum number of concurrent requests per user. Default value: 0 (no limit). |
force_index_by_date | google.protobuf.BoolValue If enabled, query is not executed if the ClickHouse can't use index by date. This setting has effect only for tables of the MergeTree family. Default value: false (setting is disabled, query executes even if ClickHouse can't use index by date). See in-depth description in ClickHouse documentation |
force_primary_key | google.protobuf.BoolValue If enabled, query is not executed if the ClickHouse can't use index by primary key. This setting has effect only for tables of the MergeTree family. Default value: false (setting is disabled, query executes even if ClickHouse can't use index by primary key). See in-depth description in ClickHouse documentation |
max_rows_to_read | google.protobuf.Int64Value Limits the maximum number of rows that can be read from a table when running a query. Minimal value and default value: 0, no limitation is set. See in-depth description in ClickHouse documentation |
max_bytes_to_read | google.protobuf.Int64Value Limits the maximum number of bytes (uncompressed data) that can be read from a table when running a query. Minimal value and default value: 0, no limitation is set. The minimum value is 0. |
read_overflow_mode | enum OverflowMode Determines the behavior on exceeding limits
|
max_rows_to_group_by | google.protobuf.Int64Value Limits the maximum number of unique keys received from aggregation function. This setting helps to reduce RAM consumption while doing aggregation. Minimal value and default value: 0, no limitation is set. The minimum value is 0. |
group_by_overflow_mode | enum GroupByOverflowMode Determines the behavior on exceeding limits
|
max_rows_to_sort | google.protobuf.Int64Value Limits the maximum number of rows that can be read from a table for sorting. This setting helps to reduce RAM consumption. Minimal value and default value: 0, no limitation is set. The minimum value is 0. |
max_bytes_to_sort | google.protobuf.Int64Value Limits the maximum number of bytes (uncompressed data) that can be read from a table for sorting. This setting helps to reduce RAM consumption. Minimal value and default value: 0, no limitation is set. The minimum value is 0. |
sort_overflow_mode | enum OverflowMode Determines the behavior on exceeding limits
|
max_result_rows | google.protobuf.Int64Value Limits the number of rows in the result. This limitation is also checked for subqueries and parts of distributed queries that run on remote servers. Minimal value and default value: 0, no limitation is set. The minimum value is 0. |
max_result_bytes | google.protobuf.Int64Value Limits the number of bytes in the result. This limitation is also checked for subqueries and parts of distributed queries that run on remote servers. Minimal value and default value: 0, no limitation is set. The minimum value is 0. |
result_overflow_mode | enum OverflowMode Determines the behavior on exceeding limits
|
max_rows_in_distinct | google.protobuf.Int64Value Limits the maximum number of different rows when using DISTINCT. Minimal value and default value: 0, no limitation is set. The minimum value is 0. |
max_bytes_in_distinct | google.protobuf.Int64Value Limits the maximum size of a hash table in bytes (uncompressed data) when using DISTINCT. The minimum value is 0. |
distinct_overflow_mode | enum OverflowMode Determines the behavior on exceeding limits
|
max_rows_to_transfer | google.protobuf.Int64Value Limits the maximum number of rows that can be passed to a remote server or saved in a temporary table when using GLOBAL IN. Minimal value and default value: 0, no limitation is set. The minimum value is 0. |
max_bytes_to_transfer | google.protobuf.Int64Value Limits the maximum number of bytes (uncompressed data) that can be passed to a remote server or saved in a temporary table when using GLOBAL IN. Minimal value and default value: 0, no limitation is set. The minimum value is 0. |
transfer_overflow_mode | enum OverflowMode Determines the behavior on exceeding limits
|
max_execution_time | google.protobuf.Int64Value Limits the maximum query execution time in milliseconds. At this moment, this limitation is not checked when passing one of the sorting stages, as well as merging and finalizing aggregation funictions. Minimal value and default value: 0, no limitation is set. The minimum value is 0. |
timeout_overflow_mode | enum OverflowMode Determines the behavior on exceeding limits
|
max_rows_in_set | google.protobuf.Int64Value Limit on the number of rows in the set resulting from the execution of the IN section. The minimum value is 0. |
max_bytes_in_set | google.protobuf.Int64Value Limit on the number of bytes in the set resulting from the execution of the IN section. The minimum value is 0. |
set_overflow_mode | enum OverflowMode Determine the behavior on exceeding max_rows_in_set or max_bytes_in_set limit. Possible values: OVERFLOW_MODE_THROW, OVERFLOW_MODE_BREAK. |
max_rows_in_join | google.protobuf.Int64Value Limit on maximum size of the hash table for JOIN, in rows. The minimum value is 0. |
max_bytes_in_join | google.protobuf.Int64Value Limit on maximum size of the hash table for JOIN, in bytes. The minimum value is 0. |
join_overflow_mode | enum OverflowMode Determine the behavior on exceeding max_rows_in_join or max_bytes_in_join limit. Possible values: OVERFLOW_MODE_THROW, OVERFLOW_MODE_BREAK. |
join_algorithm[] | enum JoinAlgorithm See in-depth description in ClickHouse documentation |
any_join_distinct_right_table_keys | google.protobuf.BoolValue See in-depth description in ClickHouse documentation |
max_columns_to_read | google.protobuf.Int64Value Limits the maximum number of columns that can be read from a table in a single query. If the query requires to read more columns to complete, then it will be aborted. Minimal value and default value: 0, no limitation is set. The minimum value is 0. |
max_temporary_columns | google.protobuf.Int64Value Limits the maximum number of temporary columns that must be kept in RAM at the same time when running a query, including constant columns. Minimal value and default value: 0, no limitation is set. The minimum value is 0. |
max_temporary_non_const_columns | google.protobuf.Int64Value Limits the maximum number of temporary columns that must be kept in RAM at the same time when running a query, excluding constant columns. Minimal value and default value: 0, no limitation is set. The minimum value is 0. |
max_query_size | google.protobuf.Int64Value Limits the size of the part of a query that can be transferred to RAM for parsing with the SQL parser, in bytes. Value must be greater than 0 (default: 262144). See in-depth description in ClickHouse documentation |
max_ast_depth | google.protobuf.Int64Value Limits the maximum depth of query syntax tree. Executing a big and complex query may result in building a syntax tree of enormous depth. By using this setting, you can prohibit execution of over-sized or non-optimized queries for huge tables. For example, the **SELECT *** query may result in more complex and deeper syntax tree, compared to the SELECT ... WHERE ... query, containing constraints and conditions, in the most cases. A user can be forced to construct more optimized queries, if this setting is used. Value must be greater than 0 (default: 1000). If a too small value is set, it may render ClickHouse unable to execute even simple queries. See in-depth description in ClickHouse documentation |
max_ast_elements | google.protobuf.Int64Value Limits the maximum size of query syntax tree in number of nodes. Executing a big and complex query may result in building a syntax tree of enormous size. By using this setting, you can prohibit execution of over-sized or non-optimized queries for huge tables. Value must be greater than 0 (default: 50000). If a too small value is set, it may render ClickHouse unable to execute even simple queries. See in-depth description in ClickHouse documentation |
max_expanded_ast_elements | google.protobuf.Int64Value Limits the maximum size of query syntax tree in number of nodes after expansion of aliases and the asterisk values. Executing a big and complex query may result in building a syntax tree of enormous size. By using this setting, you can prohibit execution of over-sized or non-optimized queries for huge tables. Value must be greater than 0 (default: 500000). If a too small value is set, it may render ClickHouse unable to execute even simple queries. Value must be greater than 0. |
min_execution_speed | google.protobuf.Int64Value Minimal execution speed in rows per second. The minimum value is 0. |
min_execution_speed_bytes | google.protobuf.Int64Value Minimal execution speed in bytes per second. The minimum value is 0. |
count_distinct_implementation | enum CountDistinctImplementation Aggregate function to use for implementation of count(DISTINCT ...). |
input_format_values_interpret_expressions | google.protobuf.BoolValue Enables or disables SQL parser if the fast stream parser cannot parse the data. Enable this setting, if the data that you want to insert into a table contains SQL expressions. For example, the stream parser is unable to parse a value that contains now() expression; therefore an INSERT query for this value will fail and no data will be inserted into a table. With enabled SQL parser, this expression is parsed correctly: the now() expression will be parsed as SQL function, interpreted, and the current date and time will be inserted into the table as a result. This setting has effect only if you use Values Default value: true (SQL parser is enabled). See in-depth description in ClickHouse documentation |
input_format_defaults_for_omitted_fields | google.protobuf.BoolValue Enables or disables replacing omitted input values with default values of the respective columns when performing INSERT queries. Default value: true (replacing is enabled). |
input_format_null_as_default | google.protobuf.BoolValue See in-depth description in ClickHouse documentation |
date_time_input_format | enum DateTimeInputFormat See in-depth description in ClickHouse documentation |
input_format_with_names_use_header | google.protobuf.BoolValue See in-depth description in ClickHouse documentation |
output_format_json_quote_64bit_integers | google.protobuf.BoolValue Enables quoting of 64-bit integers in JSON output format. If this setting is enabled, then 64-bit integers (UInt64 and Int64) will be quoted when written to JSON output in order to maintain compatibility with the most of the JavaScript engines. Otherwise, such integers will not be quoted. Default value: false (quoting 64-bit integers is disabled). |
output_format_json_quote_denormals | google.protobuf.BoolValue Enables special floating-point values (+nan, -nan, +inf and -inf) in JSON output format. Default value: false (special values do not present in output). |
date_time_output_format | enum DateTimeOutputFormat See in-depth description in ClickHouse documentation |
low_cardinality_allow_in_native_format | google.protobuf.BoolValue Determines whether to use LowCardinality type in Native format.
LowCardinality columns (aka sparse columns) store data in more effective way, compared to regular columns, by using hash tables. If data to insert suits this storage format, ClickHouse will place them into LowCardinality column. If you use a third-party ClickHouse client that can't work with LowCardinality columns, then this client will not be able to correctly interpret the result of the query that asks for data stored in LowCardinality column. Disable this setting to convert LowCardinality column to regular column when creating the result, so such clients will be able to process the result. Official ClickHouse client works with LowCardinality columns out-of-the-box. Default value: true (LowCardinality columns are used in Native format). |
allow_suspicious_low_cardinality_types | google.protobuf.BoolValue Allows specifying LowCardinality modifier for types of small fixed size (8 or less) in CREATE TABLE statements. Enabling this may increase merge times and memory consumption. More info see in ClickHouse documentation |
empty_result_for_aggregation_by_empty_set | google.protobuf.BoolValue Enables returning of empty result when aggregating without keys (with GROUP BY operation absent) on empty set (e.g., SELECT count(*) FROM table WHERE 0).
|
http_connection_timeout | google.protobuf.Int64Value HTTP connection timeout, in milliseconds. Value must be greater than 0 (default: 1000, 1 second). |
http_receive_timeout | google.protobuf.Int64Value HTTP receive timeout, in milliseconds. Value must be greater than 0 (default: 1800000, 1800 seconds, 30 minutes). |
http_send_timeout | google.protobuf.Int64Value HTTP send timeout, in milliseconds. Value must be greater than 0 (default: 1800000, 1800 seconds, 30 minutes). |
enable_http_compression | google.protobuf.BoolValue Enables or disables data compression in HTTP responses. By default, ClickHouse stores data compressed. When executing a query, its result is uncompressed. Use this setting to command ClickHouse to compress the result when sending it via HTTP. Enable this setting and add the Accept-Encoding: ClickHouse support the following compression methods: gzip, br and deflate. Default value: false (compression is disabled). See in-depth description in ClickHouse documentation |
send_progress_in_http_headers | google.protobuf.BoolValue Enables progress notifications using X-ClickHouse-Progress HTTP header. Default value: false (notifications disabled). |
http_headers_progress_interval | google.protobuf.Int64Value Minimum interval between progress notifications with X-ClickHouse-Progress HTTP header, in milliseconds. Value must be greater than 0 (default: 100). |
add_http_cors_header | google.protobuf.BoolValue Adds CORS header in HTTP responses. Default value: false (header is not added). |
cancel_http_readonly_queries_on_client_close | google.protobuf.BoolValue Cancels HTTP read-only queries (e.g. SELECT) when a client closes the connection without waiting for the response. Default value: false. More info see in ClickHouse documentation |
max_http_get_redirects | google.protobuf.Int64Value Limits the maximum number of HTTP GET redirect hops for URL-engine If the parameter is set to 0 (default), no hops is allowed. More info see in ClickHouse documentation |
joined_subquery_requires_alias | google.protobuf.BoolValue |
join_use_nulls | google.protobuf.BoolValue |
transform_null_in | google.protobuf.BoolValue |
quota_mode | enum QuotaMode Quota accounting mode. Possible values: QUOTA_MODE_DEFAULT, QUOTA_MODE_KEYED and QUOTA_MODE_KEYED_BY_IP. |
flatten_nested | google.protobuf.BoolValue Sets the data format of a nested More info see in ClickHouse documentation |
format_regexp | string Regular expression (for Regexp format) |
format_regexp_escaping_rule | enum FormatRegexpEscapingRule See in-depth description in ClickHouse documentation |
format_regexp_skip_unmatched | google.protobuf.BoolValue See in-depth description in ClickHouse documentation |
async_insert | google.protobuf.BoolValue Enables asynchronous inserts. Disabled by default. More info see in ClickHouse documentation |
async_insert_threads | google.protobuf.Int64Value The maximum number of threads for background data parsing and insertion. If the parameter is set to 0, asynchronous insertions are disabled. Default value: 16. More info see in ClickHouse documentation |
wait_for_async_insert | google.protobuf.BoolValue Enables waiting for processing of asynchronous insertion. If enabled, server returns OK only after the data is inserted. More info see in ClickHouse documentation |
wait_for_async_insert_timeout | google.protobuf.Int64Value The timeout (in seconds) for waiting for processing of asynchronous insertion. Default value: 120. More info see in ClickHouse documentation |
async_insert_max_data_size | google.protobuf.Int64Value The maximum size of the unparsed data in bytes collected per query before being inserted. If the parameter is set to 0, asynchronous insertions are disabled. Default value: 100000. More info see in ClickHouse documentation |
async_insert_busy_timeout | google.protobuf.Int64Value The maximum timeout in milliseconds since the first INSERT query before inserting collected data. If the parameter is set to 0, the timeout is disabled. Default value: 200. More info see in ClickHouse documentation |
async_insert_stale_timeout | google.protobuf.Int64Value The maximum timeout in milliseconds since the last INSERT query before dumping collected data. If enabled, the settings prolongs the async_insert_busy_timeout with every INSERT query as long as async_insert_max_data_size is not exceeded. More info see in ClickHouse documentation |
memory_profiler_step | google.protobuf.Int64Value Memory profiler step (in bytes). If the next query step requires more memory than this parameter specifies, the memory profiler collects the allocating stack trace. Values lower than a few megabytes slow down query processing. Default value: 4194304 (4 MB). Zero means disabled memory profiler. |
memory_profiler_sample_probability | google.protobuf.DoubleValue Collect random allocations and deallocations and write them into system.trace_log with 'MemorySample' trace_type. The probability is for every alloc/free regardless to the size of the allocation. Possible values: from 0 to 1. Default: 0. |
max_final_threads | google.protobuf.Int64Value Sets the maximum number of parallel threads for the SELECT query data read phase with the FINAL modifier. See in-depth description in ClickHouse documentation |
input_format_parallel_parsing | google.protobuf.BoolValue Enables or disables order-preserving parallel parsing of data formats. Supported only for TSV |
input_format_import_nested_json | google.protobuf.BoolValue Enables or disables the insertion of JSON data with nested objects. See in-depth description in ClickHouse documentation |
local_filesystem_read_method | enum LocalFilesystemReadMethod Method of reading data from local filesystem, one of: read, pread, mmap, io_uring, pread_threadpool. The 'io_uring' method is experimental and does not work for Log, TinyLog, StripeLog, File, Set and Join, and other tables with append-able files in presence of concurrent reads and writes. |
max_read_buffer_size | google.protobuf.Int64Value The maximum size of the buffer to read from the filesystem. See in-depth description in ClickHouse documentation |
insert_keeper_max_retries | google.protobuf.Int64Value The setting sets the maximum number of retries for ClickHouse Keeper (or ZooKeeper) requests during insert into replicated MergeTree. Only Keeper requests which failed due to network error, Keeper session timeout, or request timeout are considered for retries. Default: 20 from 23.2, 0(disabled) before Min_version: 22.11 See in-depth description in ClickHouse documentation |
max_temporary_data_on_disk_size_for_user | google.protobuf.Int64Value The maximum amount of data consumed by temporary files on disk in bytes for all concurrently running user queries. Zero means unlimited. Default: 0 - unlimited Min_version: 22.10 See in-depth description in ClickHouse documentation |
max_temporary_data_on_disk_size_for_query | google.protobuf.Int64Value The maximum amount of data consumed by temporary files on disk in bytes for all concurrently running queries. Zero means unlimited. Default: 0 - unlimited Min_version: 22.10 See in-depth description in ClickHouse documentation |
max_parser_depth | google.protobuf.Int64Value Limits maximum recursion depth in the recursive descent parser. Allows controlling the stack size. Default: 1000 Special: 0 - unlimited See in-depth description in ClickHouse documentation |
remote_filesystem_read_method | enum RemoteFilesystemReadMethod Method of reading data from remote filesystem, one of: read, threadpool. Default: read Min_version: 21.11 See in-depth description in ClickHouse GitHub |
memory_overcommit_ratio_denominator | google.protobuf.Int64Value It represents soft memory limit in case when hard limit is reached on user level. This value is used to compute overcommit ratio for the query. Zero means skip the query. Default: 1GiB Min_version: 22.5 See in-depth description in ClickHouse documentation |
memory_overcommit_ratio_denominator_for_user | google.protobuf.Int64Value It represents soft memory limit in case when hard limit is reached on global level. This value is used to compute overcommit ratio for the query. Zero means skip the query. Default: 1GiB Min_version: 22.5 See in-depth description in ClickHouse documentation |
memory_usage_overcommit_max_wait_microseconds | google.protobuf.Int64Value Maximum time thread will wait for memory to be freed in the case of memory overcommit on a user level. If the timeout is reached and memory is not freed, an exception is thrown. Default: 5000000 Min_version: 22.5 See in-depth description in ClickHouse documentation |
compile | google.protobuf.BoolValue The setting is deprecated and has no effect. |
min_count_to_compile | google.protobuf.Int64Value The setting is deprecated and has no effect. |
UserQuota
Field | Description |
---|---|
interval_duration | google.protobuf.Int64Value Duration of interval for quota in milliseconds. Minimal value is 1 second. The minimum value is 1000. |
queries | google.protobuf.Int64Value The total number of queries. 0 - unlimited. The minimum value is 0. |
errors | google.protobuf.Int64Value The number of queries that threw exception. 0 - unlimited. The minimum value is 0. |
result_rows | google.protobuf.Int64Value The total number of rows given as the result.. 0 - unlimited. The minimum value is 0. |
read_rows | google.protobuf.Int64Value The total number of source rows read from tables for running the query, on all remote servers. 0 - unlimited. The minimum value is 0. |
execution_time | google.protobuf.Int64Value The total query execution time, in milliseconds (wall time). 0 - unlimited. The minimum value is 0. |
HostSpec
Field | Description |
---|---|
zone_id | string ID of the availability zone where the host resides. To get a list of available zones, use the yandex.cloud.compute.v1.ZoneService.List request. The maximum string length in characters is 50. |
type | Host.Type Required. Type of the host to be deployed. |
subnet_id | string ID of the subnet that the host should belong to. This subnet should be a part of the network that the cluster belongs to. The ID of the network is set in the Cluster.network_id field. The maximum string length in characters is 50. |
assign_public_ip | bool Whether the host should get a public IP address on creation. After a host has been created, this setting cannot be changed. To remove an assigned public IP, or to assign a public IP to a host without one, recreate the host with assign_public_ip set as needed. Possible values:
|
shard_name | string Name of the shard that the host is assigned to. The maximum string length in characters is 63. Value must match the regular expression [a-zA-Z0-9_-]* . |
MaintenanceWindow
Field | Description |
---|---|
policy | oneof: anytime or weekly_maintenance_window The maintenance policy in effect. |
anytime | AnytimeMaintenanceWindow Maintenance operation can be scheduled anytime. |
weekly_maintenance_window | WeeklyMaintenanceWindow Maintenance operation can be scheduled on a weekly basis. |
AnytimeMaintenanceWindow
Empty.
WeeklyMaintenanceWindow
Field | Description |
---|---|
day | enum WeekDay Day of the week (in DDD format). |
hour | int64 Hour of the day in UTC (in HH format). Acceptable values are 1 to 24, inclusive. |
Operation
Field | Description |
---|---|
id | string ID of the operation. |
description | string Description of the operation. 0-256 characters long. |
created_at | google.protobuf.Timestamp Creation timestamp. |
created_by | string ID of the user or service account who initiated the operation. |
modified_at | google.protobuf.Timestamp The time when the Operation resource was last modified. |
done | bool If the value is false , it means the operation is still in progress. If true , the operation is completed, and either error or response is available. |
metadata | google.protobuf.Any Service-specific metadata associated with the operation. It typically contains the ID of the target resource that the operation is performed on. Any method that returns a long-running operation should document the metadata type, if any. |
result | oneof: error or response The operation result. If done == false and there was no failure detected, neither error nor response is set. If done == false and there was a failure detected, error is set. If done == true , exactly one of error or response is set. |
error | google.rpc.Status The error result of the operation in case of failure or cancellation. |
response | google.protobuf.Any if operation finished successfully. |
CreateClusterMetadata
Field | Description |
---|---|
cluster_id | string ID of the ClickHouse cluster that is being created. |
Cluster
Field | Description |
---|---|
id | string ID of the ClickHouse cluster. This ID is assigned by MDB at creation time. |
folder_id | string ID of the folder that the ClickHouse cluster belongs to. |
created_at | google.protobuf.Timestamp Creation timestamp in RFC3339 |
name | string Name of the ClickHouse cluster. The name is unique within the folder. 1-63 characters long. |
description | string Description of the ClickHouse cluster. 0-256 characters long. |
labels | map<string,string> Custom labels for the ClickHouse cluster as key:value pairs. Maximum 64 per resource. |
environment | enum Environment Deployment environment of the ClickHouse cluster.
|
monitoring[] | Monitoring Description of monitoring systems relevant to the ClickHouse cluster. |
config | ClusterConfig Configuration of the ClickHouse cluster. |
network_id | string ID of the network that the cluster belongs to. |
health | enum Health Aggregated cluster health.
|
status | enum Status Current state of the cluster.
|
service_account_id | string ID of the service account used for access to Object Storage. |
maintenance_window | MaintenanceWindow Maintenance window for the cluster. |
planned_operation | MaintenanceOperation Planned maintenance operation to be started for the cluster within the nearest maintenance_window . |
security_group_ids[] | string User security groups |
deletion_protection | bool Deletion Protection inhibits deletion of the cluster |
Monitoring
Field | Description |
---|---|
name | string Name of the monitoring system. |
description | string Description of the monitoring system. |
link | string Link to the monitoring system charts for the ClickHouse cluster. |
ClusterConfig
Field | Description |
---|---|
version | string Version of the ClickHouse server software. |
clickhouse | Clickhouse Configuration and resource allocation for ClickHouse hosts. |
zookeeper | Zookeeper Configuration and resource allocation for ZooKeeper hosts. |
backup_window_start | google.type.TimeOfDay Time to start the daily backup, in the UTC timezone. |
access | Access Access policy for external services. |
cloud_storage | CloudStorage |
sql_database_management | google.protobuf.BoolValue Whether database management through SQL commands is enabled. |
sql_user_management | google.protobuf.BoolValue Whether user management through SQL commands is enabled. |
embedded_keeper | google.protobuf.BoolValue Whether cluster should use embedded Keeper instead of Zookeeper. |
Clickhouse
Field | Description |
---|---|
config | config.ClickhouseConfigSet Configuration settings of a ClickHouse server. |
resources | Resources Resources allocated to ClickHouse hosts. |
Zookeeper
Field | Description |
---|---|
resources | Resources Resources allocated to ZooKeeper hosts. |
ClickhouseConfigSet
Field | Description |
---|---|
effective_config | ClickhouseConfig Required. Effective settings for a ClickHouse cluster (a combination of settings defined in user_config and default_config ). |
user_config | ClickhouseConfig User-defined settings for a ClickHouse cluster. |
default_config | ClickhouseConfig Default configuration for a ClickHouse cluster. |
ClickhouseConfig
Field | Description |
---|---|
log_level | enum LogLevel Logging level for the ClickHouse cluster. Possible values: TRACE, DEBUG, INFORMATION, WARNING, ERROR. |
merge_tree | MergeTree Settings for the MergeTree engine. See description in ClickHouse documentation |
compression[] | Compression Compression settings for the ClickHouse cluster. See in-depth description in ClickHouse documentation |
dictionaries[] | ExternalDictionary Configuration of external dictionaries to be used by the ClickHouse cluster. See in-depth description in ClickHouse documentation |
graphite_rollup[] | GraphiteRollup Settings for thinning Graphite data. See in-depth description in ClickHouse documentation |
kafka | Kafka |
kafka_topics[] | KafkaTopic |
rabbitmq | Rabbitmq |
max_connections | google.protobuf.Int64Value Maximum number of inbound connections. The minimum value is 10. |
max_concurrent_queries | google.protobuf.Int64Value Maximum number of simultaneously processed requests. The minimum value is 10. |
keep_alive_timeout | google.protobuf.Int64Value Number of milliseconds that ClickHouse waits for incoming requests before closing the connection. |
uncompressed_cache_size | google.protobuf.Int64Value Cache size (in bytes) for uncompressed data used by MergeTree tables. |
mark_cache_size | google.protobuf.Int64Value Approximate size (in bytes) of the cache of "marks" used by MergeTree tables. Value must be greater than 0. |
max_table_size_to_drop | google.protobuf.Int64Value Maximum size of the table that can be deleted using a DROP query. |
max_partition_size_to_drop | google.protobuf.Int64Value Maximum size of the partition that can be deleted using a DROP query. |
builtin_dictionaries_reload_interval | google.protobuf.Int64Value The setting is deprecated and has no effect. |
timezone | string The server's time zone to be used in DateTime fields conversions. Specified as an IANA identifier. |
geobase_enabled | google.protobuf.BoolValue Enable or disable geobase. |
geobase_uri | string Address of the archive with the user geobase in Object Storage. |
query_log_retention_size | google.protobuf.Int64Value The maximum size that query_log can grow to before old data will be removed. If set to 0, automatic removal of query_log data based on size is disabled. |
query_log_retention_time | google.protobuf.Int64Value The maximum time that query_log records will be retained before removal. If set to 0, automatic removal of query_log data based on time is disabled. |
query_thread_log_enabled | google.protobuf.BoolValue Whether query_thread_log system table is enabled. |
query_thread_log_retention_size | google.protobuf.Int64Value The maximum size that query_thread_log can grow to before old data will be removed. If set to 0, automatic removal of query_thread_log data based on size is disabled. |
query_thread_log_retention_time | google.protobuf.Int64Value The maximum time that query_thread_log records will be retained before removal. If set to 0, automatic removal of query_thread_log data based on time is disabled. |
part_log_retention_size | google.protobuf.Int64Value The maximum size that part_log can grow to before old data will be removed. If set to 0, automatic removal of part_log data based on size is disabled. |
part_log_retention_time | google.protobuf.Int64Value The maximum time that part_log records will be retained before removal. If set to 0, automatic removal of part_log data based on time is disabled. |
metric_log_enabled | google.protobuf.BoolValue Whether metric_log system table is enabled. |
metric_log_retention_size | google.protobuf.Int64Value The maximum size that metric_log can grow to before old data will be removed. If set to 0, automatic removal of metric_log data based on size is disabled. |
metric_log_retention_time | google.protobuf.Int64Value The maximum time that metric_log records will be retained before removal. If set to 0, automatic removal of metric_log data based on time is disabled. |
trace_log_enabled | google.protobuf.BoolValue Whether trace_log system table is enabled. |
trace_log_retention_size | google.protobuf.Int64Value The maximum size that trace_log can grow to before old data will be removed. If set to 0, automatic removal of trace_log data based on size is disabled. |
trace_log_retention_time | google.protobuf.Int64Value The maximum time that trace_log records will be retained before removal. If set to 0, automatic removal of trace_log data based on time is disabled. |
text_log_enabled | google.protobuf.BoolValue Whether text_log system table is enabled. |
text_log_retention_size | google.protobuf.Int64Value The maximum size that text_log can grow to before old data will be removed. If set to 0, automatic removal of text_log data based on size is disabled. |
text_log_retention_time | google.protobuf.Int64Value The maximum time that text_log records will be retained before removal. If set to 0, automatic removal of text_log data based on time is disabled. |
text_log_level | enum LogLevel Logging level for text_log system table. Possible values: TRACE, DEBUG, INFORMATION, WARNING, ERROR. |
opentelemetry_span_log_enabled | google.protobuf.BoolValue Enable or disable opentelemetry_span_log system table. Default value: false. |
opentelemetry_span_log_retention_size | google.protobuf.Int64Value The maximum size that opentelemetry_span_log can grow to before old data will be removed. If set to 0 (default), automatic removal of opentelemetry_span_log data based on size is disabled. The minimum value is 0. |
opentelemetry_span_log_retention_time | google.protobuf.Int64Value The maximum time that opentelemetry_span_log records will be retained before removal. If set to 0, automatic removal of opentelemetry_span_log data based on time is disabled. The minimum value is 0. |
query_views_log_enabled | google.protobuf.BoolValue Enable or disable query_views_log system table. Default value: false. |
query_views_log_retention_size | google.protobuf.Int64Value The maximum size that query_views_log can grow to before old data will be removed. If set to 0 (default), automatic removal of query_views_log data based on size is disabled. The minimum value is 0. |
query_views_log_retention_time | google.protobuf.Int64Value The maximum time that query_views_log records will be retained before removal. If set to 0, automatic removal of query_views_log data based on time is disabled. The minimum value is 0. |
asynchronous_metric_log_enabled | google.protobuf.BoolValue Enable or disable asynchronous_metric_log system table. Default value: false. |
asynchronous_metric_log_retention_size | google.protobuf.Int64Value The maximum size that asynchronous_metric_log can grow to before old data will be removed. If set to 0 (default), automatic removal of asynchronous_metric_log data based on size is disabled. The minimum value is 0. |
asynchronous_metric_log_retention_time | google.protobuf.Int64Value The maximum time that asynchronous_metric_log records will be retained before removal. If set to 0, automatic removal of asynchronous_metric_log data based on time is disabled. The minimum value is 0. |
session_log_enabled | google.protobuf.BoolValue Enable or disable session_log system table. Default value: false. |
session_log_retention_size | google.protobuf.Int64Value The maximum size that session_log can grow to before old data will be removed. If set to 0 (default), automatic removal of session_log data based on size is disabled. The minimum value is 0. |
session_log_retention_time | google.protobuf.Int64Value The maximum time that session_log records will be retained before removal. If set to 0, automatic removal of session_log data based on time is disabled. The minimum value is 0. |
zookeeper_log_enabled | google.protobuf.BoolValue Enable or disable zookeeper_log system table. Default value: false. |
zookeeper_log_retention_size | google.protobuf.Int64Value The maximum size that zookeeper_log can grow to before old data will be removed. If set to 0 (default), automatic removal of zookeeper_log data based on size is disabled. The minimum value is 0. |
zookeeper_log_retention_time | google.protobuf.Int64Value The maximum time that zookeeper_log records will be retained before removal. If set to 0, automatic removal of zookeeper_log data based on time is disabled. The minimum value is 0. |
asynchronous_insert_log_enabled | google.protobuf.BoolValue Enable or disable asynchronous_insert_log system table. Default value: false. Minimal required ClickHouse version: 22.10. |
asynchronous_insert_log_retention_size | google.protobuf.Int64Value The maximum size that asynchronous_insert_log can grow to before old data will be removed. If set to 0 (default), automatic removal of asynchronous_insert_log data based on size is disabled. The minimum value is 0. |
asynchronous_insert_log_retention_time | google.protobuf.Int64Value The maximum time that asynchronous_insert_log records will be retained before removal. If set to 0, automatic removal of asynchronous_insert_log data based on time is disabled. The minimum value is 0. |
background_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_merges_mutations_concurrency_ratio | google.protobuf.Int64Value Sets a ratio between the number of threads and the number of background merges and mutations that can be executed concurrently. For example, if the ratio equals to 2 and background_pool_size is set to 16 then ClickHouse can execute 32 background merges concurrently. This is possible, because background operations could be suspended and postponed. This is needed to give small merges more execution priority. You can only increase this ratio at runtime. To lower it you have to restart the server. The same as for background_pool_size setting background_merges_mutations_concurrency_ratio could be applied from the default profile for backward compatibility. Default: 2 See in-depth description in ClickHouse documentation |
background_schedule_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_fetches_pool_size | google.protobuf.Int64Value Sets the number of threads performing background fetches for tables with ReplicatedMergeTree engines. Default value: 8. More info see in ClickHouse documentation |
background_move_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_distributed_schedule_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_buffer_flush_schedule_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_message_broker_schedule_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_common_pool_size | google.protobuf.Int64Value The maximum number of threads that will be used for performing a variety of operations (mostly garbage collection) for *MergeTree-engine tables in a background. Default: 8 See in-depth description in ClickHouse documentation |
default_database | google.protobuf.StringValue The default database. To get a list of cluster databases, see Yandex Managed ClickHouse documentation. |
total_memory_profiler_step | google.protobuf.Int64Value Sets the memory size (in bytes) for a stack trace at every peak allocation step. Default value: 4194304. More info see in ClickHouse documentation |
total_memory_tracker_sample_probability | google.protobuf.DoubleValue |
MergeTree
Field | Description |
---|---|
replicated_deduplication_window | google.protobuf.Int64Value Number of blocks of hashes to keep in ZooKeeper. |
replicated_deduplication_window_seconds | google.protobuf.Int64Value Period of time to keep blocks of hashes for. |
parts_to_delay_insert | google.protobuf.Int64Value If table contains at least that many active parts in single partition, artificially slow down insert into table. |
parts_to_throw_insert | google.protobuf.Int64Value If more than this number active parts in single partition, throw 'Too many parts ...' exception. |
inactive_parts_to_delay_insert | google.protobuf.Int64Value |
inactive_parts_to_throw_insert | google.protobuf.Int64Value |
max_replicated_merges_in_queue | google.protobuf.Int64Value How many tasks of merging and mutating parts are allowed simultaneously in ReplicatedMergeTree queue. |
number_of_free_entries_in_pool_to_lower_max_size_of_merge | google.protobuf.Int64Value If there is less than specified number of free entries in background pool (or replicated queue), start to lower maximum size of merge to process. |
max_bytes_to_merge_at_min_space_in_pool | google.protobuf.Int64Value Maximum in total size of parts to merge, when there are minimum free threads in background pool (or entries in replication queue). |
max_bytes_to_merge_at_max_space_in_pool | google.protobuf.Int64Value |
min_bytes_for_wide_part | google.protobuf.Int64Value Minimum number of bytes in a data part that can be stored in Wide format. More info see in ClickHouse documentation |
min_rows_for_wide_part | google.protobuf.Int64Value Minimum number of rows in a data part that can be stored in Wide format. More info see in ClickHouse documentation |
ttl_only_drop_parts | google.protobuf.BoolValue Enables or disables complete dropping of data parts where all rows are expired in MergeTree tables. More info see in ClickHouse documentation |
allow_remote_fs_zero_copy_replication | google.protobuf.BoolValue |
merge_with_ttl_timeout | google.protobuf.Int64Value |
merge_with_recompression_ttl_timeout | google.protobuf.Int64Value |
max_parts_in_total | google.protobuf.Int64Value |
max_number_of_merges_with_ttl_in_pool | google.protobuf.Int64Value |
cleanup_delay_period | google.protobuf.Int64Value |
number_of_free_entries_in_pool_to_execute_mutation | google.protobuf.Int64Value |
max_avg_part_size_for_too_many_parts | google.protobuf.Int64Value The 'too many parts' check according to 'parts_to_delay_insert' and 'parts_to_throw_insert' will be active only if the average part size (in the relevant partition) is not larger than the specified threshold. If it is larger than the specified threshold, the INSERTs will be neither delayed or rejected. This allows to have hundreds of terabytes in a single table on a single server if the parts are successfully merged to larger parts. This does not affect the thresholds on inactive parts or total parts. Default: 1 GiB Min version: 22.10 See in-depth description in ClickHouse GitHub |
min_age_to_force_merge_seconds | google.protobuf.Int64Value Merge parts if every part in the range is older than the value of min_age_to_force_merge_seconds. Default: 0 - disabled Min_version: 22.10 See in-depth description in ClickHouse documentation |
min_age_to_force_merge_on_partition_only | google.protobuf.BoolValue Whether min_age_to_force_merge_seconds should be applied only on the entire partition and not on subset. Default: false Min_version: 22.11 See in-depth description in ClickHouse documentation |
merge_selecting_sleep_ms | google.protobuf.Int64Value Sleep time for merge selecting when no part is selected. A lower setting triggers selecting tasks in background_schedule_pool frequently, which results in a large number of requests to ClickHouse Keeper in large-scale clusters. Default: 5000 Min_version: 21.10 See in-depth description in ClickHouse documentation |
Kafka
Field | Description |
---|---|
security_protocol | enum SecurityProtocol |
sasl_mechanism | enum SaslMechanism |
sasl_username | string |
sasl_password | string |
enable_ssl_certificate_verification | google.protobuf.BoolValue |
max_poll_interval_ms | google.protobuf.Int64Value The minimum value is 0. |
session_timeout_ms | google.protobuf.Int64Value The minimum value is 0. |
KafkaTopic
Field | Description |
---|---|
name | string Required. |
settings | Kafka Required. |
Rabbitmq
Field | Description |
---|---|
username | string RabbitMQ |
password | string RabbitMQ |
vhost | string RabbitMQ |
Compression
Field | Description |
---|---|
method | enum Method Compression method to use for the specified combination of min_part_size and min_part_size_ratio . |
min_part_size | int64 Minimum size of a part of a table. The minimum value is 1. |
min_part_size_ratio | double Minimum ratio of a part relative to the size of all the data in the table. |
level | google.protobuf.Int64Value The minimum value is 0. |
ExternalDictionary
Field | Description |
---|---|
name | string Required. Name of the external dictionary. |
structure | Structure Required. Set of attributes for the external dictionary. For in-depth description, see ClickHouse documentation |
layout | Layout Required. Layout for storing the dictionary in memory. For in-depth description, see ClickHouse documentation |
lifetime | oneof: fixed_lifetime or lifetime_range Setting for the period of time between dictionary updates. For details, see ClickHouse documentation |
fixed_lifetime | int64 Fixed interval between dictionary updates. |
lifetime_range | Range Range of intervals between dictionary updates for ClickHouse to choose from. |
source | oneof: http_source , mysql_source , clickhouse_source , mongodb_source or postgresql_source Description of the source for the external dictionary. |
http_source | HttpSource HTTP source for the dictionary. |
mysql_source | MysqlSource MySQL source for the dictionary. |
clickhouse_source | ClickhouseSource ClickHouse source for the dictionary. |
mongodb_source | MongodbSource MongoDB source for the dictionary. |
postgresql_source | PostgresqlSource PostgreSQL source for the dictionary. |
HttpSource
Field | Description |
---|---|
url | string Required. URL of the source dictionary available over HTTP. |
format | string Required. The data format. Valid values are all formats supported by ClickHouse SQL dialect. |
MysqlSource
Field | Description |
---|---|
db | string Required. Name of the MySQL database to connect to. |
table | string Required. Name of the database table to use as a ClickHouse dictionary. |
port | int64 Default port to use when connecting to a replica of the dictionary source. Acceptable values are 0 to 65535, inclusive. |
user | string Name of the default user for replicas of the dictionary source. |
password | string Password of the default user for replicas of the dictionary source. |
replicas[] | Replica List of MySQL replicas of the database used as dictionary source. The number of elements must be greater than 0. |
where | string Selection criteria for the data in the specified MySQL table. |
invalidate_query | string Query for checking the dictionary status, to pull only updated data. For more details, see ClickHouse documentation on dictionaries |
Replica
Field | Description |
---|---|
host | string Required. MySQL host of the replica. The maximum string length in characters is 253. |
priority | int64 Required. The priority of the replica that ClickHouse takes into account when connecting. Replica with the highest priority should have this field set to the lowest number. Value must be greater than 0. |
port | int64 Port to use when connecting to the replica. If a port is not specified for a replica, ClickHouse uses the port specified for the source. Acceptable values are 0 to 65535, inclusive. |
user | string Name of the MySQL database user. |
password | string Password of the MySQL database user. |
ClickhouseSource
Field | Description |
---|---|
db | string Required. Name of the ClickHouse database. |
table | string Required. Name of the table in the specified database to be used as the dictionary source. |
host | string Required. ClickHouse host of the specified database. The maximum string length in characters is 253. |
port | int64 Port to use when connecting to the host. Acceptable values are 0 to 65535, inclusive. |
user | string Required. Name of the ClickHouse database user. |
password | string Password of the ClickHouse database user. |
where | string Selection criteria for the data in the specified ClickHouse table. |
MongodbSource
Field | Description |
---|---|
db | string Required. Name of the MongoDB database. |
collection | string Required. Name of the collection in the specified database to be used as the dictionary source. |
host | string Required. MongoDB host of the specified database. The maximum string length in characters is 253. |
port | int64 Port to use when connecting to the host. Acceptable values are 0 to 65535, inclusive. |
user | string Required. Name of the MongoDB database user. |
password | string Password of the MongoDB database user. |
options | string |
PostgresqlSource
Field | Description |
---|---|
db | string Required. Name of the PostrgreSQL database. |
table | string Required. Name of the table in the specified database to be used as the dictionary source. |
hosts[] | string Name of the PostrgreSQL host The number of elements must be greater than 0. |
port | int64 Port to use when connecting to the host. Acceptable values are 0 to 65535, inclusive. |
user | string Required. Name of the PostrgreSQL database user. |
password | string Password of the PostrgreSQL database user. |
invalidate_query | string Query for checking the dictionary status, to pull only updated data. For more details, see ClickHouse documentation on dictionaries |
ssl_mode | enum SslMode Mode of SSL TCP/IP connection to the PostgreSQL host. For more details, see PostgreSQL documentation
|
Structure
Field | Description |
---|---|
id | Id Single numeric key column for the dictionary. |
key | Key Composite key for the dictionary, containing of one or more key columns. For details, see ClickHouse documentation |
range_min | Attribute Field holding the beginning of the range for dictionaries with RANGE_HASHED layout. For details, see ClickHouse documentation |
range_max | Attribute Field holding the end of the range for dictionaries with RANGE_HASHED layout. For details, see ClickHouse documentation |
attributes[] | Attribute Description of the fields available for database queries. For details, see ClickHouse documentation |
Attribute
Field | Description |
---|---|
name | string Required. Name of the column. |
type | string Required. Type of the column. |
null_value | string Default value for an element without data (for example, an empty string). |
expression | string Expression, describing the attribute, if applicable. |
hierarchical | bool Indication of hierarchy support. Default value: false . |
injective | bool Indication of injective mapping "id -> attribute". Default value: false . |
Id
Field | Description |
---|---|
name | string Required. Name of the numeric key. |
Key
Field | Description |
---|---|
attributes[] | Attribute Attributes of a complex key. The number of elements must be greater than 0. |
Layout
Field | Description |
---|---|
type | enum Type Required. Layout type for an external dictionary.
|
size_in_cells | int64 Number of cells in the cache. Rounded up to a power of two. Applicable only for CACHE and COMPLEX_KEY_CACHE layout types. |
Range
Field | Description |
---|---|
min | int64 Minimum dictionary lifetime. |
max | int64 Maximum dictionary lifetime. |
GraphiteRollup
Field | Description |
---|---|
name | string Required. Name for the specified combination of settings for Graphite rollup. |
patterns[] | Pattern Pattern to use for the rollup. The number of elements must be greater than 0. |
Pattern
Field | Description |
---|---|
regexp | string Pattern for metric names. |
function | string Required. Name of the aggregating function to apply to data of the age specified in retention . |
retention[] | Retention Age of data to use for thinning. The number of elements must be greater than 0. |
Retention
Field | Description |
---|---|
age | int64 Minimum age of the data in seconds. The minimum value is 0. |
precision | int64 Precision of determining the age of the data, in seconds. Value must be greater than 0. |
MaintenanceOperation
Field | Description |
---|---|
info | string Information about this maintenance operation. The maximum string length in characters is 256. |
delayed_until | google.protobuf.Timestamp Time until which this maintenance operation is delayed. |
Update
Updates the specified ClickHouse cluster.
rpc Update (UpdateClusterRequest) returns (operation.Operation)
Metadata and response of Operation:
Operation.metadata:UpdateClusterMetadata
Operation.response:Cluster
UpdateClusterRequest
Field | Description |
---|---|
cluster_id | string Required. ID of the ClickHouse Cluster resource to update. To get the ClickHouse cluster ID, use a ClusterService.List request. The maximum string length in characters is 50. |
update_mask | google.protobuf.FieldMask Field mask that specifies which fields of the ClickHouse Cluster resource should be updated. |
description | string New description of the ClickHouse cluster. The maximum string length in characters is 256. |
labels | map<string,string> Custom labels for the ClickHouse cluster as key:value pairs. Maximum 64 per resource. For example, "project": "mvp" or "source": "dictionary". The new set of labels will completely replace the old ones. To add a label, request the current set with the ClusterService.Get method, then send an ClusterService.Update request with the new label added to the set. No more than 64 per resource. The maximum string length in characters for each value is 63. Each value must match the regular expression [-_0-9a-z]* . The maximum string length in characters for each key is 63. Each key must match the regular expression [a-z][-_0-9a-z]* . |
config_spec | ConfigSpec New configuration and resources for hosts in the cluster. |
name | string New name for the cluster. The maximum string length in characters is 63. Value must match the regular expression [a-zA-Z0-9_-]* . |
service_account_id | string ID of the service account used for access to Object Storage. |
maintenance_window | MaintenanceWindow New maintenance window settings for the cluster. |
security_group_ids[] | string User security groups |
deletion_protection | bool Deletion Protection inhibits deletion of the cluster |
ConfigSpec
Field | Description |
---|---|
version | string Version of the ClickHouse server software. |
clickhouse | Clickhouse Configuration and resources for a ClickHouse server. |
zookeeper | Zookeeper Configuration and resources for a ZooKeeper server. |
backup_window_start | google.type.TimeOfDay Time to start the daily backup, in the UTC timezone. |
access | Access Access policy for external services. If you want a specific service to access the ClickHouse cluster, then set the necessary values in this policy. |
cloud_storage | CloudStorage |
sql_database_management | google.protobuf.BoolValue Whether database management through SQL commands is enabled. |
sql_user_management | google.protobuf.BoolValue Whether user management through SQL commands is enabled. |
admin_password | string Password for user 'admin' that has SQL user management access. |
embedded_keeper | google.protobuf.BoolValue Whether cluster should use embedded Keeper instead of Zookeeper |
Clickhouse
Field | Description |
---|---|
config | config.ClickhouseConfig Configuration for a ClickHouse server. |
resources | Resources Resources allocated to ClickHouse hosts. |
Zookeeper
Field | Description |
---|---|
resources | Resources Resources allocated to ZooKeeper hosts. If not set, minimal available resources will be used. All available resource presets can be retrieved with a ResourcePresetService.List request. |
Access
Field | Description |
---|---|
data_lens | bool Allow to export data from the cluster to DataLens. |
web_sql | bool Allow SQL queries to the cluster databases from the management console. See SQL queries in the management console for more details. |
metrika | bool Allow to import data from Yandex Metrica and AppMetrica to the cluster. See AppMetrica documentation |
serverless | bool Allow access to cluster for Serverless. |
data_transfer | bool Allow access for DataTransfer |
yandex_query | bool Allow access for Query |
CloudStorage
Field | Description |
---|---|
enabled | bool Whether to use Object Storage for storing ClickHouse data. |
move_factor | google.protobuf.DoubleValue Acceptable values are 0 to 1, inclusive. |
data_cache_enabled | google.protobuf.BoolValue |
data_cache_max_size | google.protobuf.Int64Value |
prefer_not_to_merge | google.protobuf.BoolValue |
ClickhouseConfig
Field | Description |
---|---|
log_level | enum LogLevel Logging level for the ClickHouse cluster. Possible values: TRACE, DEBUG, INFORMATION, WARNING, ERROR. |
merge_tree | MergeTree Settings for the MergeTree engine. See description in ClickHouse documentation |
compression[] | Compression Compression settings for the ClickHouse cluster. See in-depth description in ClickHouse documentation |
dictionaries[] | ExternalDictionary Configuration of external dictionaries to be used by the ClickHouse cluster. See in-depth description in ClickHouse documentation |
graphite_rollup[] | GraphiteRollup Settings for thinning Graphite data. See in-depth description in ClickHouse documentation |
kafka | Kafka |
kafka_topics[] | KafkaTopic |
rabbitmq | Rabbitmq |
max_connections | google.protobuf.Int64Value Maximum number of inbound connections. The minimum value is 10. |
max_concurrent_queries | google.protobuf.Int64Value Maximum number of simultaneously processed requests. The minimum value is 10. |
keep_alive_timeout | google.protobuf.Int64Value Number of milliseconds that ClickHouse waits for incoming requests before closing the connection. |
uncompressed_cache_size | google.protobuf.Int64Value Cache size (in bytes) for uncompressed data used by MergeTree tables. |
mark_cache_size | google.protobuf.Int64Value Approximate size (in bytes) of the cache of "marks" used by MergeTree tables. Value must be greater than 0. |
max_table_size_to_drop | google.protobuf.Int64Value Maximum size of the table that can be deleted using a DROP query. |
max_partition_size_to_drop | google.protobuf.Int64Value Maximum size of the partition that can be deleted using a DROP query. |
builtin_dictionaries_reload_interval | google.protobuf.Int64Value The setting is deprecated and has no effect. |
timezone | string The server's time zone to be used in DateTime fields conversions. Specified as an IANA identifier. |
geobase_enabled | google.protobuf.BoolValue Enable or disable geobase. |
geobase_uri | string Address of the archive with the user geobase in Object Storage. |
query_log_retention_size | google.protobuf.Int64Value The maximum size that query_log can grow to before old data will be removed. If set to 0, automatic removal of query_log data based on size is disabled. |
query_log_retention_time | google.protobuf.Int64Value The maximum time that query_log records will be retained before removal. If set to 0, automatic removal of query_log data based on time is disabled. |
query_thread_log_enabled | google.protobuf.BoolValue Whether query_thread_log system table is enabled. |
query_thread_log_retention_size | google.protobuf.Int64Value The maximum size that query_thread_log can grow to before old data will be removed. If set to 0, automatic removal of query_thread_log data based on size is disabled. |
query_thread_log_retention_time | google.protobuf.Int64Value The maximum time that query_thread_log records will be retained before removal. If set to 0, automatic removal of query_thread_log data based on time is disabled. |
part_log_retention_size | google.protobuf.Int64Value The maximum size that part_log can grow to before old data will be removed. If set to 0, automatic removal of part_log data based on size is disabled. |
part_log_retention_time | google.protobuf.Int64Value The maximum time that part_log records will be retained before removal. If set to 0, automatic removal of part_log data based on time is disabled. |
metric_log_enabled | google.protobuf.BoolValue Whether metric_log system table is enabled. |
metric_log_retention_size | google.protobuf.Int64Value The maximum size that metric_log can grow to before old data will be removed. If set to 0, automatic removal of metric_log data based on size is disabled. |
metric_log_retention_time | google.protobuf.Int64Value The maximum time that metric_log records will be retained before removal. If set to 0, automatic removal of metric_log data based on time is disabled. |
trace_log_enabled | google.protobuf.BoolValue Whether trace_log system table is enabled. |
trace_log_retention_size | google.protobuf.Int64Value The maximum size that trace_log can grow to before old data will be removed. If set to 0, automatic removal of trace_log data based on size is disabled. |
trace_log_retention_time | google.protobuf.Int64Value The maximum time that trace_log records will be retained before removal. If set to 0, automatic removal of trace_log data based on time is disabled. |
text_log_enabled | google.protobuf.BoolValue Whether text_log system table is enabled. |
text_log_retention_size | google.protobuf.Int64Value The maximum size that text_log can grow to before old data will be removed. If set to 0, automatic removal of text_log data based on size is disabled. |
text_log_retention_time | google.protobuf.Int64Value The maximum time that text_log records will be retained before removal. If set to 0, automatic removal of text_log data based on time is disabled. |
text_log_level | enum LogLevel Logging level for text_log system table. Possible values: TRACE, DEBUG, INFORMATION, WARNING, ERROR. |
opentelemetry_span_log_enabled | google.protobuf.BoolValue Enable or disable opentelemetry_span_log system table. Default value: false. |
opentelemetry_span_log_retention_size | google.protobuf.Int64Value The maximum size that opentelemetry_span_log can grow to before old data will be removed. If set to 0 (default), automatic removal of opentelemetry_span_log data based on size is disabled. The minimum value is 0. |
opentelemetry_span_log_retention_time | google.protobuf.Int64Value The maximum time that opentelemetry_span_log records will be retained before removal. If set to 0, automatic removal of opentelemetry_span_log data based on time is disabled. The minimum value is 0. |
query_views_log_enabled | google.protobuf.BoolValue Enable or disable query_views_log system table. Default value: false. |
query_views_log_retention_size | google.protobuf.Int64Value The maximum size that query_views_log can grow to before old data will be removed. If set to 0 (default), automatic removal of query_views_log data based on size is disabled. The minimum value is 0. |
query_views_log_retention_time | google.protobuf.Int64Value The maximum time that query_views_log records will be retained before removal. If set to 0, automatic removal of query_views_log data based on time is disabled. The minimum value is 0. |
asynchronous_metric_log_enabled | google.protobuf.BoolValue Enable or disable asynchronous_metric_log system table. Default value: false. |
asynchronous_metric_log_retention_size | google.protobuf.Int64Value The maximum size that asynchronous_metric_log can grow to before old data will be removed. If set to 0 (default), automatic removal of asynchronous_metric_log data based on size is disabled. The minimum value is 0. |
asynchronous_metric_log_retention_time | google.protobuf.Int64Value The maximum time that asynchronous_metric_log records will be retained before removal. If set to 0, automatic removal of asynchronous_metric_log data based on time is disabled. The minimum value is 0. |
session_log_enabled | google.protobuf.BoolValue Enable or disable session_log system table. Default value: false. |
session_log_retention_size | google.protobuf.Int64Value The maximum size that session_log can grow to before old data will be removed. If set to 0 (default), automatic removal of session_log data based on size is disabled. The minimum value is 0. |
session_log_retention_time | google.protobuf.Int64Value The maximum time that session_log records will be retained before removal. If set to 0, automatic removal of session_log data based on time is disabled. The minimum value is 0. |
zookeeper_log_enabled | google.protobuf.BoolValue Enable or disable zookeeper_log system table. Default value: false. |
zookeeper_log_retention_size | google.protobuf.Int64Value The maximum size that zookeeper_log can grow to before old data will be removed. If set to 0 (default), automatic removal of zookeeper_log data based on size is disabled. The minimum value is 0. |
zookeeper_log_retention_time | google.protobuf.Int64Value The maximum time that zookeeper_log records will be retained before removal. If set to 0, automatic removal of zookeeper_log data based on time is disabled. The minimum value is 0. |
asynchronous_insert_log_enabled | google.protobuf.BoolValue Enable or disable asynchronous_insert_log system table. Default value: false. Minimal required ClickHouse version: 22.10. |
asynchronous_insert_log_retention_size | google.protobuf.Int64Value The maximum size that asynchronous_insert_log can grow to before old data will be removed. If set to 0 (default), automatic removal of asynchronous_insert_log data based on size is disabled. The minimum value is 0. |
asynchronous_insert_log_retention_time | google.protobuf.Int64Value The maximum time that asynchronous_insert_log records will be retained before removal. If set to 0, automatic removal of asynchronous_insert_log data based on time is disabled. The minimum value is 0. |
background_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_merges_mutations_concurrency_ratio | google.protobuf.Int64Value Sets a ratio between the number of threads and the number of background merges and mutations that can be executed concurrently. For example, if the ratio equals to 2 and background_pool_size is set to 16 then ClickHouse can execute 32 background merges concurrently. This is possible, because background operations could be suspended and postponed. This is needed to give small merges more execution priority. You can only increase this ratio at runtime. To lower it you have to restart the server. The same as for background_pool_size setting background_merges_mutations_concurrency_ratio could be applied from the default profile for backward compatibility. Default: 2 See in-depth description in ClickHouse documentation |
background_schedule_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_fetches_pool_size | google.protobuf.Int64Value Sets the number of threads performing background fetches for tables with ReplicatedMergeTree engines. Default value: 8. More info see in ClickHouse documentation |
background_move_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_distributed_schedule_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_buffer_flush_schedule_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_message_broker_schedule_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_common_pool_size | google.protobuf.Int64Value The maximum number of threads that will be used for performing a variety of operations (mostly garbage collection) for *MergeTree-engine tables in a background. Default: 8 See in-depth description in ClickHouse documentation |
default_database | google.protobuf.StringValue The default database. To get a list of cluster databases, see Yandex Managed ClickHouse documentation. |
total_memory_profiler_step | google.protobuf.Int64Value Sets the memory size (in bytes) for a stack trace at every peak allocation step. Default value: 4194304. More info see in ClickHouse documentation |
total_memory_tracker_sample_probability | google.protobuf.DoubleValue |
MergeTree
Field | Description |
---|---|
replicated_deduplication_window | google.protobuf.Int64Value Number of blocks of hashes to keep in ZooKeeper. |
replicated_deduplication_window_seconds | google.protobuf.Int64Value Period of time to keep blocks of hashes for. |
parts_to_delay_insert | google.protobuf.Int64Value If table contains at least that many active parts in single partition, artificially slow down insert into table. |
parts_to_throw_insert | google.protobuf.Int64Value If more than this number active parts in single partition, throw 'Too many parts ...' exception. |
inactive_parts_to_delay_insert | google.protobuf.Int64Value |
inactive_parts_to_throw_insert | google.protobuf.Int64Value |
max_replicated_merges_in_queue | google.protobuf.Int64Value How many tasks of merging and mutating parts are allowed simultaneously in ReplicatedMergeTree queue. |
number_of_free_entries_in_pool_to_lower_max_size_of_merge | google.protobuf.Int64Value If there is less than specified number of free entries in background pool (or replicated queue), start to lower maximum size of merge to process. |
max_bytes_to_merge_at_min_space_in_pool | google.protobuf.Int64Value Maximum in total size of parts to merge, when there are minimum free threads in background pool (or entries in replication queue). |
max_bytes_to_merge_at_max_space_in_pool | google.protobuf.Int64Value |
min_bytes_for_wide_part | google.protobuf.Int64Value Minimum number of bytes in a data part that can be stored in Wide format. More info see in ClickHouse documentation |
min_rows_for_wide_part | google.protobuf.Int64Value Minimum number of rows in a data part that can be stored in Wide format. More info see in ClickHouse documentation |
ttl_only_drop_parts | google.protobuf.BoolValue Enables or disables complete dropping of data parts where all rows are expired in MergeTree tables. More info see in ClickHouse documentation |
allow_remote_fs_zero_copy_replication | google.protobuf.BoolValue |
merge_with_ttl_timeout | google.protobuf.Int64Value |
merge_with_recompression_ttl_timeout | google.protobuf.Int64Value |
max_parts_in_total | google.protobuf.Int64Value |
max_number_of_merges_with_ttl_in_pool | google.protobuf.Int64Value |
cleanup_delay_period | google.protobuf.Int64Value |
number_of_free_entries_in_pool_to_execute_mutation | google.protobuf.Int64Value |
max_avg_part_size_for_too_many_parts | google.protobuf.Int64Value The 'too many parts' check according to 'parts_to_delay_insert' and 'parts_to_throw_insert' will be active only if the average part size (in the relevant partition) is not larger than the specified threshold. If it is larger than the specified threshold, the INSERTs will be neither delayed or rejected. This allows to have hundreds of terabytes in a single table on a single server if the parts are successfully merged to larger parts. This does not affect the thresholds on inactive parts or total parts. Default: 1 GiB Min version: 22.10 See in-depth description in ClickHouse GitHub |
min_age_to_force_merge_seconds | google.protobuf.Int64Value Merge parts if every part in the range is older than the value of min_age_to_force_merge_seconds. Default: 0 - disabled Min_version: 22.10 See in-depth description in ClickHouse documentation |
min_age_to_force_merge_on_partition_only | google.protobuf.BoolValue Whether min_age_to_force_merge_seconds should be applied only on the entire partition and not on subset. Default: false Min_version: 22.11 See in-depth description in ClickHouse documentation |
merge_selecting_sleep_ms | google.protobuf.Int64Value Sleep time for merge selecting when no part is selected. A lower setting triggers selecting tasks in background_schedule_pool frequently, which results in a large number of requests to ClickHouse Keeper in large-scale clusters. Default: 5000 Min_version: 21.10 See in-depth description in ClickHouse documentation |
Kafka
Field | Description |
---|---|
security_protocol | enum SecurityProtocol |
sasl_mechanism | enum SaslMechanism |
sasl_username | string |
sasl_password | string |
enable_ssl_certificate_verification | google.protobuf.BoolValue |
max_poll_interval_ms | google.protobuf.Int64Value The minimum value is 0. |
session_timeout_ms | google.protobuf.Int64Value The minimum value is 0. |
KafkaTopic
Field | Description |
---|---|
name | string Required. |
settings | Kafka Required. |
Rabbitmq
Field | Description |
---|---|
username | string RabbitMQ |
password | string RabbitMQ |
vhost | string RabbitMQ |
Compression
Field | Description |
---|---|
method | enum Method Compression method to use for the specified combination of min_part_size and min_part_size_ratio . |
min_part_size | int64 Minimum size of a part of a table. The minimum value is 1. |
min_part_size_ratio | double Minimum ratio of a part relative to the size of all the data in the table. |
level | google.protobuf.Int64Value The minimum value is 0. |
ExternalDictionary
Field | Description |
---|---|
name | string Required. Name of the external dictionary. |
structure | Structure Required. Set of attributes for the external dictionary. For in-depth description, see ClickHouse documentation |
layout | Layout Required. Layout for storing the dictionary in memory. For in-depth description, see ClickHouse documentation |
lifetime | oneof: fixed_lifetime or lifetime_range Setting for the period of time between dictionary updates. For details, see ClickHouse documentation |
fixed_lifetime | int64 Fixed interval between dictionary updates. |
lifetime_range | Range Range of intervals between dictionary updates for ClickHouse to choose from. |
source | oneof: http_source , mysql_source , clickhouse_source , mongodb_source or postgresql_source Description of the source for the external dictionary. |
http_source | HttpSource HTTP source for the dictionary. |
mysql_source | MysqlSource MySQL source for the dictionary. |
clickhouse_source | ClickhouseSource ClickHouse source for the dictionary. |
mongodb_source | MongodbSource MongoDB source for the dictionary. |
postgresql_source | PostgresqlSource PostgreSQL source for the dictionary. |
HttpSource
Field | Description |
---|---|
url | string Required. URL of the source dictionary available over HTTP. |
format | string Required. The data format. Valid values are all formats supported by ClickHouse SQL dialect. |
MysqlSource
Field | Description |
---|---|
db | string Required. Name of the MySQL database to connect to. |
table | string Required. Name of the database table to use as a ClickHouse dictionary. |
port | int64 Default port to use when connecting to a replica of the dictionary source. Acceptable values are 0 to 65535, inclusive. |
user | string Name of the default user for replicas of the dictionary source. |
password | string Password of the default user for replicas of the dictionary source. |
replicas[] | Replica List of MySQL replicas of the database used as dictionary source. The number of elements must be greater than 0. |
where | string Selection criteria for the data in the specified MySQL table. |
invalidate_query | string Query for checking the dictionary status, to pull only updated data. For more details, see ClickHouse documentation on dictionaries |
Replica
Field | Description |
---|---|
host | string Required. MySQL host of the replica. The maximum string length in characters is 253. |
priority | int64 Required. The priority of the replica that ClickHouse takes into account when connecting. Replica with the highest priority should have this field set to the lowest number. Value must be greater than 0. |
port | int64 Port to use when connecting to the replica. If a port is not specified for a replica, ClickHouse uses the port specified for the source. Acceptable values are 0 to 65535, inclusive. |
user | string Name of the MySQL database user. |
password | string Password of the MySQL database user. |
ClickhouseSource
Field | Description |
---|---|
db | string Required. Name of the ClickHouse database. |
table | string Required. Name of the table in the specified database to be used as the dictionary source. |
host | string Required. ClickHouse host of the specified database. The maximum string length in characters is 253. |
port | int64 Port to use when connecting to the host. Acceptable values are 0 to 65535, inclusive. |
user | string Required. Name of the ClickHouse database user. |
password | string Password of the ClickHouse database user. |
where | string Selection criteria for the data in the specified ClickHouse table. |
MongodbSource
Field | Description |
---|---|
db | string Required. Name of the MongoDB database. |
collection | string Required. Name of the collection in the specified database to be used as the dictionary source. |
host | string Required. MongoDB host of the specified database. The maximum string length in characters is 253. |
port | int64 Port to use when connecting to the host. Acceptable values are 0 to 65535, inclusive. |
user | string Required. Name of the MongoDB database user. |
password | string Password of the MongoDB database user. |
options | string |
PostgresqlSource
Field | Description |
---|---|
db | string Required. Name of the PostrgreSQL database. |
table | string Required. Name of the table in the specified database to be used as the dictionary source. |
hosts[] | string Name of the PostrgreSQL host The number of elements must be greater than 0. |
port | int64 Port to use when connecting to the host. Acceptable values are 0 to 65535, inclusive. |
user | string Required. Name of the PostrgreSQL database user. |
password | string Password of the PostrgreSQL database user. |
invalidate_query | string Query for checking the dictionary status, to pull only updated data. For more details, see ClickHouse documentation on dictionaries |
ssl_mode | enum SslMode Mode of SSL TCP/IP connection to the PostgreSQL host. For more details, see PostgreSQL documentation
|
Structure
Field | Description |
---|---|
id | Id Single numeric key column for the dictionary. |
key | Key Composite key for the dictionary, containing of one or more key columns. For details, see ClickHouse documentation |
range_min | Attribute Field holding the beginning of the range for dictionaries with RANGE_HASHED layout. For details, see ClickHouse documentation |
range_max | Attribute Field holding the end of the range for dictionaries with RANGE_HASHED layout. For details, see ClickHouse documentation |
attributes[] | Attribute Description of the fields available for database queries. For details, see ClickHouse documentation |
Attribute
Field | Description |
---|---|
name | string Required. Name of the column. |
type | string Required. Type of the column. |
null_value | string Default value for an element without data (for example, an empty string). |
expression | string Expression, describing the attribute, if applicable. |
hierarchical | bool Indication of hierarchy support. Default value: false . |
injective | bool Indication of injective mapping "id -> attribute". Default value: false . |
Id
Field | Description |
---|---|
name | string Required. Name of the numeric key. |
Key
Field | Description |
---|---|
attributes[] | Attribute Attributes of a complex key. The number of elements must be greater than 0. |
Layout
Field | Description |
---|---|
type | enum Type Required. Layout type for an external dictionary.
|
size_in_cells | int64 Number of cells in the cache. Rounded up to a power of two. Applicable only for CACHE and COMPLEX_KEY_CACHE layout types. |
Range
Field | Description |
---|---|
min | int64 Minimum dictionary lifetime. |
max | int64 Maximum dictionary lifetime. |
GraphiteRollup
Field | Description |
---|---|
name | string Required. Name for the specified combination of settings for Graphite rollup. |
patterns[] | Pattern Pattern to use for the rollup. The number of elements must be greater than 0. |
Pattern
Field | Description |
---|---|
regexp | string Pattern for metric names. |
function | string Required. Name of the aggregating function to apply to data of the age specified in retention . |
retention[] | Retention Age of data to use for thinning. The number of elements must be greater than 0. |
Retention
Field | Description |
---|---|
age | int64 Minimum age of the data in seconds. The minimum value is 0. |
precision | int64 Precision of determining the age of the data, in seconds. Value must be greater than 0. |
Resources
Field | Description |
---|---|
resource_preset_id | string ID of the preset for computational resources available to a host (CPU, memory etc.). All available presets are listed in the documentation |
disk_size | int64 Volume of the storage available to a host, in bytes. |
disk_type_id | string Type of the storage environment for the host. Possible values:
|
MaintenanceWindow
Field | Description |
---|---|
policy | oneof: anytime or weekly_maintenance_window The maintenance policy in effect. |
anytime | AnytimeMaintenanceWindow Maintenance operation can be scheduled anytime. |
weekly_maintenance_window | WeeklyMaintenanceWindow Maintenance operation can be scheduled on a weekly basis. |
AnytimeMaintenanceWindow
Empty.
WeeklyMaintenanceWindow
Field | Description |
---|---|
day | enum WeekDay Day of the week (in DDD format). |
hour | int64 Hour of the day in UTC (in HH format). Acceptable values are 1 to 24, inclusive. |
Operation
Field | Description |
---|---|
id | string ID of the operation. |
description | string Description of the operation. 0-256 characters long. |
created_at | google.protobuf.Timestamp Creation timestamp. |
created_by | string ID of the user or service account who initiated the operation. |
modified_at | google.protobuf.Timestamp The time when the Operation resource was last modified. |
done | bool If the value is false , it means the operation is still in progress. If true , the operation is completed, and either error or response is available. |
metadata | google.protobuf.Any Service-specific metadata associated with the operation. It typically contains the ID of the target resource that the operation is performed on. Any method that returns a long-running operation should document the metadata type, if any. |
result | oneof: error or response The operation result. If done == false and there was no failure detected, neither error nor response is set. If done == false and there was a failure detected, error is set. If done == true , exactly one of error or response is set. |
error | google.rpc.Status The error result of the operation in case of failure or cancellation. |
response | google.protobuf.Any if operation finished successfully. |
UpdateClusterMetadata
Field | Description |
---|---|
cluster_id | string ID of the ClickHouse Cluster resource that is being updated. |
Cluster
Field | Description |
---|---|
id | string ID of the ClickHouse cluster. This ID is assigned by MDB at creation time. |
folder_id | string ID of the folder that the ClickHouse cluster belongs to. |
created_at | google.protobuf.Timestamp Creation timestamp in RFC3339 |
name | string Name of the ClickHouse cluster. The name is unique within the folder. 1-63 characters long. |
description | string Description of the ClickHouse cluster. 0-256 characters long. |
labels | map<string,string> Custom labels for the ClickHouse cluster as key:value pairs. Maximum 64 per resource. |
environment | enum Environment Deployment environment of the ClickHouse cluster.
|
monitoring[] | Monitoring Description of monitoring systems relevant to the ClickHouse cluster. |
config | ClusterConfig Configuration of the ClickHouse cluster. |
network_id | string ID of the network that the cluster belongs to. |
health | enum Health Aggregated cluster health.
|
status | enum Status Current state of the cluster.
|
service_account_id | string ID of the service account used for access to Object Storage. |
maintenance_window | MaintenanceWindow Maintenance window for the cluster. |
planned_operation | MaintenanceOperation Planned maintenance operation to be started for the cluster within the nearest maintenance_window . |
security_group_ids[] | string User security groups |
deletion_protection | bool Deletion Protection inhibits deletion of the cluster |
Monitoring
Field | Description |
---|---|
name | string Name of the monitoring system. |
description | string Description of the monitoring system. |
link | string Link to the monitoring system charts for the ClickHouse cluster. |
ClusterConfig
Field | Description |
---|---|
version | string Version of the ClickHouse server software. |
clickhouse | Clickhouse Configuration and resource allocation for ClickHouse hosts. |
zookeeper | Zookeeper Configuration and resource allocation for ZooKeeper hosts. |
backup_window_start | google.type.TimeOfDay Time to start the daily backup, in the UTC timezone. |
access | Access Access policy for external services. |
cloud_storage | CloudStorage |
sql_database_management | google.protobuf.BoolValue Whether database management through SQL commands is enabled. |
sql_user_management | google.protobuf.BoolValue Whether user management through SQL commands is enabled. |
embedded_keeper | google.protobuf.BoolValue Whether cluster should use embedded Keeper instead of Zookeeper. |
Clickhouse
Field | Description |
---|---|
config | config.ClickhouseConfigSet Configuration settings of a ClickHouse server. |
resources | Resources Resources allocated to ClickHouse hosts. |
Zookeeper
Field | Description |
---|---|
resources | Resources Resources allocated to ZooKeeper hosts. |
ClickhouseConfigSet
Field | Description |
---|---|
effective_config | ClickhouseConfig Required. Effective settings for a ClickHouse cluster (a combination of settings defined in user_config and default_config ). |
user_config | ClickhouseConfig User-defined settings for a ClickHouse cluster. |
default_config | ClickhouseConfig Default configuration for a ClickHouse cluster. |
ClickhouseConfig
Field | Description |
---|---|
log_level | enum LogLevel Logging level for the ClickHouse cluster. Possible values: TRACE, DEBUG, INFORMATION, WARNING, ERROR. |
merge_tree | MergeTree Settings for the MergeTree engine. See description in ClickHouse documentation |
compression[] | Compression Compression settings for the ClickHouse cluster. See in-depth description in ClickHouse documentation |
dictionaries[] | ExternalDictionary Configuration of external dictionaries to be used by the ClickHouse cluster. See in-depth description in ClickHouse documentation |
graphite_rollup[] | GraphiteRollup Settings for thinning Graphite data. See in-depth description in ClickHouse documentation |
kafka | Kafka |
kafka_topics[] | KafkaTopic |
rabbitmq | Rabbitmq |
max_connections | google.protobuf.Int64Value Maximum number of inbound connections. The minimum value is 10. |
max_concurrent_queries | google.protobuf.Int64Value Maximum number of simultaneously processed requests. The minimum value is 10. |
keep_alive_timeout | google.protobuf.Int64Value Number of milliseconds that ClickHouse waits for incoming requests before closing the connection. |
uncompressed_cache_size | google.protobuf.Int64Value Cache size (in bytes) for uncompressed data used by MergeTree tables. |
mark_cache_size | google.protobuf.Int64Value Approximate size (in bytes) of the cache of "marks" used by MergeTree tables. Value must be greater than 0. |
max_table_size_to_drop | google.protobuf.Int64Value Maximum size of the table that can be deleted using a DROP query. |
max_partition_size_to_drop | google.protobuf.Int64Value Maximum size of the partition that can be deleted using a DROP query. |
builtin_dictionaries_reload_interval | google.protobuf.Int64Value The setting is deprecated and has no effect. |
timezone | string The server's time zone to be used in DateTime fields conversions. Specified as an IANA identifier. |
geobase_enabled | google.protobuf.BoolValue Enable or disable geobase. |
geobase_uri | string Address of the archive with the user geobase in Object Storage. |
query_log_retention_size | google.protobuf.Int64Value The maximum size that query_log can grow to before old data will be removed. If set to 0, automatic removal of query_log data based on size is disabled. |
query_log_retention_time | google.protobuf.Int64Value The maximum time that query_log records will be retained before removal. If set to 0, automatic removal of query_log data based on time is disabled. |
query_thread_log_enabled | google.protobuf.BoolValue Whether query_thread_log system table is enabled. |
query_thread_log_retention_size | google.protobuf.Int64Value The maximum size that query_thread_log can grow to before old data will be removed. If set to 0, automatic removal of query_thread_log data based on size is disabled. |
query_thread_log_retention_time | google.protobuf.Int64Value The maximum time that query_thread_log records will be retained before removal. If set to 0, automatic removal of query_thread_log data based on time is disabled. |
part_log_retention_size | google.protobuf.Int64Value The maximum size that part_log can grow to before old data will be removed. If set to 0, automatic removal of part_log data based on size is disabled. |
part_log_retention_time | google.protobuf.Int64Value The maximum time that part_log records will be retained before removal. If set to 0, automatic removal of part_log data based on time is disabled. |
metric_log_enabled | google.protobuf.BoolValue Whether metric_log system table is enabled. |
metric_log_retention_size | google.protobuf.Int64Value The maximum size that metric_log can grow to before old data will be removed. If set to 0, automatic removal of metric_log data based on size is disabled. |
metric_log_retention_time | google.protobuf.Int64Value The maximum time that metric_log records will be retained before removal. If set to 0, automatic removal of metric_log data based on time is disabled. |
trace_log_enabled | google.protobuf.BoolValue Whether trace_log system table is enabled. |
trace_log_retention_size | google.protobuf.Int64Value The maximum size that trace_log can grow to before old data will be removed. If set to 0, automatic removal of trace_log data based on size is disabled. |
trace_log_retention_time | google.protobuf.Int64Value The maximum time that trace_log records will be retained before removal. If set to 0, automatic removal of trace_log data based on time is disabled. |
text_log_enabled | google.protobuf.BoolValue Whether text_log system table is enabled. |
text_log_retention_size | google.protobuf.Int64Value The maximum size that text_log can grow to before old data will be removed. If set to 0, automatic removal of text_log data based on size is disabled. |
text_log_retention_time | google.protobuf.Int64Value The maximum time that text_log records will be retained before removal. If set to 0, automatic removal of text_log data based on time is disabled. |
text_log_level | enum LogLevel Logging level for text_log system table. Possible values: TRACE, DEBUG, INFORMATION, WARNING, ERROR. |
opentelemetry_span_log_enabled | google.protobuf.BoolValue Enable or disable opentelemetry_span_log system table. Default value: false. |
opentelemetry_span_log_retention_size | google.protobuf.Int64Value The maximum size that opentelemetry_span_log can grow to before old data will be removed. If set to 0 (default), automatic removal of opentelemetry_span_log data based on size is disabled. The minimum value is 0. |
opentelemetry_span_log_retention_time | google.protobuf.Int64Value The maximum time that opentelemetry_span_log records will be retained before removal. If set to 0, automatic removal of opentelemetry_span_log data based on time is disabled. The minimum value is 0. |
query_views_log_enabled | google.protobuf.BoolValue Enable or disable query_views_log system table. Default value: false. |
query_views_log_retention_size | google.protobuf.Int64Value The maximum size that query_views_log can grow to before old data will be removed. If set to 0 (default), automatic removal of query_views_log data based on size is disabled. The minimum value is 0. |
query_views_log_retention_time | google.protobuf.Int64Value The maximum time that query_views_log records will be retained before removal. If set to 0, automatic removal of query_views_log data based on time is disabled. The minimum value is 0. |
asynchronous_metric_log_enabled | google.protobuf.BoolValue Enable or disable asynchronous_metric_log system table. Default value: false. |
asynchronous_metric_log_retention_size | google.protobuf.Int64Value The maximum size that asynchronous_metric_log can grow to before old data will be removed. If set to 0 (default), automatic removal of asynchronous_metric_log data based on size is disabled. The minimum value is 0. |
asynchronous_metric_log_retention_time | google.protobuf.Int64Value The maximum time that asynchronous_metric_log records will be retained before removal. If set to 0, automatic removal of asynchronous_metric_log data based on time is disabled. The minimum value is 0. |
session_log_enabled | google.protobuf.BoolValue Enable or disable session_log system table. Default value: false. |
session_log_retention_size | google.protobuf.Int64Value The maximum size that session_log can grow to before old data will be removed. If set to 0 (default), automatic removal of session_log data based on size is disabled. The minimum value is 0. |
session_log_retention_time | google.protobuf.Int64Value The maximum time that session_log records will be retained before removal. If set to 0, automatic removal of session_log data based on time is disabled. The minimum value is 0. |
zookeeper_log_enabled | google.protobuf.BoolValue Enable or disable zookeeper_log system table. Default value: false. |
zookeeper_log_retention_size | google.protobuf.Int64Value The maximum size that zookeeper_log can grow to before old data will be removed. If set to 0 (default), automatic removal of zookeeper_log data based on size is disabled. The minimum value is 0. |
zookeeper_log_retention_time | google.protobuf.Int64Value The maximum time that zookeeper_log records will be retained before removal. If set to 0, automatic removal of zookeeper_log data based on time is disabled. The minimum value is 0. |
asynchronous_insert_log_enabled | google.protobuf.BoolValue Enable or disable asynchronous_insert_log system table. Default value: false. Minimal required ClickHouse version: 22.10. |
asynchronous_insert_log_retention_size | google.protobuf.Int64Value The maximum size that asynchronous_insert_log can grow to before old data will be removed. If set to 0 (default), automatic removal of asynchronous_insert_log data based on size is disabled. The minimum value is 0. |
asynchronous_insert_log_retention_time | google.protobuf.Int64Value The maximum time that asynchronous_insert_log records will be retained before removal. If set to 0, automatic removal of asynchronous_insert_log data based on time is disabled. The minimum value is 0. |
background_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_merges_mutations_concurrency_ratio | google.protobuf.Int64Value Sets a ratio between the number of threads and the number of background merges and mutations that can be executed concurrently. For example, if the ratio equals to 2 and background_pool_size is set to 16 then ClickHouse can execute 32 background merges concurrently. This is possible, because background operations could be suspended and postponed. This is needed to give small merges more execution priority. You can only increase this ratio at runtime. To lower it you have to restart the server. The same as for background_pool_size setting background_merges_mutations_concurrency_ratio could be applied from the default profile for backward compatibility. Default: 2 See in-depth description in ClickHouse documentation |
background_schedule_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_fetches_pool_size | google.protobuf.Int64Value Sets the number of threads performing background fetches for tables with ReplicatedMergeTree engines. Default value: 8. More info see in ClickHouse documentation |
background_move_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_distributed_schedule_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_buffer_flush_schedule_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_message_broker_schedule_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_common_pool_size | google.protobuf.Int64Value The maximum number of threads that will be used for performing a variety of operations (mostly garbage collection) for *MergeTree-engine tables in a background. Default: 8 See in-depth description in ClickHouse documentation |
default_database | google.protobuf.StringValue The default database. To get a list of cluster databases, see Yandex Managed ClickHouse documentation. |
total_memory_profiler_step | google.protobuf.Int64Value Sets the memory size (in bytes) for a stack trace at every peak allocation step. Default value: 4194304. More info see in ClickHouse documentation |
total_memory_tracker_sample_probability | google.protobuf.DoubleValue |
MergeTree
Field | Description |
---|---|
replicated_deduplication_window | google.protobuf.Int64Value Number of blocks of hashes to keep in ZooKeeper. |
replicated_deduplication_window_seconds | google.protobuf.Int64Value Period of time to keep blocks of hashes for. |
parts_to_delay_insert | google.protobuf.Int64Value If table contains at least that many active parts in single partition, artificially slow down insert into table. |
parts_to_throw_insert | google.protobuf.Int64Value If more than this number active parts in single partition, throw 'Too many parts ...' exception. |
inactive_parts_to_delay_insert | google.protobuf.Int64Value |
inactive_parts_to_throw_insert | google.protobuf.Int64Value |
max_replicated_merges_in_queue | google.protobuf.Int64Value How many tasks of merging and mutating parts are allowed simultaneously in ReplicatedMergeTree queue. |
number_of_free_entries_in_pool_to_lower_max_size_of_merge | google.protobuf.Int64Value If there is less than specified number of free entries in background pool (or replicated queue), start to lower maximum size of merge to process. |
max_bytes_to_merge_at_min_space_in_pool | google.protobuf.Int64Value Maximum in total size of parts to merge, when there are minimum free threads in background pool (or entries in replication queue). |
max_bytes_to_merge_at_max_space_in_pool | google.protobuf.Int64Value |
min_bytes_for_wide_part | google.protobuf.Int64Value Minimum number of bytes in a data part that can be stored in Wide format. More info see in ClickHouse documentation |
min_rows_for_wide_part | google.protobuf.Int64Value Minimum number of rows in a data part that can be stored in Wide format. More info see in ClickHouse documentation |
ttl_only_drop_parts | google.protobuf.BoolValue Enables or disables complete dropping of data parts where all rows are expired in MergeTree tables. More info see in ClickHouse documentation |
allow_remote_fs_zero_copy_replication | google.protobuf.BoolValue |
merge_with_ttl_timeout | google.protobuf.Int64Value |
merge_with_recompression_ttl_timeout | google.protobuf.Int64Value |
max_parts_in_total | google.protobuf.Int64Value |
max_number_of_merges_with_ttl_in_pool | google.protobuf.Int64Value |
cleanup_delay_period | google.protobuf.Int64Value |
number_of_free_entries_in_pool_to_execute_mutation | google.protobuf.Int64Value |
max_avg_part_size_for_too_many_parts | google.protobuf.Int64Value The 'too many parts' check according to 'parts_to_delay_insert' and 'parts_to_throw_insert' will be active only if the average part size (in the relevant partition) is not larger than the specified threshold. If it is larger than the specified threshold, the INSERTs will be neither delayed or rejected. This allows to have hundreds of terabytes in a single table on a single server if the parts are successfully merged to larger parts. This does not affect the thresholds on inactive parts or total parts. Default: 1 GiB Min version: 22.10 See in-depth description in ClickHouse GitHub |
min_age_to_force_merge_seconds | google.protobuf.Int64Value Merge parts if every part in the range is older than the value of min_age_to_force_merge_seconds. Default: 0 - disabled Min_version: 22.10 See in-depth description in ClickHouse documentation |
min_age_to_force_merge_on_partition_only | google.protobuf.BoolValue Whether min_age_to_force_merge_seconds should be applied only on the entire partition and not on subset. Default: false Min_version: 22.11 See in-depth description in ClickHouse documentation |
merge_selecting_sleep_ms | google.protobuf.Int64Value Sleep time for merge selecting when no part is selected. A lower setting triggers selecting tasks in background_schedule_pool frequently, which results in a large number of requests to ClickHouse Keeper in large-scale clusters. Default: 5000 Min_version: 21.10 See in-depth description in ClickHouse documentation |
Kafka
Field | Description |
---|---|
security_protocol | enum SecurityProtocol |
sasl_mechanism | enum SaslMechanism |
sasl_username | string |
sasl_password | string |
enable_ssl_certificate_verification | google.protobuf.BoolValue |
max_poll_interval_ms | google.protobuf.Int64Value The minimum value is 0. |
session_timeout_ms | google.protobuf.Int64Value The minimum value is 0. |
KafkaTopic
Field | Description |
---|---|
name | string Required. |
settings | Kafka Required. |
Rabbitmq
Field | Description |
---|---|
username | string RabbitMQ |
password | string RabbitMQ |
vhost | string RabbitMQ |
Compression
Field | Description |
---|---|
method | enum Method Compression method to use for the specified combination of min_part_size and min_part_size_ratio . |
min_part_size | int64 Minimum size of a part of a table. The minimum value is 1. |
min_part_size_ratio | double Minimum ratio of a part relative to the size of all the data in the table. |
level | google.protobuf.Int64Value The minimum value is 0. |
ExternalDictionary
Field | Description |
---|---|
name | string Required. Name of the external dictionary. |
structure | Structure Required. Set of attributes for the external dictionary. For in-depth description, see ClickHouse documentation |
layout | Layout Required. Layout for storing the dictionary in memory. For in-depth description, see ClickHouse documentation |
lifetime | oneof: fixed_lifetime or lifetime_range Setting for the period of time between dictionary updates. For details, see ClickHouse documentation |
fixed_lifetime | int64 Fixed interval between dictionary updates. |
lifetime_range | Range Range of intervals between dictionary updates for ClickHouse to choose from. |
source | oneof: http_source , mysql_source , clickhouse_source , mongodb_source or postgresql_source Description of the source for the external dictionary. |
http_source | HttpSource HTTP source for the dictionary. |
mysql_source | MysqlSource MySQL source for the dictionary. |
clickhouse_source | ClickhouseSource ClickHouse source for the dictionary. |
mongodb_source | MongodbSource MongoDB source for the dictionary. |
postgresql_source | PostgresqlSource PostgreSQL source for the dictionary. |
HttpSource
Field | Description |
---|---|
url | string Required. URL of the source dictionary available over HTTP. |
format | string Required. The data format. Valid values are all formats supported by ClickHouse SQL dialect. |
MysqlSource
Field | Description |
---|---|
db | string Required. Name of the MySQL database to connect to. |
table | string Required. Name of the database table to use as a ClickHouse dictionary. |
port | int64 Default port to use when connecting to a replica of the dictionary source. Acceptable values are 0 to 65535, inclusive. |
user | string Name of the default user for replicas of the dictionary source. |
password | string Password of the default user for replicas of the dictionary source. |
replicas[] | Replica List of MySQL replicas of the database used as dictionary source. The number of elements must be greater than 0. |
where | string Selection criteria for the data in the specified MySQL table. |
invalidate_query | string Query for checking the dictionary status, to pull only updated data. For more details, see ClickHouse documentation on dictionaries |
Replica
Field | Description |
---|---|
host | string Required. MySQL host of the replica. The maximum string length in characters is 253. |
priority | int64 Required. The priority of the replica that ClickHouse takes into account when connecting. Replica with the highest priority should have this field set to the lowest number. Value must be greater than 0. |
port | int64 Port to use when connecting to the replica. If a port is not specified for a replica, ClickHouse uses the port specified for the source. Acceptable values are 0 to 65535, inclusive. |
user | string Name of the MySQL database user. |
password | string Password of the MySQL database user. |
ClickhouseSource
Field | Description |
---|---|
db | string Required. Name of the ClickHouse database. |
table | string Required. Name of the table in the specified database to be used as the dictionary source. |
host | string Required. ClickHouse host of the specified database. The maximum string length in characters is 253. |
port | int64 Port to use when connecting to the host. Acceptable values are 0 to 65535, inclusive. |
user | string Required. Name of the ClickHouse database user. |
password | string Password of the ClickHouse database user. |
where | string Selection criteria for the data in the specified ClickHouse table. |
MongodbSource
Field | Description |
---|---|
db | string Required. Name of the MongoDB database. |
collection | string Required. Name of the collection in the specified database to be used as the dictionary source. |
host | string Required. MongoDB host of the specified database. The maximum string length in characters is 253. |
port | int64 Port to use when connecting to the host. Acceptable values are 0 to 65535, inclusive. |
user | string Required. Name of the MongoDB database user. |
password | string Password of the MongoDB database user. |
options | string |
PostgresqlSource
Field | Description |
---|---|
db | string Required. Name of the PostrgreSQL database. |
table | string Required. Name of the table in the specified database to be used as the dictionary source. |
hosts[] | string Name of the PostrgreSQL host The number of elements must be greater than 0. |
port | int64 Port to use when connecting to the host. Acceptable values are 0 to 65535, inclusive. |
user | string Required. Name of the PostrgreSQL database user. |
password | string Password of the PostrgreSQL database user. |
invalidate_query | string Query for checking the dictionary status, to pull only updated data. For more details, see ClickHouse documentation on dictionaries |
ssl_mode | enum SslMode Mode of SSL TCP/IP connection to the PostgreSQL host. For more details, see PostgreSQL documentation
|
Structure
Field | Description |
---|---|
id | Id Single numeric key column for the dictionary. |
key | Key Composite key for the dictionary, containing of one or more key columns. For details, see ClickHouse documentation |
range_min | Attribute Field holding the beginning of the range for dictionaries with RANGE_HASHED layout. For details, see ClickHouse documentation |
range_max | Attribute Field holding the end of the range for dictionaries with RANGE_HASHED layout. For details, see ClickHouse documentation |
attributes[] | Attribute Description of the fields available for database queries. For details, see ClickHouse documentation |
Attribute
Field | Description |
---|---|
name | string Required. Name of the column. |
type | string Required. Type of the column. |
null_value | string Default value for an element without data (for example, an empty string). |
expression | string Expression, describing the attribute, if applicable. |
hierarchical | bool Indication of hierarchy support. Default value: false . |
injective | bool Indication of injective mapping "id -> attribute". Default value: false . |
Id
Field | Description |
---|---|
name | string Required. Name of the numeric key. |
Key
Field | Description |
---|---|
attributes[] | Attribute Attributes of a complex key. The number of elements must be greater than 0. |
Layout
Field | Description |
---|---|
type | enum Type Required. Layout type for an external dictionary.
|
size_in_cells | int64 Number of cells in the cache. Rounded up to a power of two. Applicable only for CACHE and COMPLEX_KEY_CACHE layout types. |
Range
Field | Description |
---|---|
min | int64 Minimum dictionary lifetime. |
max | int64 Maximum dictionary lifetime. |
GraphiteRollup
Field | Description |
---|---|
name | string Required. Name for the specified combination of settings for Graphite rollup. |
patterns[] | Pattern Pattern to use for the rollup. The number of elements must be greater than 0. |
Pattern
Field | Description |
---|---|
regexp | string Pattern for metric names. |
function | string Required. Name of the aggregating function to apply to data of the age specified in retention . |
retention[] | Retention Age of data to use for thinning. The number of elements must be greater than 0. |
Retention
Field | Description |
---|---|
age | int64 Minimum age of the data in seconds. The minimum value is 0. |
precision | int64 Precision of determining the age of the data, in seconds. Value must be greater than 0. |
MaintenanceOperation
Field | Description |
---|---|
info | string Information about this maintenance operation. The maximum string length in characters is 256. |
delayed_until | google.protobuf.Timestamp Time until which this maintenance operation is delayed. |
Delete
Deletes the specified ClickHouse cluster.
rpc Delete (DeleteClusterRequest) returns (operation.Operation)
Metadata and response of Operation:
Operation.metadata:DeleteClusterMetadata
Operation.response:google.protobuf.Empty
DeleteClusterRequest
Field | Description |
---|---|
cluster_id | string Required. ID of the ClickHouse cluster to delete. To get the ClickHouse cluster ID, use a ClusterService.List request. The maximum string length in characters is 50. |
Operation
Field | Description |
---|---|
id | string ID of the operation. |
description | string Description of the operation. 0-256 characters long. |
created_at | google.protobuf.Timestamp Creation timestamp. |
created_by | string ID of the user or service account who initiated the operation. |
modified_at | google.protobuf.Timestamp The time when the Operation resource was last modified. |
done | bool If the value is false , it means the operation is still in progress. If true , the operation is completed, and either error or response is available. |
metadata | google.protobuf.Any Service-specific metadata associated with the operation. It typically contains the ID of the target resource that the operation is performed on. Any method that returns a long-running operation should document the metadata type, if any. |
result | oneof: error or response The operation result. If done == false and there was no failure detected, neither error nor response is set. If done == false and there was a failure detected, error is set. If done == true , exactly one of error or response is set. |
error | google.rpc.Status The error result of the operation in case of failure or cancellation. |
response | google.protobuf.Any if operation finished successfully. |
DeleteClusterMetadata
Field | Description |
---|---|
cluster_id | string ID of the ClickHouse cluster that is being deleted. |
Start
Starts the specified ClickHouse cluster.
rpc Start (StartClusterRequest) returns (operation.Operation)
Metadata and response of Operation:
Operation.metadata:StartClusterMetadata
Operation.response:Cluster
StartClusterRequest
Field | Description |
---|---|
cluster_id | string Required. ID of the ClickHouse cluster to start. The maximum string length in characters is 50. |
Operation
Field | Description |
---|---|
id | string ID of the operation. |
description | string Description of the operation. 0-256 characters long. |
created_at | google.protobuf.Timestamp Creation timestamp. |
created_by | string ID of the user or service account who initiated the operation. |
modified_at | google.protobuf.Timestamp The time when the Operation resource was last modified. |
done | bool If the value is false , it means the operation is still in progress. If true , the operation is completed, and either error or response is available. |
metadata | google.protobuf.Any Service-specific metadata associated with the operation. It typically contains the ID of the target resource that the operation is performed on. Any method that returns a long-running operation should document the metadata type, if any. |
result | oneof: error or response The operation result. If done == false and there was no failure detected, neither error nor response is set. If done == false and there was a failure detected, error is set. If done == true , exactly one of error or response is set. |
error | google.rpc.Status The error result of the operation in case of failure or cancellation. |
response | google.protobuf.Any if operation finished successfully. |
StartClusterMetadata
Field | Description |
---|---|
cluster_id | string ID of the ClickHouse cluster being started. |
Cluster
Field | Description |
---|---|
id | string ID of the ClickHouse cluster. This ID is assigned by MDB at creation time. |
folder_id | string ID of the folder that the ClickHouse cluster belongs to. |
created_at | google.protobuf.Timestamp Creation timestamp in RFC3339 |
name | string Name of the ClickHouse cluster. The name is unique within the folder. 1-63 characters long. |
description | string Description of the ClickHouse cluster. 0-256 characters long. |
labels | map<string,string> Custom labels for the ClickHouse cluster as key:value pairs. Maximum 64 per resource. |
environment | enum Environment Deployment environment of the ClickHouse cluster.
|
monitoring[] | Monitoring Description of monitoring systems relevant to the ClickHouse cluster. |
config | ClusterConfig Configuration of the ClickHouse cluster. |
network_id | string ID of the network that the cluster belongs to. |
health | enum Health Aggregated cluster health.
|
status | enum Status Current state of the cluster.
|
service_account_id | string ID of the service account used for access to Object Storage. |
maintenance_window | MaintenanceWindow Maintenance window for the cluster. |
planned_operation | MaintenanceOperation Planned maintenance operation to be started for the cluster within the nearest maintenance_window . |
security_group_ids[] | string User security groups |
deletion_protection | bool Deletion Protection inhibits deletion of the cluster |
Monitoring
Field | Description |
---|---|
name | string Name of the monitoring system. |
description | string Description of the monitoring system. |
link | string Link to the monitoring system charts for the ClickHouse cluster. |
ClusterConfig
Field | Description |
---|---|
version | string Version of the ClickHouse server software. |
clickhouse | Clickhouse Configuration and resource allocation for ClickHouse hosts. |
zookeeper | Zookeeper Configuration and resource allocation for ZooKeeper hosts. |
backup_window_start | google.type.TimeOfDay Time to start the daily backup, in the UTC timezone. |
access | Access Access policy for external services. |
cloud_storage | CloudStorage |
sql_database_management | google.protobuf.BoolValue Whether database management through SQL commands is enabled. |
sql_user_management | google.protobuf.BoolValue Whether user management through SQL commands is enabled. |
embedded_keeper | google.protobuf.BoolValue Whether cluster should use embedded Keeper instead of Zookeeper. |
Clickhouse
Field | Description |
---|---|
config | config.ClickhouseConfigSet Configuration settings of a ClickHouse server. |
resources | Resources Resources allocated to ClickHouse hosts. |
Zookeeper
Field | Description |
---|---|
resources | Resources Resources allocated to ZooKeeper hosts. |
Access
Field | Description |
---|---|
data_lens | bool Allow to export data from the cluster to DataLens. |
web_sql | bool Allow SQL queries to the cluster databases from the management console. See SQL queries in the management console for more details. |
metrika | bool Allow to import data from Yandex Metrica and AppMetrica to the cluster. See AppMetrica documentation |
serverless | bool Allow access to cluster for Serverless. |
data_transfer | bool Allow access for DataTransfer |
yandex_query | bool Allow access for Query |
CloudStorage
Field | Description |
---|---|
enabled | bool Whether to use Object Storage for storing ClickHouse data. |
move_factor | google.protobuf.DoubleValue Acceptable values are 0 to 1, inclusive. |
data_cache_enabled | google.protobuf.BoolValue |
data_cache_max_size | google.protobuf.Int64Value |
prefer_not_to_merge | google.protobuf.BoolValue |
ClickhouseConfigSet
Field | Description |
---|---|
effective_config | ClickhouseConfig Required. Effective settings for a ClickHouse cluster (a combination of settings defined in user_config and default_config ). |
user_config | ClickhouseConfig User-defined settings for a ClickHouse cluster. |
default_config | ClickhouseConfig Default configuration for a ClickHouse cluster. |
ClickhouseConfig
Field | Description |
---|---|
log_level | enum LogLevel Logging level for the ClickHouse cluster. Possible values: TRACE, DEBUG, INFORMATION, WARNING, ERROR. |
merge_tree | MergeTree Settings for the MergeTree engine. See description in ClickHouse documentation |
compression[] | Compression Compression settings for the ClickHouse cluster. See in-depth description in ClickHouse documentation |
dictionaries[] | ExternalDictionary Configuration of external dictionaries to be used by the ClickHouse cluster. See in-depth description in ClickHouse documentation |
graphite_rollup[] | GraphiteRollup Settings for thinning Graphite data. See in-depth description in ClickHouse documentation |
kafka | Kafka |
kafka_topics[] | KafkaTopic |
rabbitmq | Rabbitmq |
max_connections | google.protobuf.Int64Value Maximum number of inbound connections. The minimum value is 10. |
max_concurrent_queries | google.protobuf.Int64Value Maximum number of simultaneously processed requests. The minimum value is 10. |
keep_alive_timeout | google.protobuf.Int64Value Number of milliseconds that ClickHouse waits for incoming requests before closing the connection. |
uncompressed_cache_size | google.protobuf.Int64Value Cache size (in bytes) for uncompressed data used by MergeTree tables. |
mark_cache_size | google.protobuf.Int64Value Approximate size (in bytes) of the cache of "marks" used by MergeTree tables. Value must be greater than 0. |
max_table_size_to_drop | google.protobuf.Int64Value Maximum size of the table that can be deleted using a DROP query. |
max_partition_size_to_drop | google.protobuf.Int64Value Maximum size of the partition that can be deleted using a DROP query. |
builtin_dictionaries_reload_interval | google.protobuf.Int64Value The setting is deprecated and has no effect. |
timezone | string The server's time zone to be used in DateTime fields conversions. Specified as an IANA identifier. |
geobase_enabled | google.protobuf.BoolValue Enable or disable geobase. |
geobase_uri | string Address of the archive with the user geobase in Object Storage. |
query_log_retention_size | google.protobuf.Int64Value The maximum size that query_log can grow to before old data will be removed. If set to 0, automatic removal of query_log data based on size is disabled. |
query_log_retention_time | google.protobuf.Int64Value The maximum time that query_log records will be retained before removal. If set to 0, automatic removal of query_log data based on time is disabled. |
query_thread_log_enabled | google.protobuf.BoolValue Whether query_thread_log system table is enabled. |
query_thread_log_retention_size | google.protobuf.Int64Value The maximum size that query_thread_log can grow to before old data will be removed. If set to 0, automatic removal of query_thread_log data based on size is disabled. |
query_thread_log_retention_time | google.protobuf.Int64Value The maximum time that query_thread_log records will be retained before removal. If set to 0, automatic removal of query_thread_log data based on time is disabled. |
part_log_retention_size | google.protobuf.Int64Value The maximum size that part_log can grow to before old data will be removed. If set to 0, automatic removal of part_log data based on size is disabled. |
part_log_retention_time | google.protobuf.Int64Value The maximum time that part_log records will be retained before removal. If set to 0, automatic removal of part_log data based on time is disabled. |
metric_log_enabled | google.protobuf.BoolValue Whether metric_log system table is enabled. |
metric_log_retention_size | google.protobuf.Int64Value The maximum size that metric_log can grow to before old data will be removed. If set to 0, automatic removal of metric_log data based on size is disabled. |
metric_log_retention_time | google.protobuf.Int64Value The maximum time that metric_log records will be retained before removal. If set to 0, automatic removal of metric_log data based on time is disabled. |
trace_log_enabled | google.protobuf.BoolValue Whether trace_log system table is enabled. |
trace_log_retention_size | google.protobuf.Int64Value The maximum size that trace_log can grow to before old data will be removed. If set to 0, automatic removal of trace_log data based on size is disabled. |
trace_log_retention_time | google.protobuf.Int64Value The maximum time that trace_log records will be retained before removal. If set to 0, automatic removal of trace_log data based on time is disabled. |
text_log_enabled | google.protobuf.BoolValue Whether text_log system table is enabled. |
text_log_retention_size | google.protobuf.Int64Value The maximum size that text_log can grow to before old data will be removed. If set to 0, automatic removal of text_log data based on size is disabled. |
text_log_retention_time | google.protobuf.Int64Value The maximum time that text_log records will be retained before removal. If set to 0, automatic removal of text_log data based on time is disabled. |
text_log_level | enum LogLevel Logging level for text_log system table. Possible values: TRACE, DEBUG, INFORMATION, WARNING, ERROR. |
opentelemetry_span_log_enabled | google.protobuf.BoolValue Enable or disable opentelemetry_span_log system table. Default value: false. |
opentelemetry_span_log_retention_size | google.protobuf.Int64Value The maximum size that opentelemetry_span_log can grow to before old data will be removed. If set to 0 (default), automatic removal of opentelemetry_span_log data based on size is disabled. The minimum value is 0. |
opentelemetry_span_log_retention_time | google.protobuf.Int64Value The maximum time that opentelemetry_span_log records will be retained before removal. If set to 0, automatic removal of opentelemetry_span_log data based on time is disabled. The minimum value is 0. |
query_views_log_enabled | google.protobuf.BoolValue Enable or disable query_views_log system table. Default value: false. |
query_views_log_retention_size | google.protobuf.Int64Value The maximum size that query_views_log can grow to before old data will be removed. If set to 0 (default), automatic removal of query_views_log data based on size is disabled. The minimum value is 0. |
query_views_log_retention_time | google.protobuf.Int64Value The maximum time that query_views_log records will be retained before removal. If set to 0, automatic removal of query_views_log data based on time is disabled. The minimum value is 0. |
asynchronous_metric_log_enabled | google.protobuf.BoolValue Enable or disable asynchronous_metric_log system table. Default value: false. |
asynchronous_metric_log_retention_size | google.protobuf.Int64Value The maximum size that asynchronous_metric_log can grow to before old data will be removed. If set to 0 (default), automatic removal of asynchronous_metric_log data based on size is disabled. The minimum value is 0. |
asynchronous_metric_log_retention_time | google.protobuf.Int64Value The maximum time that asynchronous_metric_log records will be retained before removal. If set to 0, automatic removal of asynchronous_metric_log data based on time is disabled. The minimum value is 0. |
session_log_enabled | google.protobuf.BoolValue Enable or disable session_log system table. Default value: false. |
session_log_retention_size | google.protobuf.Int64Value The maximum size that session_log can grow to before old data will be removed. If set to 0 (default), automatic removal of session_log data based on size is disabled. The minimum value is 0. |
session_log_retention_time | google.protobuf.Int64Value The maximum time that session_log records will be retained before removal. If set to 0, automatic removal of session_log data based on time is disabled. The minimum value is 0. |
zookeeper_log_enabled | google.protobuf.BoolValue Enable or disable zookeeper_log system table. Default value: false. |
zookeeper_log_retention_size | google.protobuf.Int64Value The maximum size that zookeeper_log can grow to before old data will be removed. If set to 0 (default), automatic removal of zookeeper_log data based on size is disabled. The minimum value is 0. |
zookeeper_log_retention_time | google.protobuf.Int64Value The maximum time that zookeeper_log records will be retained before removal. If set to 0, automatic removal of zookeeper_log data based on time is disabled. The minimum value is 0. |
asynchronous_insert_log_enabled | google.protobuf.BoolValue Enable or disable asynchronous_insert_log system table. Default value: false. Minimal required ClickHouse version: 22.10. |
asynchronous_insert_log_retention_size | google.protobuf.Int64Value The maximum size that asynchronous_insert_log can grow to before old data will be removed. If set to 0 (default), automatic removal of asynchronous_insert_log data based on size is disabled. The minimum value is 0. |
asynchronous_insert_log_retention_time | google.protobuf.Int64Value The maximum time that asynchronous_insert_log records will be retained before removal. If set to 0, automatic removal of asynchronous_insert_log data based on time is disabled. The minimum value is 0. |
background_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_merges_mutations_concurrency_ratio | google.protobuf.Int64Value Sets a ratio between the number of threads and the number of background merges and mutations that can be executed concurrently. For example, if the ratio equals to 2 and background_pool_size is set to 16 then ClickHouse can execute 32 background merges concurrently. This is possible, because background operations could be suspended and postponed. This is needed to give small merges more execution priority. You can only increase this ratio at runtime. To lower it you have to restart the server. The same as for background_pool_size setting background_merges_mutations_concurrency_ratio could be applied from the default profile for backward compatibility. Default: 2 See in-depth description in ClickHouse documentation |
background_schedule_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_fetches_pool_size | google.protobuf.Int64Value Sets the number of threads performing background fetches for tables with ReplicatedMergeTree engines. Default value: 8. More info see in ClickHouse documentation |
background_move_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_distributed_schedule_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_buffer_flush_schedule_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_message_broker_schedule_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_common_pool_size | google.protobuf.Int64Value The maximum number of threads that will be used for performing a variety of operations (mostly garbage collection) for *MergeTree-engine tables in a background. Default: 8 See in-depth description in ClickHouse documentation |
default_database | google.protobuf.StringValue The default database. To get a list of cluster databases, see Yandex Managed ClickHouse documentation. |
total_memory_profiler_step | google.protobuf.Int64Value Sets the memory size (in bytes) for a stack trace at every peak allocation step. Default value: 4194304. More info see in ClickHouse documentation |
total_memory_tracker_sample_probability | google.protobuf.DoubleValue |
MergeTree
Field | Description |
---|---|
replicated_deduplication_window | google.protobuf.Int64Value Number of blocks of hashes to keep in ZooKeeper. |
replicated_deduplication_window_seconds | google.protobuf.Int64Value Period of time to keep blocks of hashes for. |
parts_to_delay_insert | google.protobuf.Int64Value If table contains at least that many active parts in single partition, artificially slow down insert into table. |
parts_to_throw_insert | google.protobuf.Int64Value If more than this number active parts in single partition, throw 'Too many parts ...' exception. |
inactive_parts_to_delay_insert | google.protobuf.Int64Value |
inactive_parts_to_throw_insert | google.protobuf.Int64Value |
max_replicated_merges_in_queue | google.protobuf.Int64Value How many tasks of merging and mutating parts are allowed simultaneously in ReplicatedMergeTree queue. |
number_of_free_entries_in_pool_to_lower_max_size_of_merge | google.protobuf.Int64Value If there is less than specified number of free entries in background pool (or replicated queue), start to lower maximum size of merge to process. |
max_bytes_to_merge_at_min_space_in_pool | google.protobuf.Int64Value Maximum in total size of parts to merge, when there are minimum free threads in background pool (or entries in replication queue). |
max_bytes_to_merge_at_max_space_in_pool | google.protobuf.Int64Value |
min_bytes_for_wide_part | google.protobuf.Int64Value Minimum number of bytes in a data part that can be stored in Wide format. More info see in ClickHouse documentation |
min_rows_for_wide_part | google.protobuf.Int64Value Minimum number of rows in a data part that can be stored in Wide format. More info see in ClickHouse documentation |
ttl_only_drop_parts | google.protobuf.BoolValue Enables or disables complete dropping of data parts where all rows are expired in MergeTree tables. More info see in ClickHouse documentation |
allow_remote_fs_zero_copy_replication | google.protobuf.BoolValue |
merge_with_ttl_timeout | google.protobuf.Int64Value |
merge_with_recompression_ttl_timeout | google.protobuf.Int64Value |
max_parts_in_total | google.protobuf.Int64Value |
max_number_of_merges_with_ttl_in_pool | google.protobuf.Int64Value |
cleanup_delay_period | google.protobuf.Int64Value |
number_of_free_entries_in_pool_to_execute_mutation | google.protobuf.Int64Value |
max_avg_part_size_for_too_many_parts | google.protobuf.Int64Value The 'too many parts' check according to 'parts_to_delay_insert' and 'parts_to_throw_insert' will be active only if the average part size (in the relevant partition) is not larger than the specified threshold. If it is larger than the specified threshold, the INSERTs will be neither delayed or rejected. This allows to have hundreds of terabytes in a single table on a single server if the parts are successfully merged to larger parts. This does not affect the thresholds on inactive parts or total parts. Default: 1 GiB Min version: 22.10 See in-depth description in ClickHouse GitHub |
min_age_to_force_merge_seconds | google.protobuf.Int64Value Merge parts if every part in the range is older than the value of min_age_to_force_merge_seconds. Default: 0 - disabled Min_version: 22.10 See in-depth description in ClickHouse documentation |
min_age_to_force_merge_on_partition_only | google.protobuf.BoolValue Whether min_age_to_force_merge_seconds should be applied only on the entire partition and not on subset. Default: false Min_version: 22.11 See in-depth description in ClickHouse documentation |
merge_selecting_sleep_ms | google.protobuf.Int64Value Sleep time for merge selecting when no part is selected. A lower setting triggers selecting tasks in background_schedule_pool frequently, which results in a large number of requests to ClickHouse Keeper in large-scale clusters. Default: 5000 Min_version: 21.10 See in-depth description in ClickHouse documentation |
Kafka
Field | Description |
---|---|
security_protocol | enum SecurityProtocol |
sasl_mechanism | enum SaslMechanism |
sasl_username | string |
sasl_password | string |
enable_ssl_certificate_verification | google.protobuf.BoolValue |
max_poll_interval_ms | google.protobuf.Int64Value The minimum value is 0. |
session_timeout_ms | google.protobuf.Int64Value The minimum value is 0. |
KafkaTopic
Field | Description |
---|---|
name | string Required. |
settings | Kafka Required. |
Rabbitmq
Field | Description |
---|---|
username | string RabbitMQ |
password | string RabbitMQ |
vhost | string RabbitMQ |
Compression
Field | Description |
---|---|
method | enum Method Compression method to use for the specified combination of min_part_size and min_part_size_ratio . |
min_part_size | int64 Minimum size of a part of a table. The minimum value is 1. |
min_part_size_ratio | double Minimum ratio of a part relative to the size of all the data in the table. |
level | google.protobuf.Int64Value The minimum value is 0. |
ExternalDictionary
Field | Description |
---|---|
name | string Required. Name of the external dictionary. |
structure | Structure Required. Set of attributes for the external dictionary. For in-depth description, see ClickHouse documentation |
layout | Layout Required. Layout for storing the dictionary in memory. For in-depth description, see ClickHouse documentation |
lifetime | oneof: fixed_lifetime or lifetime_range Setting for the period of time between dictionary updates. For details, see ClickHouse documentation |
fixed_lifetime | int64 Fixed interval between dictionary updates. |
lifetime_range | Range Range of intervals between dictionary updates for ClickHouse to choose from. |
source | oneof: http_source , mysql_source , clickhouse_source , mongodb_source or postgresql_source Description of the source for the external dictionary. |
http_source | HttpSource HTTP source for the dictionary. |
mysql_source | MysqlSource MySQL source for the dictionary. |
clickhouse_source | ClickhouseSource ClickHouse source for the dictionary. |
mongodb_source | MongodbSource MongoDB source for the dictionary. |
postgresql_source | PostgresqlSource PostgreSQL source for the dictionary. |
HttpSource
Field | Description |
---|---|
url | string Required. URL of the source dictionary available over HTTP. |
format | string Required. The data format. Valid values are all formats supported by ClickHouse SQL dialect. |
MysqlSource
Field | Description |
---|---|
db | string Required. Name of the MySQL database to connect to. |
table | string Required. Name of the database table to use as a ClickHouse dictionary. |
port | int64 Default port to use when connecting to a replica of the dictionary source. Acceptable values are 0 to 65535, inclusive. |
user | string Name of the default user for replicas of the dictionary source. |
password | string Password of the default user for replicas of the dictionary source. |
replicas[] | Replica List of MySQL replicas of the database used as dictionary source. The number of elements must be greater than 0. |
where | string Selection criteria for the data in the specified MySQL table. |
invalidate_query | string Query for checking the dictionary status, to pull only updated data. For more details, see ClickHouse documentation on dictionaries |
Replica
Field | Description |
---|---|
host | string Required. MySQL host of the replica. The maximum string length in characters is 253. |
priority | int64 Required. The priority of the replica that ClickHouse takes into account when connecting. Replica with the highest priority should have this field set to the lowest number. Value must be greater than 0. |
port | int64 Port to use when connecting to the replica. If a port is not specified for a replica, ClickHouse uses the port specified for the source. Acceptable values are 0 to 65535, inclusive. |
user | string Name of the MySQL database user. |
password | string Password of the MySQL database user. |
ClickhouseSource
Field | Description |
---|---|
db | string Required. Name of the ClickHouse database. |
table | string Required. Name of the table in the specified database to be used as the dictionary source. |
host | string Required. ClickHouse host of the specified database. The maximum string length in characters is 253. |
port | int64 Port to use when connecting to the host. Acceptable values are 0 to 65535, inclusive. |
user | string Required. Name of the ClickHouse database user. |
password | string Password of the ClickHouse database user. |
where | string Selection criteria for the data in the specified ClickHouse table. |
MongodbSource
Field | Description |
---|---|
db | string Required. Name of the MongoDB database. |
collection | string Required. Name of the collection in the specified database to be used as the dictionary source. |
host | string Required. MongoDB host of the specified database. The maximum string length in characters is 253. |
port | int64 Port to use when connecting to the host. Acceptable values are 0 to 65535, inclusive. |
user | string Required. Name of the MongoDB database user. |
password | string Password of the MongoDB database user. |
options | string |
PostgresqlSource
Field | Description |
---|---|
db | string Required. Name of the PostrgreSQL database. |
table | string Required. Name of the table in the specified database to be used as the dictionary source. |
hosts[] | string Name of the PostrgreSQL host The number of elements must be greater than 0. |
port | int64 Port to use when connecting to the host. Acceptable values are 0 to 65535, inclusive. |
user | string Required. Name of the PostrgreSQL database user. |
password | string Password of the PostrgreSQL database user. |
invalidate_query | string Query for checking the dictionary status, to pull only updated data. For more details, see ClickHouse documentation on dictionaries |
ssl_mode | enum SslMode Mode of SSL TCP/IP connection to the PostgreSQL host. For more details, see PostgreSQL documentation
|
Structure
Field | Description |
---|---|
id | Id Single numeric key column for the dictionary. |
key | Key Composite key for the dictionary, containing of one or more key columns. For details, see ClickHouse documentation |
range_min | Attribute Field holding the beginning of the range for dictionaries with RANGE_HASHED layout. For details, see ClickHouse documentation |
range_max | Attribute Field holding the end of the range for dictionaries with RANGE_HASHED layout. For details, see ClickHouse documentation |
attributes[] | Attribute Description of the fields available for database queries. For details, see ClickHouse documentation |
Attribute
Field | Description |
---|---|
name | string Required. Name of the column. |
type | string Required. Type of the column. |
null_value | string Default value for an element without data (for example, an empty string). |
expression | string Expression, describing the attribute, if applicable. |
hierarchical | bool Indication of hierarchy support. Default value: false . |
injective | bool Indication of injective mapping "id -> attribute". Default value: false . |
Id
Field | Description |
---|---|
name | string Required. Name of the numeric key. |
Key
Field | Description |
---|---|
attributes[] | Attribute Attributes of a complex key. The number of elements must be greater than 0. |
Layout
Field | Description |
---|---|
type | enum Type Required. Layout type for an external dictionary.
|
size_in_cells | int64 Number of cells in the cache. Rounded up to a power of two. Applicable only for CACHE and COMPLEX_KEY_CACHE layout types. |
Range
Field | Description |
---|---|
min | int64 Minimum dictionary lifetime. |
max | int64 Maximum dictionary lifetime. |
GraphiteRollup
Field | Description |
---|---|
name | string Required. Name for the specified combination of settings for Graphite rollup. |
patterns[] | Pattern Pattern to use for the rollup. The number of elements must be greater than 0. |
Pattern
Field | Description |
---|---|
regexp | string Pattern for metric names. |
function | string Required. Name of the aggregating function to apply to data of the age specified in retention . |
retention[] | Retention Age of data to use for thinning. The number of elements must be greater than 0. |
Retention
Field | Description |
---|---|
age | int64 Minimum age of the data in seconds. The minimum value is 0. |
precision | int64 Precision of determining the age of the data, in seconds. Value must be greater than 0. |
Resources
Field | Description |
---|---|
resource_preset_id | string ID of the preset for computational resources available to a host (CPU, memory etc.). All available presets are listed in the documentation |
disk_size | int64 Volume of the storage available to a host, in bytes. |
disk_type_id | string Type of the storage environment for the host. Possible values:
|
MaintenanceWindow
Field | Description |
---|---|
policy | oneof: anytime or weekly_maintenance_window The maintenance policy in effect. |
anytime | AnytimeMaintenanceWindow Maintenance operation can be scheduled anytime. |
weekly_maintenance_window | WeeklyMaintenanceWindow Maintenance operation can be scheduled on a weekly basis. |
AnytimeMaintenanceWindow
Empty.
WeeklyMaintenanceWindow
Field | Description |
---|---|
day | enum WeekDay Day of the week (in DDD format). |
hour | int64 Hour of the day in UTC (in HH format). Acceptable values are 1 to 24, inclusive. |
MaintenanceOperation
Field | Description |
---|---|
info | string Information about this maintenance operation. The maximum string length in characters is 256. |
delayed_until | google.protobuf.Timestamp Time until which this maintenance operation is delayed. |
Stop
Stops the specified ClickHouse cluster.
rpc Stop (StopClusterRequest) returns (operation.Operation)
Metadata and response of Operation:
Operation.metadata:StopClusterMetadata
Operation.response:Cluster
StopClusterRequest
Field | Description |
---|---|
cluster_id | string Required. ID of the ClickHouse cluster to stop. The maximum string length in characters is 50. |
Operation
Field | Description |
---|---|
id | string ID of the operation. |
description | string Description of the operation. 0-256 characters long. |
created_at | google.protobuf.Timestamp Creation timestamp. |
created_by | string ID of the user or service account who initiated the operation. |
modified_at | google.protobuf.Timestamp The time when the Operation resource was last modified. |
done | bool If the value is false , it means the operation is still in progress. If true , the operation is completed, and either error or response is available. |
metadata | google.protobuf.Any Service-specific metadata associated with the operation. It typically contains the ID of the target resource that the operation is performed on. Any method that returns a long-running operation should document the metadata type, if any. |
result | oneof: error or response The operation result. If done == false and there was no failure detected, neither error nor response is set. If done == false and there was a failure detected, error is set. If done == true , exactly one of error or response is set. |
error | google.rpc.Status The error result of the operation in case of failure or cancellation. |
response | google.protobuf.Any if operation finished successfully. |
StopClusterMetadata
Field | Description |
---|---|
cluster_id | string ID of the ClickHouse cluster being stopped. |
Cluster
Field | Description |
---|---|
id | string ID of the ClickHouse cluster. This ID is assigned by MDB at creation time. |
folder_id | string ID of the folder that the ClickHouse cluster belongs to. |
created_at | google.protobuf.Timestamp Creation timestamp in RFC3339 |
name | string Name of the ClickHouse cluster. The name is unique within the folder. 1-63 characters long. |
description | string Description of the ClickHouse cluster. 0-256 characters long. |
labels | map<string,string> Custom labels for the ClickHouse cluster as key:value pairs. Maximum 64 per resource. |
environment | enum Environment Deployment environment of the ClickHouse cluster.
|
monitoring[] | Monitoring Description of monitoring systems relevant to the ClickHouse cluster. |
config | ClusterConfig Configuration of the ClickHouse cluster. |
network_id | string ID of the network that the cluster belongs to. |
health | enum Health Aggregated cluster health.
|
status | enum Status Current state of the cluster.
|
service_account_id | string ID of the service account used for access to Object Storage. |
maintenance_window | MaintenanceWindow Maintenance window for the cluster. |
planned_operation | MaintenanceOperation Planned maintenance operation to be started for the cluster within the nearest maintenance_window . |
security_group_ids[] | string User security groups |
deletion_protection | bool Deletion Protection inhibits deletion of the cluster |
Monitoring
Field | Description |
---|---|
name | string Name of the monitoring system. |
description | string Description of the monitoring system. |
link | string Link to the monitoring system charts for the ClickHouse cluster. |
ClusterConfig
Field | Description |
---|---|
version | string Version of the ClickHouse server software. |
clickhouse | Clickhouse Configuration and resource allocation for ClickHouse hosts. |
zookeeper | Zookeeper Configuration and resource allocation for ZooKeeper hosts. |
backup_window_start | google.type.TimeOfDay Time to start the daily backup, in the UTC timezone. |
access | Access Access policy for external services. |
cloud_storage | CloudStorage |
sql_database_management | google.protobuf.BoolValue Whether database management through SQL commands is enabled. |
sql_user_management | google.protobuf.BoolValue Whether user management through SQL commands is enabled. |
embedded_keeper | google.protobuf.BoolValue Whether cluster should use embedded Keeper instead of Zookeeper. |
Clickhouse
Field | Description |
---|---|
config | config.ClickhouseConfigSet Configuration settings of a ClickHouse server. |
resources | Resources Resources allocated to ClickHouse hosts. |
Zookeeper
Field | Description |
---|---|
resources | Resources Resources allocated to ZooKeeper hosts. |
Access
Field | Description |
---|---|
data_lens | bool Allow to export data from the cluster to DataLens. |
web_sql | bool Allow SQL queries to the cluster databases from the management console. See SQL queries in the management console for more details. |
metrika | bool Allow to import data from Yandex Metrica and AppMetrica to the cluster. See AppMetrica documentation |
serverless | bool Allow access to cluster for Serverless. |
data_transfer | bool Allow access for DataTransfer |
yandex_query | bool Allow access for Query |
CloudStorage
Field | Description |
---|---|
enabled | bool Whether to use Object Storage for storing ClickHouse data. |
move_factor | google.protobuf.DoubleValue Acceptable values are 0 to 1, inclusive. |
data_cache_enabled | google.protobuf.BoolValue |
data_cache_max_size | google.protobuf.Int64Value |
prefer_not_to_merge | google.protobuf.BoolValue |
ClickhouseConfigSet
Field | Description |
---|---|
effective_config | ClickhouseConfig Required. Effective settings for a ClickHouse cluster (a combination of settings defined in user_config and default_config ). |
user_config | ClickhouseConfig User-defined settings for a ClickHouse cluster. |
default_config | ClickhouseConfig Default configuration for a ClickHouse cluster. |
ClickhouseConfig
Field | Description |
---|---|
log_level | enum LogLevel Logging level for the ClickHouse cluster. Possible values: TRACE, DEBUG, INFORMATION, WARNING, ERROR. |
merge_tree | MergeTree Settings for the MergeTree engine. See description in ClickHouse documentation |
compression[] | Compression Compression settings for the ClickHouse cluster. See in-depth description in ClickHouse documentation |
dictionaries[] | ExternalDictionary Configuration of external dictionaries to be used by the ClickHouse cluster. See in-depth description in ClickHouse documentation |
graphite_rollup[] | GraphiteRollup Settings for thinning Graphite data. See in-depth description in ClickHouse documentation |
kafka | Kafka |
kafka_topics[] | KafkaTopic |
rabbitmq | Rabbitmq |
max_connections | google.protobuf.Int64Value Maximum number of inbound connections. The minimum value is 10. |
max_concurrent_queries | google.protobuf.Int64Value Maximum number of simultaneously processed requests. The minimum value is 10. |
keep_alive_timeout | google.protobuf.Int64Value Number of milliseconds that ClickHouse waits for incoming requests before closing the connection. |
uncompressed_cache_size | google.protobuf.Int64Value Cache size (in bytes) for uncompressed data used by MergeTree tables. |
mark_cache_size | google.protobuf.Int64Value Approximate size (in bytes) of the cache of "marks" used by MergeTree tables. Value must be greater than 0. |
max_table_size_to_drop | google.protobuf.Int64Value Maximum size of the table that can be deleted using a DROP query. |
max_partition_size_to_drop | google.protobuf.Int64Value Maximum size of the partition that can be deleted using a DROP query. |
builtin_dictionaries_reload_interval | google.protobuf.Int64Value The setting is deprecated and has no effect. |
timezone | string The server's time zone to be used in DateTime fields conversions. Specified as an IANA identifier. |
geobase_enabled | google.protobuf.BoolValue Enable or disable geobase. |
geobase_uri | string Address of the archive with the user geobase in Object Storage. |
query_log_retention_size | google.protobuf.Int64Value The maximum size that query_log can grow to before old data will be removed. If set to 0, automatic removal of query_log data based on size is disabled. |
query_log_retention_time | google.protobuf.Int64Value The maximum time that query_log records will be retained before removal. If set to 0, automatic removal of query_log data based on time is disabled. |
query_thread_log_enabled | google.protobuf.BoolValue Whether query_thread_log system table is enabled. |
query_thread_log_retention_size | google.protobuf.Int64Value The maximum size that query_thread_log can grow to before old data will be removed. If set to 0, automatic removal of query_thread_log data based on size is disabled. |
query_thread_log_retention_time | google.protobuf.Int64Value The maximum time that query_thread_log records will be retained before removal. If set to 0, automatic removal of query_thread_log data based on time is disabled. |
part_log_retention_size | google.protobuf.Int64Value The maximum size that part_log can grow to before old data will be removed. If set to 0, automatic removal of part_log data based on size is disabled. |
part_log_retention_time | google.protobuf.Int64Value The maximum time that part_log records will be retained before removal. If set to 0, automatic removal of part_log data based on time is disabled. |
metric_log_enabled | google.protobuf.BoolValue Whether metric_log system table is enabled. |
metric_log_retention_size | google.protobuf.Int64Value The maximum size that metric_log can grow to before old data will be removed. If set to 0, automatic removal of metric_log data based on size is disabled. |
metric_log_retention_time | google.protobuf.Int64Value The maximum time that metric_log records will be retained before removal. If set to 0, automatic removal of metric_log data based on time is disabled. |
trace_log_enabled | google.protobuf.BoolValue Whether trace_log system table is enabled. |
trace_log_retention_size | google.protobuf.Int64Value The maximum size that trace_log can grow to before old data will be removed. If set to 0, automatic removal of trace_log data based on size is disabled. |
trace_log_retention_time | google.protobuf.Int64Value The maximum time that trace_log records will be retained before removal. If set to 0, automatic removal of trace_log data based on time is disabled. |
text_log_enabled | google.protobuf.BoolValue Whether text_log system table is enabled. |
text_log_retention_size | google.protobuf.Int64Value The maximum size that text_log can grow to before old data will be removed. If set to 0, automatic removal of text_log data based on size is disabled. |
text_log_retention_time | google.protobuf.Int64Value The maximum time that text_log records will be retained before removal. If set to 0, automatic removal of text_log data based on time is disabled. |
text_log_level | enum LogLevel Logging level for text_log system table. Possible values: TRACE, DEBUG, INFORMATION, WARNING, ERROR. |
opentelemetry_span_log_enabled | google.protobuf.BoolValue Enable or disable opentelemetry_span_log system table. Default value: false. |
opentelemetry_span_log_retention_size | google.protobuf.Int64Value The maximum size that opentelemetry_span_log can grow to before old data will be removed. If set to 0 (default), automatic removal of opentelemetry_span_log data based on size is disabled. The minimum value is 0. |
opentelemetry_span_log_retention_time | google.protobuf.Int64Value The maximum time that opentelemetry_span_log records will be retained before removal. If set to 0, automatic removal of opentelemetry_span_log data based on time is disabled. The minimum value is 0. |
query_views_log_enabled | google.protobuf.BoolValue Enable or disable query_views_log system table. Default value: false. |
query_views_log_retention_size | google.protobuf.Int64Value The maximum size that query_views_log can grow to before old data will be removed. If set to 0 (default), automatic removal of query_views_log data based on size is disabled. The minimum value is 0. |
query_views_log_retention_time | google.protobuf.Int64Value The maximum time that query_views_log records will be retained before removal. If set to 0, automatic removal of query_views_log data based on time is disabled. The minimum value is 0. |
asynchronous_metric_log_enabled | google.protobuf.BoolValue Enable or disable asynchronous_metric_log system table. Default value: false. |
asynchronous_metric_log_retention_size | google.protobuf.Int64Value The maximum size that asynchronous_metric_log can grow to before old data will be removed. If set to 0 (default), automatic removal of asynchronous_metric_log data based on size is disabled. The minimum value is 0. |
asynchronous_metric_log_retention_time | google.protobuf.Int64Value The maximum time that asynchronous_metric_log records will be retained before removal. If set to 0, automatic removal of asynchronous_metric_log data based on time is disabled. The minimum value is 0. |
session_log_enabled | google.protobuf.BoolValue Enable or disable session_log system table. Default value: false. |
session_log_retention_size | google.protobuf.Int64Value The maximum size that session_log can grow to before old data will be removed. If set to 0 (default), automatic removal of session_log data based on size is disabled. The minimum value is 0. |
session_log_retention_time | google.protobuf.Int64Value The maximum time that session_log records will be retained before removal. If set to 0, automatic removal of session_log data based on time is disabled. The minimum value is 0. |
zookeeper_log_enabled | google.protobuf.BoolValue Enable or disable zookeeper_log system table. Default value: false. |
zookeeper_log_retention_size | google.protobuf.Int64Value The maximum size that zookeeper_log can grow to before old data will be removed. If set to 0 (default), automatic removal of zookeeper_log data based on size is disabled. The minimum value is 0. |
zookeeper_log_retention_time | google.protobuf.Int64Value The maximum time that zookeeper_log records will be retained before removal. If set to 0, automatic removal of zookeeper_log data based on time is disabled. The minimum value is 0. |
asynchronous_insert_log_enabled | google.protobuf.BoolValue Enable or disable asynchronous_insert_log system table. Default value: false. Minimal required ClickHouse version: 22.10. |
asynchronous_insert_log_retention_size | google.protobuf.Int64Value The maximum size that asynchronous_insert_log can grow to before old data will be removed. If set to 0 (default), automatic removal of asynchronous_insert_log data based on size is disabled. The minimum value is 0. |
asynchronous_insert_log_retention_time | google.protobuf.Int64Value The maximum time that asynchronous_insert_log records will be retained before removal. If set to 0, automatic removal of asynchronous_insert_log data based on time is disabled. The minimum value is 0. |
background_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_merges_mutations_concurrency_ratio | google.protobuf.Int64Value Sets a ratio between the number of threads and the number of background merges and mutations that can be executed concurrently. For example, if the ratio equals to 2 and background_pool_size is set to 16 then ClickHouse can execute 32 background merges concurrently. This is possible, because background operations could be suspended and postponed. This is needed to give small merges more execution priority. You can only increase this ratio at runtime. To lower it you have to restart the server. The same as for background_pool_size setting background_merges_mutations_concurrency_ratio could be applied from the default profile for backward compatibility. Default: 2 See in-depth description in ClickHouse documentation |
background_schedule_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_fetches_pool_size | google.protobuf.Int64Value Sets the number of threads performing background fetches for tables with ReplicatedMergeTree engines. Default value: 8. More info see in ClickHouse documentation |
background_move_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_distributed_schedule_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_buffer_flush_schedule_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_message_broker_schedule_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_common_pool_size | google.protobuf.Int64Value The maximum number of threads that will be used for performing a variety of operations (mostly garbage collection) for *MergeTree-engine tables in a background. Default: 8 See in-depth description in ClickHouse documentation |
default_database | google.protobuf.StringValue The default database. To get a list of cluster databases, see Yandex Managed ClickHouse documentation. |
total_memory_profiler_step | google.protobuf.Int64Value Sets the memory size (in bytes) for a stack trace at every peak allocation step. Default value: 4194304. More info see in ClickHouse documentation |
total_memory_tracker_sample_probability | google.protobuf.DoubleValue |
MergeTree
Field | Description |
---|---|
replicated_deduplication_window | google.protobuf.Int64Value Number of blocks of hashes to keep in ZooKeeper. |
replicated_deduplication_window_seconds | google.protobuf.Int64Value Period of time to keep blocks of hashes for. |
parts_to_delay_insert | google.protobuf.Int64Value If table contains at least that many active parts in single partition, artificially slow down insert into table. |
parts_to_throw_insert | google.protobuf.Int64Value If more than this number active parts in single partition, throw 'Too many parts ...' exception. |
inactive_parts_to_delay_insert | google.protobuf.Int64Value |
inactive_parts_to_throw_insert | google.protobuf.Int64Value |
max_replicated_merges_in_queue | google.protobuf.Int64Value How many tasks of merging and mutating parts are allowed simultaneously in ReplicatedMergeTree queue. |
number_of_free_entries_in_pool_to_lower_max_size_of_merge | google.protobuf.Int64Value If there is less than specified number of free entries in background pool (or replicated queue), start to lower maximum size of merge to process. |
max_bytes_to_merge_at_min_space_in_pool | google.protobuf.Int64Value Maximum in total size of parts to merge, when there are minimum free threads in background pool (or entries in replication queue). |
max_bytes_to_merge_at_max_space_in_pool | google.protobuf.Int64Value |
min_bytes_for_wide_part | google.protobuf.Int64Value Minimum number of bytes in a data part that can be stored in Wide format. More info see in ClickHouse documentation |
min_rows_for_wide_part | google.protobuf.Int64Value Minimum number of rows in a data part that can be stored in Wide format. More info see in ClickHouse documentation |
ttl_only_drop_parts | google.protobuf.BoolValue Enables or disables complete dropping of data parts where all rows are expired in MergeTree tables. More info see in ClickHouse documentation |
allow_remote_fs_zero_copy_replication | google.protobuf.BoolValue |
merge_with_ttl_timeout | google.protobuf.Int64Value |
merge_with_recompression_ttl_timeout | google.protobuf.Int64Value |
max_parts_in_total | google.protobuf.Int64Value |
max_number_of_merges_with_ttl_in_pool | google.protobuf.Int64Value |
cleanup_delay_period | google.protobuf.Int64Value |
number_of_free_entries_in_pool_to_execute_mutation | google.protobuf.Int64Value |
max_avg_part_size_for_too_many_parts | google.protobuf.Int64Value The 'too many parts' check according to 'parts_to_delay_insert' and 'parts_to_throw_insert' will be active only if the average part size (in the relevant partition) is not larger than the specified threshold. If it is larger than the specified threshold, the INSERTs will be neither delayed or rejected. This allows to have hundreds of terabytes in a single table on a single server if the parts are successfully merged to larger parts. This does not affect the thresholds on inactive parts or total parts. Default: 1 GiB Min version: 22.10 See in-depth description in ClickHouse GitHub |
min_age_to_force_merge_seconds | google.protobuf.Int64Value Merge parts if every part in the range is older than the value of min_age_to_force_merge_seconds. Default: 0 - disabled Min_version: 22.10 See in-depth description in ClickHouse documentation |
min_age_to_force_merge_on_partition_only | google.protobuf.BoolValue Whether min_age_to_force_merge_seconds should be applied only on the entire partition and not on subset. Default: false Min_version: 22.11 See in-depth description in ClickHouse documentation |
merge_selecting_sleep_ms | google.protobuf.Int64Value Sleep time for merge selecting when no part is selected. A lower setting triggers selecting tasks in background_schedule_pool frequently, which results in a large number of requests to ClickHouse Keeper in large-scale clusters. Default: 5000 Min_version: 21.10 See in-depth description in ClickHouse documentation |
Kafka
Field | Description |
---|---|
security_protocol | enum SecurityProtocol |
sasl_mechanism | enum SaslMechanism |
sasl_username | string |
sasl_password | string |
enable_ssl_certificate_verification | google.protobuf.BoolValue |
max_poll_interval_ms | google.protobuf.Int64Value The minimum value is 0. |
session_timeout_ms | google.protobuf.Int64Value The minimum value is 0. |
KafkaTopic
Field | Description |
---|---|
name | string Required. |
settings | Kafka Required. |
Rabbitmq
Field | Description |
---|---|
username | string RabbitMQ |
password | string RabbitMQ |
vhost | string RabbitMQ |
Compression
Field | Description |
---|---|
method | enum Method Compression method to use for the specified combination of min_part_size and min_part_size_ratio . |
min_part_size | int64 Minimum size of a part of a table. The minimum value is 1. |
min_part_size_ratio | double Minimum ratio of a part relative to the size of all the data in the table. |
level | google.protobuf.Int64Value The minimum value is 0. |
ExternalDictionary
Field | Description |
---|---|
name | string Required. Name of the external dictionary. |
structure | Structure Required. Set of attributes for the external dictionary. For in-depth description, see ClickHouse documentation |
layout | Layout Required. Layout for storing the dictionary in memory. For in-depth description, see ClickHouse documentation |
lifetime | oneof: fixed_lifetime or lifetime_range Setting for the period of time between dictionary updates. For details, see ClickHouse documentation |
fixed_lifetime | int64 Fixed interval between dictionary updates. |
lifetime_range | Range Range of intervals between dictionary updates for ClickHouse to choose from. |
source | oneof: http_source , mysql_source , clickhouse_source , mongodb_source or postgresql_source Description of the source for the external dictionary. |
http_source | HttpSource HTTP source for the dictionary. |
mysql_source | MysqlSource MySQL source for the dictionary. |
clickhouse_source | ClickhouseSource ClickHouse source for the dictionary. |
mongodb_source | MongodbSource MongoDB source for the dictionary. |
postgresql_source | PostgresqlSource PostgreSQL source for the dictionary. |
HttpSource
Field | Description |
---|---|
url | string Required. URL of the source dictionary available over HTTP. |
format | string Required. The data format. Valid values are all formats supported by ClickHouse SQL dialect. |
MysqlSource
Field | Description |
---|---|
db | string Required. Name of the MySQL database to connect to. |
table | string Required. Name of the database table to use as a ClickHouse dictionary. |
port | int64 Default port to use when connecting to a replica of the dictionary source. Acceptable values are 0 to 65535, inclusive. |
user | string Name of the default user for replicas of the dictionary source. |
password | string Password of the default user for replicas of the dictionary source. |
replicas[] | Replica List of MySQL replicas of the database used as dictionary source. The number of elements must be greater than 0. |
where | string Selection criteria for the data in the specified MySQL table. |
invalidate_query | string Query for checking the dictionary status, to pull only updated data. For more details, see ClickHouse documentation on dictionaries |
Replica
Field | Description |
---|---|
host | string Required. MySQL host of the replica. The maximum string length in characters is 253. |
priority | int64 Required. The priority of the replica that ClickHouse takes into account when connecting. Replica with the highest priority should have this field set to the lowest number. Value must be greater than 0. |
port | int64 Port to use when connecting to the replica. If a port is not specified for a replica, ClickHouse uses the port specified for the source. Acceptable values are 0 to 65535, inclusive. |
user | string Name of the MySQL database user. |
password | string Password of the MySQL database user. |
ClickhouseSource
Field | Description |
---|---|
db | string Required. Name of the ClickHouse database. |
table | string Required. Name of the table in the specified database to be used as the dictionary source. |
host | string Required. ClickHouse host of the specified database. The maximum string length in characters is 253. |
port | int64 Port to use when connecting to the host. Acceptable values are 0 to 65535, inclusive. |
user | string Required. Name of the ClickHouse database user. |
password | string Password of the ClickHouse database user. |
where | string Selection criteria for the data in the specified ClickHouse table. |
MongodbSource
Field | Description |
---|---|
db | string Required. Name of the MongoDB database. |
collection | string Required. Name of the collection in the specified database to be used as the dictionary source. |
host | string Required. MongoDB host of the specified database. The maximum string length in characters is 253. |
port | int64 Port to use when connecting to the host. Acceptable values are 0 to 65535, inclusive. |
user | string Required. Name of the MongoDB database user. |
password | string Password of the MongoDB database user. |
options | string |
PostgresqlSource
Field | Description |
---|---|
db | string Required. Name of the PostrgreSQL database. |
table | string Required. Name of the table in the specified database to be used as the dictionary source. |
hosts[] | string Name of the PostrgreSQL host The number of elements must be greater than 0. |
port | int64 Port to use when connecting to the host. Acceptable values are 0 to 65535, inclusive. |
user | string Required. Name of the PostrgreSQL database user. |
password | string Password of the PostrgreSQL database user. |
invalidate_query | string Query for checking the dictionary status, to pull only updated data. For more details, see ClickHouse documentation on dictionaries |
ssl_mode | enum SslMode Mode of SSL TCP/IP connection to the PostgreSQL host. For more details, see PostgreSQL documentation
|
Structure
Field | Description |
---|---|
id | Id Single numeric key column for the dictionary. |
key | Key Composite key for the dictionary, containing of one or more key columns. For details, see ClickHouse documentation |
range_min | Attribute Field holding the beginning of the range for dictionaries with RANGE_HASHED layout. For details, see ClickHouse documentation |
range_max | Attribute Field holding the end of the range for dictionaries with RANGE_HASHED layout. For details, see ClickHouse documentation |
attributes[] | Attribute Description of the fields available for database queries. For details, see ClickHouse documentation |
Attribute
Field | Description |
---|---|
name | string Required. Name of the column. |
type | string Required. Type of the column. |
null_value | string Default value for an element without data (for example, an empty string). |
expression | string Expression, describing the attribute, if applicable. |
hierarchical | bool Indication of hierarchy support. Default value: false . |
injective | bool Indication of injective mapping "id -> attribute". Default value: false . |
Id
Field | Description |
---|---|
name | string Required. Name of the numeric key. |
Key
Field | Description |
---|---|
attributes[] | Attribute Attributes of a complex key. The number of elements must be greater than 0. |
Layout
Field | Description |
---|---|
type | enum Type Required. Layout type for an external dictionary.
|
size_in_cells | int64 Number of cells in the cache. Rounded up to a power of two. Applicable only for CACHE and COMPLEX_KEY_CACHE layout types. |
Range
Field | Description |
---|---|
min | int64 Minimum dictionary lifetime. |
max | int64 Maximum dictionary lifetime. |
GraphiteRollup
Field | Description |
---|---|
name | string Required. Name for the specified combination of settings for Graphite rollup. |
patterns[] | Pattern Pattern to use for the rollup. The number of elements must be greater than 0. |
Pattern
Field | Description |
---|---|
regexp | string Pattern for metric names. |
function | string Required. Name of the aggregating function to apply to data of the age specified in retention . |
retention[] | Retention Age of data to use for thinning. The number of elements must be greater than 0. |
Retention
Field | Description |
---|---|
age | int64 Minimum age of the data in seconds. The minimum value is 0. |
precision | int64 Precision of determining the age of the data, in seconds. Value must be greater than 0. |
Resources
Field | Description |
---|---|
resource_preset_id | string ID of the preset for computational resources available to a host (CPU, memory etc.). All available presets are listed in the documentation |
disk_size | int64 Volume of the storage available to a host, in bytes. |
disk_type_id | string Type of the storage environment for the host. Possible values:
|
MaintenanceWindow
Field | Description |
---|---|
policy | oneof: anytime or weekly_maintenance_window The maintenance policy in effect. |
anytime | AnytimeMaintenanceWindow Maintenance operation can be scheduled anytime. |
weekly_maintenance_window | WeeklyMaintenanceWindow Maintenance operation can be scheduled on a weekly basis. |
AnytimeMaintenanceWindow
Empty.
WeeklyMaintenanceWindow
Field | Description |
---|---|
day | enum WeekDay Day of the week (in DDD format). |
hour | int64 Hour of the day in UTC (in HH format). Acceptable values are 1 to 24, inclusive. |
MaintenanceOperation
Field | Description |
---|---|
info | string Information about this maintenance operation. The maximum string length in characters is 256. |
delayed_until | google.protobuf.Timestamp Time until which this maintenance operation is delayed. |
Move
Moves a ClickHouse cluster to the specified folder.
rpc Move (MoveClusterRequest) returns (operation.Operation)
Metadata and response of Operation:
Operation.metadata:MoveClusterMetadata
Operation.response:Cluster
MoveClusterRequest
Field | Description |
---|---|
cluster_id | string Required. ID of the ClickHouse cluster to move. The maximum string length in characters is 50. |
destination_folder_id | string Required. ID of the destination folder. The maximum string length in characters is 50. |
Operation
Field | Description |
---|---|
id | string ID of the operation. |
description | string Description of the operation. 0-256 characters long. |
created_at | google.protobuf.Timestamp Creation timestamp. |
created_by | string ID of the user or service account who initiated the operation. |
modified_at | google.protobuf.Timestamp The time when the Operation resource was last modified. |
done | bool If the value is false , it means the operation is still in progress. If true , the operation is completed, and either error or response is available. |
metadata | google.protobuf.Any Service-specific metadata associated with the operation. It typically contains the ID of the target resource that the operation is performed on. Any method that returns a long-running operation should document the metadata type, if any. |
result | oneof: error or response The operation result. If done == false and there was no failure detected, neither error nor response is set. If done == false and there was a failure detected, error is set. If done == true , exactly one of error or response is set. |
error | google.rpc.Status The error result of the operation in case of failure or cancellation. |
response | google.protobuf.Any if operation finished successfully. |
MoveClusterMetadata
Field | Description |
---|---|
cluster_id | string ID of the ClickHouse cluster being moved. |
source_folder_id | string ID of the source folder. |
destination_folder_id | string ID of the destination folder. |
Cluster
Field | Description |
---|---|
id | string ID of the ClickHouse cluster. This ID is assigned by MDB at creation time. |
folder_id | string ID of the folder that the ClickHouse cluster belongs to. |
created_at | google.protobuf.Timestamp Creation timestamp in RFC3339 |
name | string Name of the ClickHouse cluster. The name is unique within the folder. 1-63 characters long. |
description | string Description of the ClickHouse cluster. 0-256 characters long. |
labels | map<string,string> Custom labels for the ClickHouse cluster as key:value pairs. Maximum 64 per resource. |
environment | enum Environment Deployment environment of the ClickHouse cluster.
|
monitoring[] | Monitoring Description of monitoring systems relevant to the ClickHouse cluster. |
config | ClusterConfig Configuration of the ClickHouse cluster. |
network_id | string ID of the network that the cluster belongs to. |
health | enum Health Aggregated cluster health.
|
status | enum Status Current state of the cluster.
|
service_account_id | string ID of the service account used for access to Object Storage. |
maintenance_window | MaintenanceWindow Maintenance window for the cluster. |
planned_operation | MaintenanceOperation Planned maintenance operation to be started for the cluster within the nearest maintenance_window . |
security_group_ids[] | string User security groups |
deletion_protection | bool Deletion Protection inhibits deletion of the cluster |
Monitoring
Field | Description |
---|---|
name | string Name of the monitoring system. |
description | string Description of the monitoring system. |
link | string Link to the monitoring system charts for the ClickHouse cluster. |
ClusterConfig
Field | Description |
---|---|
version | string Version of the ClickHouse server software. |
clickhouse | Clickhouse Configuration and resource allocation for ClickHouse hosts. |
zookeeper | Zookeeper Configuration and resource allocation for ZooKeeper hosts. |
backup_window_start | google.type.TimeOfDay Time to start the daily backup, in the UTC timezone. |
access | Access Access policy for external services. |
cloud_storage | CloudStorage |
sql_database_management | google.protobuf.BoolValue Whether database management through SQL commands is enabled. |
sql_user_management | google.protobuf.BoolValue Whether user management through SQL commands is enabled. |
embedded_keeper | google.protobuf.BoolValue Whether cluster should use embedded Keeper instead of Zookeeper. |
Clickhouse
Field | Description |
---|---|
config | config.ClickhouseConfigSet Configuration settings of a ClickHouse server. |
resources | Resources Resources allocated to ClickHouse hosts. |
Zookeeper
Field | Description |
---|---|
resources | Resources Resources allocated to ZooKeeper hosts. |
Access
Field | Description |
---|---|
data_lens | bool Allow to export data from the cluster to DataLens. |
web_sql | bool Allow SQL queries to the cluster databases from the management console. See SQL queries in the management console for more details. |
metrika | bool Allow to import data from Yandex Metrica and AppMetrica to the cluster. See AppMetrica documentation |
serverless | bool Allow access to cluster for Serverless. |
data_transfer | bool Allow access for DataTransfer |
yandex_query | bool Allow access for Query |
CloudStorage
Field | Description |
---|---|
enabled | bool Whether to use Object Storage for storing ClickHouse data. |
move_factor | google.protobuf.DoubleValue Acceptable values are 0 to 1, inclusive. |
data_cache_enabled | google.protobuf.BoolValue |
data_cache_max_size | google.protobuf.Int64Value |
prefer_not_to_merge | google.protobuf.BoolValue |
ClickhouseConfigSet
Field | Description |
---|---|
effective_config | ClickhouseConfig Required. Effective settings for a ClickHouse cluster (a combination of settings defined in user_config and default_config ). |
user_config | ClickhouseConfig User-defined settings for a ClickHouse cluster. |
default_config | ClickhouseConfig Default configuration for a ClickHouse cluster. |
ClickhouseConfig
Field | Description |
---|---|
log_level | enum LogLevel Logging level for the ClickHouse cluster. Possible values: TRACE, DEBUG, INFORMATION, WARNING, ERROR. |
merge_tree | MergeTree Settings for the MergeTree engine. See description in ClickHouse documentation |
compression[] | Compression Compression settings for the ClickHouse cluster. See in-depth description in ClickHouse documentation |
dictionaries[] | ExternalDictionary Configuration of external dictionaries to be used by the ClickHouse cluster. See in-depth description in ClickHouse documentation |
graphite_rollup[] | GraphiteRollup Settings for thinning Graphite data. See in-depth description in ClickHouse documentation |
kafka | Kafka |
kafka_topics[] | KafkaTopic |
rabbitmq | Rabbitmq |
max_connections | google.protobuf.Int64Value Maximum number of inbound connections. The minimum value is 10. |
max_concurrent_queries | google.protobuf.Int64Value Maximum number of simultaneously processed requests. The minimum value is 10. |
keep_alive_timeout | google.protobuf.Int64Value Number of milliseconds that ClickHouse waits for incoming requests before closing the connection. |
uncompressed_cache_size | google.protobuf.Int64Value Cache size (in bytes) for uncompressed data used by MergeTree tables. |
mark_cache_size | google.protobuf.Int64Value Approximate size (in bytes) of the cache of "marks" used by MergeTree tables. Value must be greater than 0. |
max_table_size_to_drop | google.protobuf.Int64Value Maximum size of the table that can be deleted using a DROP query. |
max_partition_size_to_drop | google.protobuf.Int64Value Maximum size of the partition that can be deleted using a DROP query. |
builtin_dictionaries_reload_interval | google.protobuf.Int64Value The setting is deprecated and has no effect. |
timezone | string The server's time zone to be used in DateTime fields conversions. Specified as an IANA identifier. |
geobase_enabled | google.protobuf.BoolValue Enable or disable geobase. |
geobase_uri | string Address of the archive with the user geobase in Object Storage. |
query_log_retention_size | google.protobuf.Int64Value The maximum size that query_log can grow to before old data will be removed. If set to 0, automatic removal of query_log data based on size is disabled. |
query_log_retention_time | google.protobuf.Int64Value The maximum time that query_log records will be retained before removal. If set to 0, automatic removal of query_log data based on time is disabled. |
query_thread_log_enabled | google.protobuf.BoolValue Whether query_thread_log system table is enabled. |
query_thread_log_retention_size | google.protobuf.Int64Value The maximum size that query_thread_log can grow to before old data will be removed. If set to 0, automatic removal of query_thread_log data based on size is disabled. |
query_thread_log_retention_time | google.protobuf.Int64Value The maximum time that query_thread_log records will be retained before removal. If set to 0, automatic removal of query_thread_log data based on time is disabled. |
part_log_retention_size | google.protobuf.Int64Value The maximum size that part_log can grow to before old data will be removed. If set to 0, automatic removal of part_log data based on size is disabled. |
part_log_retention_time | google.protobuf.Int64Value The maximum time that part_log records will be retained before removal. If set to 0, automatic removal of part_log data based on time is disabled. |
metric_log_enabled | google.protobuf.BoolValue Whether metric_log system table is enabled. |
metric_log_retention_size | google.protobuf.Int64Value The maximum size that metric_log can grow to before old data will be removed. If set to 0, automatic removal of metric_log data based on size is disabled. |
metric_log_retention_time | google.protobuf.Int64Value The maximum time that metric_log records will be retained before removal. If set to 0, automatic removal of metric_log data based on time is disabled. |
trace_log_enabled | google.protobuf.BoolValue Whether trace_log system table is enabled. |
trace_log_retention_size | google.protobuf.Int64Value The maximum size that trace_log can grow to before old data will be removed. If set to 0, automatic removal of trace_log data based on size is disabled. |
trace_log_retention_time | google.protobuf.Int64Value The maximum time that trace_log records will be retained before removal. If set to 0, automatic removal of trace_log data based on time is disabled. |
text_log_enabled | google.protobuf.BoolValue Whether text_log system table is enabled. |
text_log_retention_size | google.protobuf.Int64Value The maximum size that text_log can grow to before old data will be removed. If set to 0, automatic removal of text_log data based on size is disabled. |
text_log_retention_time | google.protobuf.Int64Value The maximum time that text_log records will be retained before removal. If set to 0, automatic removal of text_log data based on time is disabled. |
text_log_level | enum LogLevel Logging level for text_log system table. Possible values: TRACE, DEBUG, INFORMATION, WARNING, ERROR. |
opentelemetry_span_log_enabled | google.protobuf.BoolValue Enable or disable opentelemetry_span_log system table. Default value: false. |
opentelemetry_span_log_retention_size | google.protobuf.Int64Value The maximum size that opentelemetry_span_log can grow to before old data will be removed. If set to 0 (default), automatic removal of opentelemetry_span_log data based on size is disabled. The minimum value is 0. |
opentelemetry_span_log_retention_time | google.protobuf.Int64Value The maximum time that opentelemetry_span_log records will be retained before removal. If set to 0, automatic removal of opentelemetry_span_log data based on time is disabled. The minimum value is 0. |
query_views_log_enabled | google.protobuf.BoolValue Enable or disable query_views_log system table. Default value: false. |
query_views_log_retention_size | google.protobuf.Int64Value The maximum size that query_views_log can grow to before old data will be removed. If set to 0 (default), automatic removal of query_views_log data based on size is disabled. The minimum value is 0. |
query_views_log_retention_time | google.protobuf.Int64Value The maximum time that query_views_log records will be retained before removal. If set to 0, automatic removal of query_views_log data based on time is disabled. The minimum value is 0. |
asynchronous_metric_log_enabled | google.protobuf.BoolValue Enable or disable asynchronous_metric_log system table. Default value: false. |
asynchronous_metric_log_retention_size | google.protobuf.Int64Value The maximum size that asynchronous_metric_log can grow to before old data will be removed. If set to 0 (default), automatic removal of asynchronous_metric_log data based on size is disabled. The minimum value is 0. |
asynchronous_metric_log_retention_time | google.protobuf.Int64Value The maximum time that asynchronous_metric_log records will be retained before removal. If set to 0, automatic removal of asynchronous_metric_log data based on time is disabled. The minimum value is 0. |
session_log_enabled | google.protobuf.BoolValue Enable or disable session_log system table. Default value: false. |
session_log_retention_size | google.protobuf.Int64Value The maximum size that session_log can grow to before old data will be removed. If set to 0 (default), automatic removal of session_log data based on size is disabled. The minimum value is 0. |
session_log_retention_time | google.protobuf.Int64Value The maximum time that session_log records will be retained before removal. If set to 0, automatic removal of session_log data based on time is disabled. The minimum value is 0. |
zookeeper_log_enabled | google.protobuf.BoolValue Enable or disable zookeeper_log system table. Default value: false. |
zookeeper_log_retention_size | google.protobuf.Int64Value The maximum size that zookeeper_log can grow to before old data will be removed. If set to 0 (default), automatic removal of zookeeper_log data based on size is disabled. The minimum value is 0. |
zookeeper_log_retention_time | google.protobuf.Int64Value The maximum time that zookeeper_log records will be retained before removal. If set to 0, automatic removal of zookeeper_log data based on time is disabled. The minimum value is 0. |
asynchronous_insert_log_enabled | google.protobuf.BoolValue Enable or disable asynchronous_insert_log system table. Default value: false. Minimal required ClickHouse version: 22.10. |
asynchronous_insert_log_retention_size | google.protobuf.Int64Value The maximum size that asynchronous_insert_log can grow to before old data will be removed. If set to 0 (default), automatic removal of asynchronous_insert_log data based on size is disabled. The minimum value is 0. |
asynchronous_insert_log_retention_time | google.protobuf.Int64Value The maximum time that asynchronous_insert_log records will be retained before removal. If set to 0, automatic removal of asynchronous_insert_log data based on time is disabled. The minimum value is 0. |
background_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_merges_mutations_concurrency_ratio | google.protobuf.Int64Value Sets a ratio between the number of threads and the number of background merges and mutations that can be executed concurrently. For example, if the ratio equals to 2 and background_pool_size is set to 16 then ClickHouse can execute 32 background merges concurrently. This is possible, because background operations could be suspended and postponed. This is needed to give small merges more execution priority. You can only increase this ratio at runtime. To lower it you have to restart the server. The same as for background_pool_size setting background_merges_mutations_concurrency_ratio could be applied from the default profile for backward compatibility. Default: 2 See in-depth description in ClickHouse documentation |
background_schedule_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_fetches_pool_size | google.protobuf.Int64Value Sets the number of threads performing background fetches for tables with ReplicatedMergeTree engines. Default value: 8. More info see in ClickHouse documentation |
background_move_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_distributed_schedule_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_buffer_flush_schedule_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_message_broker_schedule_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_common_pool_size | google.protobuf.Int64Value The maximum number of threads that will be used for performing a variety of operations (mostly garbage collection) for *MergeTree-engine tables in a background. Default: 8 See in-depth description in ClickHouse documentation |
default_database | google.protobuf.StringValue The default database. To get a list of cluster databases, see Yandex Managed ClickHouse documentation. |
total_memory_profiler_step | google.protobuf.Int64Value Sets the memory size (in bytes) for a stack trace at every peak allocation step. Default value: 4194304. More info see in ClickHouse documentation |
total_memory_tracker_sample_probability | google.protobuf.DoubleValue |
MergeTree
Field | Description |
---|---|
replicated_deduplication_window | google.protobuf.Int64Value Number of blocks of hashes to keep in ZooKeeper. |
replicated_deduplication_window_seconds | google.protobuf.Int64Value Period of time to keep blocks of hashes for. |
parts_to_delay_insert | google.protobuf.Int64Value If table contains at least that many active parts in single partition, artificially slow down insert into table. |
parts_to_throw_insert | google.protobuf.Int64Value If more than this number active parts in single partition, throw 'Too many parts ...' exception. |
inactive_parts_to_delay_insert | google.protobuf.Int64Value |
inactive_parts_to_throw_insert | google.protobuf.Int64Value |
max_replicated_merges_in_queue | google.protobuf.Int64Value How many tasks of merging and mutating parts are allowed simultaneously in ReplicatedMergeTree queue. |
number_of_free_entries_in_pool_to_lower_max_size_of_merge | google.protobuf.Int64Value If there is less than specified number of free entries in background pool (or replicated queue), start to lower maximum size of merge to process. |
max_bytes_to_merge_at_min_space_in_pool | google.protobuf.Int64Value Maximum in total size of parts to merge, when there are minimum free threads in background pool (or entries in replication queue). |
max_bytes_to_merge_at_max_space_in_pool | google.protobuf.Int64Value |
min_bytes_for_wide_part | google.protobuf.Int64Value Minimum number of bytes in a data part that can be stored in Wide format. More info see in ClickHouse documentation |
min_rows_for_wide_part | google.protobuf.Int64Value Minimum number of rows in a data part that can be stored in Wide format. More info see in ClickHouse documentation |
ttl_only_drop_parts | google.protobuf.BoolValue Enables or disables complete dropping of data parts where all rows are expired in MergeTree tables. More info see in ClickHouse documentation |
allow_remote_fs_zero_copy_replication | google.protobuf.BoolValue |
merge_with_ttl_timeout | google.protobuf.Int64Value |
merge_with_recompression_ttl_timeout | google.protobuf.Int64Value |
max_parts_in_total | google.protobuf.Int64Value |
max_number_of_merges_with_ttl_in_pool | google.protobuf.Int64Value |
cleanup_delay_period | google.protobuf.Int64Value |
number_of_free_entries_in_pool_to_execute_mutation | google.protobuf.Int64Value |
max_avg_part_size_for_too_many_parts | google.protobuf.Int64Value The 'too many parts' check according to 'parts_to_delay_insert' and 'parts_to_throw_insert' will be active only if the average part size (in the relevant partition) is not larger than the specified threshold. If it is larger than the specified threshold, the INSERTs will be neither delayed or rejected. This allows to have hundreds of terabytes in a single table on a single server if the parts are successfully merged to larger parts. This does not affect the thresholds on inactive parts or total parts. Default: 1 GiB Min version: 22.10 See in-depth description in ClickHouse GitHub |
min_age_to_force_merge_seconds | google.protobuf.Int64Value Merge parts if every part in the range is older than the value of min_age_to_force_merge_seconds. Default: 0 - disabled Min_version: 22.10 See in-depth description in ClickHouse documentation |
min_age_to_force_merge_on_partition_only | google.protobuf.BoolValue Whether min_age_to_force_merge_seconds should be applied only on the entire partition and not on subset. Default: false Min_version: 22.11 See in-depth description in ClickHouse documentation |
merge_selecting_sleep_ms | google.protobuf.Int64Value Sleep time for merge selecting when no part is selected. A lower setting triggers selecting tasks in background_schedule_pool frequently, which results in a large number of requests to ClickHouse Keeper in large-scale clusters. Default: 5000 Min_version: 21.10 See in-depth description in ClickHouse documentation |
Kafka
Field | Description |
---|---|
security_protocol | enum SecurityProtocol |
sasl_mechanism | enum SaslMechanism |
sasl_username | string |
sasl_password | string |
enable_ssl_certificate_verification | google.protobuf.BoolValue |
max_poll_interval_ms | google.protobuf.Int64Value The minimum value is 0. |
session_timeout_ms | google.protobuf.Int64Value The minimum value is 0. |
KafkaTopic
Field | Description |
---|---|
name | string Required. |
settings | Kafka Required. |
Rabbitmq
Field | Description |
---|---|
username | string RabbitMQ |
password | string RabbitMQ |
vhost | string RabbitMQ |
Compression
Field | Description |
---|---|
method | enum Method Compression method to use for the specified combination of min_part_size and min_part_size_ratio . |
min_part_size | int64 Minimum size of a part of a table. The minimum value is 1. |
min_part_size_ratio | double Minimum ratio of a part relative to the size of all the data in the table. |
level | google.protobuf.Int64Value The minimum value is 0. |
ExternalDictionary
Field | Description |
---|---|
name | string Required. Name of the external dictionary. |
structure | Structure Required. Set of attributes for the external dictionary. For in-depth description, see ClickHouse documentation |
layout | Layout Required. Layout for storing the dictionary in memory. For in-depth description, see ClickHouse documentation |
lifetime | oneof: fixed_lifetime or lifetime_range Setting for the period of time between dictionary updates. For details, see ClickHouse documentation |
fixed_lifetime | int64 Fixed interval between dictionary updates. |
lifetime_range | Range Range of intervals between dictionary updates for ClickHouse to choose from. |
source | oneof: http_source , mysql_source , clickhouse_source , mongodb_source or postgresql_source Description of the source for the external dictionary. |
http_source | HttpSource HTTP source for the dictionary. |
mysql_source | MysqlSource MySQL source for the dictionary. |
clickhouse_source | ClickhouseSource ClickHouse source for the dictionary. |
mongodb_source | MongodbSource MongoDB source for the dictionary. |
postgresql_source | PostgresqlSource PostgreSQL source for the dictionary. |
HttpSource
Field | Description |
---|---|
url | string Required. URL of the source dictionary available over HTTP. |
format | string Required. The data format. Valid values are all formats supported by ClickHouse SQL dialect. |
MysqlSource
Field | Description |
---|---|
db | string Required. Name of the MySQL database to connect to. |
table | string Required. Name of the database table to use as a ClickHouse dictionary. |
port | int64 Default port to use when connecting to a replica of the dictionary source. Acceptable values are 0 to 65535, inclusive. |
user | string Name of the default user for replicas of the dictionary source. |
password | string Password of the default user for replicas of the dictionary source. |
replicas[] | Replica List of MySQL replicas of the database used as dictionary source. The number of elements must be greater than 0. |
where | string Selection criteria for the data in the specified MySQL table. |
invalidate_query | string Query for checking the dictionary status, to pull only updated data. For more details, see ClickHouse documentation on dictionaries |
Replica
Field | Description |
---|---|
host | string Required. MySQL host of the replica. The maximum string length in characters is 253. |
priority | int64 Required. The priority of the replica that ClickHouse takes into account when connecting. Replica with the highest priority should have this field set to the lowest number. Value must be greater than 0. |
port | int64 Port to use when connecting to the replica. If a port is not specified for a replica, ClickHouse uses the port specified for the source. Acceptable values are 0 to 65535, inclusive. |
user | string Name of the MySQL database user. |
password | string Password of the MySQL database user. |
ClickhouseSource
Field | Description |
---|---|
db | string Required. Name of the ClickHouse database. |
table | string Required. Name of the table in the specified database to be used as the dictionary source. |
host | string Required. ClickHouse host of the specified database. The maximum string length in characters is 253. |
port | int64 Port to use when connecting to the host. Acceptable values are 0 to 65535, inclusive. |
user | string Required. Name of the ClickHouse database user. |
password | string Password of the ClickHouse database user. |
where | string Selection criteria for the data in the specified ClickHouse table. |
MongodbSource
Field | Description |
---|---|
db | string Required. Name of the MongoDB database. |
collection | string Required. Name of the collection in the specified database to be used as the dictionary source. |
host | string Required. MongoDB host of the specified database. The maximum string length in characters is 253. |
port | int64 Port to use when connecting to the host. Acceptable values are 0 to 65535, inclusive. |
user | string Required. Name of the MongoDB database user. |
password | string Password of the MongoDB database user. |
options | string |
PostgresqlSource
Field | Description |
---|---|
db | string Required. Name of the PostrgreSQL database. |
table | string Required. Name of the table in the specified database to be used as the dictionary source. |
hosts[] | string Name of the PostrgreSQL host The number of elements must be greater than 0. |
port | int64 Port to use when connecting to the host. Acceptable values are 0 to 65535, inclusive. |
user | string Required. Name of the PostrgreSQL database user. |
password | string Password of the PostrgreSQL database user. |
invalidate_query | string Query for checking the dictionary status, to pull only updated data. For more details, see ClickHouse documentation on dictionaries |
ssl_mode | enum SslMode Mode of SSL TCP/IP connection to the PostgreSQL host. For more details, see PostgreSQL documentation
|
Structure
Field | Description |
---|---|
id | Id Single numeric key column for the dictionary. |
key | Key Composite key for the dictionary, containing of one or more key columns. For details, see ClickHouse documentation |
range_min | Attribute Field holding the beginning of the range for dictionaries with RANGE_HASHED layout. For details, see ClickHouse documentation |
range_max | Attribute Field holding the end of the range for dictionaries with RANGE_HASHED layout. For details, see ClickHouse documentation |
attributes[] | Attribute Description of the fields available for database queries. For details, see ClickHouse documentation |
Attribute
Field | Description |
---|---|
name | string Required. Name of the column. |
type | string Required. Type of the column. |
null_value | string Default value for an element without data (for example, an empty string). |
expression | string Expression, describing the attribute, if applicable. |
hierarchical | bool Indication of hierarchy support. Default value: false . |
injective | bool Indication of injective mapping "id -> attribute". Default value: false . |
Id
Field | Description |
---|---|
name | string Required. Name of the numeric key. |
Key
Field | Description |
---|---|
attributes[] | Attribute Attributes of a complex key. The number of elements must be greater than 0. |
Layout
Field | Description |
---|---|
type | enum Type Required. Layout type for an external dictionary.
|
size_in_cells | int64 Number of cells in the cache. Rounded up to a power of two. Applicable only for CACHE and COMPLEX_KEY_CACHE layout types. |
Range
Field | Description |
---|---|
min | int64 Minimum dictionary lifetime. |
max | int64 Maximum dictionary lifetime. |
GraphiteRollup
Field | Description |
---|---|
name | string Required. Name for the specified combination of settings for Graphite rollup. |
patterns[] | Pattern Pattern to use for the rollup. The number of elements must be greater than 0. |
Pattern
Field | Description |
---|---|
regexp | string Pattern for metric names. |
function | string Required. Name of the aggregating function to apply to data of the age specified in retention . |
retention[] | Retention Age of data to use for thinning. The number of elements must be greater than 0. |
Retention
Field | Description |
---|---|
age | int64 Minimum age of the data in seconds. The minimum value is 0. |
precision | int64 Precision of determining the age of the data, in seconds. Value must be greater than 0. |
Resources
Field | Description |
---|---|
resource_preset_id | string ID of the preset for computational resources available to a host (CPU, memory etc.). All available presets are listed in the documentation |
disk_size | int64 Volume of the storage available to a host, in bytes. |
disk_type_id | string Type of the storage environment for the host. Possible values:
|
MaintenanceWindow
Field | Description |
---|---|
policy | oneof: anytime or weekly_maintenance_window The maintenance policy in effect. |
anytime | AnytimeMaintenanceWindow Maintenance operation can be scheduled anytime. |
weekly_maintenance_window | WeeklyMaintenanceWindow Maintenance operation can be scheduled on a weekly basis. |
AnytimeMaintenanceWindow
Empty.
WeeklyMaintenanceWindow
Field | Description |
---|---|
day | enum WeekDay Day of the week (in DDD format). |
hour | int64 Hour of the day in UTC (in HH format). Acceptable values are 1 to 24, inclusive. |
MaintenanceOperation
Field | Description |
---|---|
info | string Information about this maintenance operation. The maximum string length in characters is 256. |
delayed_until | google.protobuf.Timestamp Time until which this maintenance operation is delayed. |
AddZookeeper
Adds a ZooKeeper subcluster to the specified ClickHouse cluster.
rpc AddZookeeper (AddClusterZookeeperRequest) returns (operation.Operation)
Metadata and response of Operation:
Operation.metadata:AddClusterZookeeperMetadata
Operation.response:Cluster
AddClusterZookeeperRequest
Field | Description |
---|---|
cluster_id | string Required. ID of the ClickHouse cluster to modify. The maximum string length in characters is 50. |
resources | Resources Resources allocated to Zookeeper hosts. |
host_specs[] | HostSpec Configuration of ZooKeeper hosts. |
Resources
Field | Description |
---|---|
resource_preset_id | string ID of the preset for computational resources available to a host (CPU, memory etc.). All available presets are listed in the documentation |
disk_size | int64 Volume of the storage available to a host, in bytes. |
disk_type_id | string Type of the storage environment for the host. Possible values:
|
HostSpec
Field | Description |
---|---|
zone_id | string ID of the availability zone where the host resides. To get a list of available zones, use the yandex.cloud.compute.v1.ZoneService.List request. The maximum string length in characters is 50. |
type | Host.Type Required. Type of the host to be deployed. |
subnet_id | string ID of the subnet that the host should belong to. This subnet should be a part of the network that the cluster belongs to. The ID of the network is set in the Cluster.network_id field. The maximum string length in characters is 50. |
assign_public_ip | bool Whether the host should get a public IP address on creation. After a host has been created, this setting cannot be changed. To remove an assigned public IP, or to assign a public IP to a host without one, recreate the host with assign_public_ip set as needed. Possible values:
|
shard_name | string Name of the shard that the host is assigned to. The maximum string length in characters is 63. Value must match the regular expression [a-zA-Z0-9_-]* . |
Operation
Field | Description |
---|---|
id | string ID of the operation. |
description | string Description of the operation. 0-256 characters long. |
created_at | google.protobuf.Timestamp Creation timestamp. |
created_by | string ID of the user or service account who initiated the operation. |
modified_at | google.protobuf.Timestamp The time when the Operation resource was last modified. |
done | bool If the value is false , it means the operation is still in progress. If true , the operation is completed, and either error or response is available. |
metadata | google.protobuf.Any Service-specific metadata associated with the operation. It typically contains the ID of the target resource that the operation is performed on. Any method that returns a long-running operation should document the metadata type, if any. |
result | oneof: error or response The operation result. If done == false and there was no failure detected, neither error nor response is set. If done == false and there was a failure detected, error is set. If done == true , exactly one of error or response is set. |
error | google.rpc.Status The error result of the operation in case of failure or cancellation. |
response | google.protobuf.Any if operation finished successfully. |
AddClusterZookeeperMetadata
Field | Description |
---|---|
cluster_id | string ID of the ClickHouse cluster. |
Cluster
Field | Description |
---|---|
id | string ID of the ClickHouse cluster. This ID is assigned by MDB at creation time. |
folder_id | string ID of the folder that the ClickHouse cluster belongs to. |
created_at | google.protobuf.Timestamp Creation timestamp in RFC3339 |
name | string Name of the ClickHouse cluster. The name is unique within the folder. 1-63 characters long. |
description | string Description of the ClickHouse cluster. 0-256 characters long. |
labels | map<string,string> Custom labels for the ClickHouse cluster as key:value pairs. Maximum 64 per resource. |
environment | enum Environment Deployment environment of the ClickHouse cluster.
|
monitoring[] | Monitoring Description of monitoring systems relevant to the ClickHouse cluster. |
config | ClusterConfig Configuration of the ClickHouse cluster. |
network_id | string ID of the network that the cluster belongs to. |
health | enum Health Aggregated cluster health.
|
status | enum Status Current state of the cluster.
|
service_account_id | string ID of the service account used for access to Object Storage. |
maintenance_window | MaintenanceWindow Maintenance window for the cluster. |
planned_operation | MaintenanceOperation Planned maintenance operation to be started for the cluster within the nearest maintenance_window . |
security_group_ids[] | string User security groups |
deletion_protection | bool Deletion Protection inhibits deletion of the cluster |
Monitoring
Field | Description |
---|---|
name | string Name of the monitoring system. |
description | string Description of the monitoring system. |
link | string Link to the monitoring system charts for the ClickHouse cluster. |
ClusterConfig
Field | Description |
---|---|
version | string Version of the ClickHouse server software. |
clickhouse | Clickhouse Configuration and resource allocation for ClickHouse hosts. |
zookeeper | Zookeeper Configuration and resource allocation for ZooKeeper hosts. |
backup_window_start | google.type.TimeOfDay Time to start the daily backup, in the UTC timezone. |
access | Access Access policy for external services. |
cloud_storage | CloudStorage |
sql_database_management | google.protobuf.BoolValue Whether database management through SQL commands is enabled. |
sql_user_management | google.protobuf.BoolValue Whether user management through SQL commands is enabled. |
embedded_keeper | google.protobuf.BoolValue Whether cluster should use embedded Keeper instead of Zookeeper. |
Clickhouse
Field | Description |
---|---|
config | config.ClickhouseConfigSet Configuration settings of a ClickHouse server. |
resources | Resources Resources allocated to ClickHouse hosts. |
Zookeeper
Field | Description |
---|---|
resources | Resources Resources allocated to ZooKeeper hosts. |
Access
Field | Description |
---|---|
data_lens | bool Allow to export data from the cluster to DataLens. |
web_sql | bool Allow SQL queries to the cluster databases from the management console. See SQL queries in the management console for more details. |
metrika | bool Allow to import data from Yandex Metrica and AppMetrica to the cluster. See AppMetrica documentation |
serverless | bool Allow access to cluster for Serverless. |
data_transfer | bool Allow access for DataTransfer |
yandex_query | bool Allow access for Query |
CloudStorage
Field | Description |
---|---|
enabled | bool Whether to use Object Storage for storing ClickHouse data. |
move_factor | google.protobuf.DoubleValue Acceptable values are 0 to 1, inclusive. |
data_cache_enabled | google.protobuf.BoolValue |
data_cache_max_size | google.protobuf.Int64Value |
prefer_not_to_merge | google.protobuf.BoolValue |
ClickhouseConfigSet
Field | Description |
---|---|
effective_config | ClickhouseConfig Required. Effective settings for a ClickHouse cluster (a combination of settings defined in user_config and default_config ). |
user_config | ClickhouseConfig User-defined settings for a ClickHouse cluster. |
default_config | ClickhouseConfig Default configuration for a ClickHouse cluster. |
ClickhouseConfig
Field | Description |
---|---|
log_level | enum LogLevel Logging level for the ClickHouse cluster. Possible values: TRACE, DEBUG, INFORMATION, WARNING, ERROR. |
merge_tree | MergeTree Settings for the MergeTree engine. See description in ClickHouse documentation |
compression[] | Compression Compression settings for the ClickHouse cluster. See in-depth description in ClickHouse documentation |
dictionaries[] | ExternalDictionary Configuration of external dictionaries to be used by the ClickHouse cluster. See in-depth description in ClickHouse documentation |
graphite_rollup[] | GraphiteRollup Settings for thinning Graphite data. See in-depth description in ClickHouse documentation |
kafka | Kafka |
kafka_topics[] | KafkaTopic |
rabbitmq | Rabbitmq |
max_connections | google.protobuf.Int64Value Maximum number of inbound connections. The minimum value is 10. |
max_concurrent_queries | google.protobuf.Int64Value Maximum number of simultaneously processed requests. The minimum value is 10. |
keep_alive_timeout | google.protobuf.Int64Value Number of milliseconds that ClickHouse waits for incoming requests before closing the connection. |
uncompressed_cache_size | google.protobuf.Int64Value Cache size (in bytes) for uncompressed data used by MergeTree tables. |
mark_cache_size | google.protobuf.Int64Value Approximate size (in bytes) of the cache of "marks" used by MergeTree tables. Value must be greater than 0. |
max_table_size_to_drop | google.protobuf.Int64Value Maximum size of the table that can be deleted using a DROP query. |
max_partition_size_to_drop | google.protobuf.Int64Value Maximum size of the partition that can be deleted using a DROP query. |
builtin_dictionaries_reload_interval | google.protobuf.Int64Value The setting is deprecated and has no effect. |
timezone | string The server's time zone to be used in DateTime fields conversions. Specified as an IANA identifier. |
geobase_enabled | google.protobuf.BoolValue Enable or disable geobase. |
geobase_uri | string Address of the archive with the user geobase in Object Storage. |
query_log_retention_size | google.protobuf.Int64Value The maximum size that query_log can grow to before old data will be removed. If set to 0, automatic removal of query_log data based on size is disabled. |
query_log_retention_time | google.protobuf.Int64Value The maximum time that query_log records will be retained before removal. If set to 0, automatic removal of query_log data based on time is disabled. |
query_thread_log_enabled | google.protobuf.BoolValue Whether query_thread_log system table is enabled. |
query_thread_log_retention_size | google.protobuf.Int64Value The maximum size that query_thread_log can grow to before old data will be removed. If set to 0, automatic removal of query_thread_log data based on size is disabled. |
query_thread_log_retention_time | google.protobuf.Int64Value The maximum time that query_thread_log records will be retained before removal. If set to 0, automatic removal of query_thread_log data based on time is disabled. |
part_log_retention_size | google.protobuf.Int64Value The maximum size that part_log can grow to before old data will be removed. If set to 0, automatic removal of part_log data based on size is disabled. |
part_log_retention_time | google.protobuf.Int64Value The maximum time that part_log records will be retained before removal. If set to 0, automatic removal of part_log data based on time is disabled. |
metric_log_enabled | google.protobuf.BoolValue Whether metric_log system table is enabled. |
metric_log_retention_size | google.protobuf.Int64Value The maximum size that metric_log can grow to before old data will be removed. If set to 0, automatic removal of metric_log data based on size is disabled. |
metric_log_retention_time | google.protobuf.Int64Value The maximum time that metric_log records will be retained before removal. If set to 0, automatic removal of metric_log data based on time is disabled. |
trace_log_enabled | google.protobuf.BoolValue Whether trace_log system table is enabled. |
trace_log_retention_size | google.protobuf.Int64Value The maximum size that trace_log can grow to before old data will be removed. If set to 0, automatic removal of trace_log data based on size is disabled. |
trace_log_retention_time | google.protobuf.Int64Value The maximum time that trace_log records will be retained before removal. If set to 0, automatic removal of trace_log data based on time is disabled. |
text_log_enabled | google.protobuf.BoolValue Whether text_log system table is enabled. |
text_log_retention_size | google.protobuf.Int64Value The maximum size that text_log can grow to before old data will be removed. If set to 0, automatic removal of text_log data based on size is disabled. |
text_log_retention_time | google.protobuf.Int64Value The maximum time that text_log records will be retained before removal. If set to 0, automatic removal of text_log data based on time is disabled. |
text_log_level | enum LogLevel Logging level for text_log system table. Possible values: TRACE, DEBUG, INFORMATION, WARNING, ERROR. |
opentelemetry_span_log_enabled | google.protobuf.BoolValue Enable or disable opentelemetry_span_log system table. Default value: false. |
opentelemetry_span_log_retention_size | google.protobuf.Int64Value The maximum size that opentelemetry_span_log can grow to before old data will be removed. If set to 0 (default), automatic removal of opentelemetry_span_log data based on size is disabled. The minimum value is 0. |
opentelemetry_span_log_retention_time | google.protobuf.Int64Value The maximum time that opentelemetry_span_log records will be retained before removal. If set to 0, automatic removal of opentelemetry_span_log data based on time is disabled. The minimum value is 0. |
query_views_log_enabled | google.protobuf.BoolValue Enable or disable query_views_log system table. Default value: false. |
query_views_log_retention_size | google.protobuf.Int64Value The maximum size that query_views_log can grow to before old data will be removed. If set to 0 (default), automatic removal of query_views_log data based on size is disabled. The minimum value is 0. |
query_views_log_retention_time | google.protobuf.Int64Value The maximum time that query_views_log records will be retained before removal. If set to 0, automatic removal of query_views_log data based on time is disabled. The minimum value is 0. |
asynchronous_metric_log_enabled | google.protobuf.BoolValue Enable or disable asynchronous_metric_log system table. Default value: false. |
asynchronous_metric_log_retention_size | google.protobuf.Int64Value The maximum size that asynchronous_metric_log can grow to before old data will be removed. If set to 0 (default), automatic removal of asynchronous_metric_log data based on size is disabled. The minimum value is 0. |
asynchronous_metric_log_retention_time | google.protobuf.Int64Value The maximum time that asynchronous_metric_log records will be retained before removal. If set to 0, automatic removal of asynchronous_metric_log data based on time is disabled. The minimum value is 0. |
session_log_enabled | google.protobuf.BoolValue Enable or disable session_log system table. Default value: false. |
session_log_retention_size | google.protobuf.Int64Value The maximum size that session_log can grow to before old data will be removed. If set to 0 (default), automatic removal of session_log data based on size is disabled. The minimum value is 0. |
session_log_retention_time | google.protobuf.Int64Value The maximum time that session_log records will be retained before removal. If set to 0, automatic removal of session_log data based on time is disabled. The minimum value is 0. |
zookeeper_log_enabled | google.protobuf.BoolValue Enable or disable zookeeper_log system table. Default value: false. |
zookeeper_log_retention_size | google.protobuf.Int64Value The maximum size that zookeeper_log can grow to before old data will be removed. If set to 0 (default), automatic removal of zookeeper_log data based on size is disabled. The minimum value is 0. |
zookeeper_log_retention_time | google.protobuf.Int64Value The maximum time that zookeeper_log records will be retained before removal. If set to 0, automatic removal of zookeeper_log data based on time is disabled. The minimum value is 0. |
asynchronous_insert_log_enabled | google.protobuf.BoolValue Enable or disable asynchronous_insert_log system table. Default value: false. Minimal required ClickHouse version: 22.10. |
asynchronous_insert_log_retention_size | google.protobuf.Int64Value The maximum size that asynchronous_insert_log can grow to before old data will be removed. If set to 0 (default), automatic removal of asynchronous_insert_log data based on size is disabled. The minimum value is 0. |
asynchronous_insert_log_retention_time | google.protobuf.Int64Value The maximum time that asynchronous_insert_log records will be retained before removal. If set to 0, automatic removal of asynchronous_insert_log data based on time is disabled. The minimum value is 0. |
background_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_merges_mutations_concurrency_ratio | google.protobuf.Int64Value Sets a ratio between the number of threads and the number of background merges and mutations that can be executed concurrently. For example, if the ratio equals to 2 and background_pool_size is set to 16 then ClickHouse can execute 32 background merges concurrently. This is possible, because background operations could be suspended and postponed. This is needed to give small merges more execution priority. You can only increase this ratio at runtime. To lower it you have to restart the server. The same as for background_pool_size setting background_merges_mutations_concurrency_ratio could be applied from the default profile for backward compatibility. Default: 2 See in-depth description in ClickHouse documentation |
background_schedule_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_fetches_pool_size | google.protobuf.Int64Value Sets the number of threads performing background fetches for tables with ReplicatedMergeTree engines. Default value: 8. More info see in ClickHouse documentation |
background_move_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_distributed_schedule_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_buffer_flush_schedule_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_message_broker_schedule_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_common_pool_size | google.protobuf.Int64Value The maximum number of threads that will be used for performing a variety of operations (mostly garbage collection) for *MergeTree-engine tables in a background. Default: 8 See in-depth description in ClickHouse documentation |
default_database | google.protobuf.StringValue The default database. To get a list of cluster databases, see Yandex Managed ClickHouse documentation. |
total_memory_profiler_step | google.protobuf.Int64Value Sets the memory size (in bytes) for a stack trace at every peak allocation step. Default value: 4194304. More info see in ClickHouse documentation |
total_memory_tracker_sample_probability | google.protobuf.DoubleValue |
MergeTree
Field | Description |
---|---|
replicated_deduplication_window | google.protobuf.Int64Value Number of blocks of hashes to keep in ZooKeeper. |
replicated_deduplication_window_seconds | google.protobuf.Int64Value Period of time to keep blocks of hashes for. |
parts_to_delay_insert | google.protobuf.Int64Value If table contains at least that many active parts in single partition, artificially slow down insert into table. |
parts_to_throw_insert | google.protobuf.Int64Value If more than this number active parts in single partition, throw 'Too many parts ...' exception. |
inactive_parts_to_delay_insert | google.protobuf.Int64Value |
inactive_parts_to_throw_insert | google.protobuf.Int64Value |
max_replicated_merges_in_queue | google.protobuf.Int64Value How many tasks of merging and mutating parts are allowed simultaneously in ReplicatedMergeTree queue. |
number_of_free_entries_in_pool_to_lower_max_size_of_merge | google.protobuf.Int64Value If there is less than specified number of free entries in background pool (or replicated queue), start to lower maximum size of merge to process. |
max_bytes_to_merge_at_min_space_in_pool | google.protobuf.Int64Value Maximum in total size of parts to merge, when there are minimum free threads in background pool (or entries in replication queue). |
max_bytes_to_merge_at_max_space_in_pool | google.protobuf.Int64Value |
min_bytes_for_wide_part | google.protobuf.Int64Value Minimum number of bytes in a data part that can be stored in Wide format. More info see in ClickHouse documentation |
min_rows_for_wide_part | google.protobuf.Int64Value Minimum number of rows in a data part that can be stored in Wide format. More info see in ClickHouse documentation |
ttl_only_drop_parts | google.protobuf.BoolValue Enables or disables complete dropping of data parts where all rows are expired in MergeTree tables. More info see in ClickHouse documentation |
allow_remote_fs_zero_copy_replication | google.protobuf.BoolValue |
merge_with_ttl_timeout | google.protobuf.Int64Value |
merge_with_recompression_ttl_timeout | google.protobuf.Int64Value |
max_parts_in_total | google.protobuf.Int64Value |
max_number_of_merges_with_ttl_in_pool | google.protobuf.Int64Value |
cleanup_delay_period | google.protobuf.Int64Value |
number_of_free_entries_in_pool_to_execute_mutation | google.protobuf.Int64Value |
max_avg_part_size_for_too_many_parts | google.protobuf.Int64Value The 'too many parts' check according to 'parts_to_delay_insert' and 'parts_to_throw_insert' will be active only if the average part size (in the relevant partition) is not larger than the specified threshold. If it is larger than the specified threshold, the INSERTs will be neither delayed or rejected. This allows to have hundreds of terabytes in a single table on a single server if the parts are successfully merged to larger parts. This does not affect the thresholds on inactive parts or total parts. Default: 1 GiB Min version: 22.10 See in-depth description in ClickHouse GitHub |
min_age_to_force_merge_seconds | google.protobuf.Int64Value Merge parts if every part in the range is older than the value of min_age_to_force_merge_seconds. Default: 0 - disabled Min_version: 22.10 See in-depth description in ClickHouse documentation |
min_age_to_force_merge_on_partition_only | google.protobuf.BoolValue Whether min_age_to_force_merge_seconds should be applied only on the entire partition and not on subset. Default: false Min_version: 22.11 See in-depth description in ClickHouse documentation |
merge_selecting_sleep_ms | google.protobuf.Int64Value Sleep time for merge selecting when no part is selected. A lower setting triggers selecting tasks in background_schedule_pool frequently, which results in a large number of requests to ClickHouse Keeper in large-scale clusters. Default: 5000 Min_version: 21.10 See in-depth description in ClickHouse documentation |
Kafka
Field | Description |
---|---|
security_protocol | enum SecurityProtocol |
sasl_mechanism | enum SaslMechanism |
sasl_username | string |
sasl_password | string |
enable_ssl_certificate_verification | google.protobuf.BoolValue |
max_poll_interval_ms | google.protobuf.Int64Value The minimum value is 0. |
session_timeout_ms | google.protobuf.Int64Value The minimum value is 0. |
KafkaTopic
Field | Description |
---|---|
name | string Required. |
settings | Kafka Required. |
Rabbitmq
Field | Description |
---|---|
username | string RabbitMQ |
password | string RabbitMQ |
vhost | string RabbitMQ |
Compression
Field | Description |
---|---|
method | enum Method Compression method to use for the specified combination of min_part_size and min_part_size_ratio . |
min_part_size | int64 Minimum size of a part of a table. The minimum value is 1. |
min_part_size_ratio | double Minimum ratio of a part relative to the size of all the data in the table. |
level | google.protobuf.Int64Value The minimum value is 0. |
ExternalDictionary
Field | Description |
---|---|
name | string Required. Name of the external dictionary. |
structure | Structure Required. Set of attributes for the external dictionary. For in-depth description, see ClickHouse documentation |
layout | Layout Required. Layout for storing the dictionary in memory. For in-depth description, see ClickHouse documentation |
lifetime | oneof: fixed_lifetime or lifetime_range Setting for the period of time between dictionary updates. For details, see ClickHouse documentation |
fixed_lifetime | int64 Fixed interval between dictionary updates. |
lifetime_range | Range Range of intervals between dictionary updates for ClickHouse to choose from. |
source | oneof: http_source , mysql_source , clickhouse_source , mongodb_source or postgresql_source Description of the source for the external dictionary. |
http_source | HttpSource HTTP source for the dictionary. |
mysql_source | MysqlSource MySQL source for the dictionary. |
clickhouse_source | ClickhouseSource ClickHouse source for the dictionary. |
mongodb_source | MongodbSource MongoDB source for the dictionary. |
postgresql_source | PostgresqlSource PostgreSQL source for the dictionary. |
HttpSource
Field | Description |
---|---|
url | string Required. URL of the source dictionary available over HTTP. |
format | string Required. The data format. Valid values are all formats supported by ClickHouse SQL dialect. |
MysqlSource
Field | Description |
---|---|
db | string Required. Name of the MySQL database to connect to. |
table | string Required. Name of the database table to use as a ClickHouse dictionary. |
port | int64 Default port to use when connecting to a replica of the dictionary source. Acceptable values are 0 to 65535, inclusive. |
user | string Name of the default user for replicas of the dictionary source. |
password | string Password of the default user for replicas of the dictionary source. |
replicas[] | Replica List of MySQL replicas of the database used as dictionary source. The number of elements must be greater than 0. |
where | string Selection criteria for the data in the specified MySQL table. |
invalidate_query | string Query for checking the dictionary status, to pull only updated data. For more details, see ClickHouse documentation on dictionaries |
Replica
Field | Description |
---|---|
host | string Required. MySQL host of the replica. The maximum string length in characters is 253. |
priority | int64 Required. The priority of the replica that ClickHouse takes into account when connecting. Replica with the highest priority should have this field set to the lowest number. Value must be greater than 0. |
port | int64 Port to use when connecting to the replica. If a port is not specified for a replica, ClickHouse uses the port specified for the source. Acceptable values are 0 to 65535, inclusive. |
user | string Name of the MySQL database user. |
password | string Password of the MySQL database user. |
ClickhouseSource
Field | Description |
---|---|
db | string Required. Name of the ClickHouse database. |
table | string Required. Name of the table in the specified database to be used as the dictionary source. |
host | string Required. ClickHouse host of the specified database. The maximum string length in characters is 253. |
port | int64 Port to use when connecting to the host. Acceptable values are 0 to 65535, inclusive. |
user | string Required. Name of the ClickHouse database user. |
password | string Password of the ClickHouse database user. |
where | string Selection criteria for the data in the specified ClickHouse table. |
MongodbSource
Field | Description |
---|---|
db | string Required. Name of the MongoDB database. |
collection | string Required. Name of the collection in the specified database to be used as the dictionary source. |
host | string Required. MongoDB host of the specified database. The maximum string length in characters is 253. |
port | int64 Port to use when connecting to the host. Acceptable values are 0 to 65535, inclusive. |
user | string Required. Name of the MongoDB database user. |
password | string Password of the MongoDB database user. |
options | string |
PostgresqlSource
Field | Description |
---|---|
db | string Required. Name of the PostrgreSQL database. |
table | string Required. Name of the table in the specified database to be used as the dictionary source. |
hosts[] | string Name of the PostrgreSQL host The number of elements must be greater than 0. |
port | int64 Port to use when connecting to the host. Acceptable values are 0 to 65535, inclusive. |
user | string Required. Name of the PostrgreSQL database user. |
password | string Password of the PostrgreSQL database user. |
invalidate_query | string Query for checking the dictionary status, to pull only updated data. For more details, see ClickHouse documentation on dictionaries |
ssl_mode | enum SslMode Mode of SSL TCP/IP connection to the PostgreSQL host. For more details, see PostgreSQL documentation
|
Structure
Field | Description |
---|---|
id | Id Single numeric key column for the dictionary. |
key | Key Composite key for the dictionary, containing of one or more key columns. For details, see ClickHouse documentation |
range_min | Attribute Field holding the beginning of the range for dictionaries with RANGE_HASHED layout. For details, see ClickHouse documentation |
range_max | Attribute Field holding the end of the range for dictionaries with RANGE_HASHED layout. For details, see ClickHouse documentation |
attributes[] | Attribute Description of the fields available for database queries. For details, see ClickHouse documentation |
Attribute
Field | Description |
---|---|
name | string Required. Name of the column. |
type | string Required. Type of the column. |
null_value | string Default value for an element without data (for example, an empty string). |
expression | string Expression, describing the attribute, if applicable. |
hierarchical | bool Indication of hierarchy support. Default value: false . |
injective | bool Indication of injective mapping "id -> attribute". Default value: false . |
Id
Field | Description |
---|---|
name | string Required. Name of the numeric key. |
Key
Field | Description |
---|---|
attributes[] | Attribute Attributes of a complex key. The number of elements must be greater than 0. |
Layout
Field | Description |
---|---|
type | enum Type Required. Layout type for an external dictionary.
|
size_in_cells | int64 Number of cells in the cache. Rounded up to a power of two. Applicable only for CACHE and COMPLEX_KEY_CACHE layout types. |
Range
Field | Description |
---|---|
min | int64 Minimum dictionary lifetime. |
max | int64 Maximum dictionary lifetime. |
GraphiteRollup
Field | Description |
---|---|
name | string Required. Name for the specified combination of settings for Graphite rollup. |
patterns[] | Pattern Pattern to use for the rollup. The number of elements must be greater than 0. |
Pattern
Field | Description |
---|---|
regexp | string Pattern for metric names. |
function | string Required. Name of the aggregating function to apply to data of the age specified in retention . |
retention[] | Retention Age of data to use for thinning. The number of elements must be greater than 0. |
Retention
Field | Description |
---|---|
age | int64 Minimum age of the data in seconds. The minimum value is 0. |
precision | int64 Precision of determining the age of the data, in seconds. Value must be greater than 0. |
MaintenanceWindow
Field | Description |
---|---|
policy | oneof: anytime or weekly_maintenance_window The maintenance policy in effect. |
anytime | AnytimeMaintenanceWindow Maintenance operation can be scheduled anytime. |
weekly_maintenance_window | WeeklyMaintenanceWindow Maintenance operation can be scheduled on a weekly basis. |
AnytimeMaintenanceWindow
Empty.
WeeklyMaintenanceWindow
Field | Description |
---|---|
day | enum WeekDay Day of the week (in DDD format). |
hour | int64 Hour of the day in UTC (in HH format). Acceptable values are 1 to 24, inclusive. |
MaintenanceOperation
Field | Description |
---|---|
info | string Information about this maintenance operation. The maximum string length in characters is 256. |
delayed_until | google.protobuf.Timestamp Time until which this maintenance operation is delayed. |
Backup
Creates a backup for the specified ClickHouse cluster.
rpc Backup (BackupClusterRequest) returns (operation.Operation)
Metadata and response of Operation:
Operation.metadata:BackupClusterMetadata
Operation.response:Cluster
BackupClusterRequest
Field | Description |
---|---|
cluster_id | string Required. ID of the ClickHouse cluster to back up. To get the ClickHouse cluster ID use a ClusterService.List request. The maximum string length in characters is 50. |
Operation
Field | Description |
---|---|
id | string ID of the operation. |
description | string Description of the operation. 0-256 characters long. |
created_at | google.protobuf.Timestamp Creation timestamp. |
created_by | string ID of the user or service account who initiated the operation. |
modified_at | google.protobuf.Timestamp The time when the Operation resource was last modified. |
done | bool If the value is false , it means the operation is still in progress. If true , the operation is completed, and either error or response is available. |
metadata | google.protobuf.Any Service-specific metadata associated with the operation. It typically contains the ID of the target resource that the operation is performed on. Any method that returns a long-running operation should document the metadata type, if any. |
result | oneof: error or response The operation result. If done == false and there was no failure detected, neither error nor response is set. If done == false and there was a failure detected, error is set. If done == true , exactly one of error or response is set. |
error | google.rpc.Status The error result of the operation in case of failure or cancellation. |
response | google.protobuf.Any if operation finished successfully. |
BackupClusterMetadata
Field | Description |
---|---|
cluster_id | string ID of the ClickHouse cluster that is being backed up. |
Cluster
Field | Description |
---|---|
id | string ID of the ClickHouse cluster. This ID is assigned by MDB at creation time. |
folder_id | string ID of the folder that the ClickHouse cluster belongs to. |
created_at | google.protobuf.Timestamp Creation timestamp in RFC3339 |
name | string Name of the ClickHouse cluster. The name is unique within the folder. 1-63 characters long. |
description | string Description of the ClickHouse cluster. 0-256 characters long. |
labels | map<string,string> Custom labels for the ClickHouse cluster as key:value pairs. Maximum 64 per resource. |
environment | enum Environment Deployment environment of the ClickHouse cluster.
|
monitoring[] | Monitoring Description of monitoring systems relevant to the ClickHouse cluster. |
config | ClusterConfig Configuration of the ClickHouse cluster. |
network_id | string ID of the network that the cluster belongs to. |
health | enum Health Aggregated cluster health.
|
status | enum Status Current state of the cluster.
|
service_account_id | string ID of the service account used for access to Object Storage. |
maintenance_window | MaintenanceWindow Maintenance window for the cluster. |
planned_operation | MaintenanceOperation Planned maintenance operation to be started for the cluster within the nearest maintenance_window . |
security_group_ids[] | string User security groups |
deletion_protection | bool Deletion Protection inhibits deletion of the cluster |
Monitoring
Field | Description |
---|---|
name | string Name of the monitoring system. |
description | string Description of the monitoring system. |
link | string Link to the monitoring system charts for the ClickHouse cluster. |
ClusterConfig
Field | Description |
---|---|
version | string Version of the ClickHouse server software. |
clickhouse | Clickhouse Configuration and resource allocation for ClickHouse hosts. |
zookeeper | Zookeeper Configuration and resource allocation for ZooKeeper hosts. |
backup_window_start | google.type.TimeOfDay Time to start the daily backup, in the UTC timezone. |
access | Access Access policy for external services. |
cloud_storage | CloudStorage |
sql_database_management | google.protobuf.BoolValue Whether database management through SQL commands is enabled. |
sql_user_management | google.protobuf.BoolValue Whether user management through SQL commands is enabled. |
embedded_keeper | google.protobuf.BoolValue Whether cluster should use embedded Keeper instead of Zookeeper. |
Clickhouse
Field | Description |
---|---|
config | config.ClickhouseConfigSet Configuration settings of a ClickHouse server. |
resources | Resources Resources allocated to ClickHouse hosts. |
Zookeeper
Field | Description |
---|---|
resources | Resources Resources allocated to ZooKeeper hosts. |
Access
Field | Description |
---|---|
data_lens | bool Allow to export data from the cluster to DataLens. |
web_sql | bool Allow SQL queries to the cluster databases from the management console. See SQL queries in the management console for more details. |
metrika | bool Allow to import data from Yandex Metrica and AppMetrica to the cluster. See AppMetrica documentation |
serverless | bool Allow access to cluster for Serverless. |
data_transfer | bool Allow access for DataTransfer |
yandex_query | bool Allow access for Query |
CloudStorage
Field | Description |
---|---|
enabled | bool Whether to use Object Storage for storing ClickHouse data. |
move_factor | google.protobuf.DoubleValue Acceptable values are 0 to 1, inclusive. |
data_cache_enabled | google.protobuf.BoolValue |
data_cache_max_size | google.protobuf.Int64Value |
prefer_not_to_merge | google.protobuf.BoolValue |
ClickhouseConfigSet
Field | Description |
---|---|
effective_config | ClickhouseConfig Required. Effective settings for a ClickHouse cluster (a combination of settings defined in user_config and default_config ). |
user_config | ClickhouseConfig User-defined settings for a ClickHouse cluster. |
default_config | ClickhouseConfig Default configuration for a ClickHouse cluster. |
ClickhouseConfig
Field | Description |
---|---|
log_level | enum LogLevel Logging level for the ClickHouse cluster. Possible values: TRACE, DEBUG, INFORMATION, WARNING, ERROR. |
merge_tree | MergeTree Settings for the MergeTree engine. See description in ClickHouse documentation |
compression[] | Compression Compression settings for the ClickHouse cluster. See in-depth description in ClickHouse documentation |
dictionaries[] | ExternalDictionary Configuration of external dictionaries to be used by the ClickHouse cluster. See in-depth description in ClickHouse documentation |
graphite_rollup[] | GraphiteRollup Settings for thinning Graphite data. See in-depth description in ClickHouse documentation |
kafka | Kafka |
kafka_topics[] | KafkaTopic |
rabbitmq | Rabbitmq |
max_connections | google.protobuf.Int64Value Maximum number of inbound connections. The minimum value is 10. |
max_concurrent_queries | google.protobuf.Int64Value Maximum number of simultaneously processed requests. The minimum value is 10. |
keep_alive_timeout | google.protobuf.Int64Value Number of milliseconds that ClickHouse waits for incoming requests before closing the connection. |
uncompressed_cache_size | google.protobuf.Int64Value Cache size (in bytes) for uncompressed data used by MergeTree tables. |
mark_cache_size | google.protobuf.Int64Value Approximate size (in bytes) of the cache of "marks" used by MergeTree tables. Value must be greater than 0. |
max_table_size_to_drop | google.protobuf.Int64Value Maximum size of the table that can be deleted using a DROP query. |
max_partition_size_to_drop | google.protobuf.Int64Value Maximum size of the partition that can be deleted using a DROP query. |
builtin_dictionaries_reload_interval | google.protobuf.Int64Value The setting is deprecated and has no effect. |
timezone | string The server's time zone to be used in DateTime fields conversions. Specified as an IANA identifier. |
geobase_enabled | google.protobuf.BoolValue Enable or disable geobase. |
geobase_uri | string Address of the archive with the user geobase in Object Storage. |
query_log_retention_size | google.protobuf.Int64Value The maximum size that query_log can grow to before old data will be removed. If set to 0, automatic removal of query_log data based on size is disabled. |
query_log_retention_time | google.protobuf.Int64Value The maximum time that query_log records will be retained before removal. If set to 0, automatic removal of query_log data based on time is disabled. |
query_thread_log_enabled | google.protobuf.BoolValue Whether query_thread_log system table is enabled. |
query_thread_log_retention_size | google.protobuf.Int64Value The maximum size that query_thread_log can grow to before old data will be removed. If set to 0, automatic removal of query_thread_log data based on size is disabled. |
query_thread_log_retention_time | google.protobuf.Int64Value The maximum time that query_thread_log records will be retained before removal. If set to 0, automatic removal of query_thread_log data based on time is disabled. |
part_log_retention_size | google.protobuf.Int64Value The maximum size that part_log can grow to before old data will be removed. If set to 0, automatic removal of part_log data based on size is disabled. |
part_log_retention_time | google.protobuf.Int64Value The maximum time that part_log records will be retained before removal. If set to 0, automatic removal of part_log data based on time is disabled. |
metric_log_enabled | google.protobuf.BoolValue Whether metric_log system table is enabled. |
metric_log_retention_size | google.protobuf.Int64Value The maximum size that metric_log can grow to before old data will be removed. If set to 0, automatic removal of metric_log data based on size is disabled. |
metric_log_retention_time | google.protobuf.Int64Value The maximum time that metric_log records will be retained before removal. If set to 0, automatic removal of metric_log data based on time is disabled. |
trace_log_enabled | google.protobuf.BoolValue Whether trace_log system table is enabled. |
trace_log_retention_size | google.protobuf.Int64Value The maximum size that trace_log can grow to before old data will be removed. If set to 0, automatic removal of trace_log data based on size is disabled. |
trace_log_retention_time | google.protobuf.Int64Value The maximum time that trace_log records will be retained before removal. If set to 0, automatic removal of trace_log data based on time is disabled. |
text_log_enabled | google.protobuf.BoolValue Whether text_log system table is enabled. |
text_log_retention_size | google.protobuf.Int64Value The maximum size that text_log can grow to before old data will be removed. If set to 0, automatic removal of text_log data based on size is disabled. |
text_log_retention_time | google.protobuf.Int64Value The maximum time that text_log records will be retained before removal. If set to 0, automatic removal of text_log data based on time is disabled. |
text_log_level | enum LogLevel Logging level for text_log system table. Possible values: TRACE, DEBUG, INFORMATION, WARNING, ERROR. |
opentelemetry_span_log_enabled | google.protobuf.BoolValue Enable or disable opentelemetry_span_log system table. Default value: false. |
opentelemetry_span_log_retention_size | google.protobuf.Int64Value The maximum size that opentelemetry_span_log can grow to before old data will be removed. If set to 0 (default), automatic removal of opentelemetry_span_log data based on size is disabled. The minimum value is 0. |
opentelemetry_span_log_retention_time | google.protobuf.Int64Value The maximum time that opentelemetry_span_log records will be retained before removal. If set to 0, automatic removal of opentelemetry_span_log data based on time is disabled. The minimum value is 0. |
query_views_log_enabled | google.protobuf.BoolValue Enable or disable query_views_log system table. Default value: false. |
query_views_log_retention_size | google.protobuf.Int64Value The maximum size that query_views_log can grow to before old data will be removed. If set to 0 (default), automatic removal of query_views_log data based on size is disabled. The minimum value is 0. |
query_views_log_retention_time | google.protobuf.Int64Value The maximum time that query_views_log records will be retained before removal. If set to 0, automatic removal of query_views_log data based on time is disabled. The minimum value is 0. |
asynchronous_metric_log_enabled | google.protobuf.BoolValue Enable or disable asynchronous_metric_log system table. Default value: false. |
asynchronous_metric_log_retention_size | google.protobuf.Int64Value The maximum size that asynchronous_metric_log can grow to before old data will be removed. If set to 0 (default), automatic removal of asynchronous_metric_log data based on size is disabled. The minimum value is 0. |
asynchronous_metric_log_retention_time | google.protobuf.Int64Value The maximum time that asynchronous_metric_log records will be retained before removal. If set to 0, automatic removal of asynchronous_metric_log data based on time is disabled. The minimum value is 0. |
session_log_enabled | google.protobuf.BoolValue Enable or disable session_log system table. Default value: false. |
session_log_retention_size | google.protobuf.Int64Value The maximum size that session_log can grow to before old data will be removed. If set to 0 (default), automatic removal of session_log data based on size is disabled. The minimum value is 0. |
session_log_retention_time | google.protobuf.Int64Value The maximum time that session_log records will be retained before removal. If set to 0, automatic removal of session_log data based on time is disabled. The minimum value is 0. |
zookeeper_log_enabled | google.protobuf.BoolValue Enable or disable zookeeper_log system table. Default value: false. |
zookeeper_log_retention_size | google.protobuf.Int64Value The maximum size that zookeeper_log can grow to before old data will be removed. If set to 0 (default), automatic removal of zookeeper_log data based on size is disabled. The minimum value is 0. |
zookeeper_log_retention_time | google.protobuf.Int64Value The maximum time that zookeeper_log records will be retained before removal. If set to 0, automatic removal of zookeeper_log data based on time is disabled. The minimum value is 0. |
asynchronous_insert_log_enabled | google.protobuf.BoolValue Enable or disable asynchronous_insert_log system table. Default value: false. Minimal required ClickHouse version: 22.10. |
asynchronous_insert_log_retention_size | google.protobuf.Int64Value The maximum size that asynchronous_insert_log can grow to before old data will be removed. If set to 0 (default), automatic removal of asynchronous_insert_log data based on size is disabled. The minimum value is 0. |
asynchronous_insert_log_retention_time | google.protobuf.Int64Value The maximum time that asynchronous_insert_log records will be retained before removal. If set to 0, automatic removal of asynchronous_insert_log data based on time is disabled. The minimum value is 0. |
background_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_merges_mutations_concurrency_ratio | google.protobuf.Int64Value Sets a ratio between the number of threads and the number of background merges and mutations that can be executed concurrently. For example, if the ratio equals to 2 and background_pool_size is set to 16 then ClickHouse can execute 32 background merges concurrently. This is possible, because background operations could be suspended and postponed. This is needed to give small merges more execution priority. You can only increase this ratio at runtime. To lower it you have to restart the server. The same as for background_pool_size setting background_merges_mutations_concurrency_ratio could be applied from the default profile for backward compatibility. Default: 2 See in-depth description in ClickHouse documentation |
background_schedule_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_fetches_pool_size | google.protobuf.Int64Value Sets the number of threads performing background fetches for tables with ReplicatedMergeTree engines. Default value: 8. More info see in ClickHouse documentation |
background_move_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_distributed_schedule_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_buffer_flush_schedule_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_message_broker_schedule_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_common_pool_size | google.protobuf.Int64Value The maximum number of threads that will be used for performing a variety of operations (mostly garbage collection) for *MergeTree-engine tables in a background. Default: 8 See in-depth description in ClickHouse documentation |
default_database | google.protobuf.StringValue The default database. To get a list of cluster databases, see Yandex Managed ClickHouse documentation. |
total_memory_profiler_step | google.protobuf.Int64Value Sets the memory size (in bytes) for a stack trace at every peak allocation step. Default value: 4194304. More info see in ClickHouse documentation |
total_memory_tracker_sample_probability | google.protobuf.DoubleValue |
MergeTree
Field | Description |
---|---|
replicated_deduplication_window | google.protobuf.Int64Value Number of blocks of hashes to keep in ZooKeeper. |
replicated_deduplication_window_seconds | google.protobuf.Int64Value Period of time to keep blocks of hashes for. |
parts_to_delay_insert | google.protobuf.Int64Value If table contains at least that many active parts in single partition, artificially slow down insert into table. |
parts_to_throw_insert | google.protobuf.Int64Value If more than this number active parts in single partition, throw 'Too many parts ...' exception. |
inactive_parts_to_delay_insert | google.protobuf.Int64Value |
inactive_parts_to_throw_insert | google.protobuf.Int64Value |
max_replicated_merges_in_queue | google.protobuf.Int64Value How many tasks of merging and mutating parts are allowed simultaneously in ReplicatedMergeTree queue. |
number_of_free_entries_in_pool_to_lower_max_size_of_merge | google.protobuf.Int64Value If there is less than specified number of free entries in background pool (or replicated queue), start to lower maximum size of merge to process. |
max_bytes_to_merge_at_min_space_in_pool | google.protobuf.Int64Value Maximum in total size of parts to merge, when there are minimum free threads in background pool (or entries in replication queue). |
max_bytes_to_merge_at_max_space_in_pool | google.protobuf.Int64Value |
min_bytes_for_wide_part | google.protobuf.Int64Value Minimum number of bytes in a data part that can be stored in Wide format. More info see in ClickHouse documentation |
min_rows_for_wide_part | google.protobuf.Int64Value Minimum number of rows in a data part that can be stored in Wide format. More info see in ClickHouse documentation |
ttl_only_drop_parts | google.protobuf.BoolValue Enables or disables complete dropping of data parts where all rows are expired in MergeTree tables. More info see in ClickHouse documentation |
allow_remote_fs_zero_copy_replication | google.protobuf.BoolValue |
merge_with_ttl_timeout | google.protobuf.Int64Value |
merge_with_recompression_ttl_timeout | google.protobuf.Int64Value |
max_parts_in_total | google.protobuf.Int64Value |
max_number_of_merges_with_ttl_in_pool | google.protobuf.Int64Value |
cleanup_delay_period | google.protobuf.Int64Value |
number_of_free_entries_in_pool_to_execute_mutation | google.protobuf.Int64Value |
max_avg_part_size_for_too_many_parts | google.protobuf.Int64Value The 'too many parts' check according to 'parts_to_delay_insert' and 'parts_to_throw_insert' will be active only if the average part size (in the relevant partition) is not larger than the specified threshold. If it is larger than the specified threshold, the INSERTs will be neither delayed or rejected. This allows to have hundreds of terabytes in a single table on a single server if the parts are successfully merged to larger parts. This does not affect the thresholds on inactive parts or total parts. Default: 1 GiB Min version: 22.10 See in-depth description in ClickHouse GitHub |
min_age_to_force_merge_seconds | google.protobuf.Int64Value Merge parts if every part in the range is older than the value of min_age_to_force_merge_seconds. Default: 0 - disabled Min_version: 22.10 See in-depth description in ClickHouse documentation |
min_age_to_force_merge_on_partition_only | google.protobuf.BoolValue Whether min_age_to_force_merge_seconds should be applied only on the entire partition and not on subset. Default: false Min_version: 22.11 See in-depth description in ClickHouse documentation |
merge_selecting_sleep_ms | google.protobuf.Int64Value Sleep time for merge selecting when no part is selected. A lower setting triggers selecting tasks in background_schedule_pool frequently, which results in a large number of requests to ClickHouse Keeper in large-scale clusters. Default: 5000 Min_version: 21.10 See in-depth description in ClickHouse documentation |
Kafka
Field | Description |
---|---|
security_protocol | enum SecurityProtocol |
sasl_mechanism | enum SaslMechanism |
sasl_username | string |
sasl_password | string |
enable_ssl_certificate_verification | google.protobuf.BoolValue |
max_poll_interval_ms | google.protobuf.Int64Value The minimum value is 0. |
session_timeout_ms | google.protobuf.Int64Value The minimum value is 0. |
KafkaTopic
Field | Description |
---|---|
name | string Required. |
settings | Kafka Required. |
Rabbitmq
Field | Description |
---|---|
username | string RabbitMQ |
password | string RabbitMQ |
vhost | string RabbitMQ |
Compression
Field | Description |
---|---|
method | enum Method Compression method to use for the specified combination of min_part_size and min_part_size_ratio . |
min_part_size | int64 Minimum size of a part of a table. The minimum value is 1. |
min_part_size_ratio | double Minimum ratio of a part relative to the size of all the data in the table. |
level | google.protobuf.Int64Value The minimum value is 0. |
ExternalDictionary
Field | Description |
---|---|
name | string Required. Name of the external dictionary. |
structure | Structure Required. Set of attributes for the external dictionary. For in-depth description, see ClickHouse documentation |
layout | Layout Required. Layout for storing the dictionary in memory. For in-depth description, see ClickHouse documentation |
lifetime | oneof: fixed_lifetime or lifetime_range Setting for the period of time between dictionary updates. For details, see ClickHouse documentation |
fixed_lifetime | int64 Fixed interval between dictionary updates. |
lifetime_range | Range Range of intervals between dictionary updates for ClickHouse to choose from. |
source | oneof: http_source , mysql_source , clickhouse_source , mongodb_source or postgresql_source Description of the source for the external dictionary. |
http_source | HttpSource HTTP source for the dictionary. |
mysql_source | MysqlSource MySQL source for the dictionary. |
clickhouse_source | ClickhouseSource ClickHouse source for the dictionary. |
mongodb_source | MongodbSource MongoDB source for the dictionary. |
postgresql_source | PostgresqlSource PostgreSQL source for the dictionary. |
HttpSource
Field | Description |
---|---|
url | string Required. URL of the source dictionary available over HTTP. |
format | string Required. The data format. Valid values are all formats supported by ClickHouse SQL dialect. |
MysqlSource
Field | Description |
---|---|
db | string Required. Name of the MySQL database to connect to. |
table | string Required. Name of the database table to use as a ClickHouse dictionary. |
port | int64 Default port to use when connecting to a replica of the dictionary source. Acceptable values are 0 to 65535, inclusive. |
user | string Name of the default user for replicas of the dictionary source. |
password | string Password of the default user for replicas of the dictionary source. |
replicas[] | Replica List of MySQL replicas of the database used as dictionary source. The number of elements must be greater than 0. |
where | string Selection criteria for the data in the specified MySQL table. |
invalidate_query | string Query for checking the dictionary status, to pull only updated data. For more details, see ClickHouse documentation on dictionaries |
Replica
Field | Description |
---|---|
host | string Required. MySQL host of the replica. The maximum string length in characters is 253. |
priority | int64 Required. The priority of the replica that ClickHouse takes into account when connecting. Replica with the highest priority should have this field set to the lowest number. Value must be greater than 0. |
port | int64 Port to use when connecting to the replica. If a port is not specified for a replica, ClickHouse uses the port specified for the source. Acceptable values are 0 to 65535, inclusive. |
user | string Name of the MySQL database user. |
password | string Password of the MySQL database user. |
ClickhouseSource
Field | Description |
---|---|
db | string Required. Name of the ClickHouse database. |
table | string Required. Name of the table in the specified database to be used as the dictionary source. |
host | string Required. ClickHouse host of the specified database. The maximum string length in characters is 253. |
port | int64 Port to use when connecting to the host. Acceptable values are 0 to 65535, inclusive. |
user | string Required. Name of the ClickHouse database user. |
password | string Password of the ClickHouse database user. |
where | string Selection criteria for the data in the specified ClickHouse table. |
MongodbSource
Field | Description |
---|---|
db | string Required. Name of the MongoDB database. |
collection | string Required. Name of the collection in the specified database to be used as the dictionary source. |
host | string Required. MongoDB host of the specified database. The maximum string length in characters is 253. |
port | int64 Port to use when connecting to the host. Acceptable values are 0 to 65535, inclusive. |
user | string Required. Name of the MongoDB database user. |
password | string Password of the MongoDB database user. |
options | string |
PostgresqlSource
Field | Description |
---|---|
db | string Required. Name of the PostrgreSQL database. |
table | string Required. Name of the table in the specified database to be used as the dictionary source. |
hosts[] | string Name of the PostrgreSQL host The number of elements must be greater than 0. |
port | int64 Port to use when connecting to the host. Acceptable values are 0 to 65535, inclusive. |
user | string Required. Name of the PostrgreSQL database user. |
password | string Password of the PostrgreSQL database user. |
invalidate_query | string Query for checking the dictionary status, to pull only updated data. For more details, see ClickHouse documentation on dictionaries |
ssl_mode | enum SslMode Mode of SSL TCP/IP connection to the PostgreSQL host. For more details, see PostgreSQL documentation
|
Structure
Field | Description |
---|---|
id | Id Single numeric key column for the dictionary. |
key | Key Composite key for the dictionary, containing of one or more key columns. For details, see ClickHouse documentation |
range_min | Attribute Field holding the beginning of the range for dictionaries with RANGE_HASHED layout. For details, see ClickHouse documentation |
range_max | Attribute Field holding the end of the range for dictionaries with RANGE_HASHED layout. For details, see ClickHouse documentation |
attributes[] | Attribute Description of the fields available for database queries. For details, see ClickHouse documentation |
Attribute
Field | Description |
---|---|
name | string Required. Name of the column. |
type | string Required. Type of the column. |
null_value | string Default value for an element without data (for example, an empty string). |
expression | string Expression, describing the attribute, if applicable. |
hierarchical | bool Indication of hierarchy support. Default value: false . |
injective | bool Indication of injective mapping "id -> attribute". Default value: false . |
Id
Field | Description |
---|---|
name | string Required. Name of the numeric key. |
Key
Field | Description |
---|---|
attributes[] | Attribute Attributes of a complex key. The number of elements must be greater than 0. |
Layout
Field | Description |
---|---|
type | enum Type Required. Layout type for an external dictionary.
|
size_in_cells | int64 Number of cells in the cache. Rounded up to a power of two. Applicable only for CACHE and COMPLEX_KEY_CACHE layout types. |
Range
Field | Description |
---|---|
min | int64 Minimum dictionary lifetime. |
max | int64 Maximum dictionary lifetime. |
GraphiteRollup
Field | Description |
---|---|
name | string Required. Name for the specified combination of settings for Graphite rollup. |
patterns[] | Pattern Pattern to use for the rollup. The number of elements must be greater than 0. |
Pattern
Field | Description |
---|---|
regexp | string Pattern for metric names. |
function | string Required. Name of the aggregating function to apply to data of the age specified in retention . |
retention[] | Retention Age of data to use for thinning. The number of elements must be greater than 0. |
Retention
Field | Description |
---|---|
age | int64 Minimum age of the data in seconds. The minimum value is 0. |
precision | int64 Precision of determining the age of the data, in seconds. Value must be greater than 0. |
Resources
Field | Description |
---|---|
resource_preset_id | string ID of the preset for computational resources available to a host (CPU, memory etc.). All available presets are listed in the documentation |
disk_size | int64 Volume of the storage available to a host, in bytes. |
disk_type_id | string Type of the storage environment for the host. Possible values:
|
MaintenanceWindow
Field | Description |
---|---|
policy | oneof: anytime or weekly_maintenance_window The maintenance policy in effect. |
anytime | AnytimeMaintenanceWindow Maintenance operation can be scheduled anytime. |
weekly_maintenance_window | WeeklyMaintenanceWindow Maintenance operation can be scheduled on a weekly basis. |
AnytimeMaintenanceWindow
Empty.
WeeklyMaintenanceWindow
Field | Description |
---|---|
day | enum WeekDay Day of the week (in DDD format). |
hour | int64 Hour of the day in UTC (in HH format). Acceptable values are 1 to 24, inclusive. |
MaintenanceOperation
Field | Description |
---|---|
info | string Information about this maintenance operation. The maximum string length in characters is 256. |
delayed_until | google.protobuf.Timestamp Time until which this maintenance operation is delayed. |
Restore
Creates a new ClickHouse cluster using the specified backup.
rpc Restore (RestoreClusterRequest) returns (operation.Operation)
Metadata and response of Operation:
Operation.metadata:RestoreClusterMetadata
Operation.response:Cluster
RestoreClusterRequest
Field | Description |
---|---|
backup_id | string Required. ID of the backup to restore from. This backup will be used to create one cluster shard. To get the backup ID, use a ClusterService.ListBackups request. |
additional_backup_ids[] | string Additional IDs of the backups to restore from. Each additional backup is responsible for restoring separate shard. Restored cluster will have len(additional_backup_ids)+1 shards in total. To get the backup ID, use a ClusterService.ListBackups request. |
name | string Required. Name of the new ClickHouse cluster. The name must be unique within the folder. The maximum string length in characters is 63. Value must match the regular expression [a-zA-Z0-9_-]* . |
description | string Description of the new ClickHouse cluster. The maximum string length in characters is 256. |
labels | map<string,string> Custom labels for the ClickHouse cluster as key:value pairs. Maximum 64 per resource. For example, "project": "mvp" or "source": "dictionary". No more than 64 per resource. The maximum string length in characters for each value is 63. Each value must match the regular expression [-_0-9a-z]* . The maximum string length in characters for each key is 63. Each key must match the regular expression [a-z][-_0-9a-z]* . |
environment | Cluster.Environment Required. Deployment environment of the new ClickHouse cluster. |
config_spec | ConfigSpec Required. Configuration for the ClickHouse cluster to be created. |
host_specs[] | HostSpec Configurations for ClickHouse hosts that should be created for the cluster that is being created from the backup. The number of elements must be greater than 0. |
network_id | string Required. ID of the network to create the ClickHouse cluster in. The maximum string length in characters is 50. |
folder_id | string ID of the folder to create the ClickHouse cluster in. The maximum string length in characters is 50. |
service_account_id | string ID of the service account used for access to Object Storage. |
security_group_ids[] | string User security groups |
ConfigSpec
Field | Description |
---|---|
version | string Version of the ClickHouse server software. |
clickhouse | Clickhouse Configuration and resources for a ClickHouse server. |
zookeeper | Zookeeper Configuration and resources for a ZooKeeper server. |
backup_window_start | google.type.TimeOfDay Time to start the daily backup, in the UTC timezone. |
access | Access Access policy for external services. If you want a specific service to access the ClickHouse cluster, then set the necessary values in this policy. |
cloud_storage | CloudStorage |
sql_database_management | google.protobuf.BoolValue Whether database management through SQL commands is enabled. |
sql_user_management | google.protobuf.BoolValue Whether user management through SQL commands is enabled. |
admin_password | string Password for user 'admin' that has SQL user management access. |
embedded_keeper | google.protobuf.BoolValue Whether cluster should use embedded Keeper instead of Zookeeper |
Clickhouse
Field | Description |
---|---|
config | config.ClickhouseConfig Configuration for a ClickHouse server. |
resources | Resources Resources allocated to ClickHouse hosts. |
Zookeeper
Field | Description |
---|---|
resources | Resources Resources allocated to ZooKeeper hosts. If not set, minimal available resources will be used. All available resource presets can be retrieved with a ResourcePresetService.List request. |
Access
Field | Description |
---|---|
data_lens | bool Allow to export data from the cluster to DataLens. |
web_sql | bool Allow SQL queries to the cluster databases from the management console. See SQL queries in the management console for more details. |
metrika | bool Allow to import data from Yandex Metrica and AppMetrica to the cluster. See AppMetrica documentation |
serverless | bool Allow access to cluster for Serverless. |
data_transfer | bool Allow access for DataTransfer |
yandex_query | bool Allow access for Query |
CloudStorage
Field | Description |
---|---|
enabled | bool Whether to use Object Storage for storing ClickHouse data. |
move_factor | google.protobuf.DoubleValue Acceptable values are 0 to 1, inclusive. |
data_cache_enabled | google.protobuf.BoolValue |
data_cache_max_size | google.protobuf.Int64Value |
prefer_not_to_merge | google.protobuf.BoolValue |
ClickhouseConfig
Field | Description |
---|---|
log_level | enum LogLevel Logging level for the ClickHouse cluster. Possible values: TRACE, DEBUG, INFORMATION, WARNING, ERROR. |
merge_tree | MergeTree Settings for the MergeTree engine. See description in ClickHouse documentation |
compression[] | Compression Compression settings for the ClickHouse cluster. See in-depth description in ClickHouse documentation |
dictionaries[] | ExternalDictionary Configuration of external dictionaries to be used by the ClickHouse cluster. See in-depth description in ClickHouse documentation |
graphite_rollup[] | GraphiteRollup Settings for thinning Graphite data. See in-depth description in ClickHouse documentation |
kafka | Kafka |
kafka_topics[] | KafkaTopic |
rabbitmq | Rabbitmq |
max_connections | google.protobuf.Int64Value Maximum number of inbound connections. The minimum value is 10. |
max_concurrent_queries | google.protobuf.Int64Value Maximum number of simultaneously processed requests. The minimum value is 10. |
keep_alive_timeout | google.protobuf.Int64Value Number of milliseconds that ClickHouse waits for incoming requests before closing the connection. |
uncompressed_cache_size | google.protobuf.Int64Value Cache size (in bytes) for uncompressed data used by MergeTree tables. |
mark_cache_size | google.protobuf.Int64Value Approximate size (in bytes) of the cache of "marks" used by MergeTree tables. Value must be greater than 0. |
max_table_size_to_drop | google.protobuf.Int64Value Maximum size of the table that can be deleted using a DROP query. |
max_partition_size_to_drop | google.protobuf.Int64Value Maximum size of the partition that can be deleted using a DROP query. |
builtin_dictionaries_reload_interval | google.protobuf.Int64Value The setting is deprecated and has no effect. |
timezone | string The server's time zone to be used in DateTime fields conversions. Specified as an IANA identifier. |
geobase_enabled | google.protobuf.BoolValue Enable or disable geobase. |
geobase_uri | string Address of the archive with the user geobase in Object Storage. |
query_log_retention_size | google.protobuf.Int64Value The maximum size that query_log can grow to before old data will be removed. If set to 0, automatic removal of query_log data based on size is disabled. |
query_log_retention_time | google.protobuf.Int64Value The maximum time that query_log records will be retained before removal. If set to 0, automatic removal of query_log data based on time is disabled. |
query_thread_log_enabled | google.protobuf.BoolValue Whether query_thread_log system table is enabled. |
query_thread_log_retention_size | google.protobuf.Int64Value The maximum size that query_thread_log can grow to before old data will be removed. If set to 0, automatic removal of query_thread_log data based on size is disabled. |
query_thread_log_retention_time | google.protobuf.Int64Value The maximum time that query_thread_log records will be retained before removal. If set to 0, automatic removal of query_thread_log data based on time is disabled. |
part_log_retention_size | google.protobuf.Int64Value The maximum size that part_log can grow to before old data will be removed. If set to 0, automatic removal of part_log data based on size is disabled. |
part_log_retention_time | google.protobuf.Int64Value The maximum time that part_log records will be retained before removal. If set to 0, automatic removal of part_log data based on time is disabled. |
metric_log_enabled | google.protobuf.BoolValue Whether metric_log system table is enabled. |
metric_log_retention_size | google.protobuf.Int64Value The maximum size that metric_log can grow to before old data will be removed. If set to 0, automatic removal of metric_log data based on size is disabled. |
metric_log_retention_time | google.protobuf.Int64Value The maximum time that metric_log records will be retained before removal. If set to 0, automatic removal of metric_log data based on time is disabled. |
trace_log_enabled | google.protobuf.BoolValue Whether trace_log system table is enabled. |
trace_log_retention_size | google.protobuf.Int64Value The maximum size that trace_log can grow to before old data will be removed. If set to 0, automatic removal of trace_log data based on size is disabled. |
trace_log_retention_time | google.protobuf.Int64Value The maximum time that trace_log records will be retained before removal. If set to 0, automatic removal of trace_log data based on time is disabled. |
text_log_enabled | google.protobuf.BoolValue Whether text_log system table is enabled. |
text_log_retention_size | google.protobuf.Int64Value The maximum size that text_log can grow to before old data will be removed. If set to 0, automatic removal of text_log data based on size is disabled. |
text_log_retention_time | google.protobuf.Int64Value The maximum time that text_log records will be retained before removal. If set to 0, automatic removal of text_log data based on time is disabled. |
text_log_level | enum LogLevel Logging level for text_log system table. Possible values: TRACE, DEBUG, INFORMATION, WARNING, ERROR. |
opentelemetry_span_log_enabled | google.protobuf.BoolValue Enable or disable opentelemetry_span_log system table. Default value: false. |
opentelemetry_span_log_retention_size | google.protobuf.Int64Value The maximum size that opentelemetry_span_log can grow to before old data will be removed. If set to 0 (default), automatic removal of opentelemetry_span_log data based on size is disabled. The minimum value is 0. |
opentelemetry_span_log_retention_time | google.protobuf.Int64Value The maximum time that opentelemetry_span_log records will be retained before removal. If set to 0, automatic removal of opentelemetry_span_log data based on time is disabled. The minimum value is 0. |
query_views_log_enabled | google.protobuf.BoolValue Enable or disable query_views_log system table. Default value: false. |
query_views_log_retention_size | google.protobuf.Int64Value The maximum size that query_views_log can grow to before old data will be removed. If set to 0 (default), automatic removal of query_views_log data based on size is disabled. The minimum value is 0. |
query_views_log_retention_time | google.protobuf.Int64Value The maximum time that query_views_log records will be retained before removal. If set to 0, automatic removal of query_views_log data based on time is disabled. The minimum value is 0. |
asynchronous_metric_log_enabled | google.protobuf.BoolValue Enable or disable asynchronous_metric_log system table. Default value: false. |
asynchronous_metric_log_retention_size | google.protobuf.Int64Value The maximum size that asynchronous_metric_log can grow to before old data will be removed. If set to 0 (default), automatic removal of asynchronous_metric_log data based on size is disabled. The minimum value is 0. |
asynchronous_metric_log_retention_time | google.protobuf.Int64Value The maximum time that asynchronous_metric_log records will be retained before removal. If set to 0, automatic removal of asynchronous_metric_log data based on time is disabled. The minimum value is 0. |
session_log_enabled | google.protobuf.BoolValue Enable or disable session_log system table. Default value: false. |
session_log_retention_size | google.protobuf.Int64Value The maximum size that session_log can grow to before old data will be removed. If set to 0 (default), automatic removal of session_log data based on size is disabled. The minimum value is 0. |
session_log_retention_time | google.protobuf.Int64Value The maximum time that session_log records will be retained before removal. If set to 0, automatic removal of session_log data based on time is disabled. The minimum value is 0. |
zookeeper_log_enabled | google.protobuf.BoolValue Enable or disable zookeeper_log system table. Default value: false. |
zookeeper_log_retention_size | google.protobuf.Int64Value The maximum size that zookeeper_log can grow to before old data will be removed. If set to 0 (default), automatic removal of zookeeper_log data based on size is disabled. The minimum value is 0. |
zookeeper_log_retention_time | google.protobuf.Int64Value The maximum time that zookeeper_log records will be retained before removal. If set to 0, automatic removal of zookeeper_log data based on time is disabled. The minimum value is 0. |
asynchronous_insert_log_enabled | google.protobuf.BoolValue Enable or disable asynchronous_insert_log system table. Default value: false. Minimal required ClickHouse version: 22.10. |
asynchronous_insert_log_retention_size | google.protobuf.Int64Value The maximum size that asynchronous_insert_log can grow to before old data will be removed. If set to 0 (default), automatic removal of asynchronous_insert_log data based on size is disabled. The minimum value is 0. |
asynchronous_insert_log_retention_time | google.protobuf.Int64Value The maximum time that asynchronous_insert_log records will be retained before removal. If set to 0, automatic removal of asynchronous_insert_log data based on time is disabled. The minimum value is 0. |
background_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_merges_mutations_concurrency_ratio | google.protobuf.Int64Value Sets a ratio between the number of threads and the number of background merges and mutations that can be executed concurrently. For example, if the ratio equals to 2 and background_pool_size is set to 16 then ClickHouse can execute 32 background merges concurrently. This is possible, because background operations could be suspended and postponed. This is needed to give small merges more execution priority. You can only increase this ratio at runtime. To lower it you have to restart the server. The same as for background_pool_size setting background_merges_mutations_concurrency_ratio could be applied from the default profile for backward compatibility. Default: 2 See in-depth description in ClickHouse documentation |
background_schedule_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_fetches_pool_size | google.protobuf.Int64Value Sets the number of threads performing background fetches for tables with ReplicatedMergeTree engines. Default value: 8. More info see in ClickHouse documentation |
background_move_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_distributed_schedule_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_buffer_flush_schedule_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_message_broker_schedule_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_common_pool_size | google.protobuf.Int64Value The maximum number of threads that will be used for performing a variety of operations (mostly garbage collection) for *MergeTree-engine tables in a background. Default: 8 See in-depth description in ClickHouse documentation |
default_database | google.protobuf.StringValue The default database. To get a list of cluster databases, see Yandex Managed ClickHouse documentation. |
total_memory_profiler_step | google.protobuf.Int64Value Sets the memory size (in bytes) for a stack trace at every peak allocation step. Default value: 4194304. More info see in ClickHouse documentation |
total_memory_tracker_sample_probability | google.protobuf.DoubleValue |
MergeTree
Field | Description |
---|---|
replicated_deduplication_window | google.protobuf.Int64Value Number of blocks of hashes to keep in ZooKeeper. |
replicated_deduplication_window_seconds | google.protobuf.Int64Value Period of time to keep blocks of hashes for. |
parts_to_delay_insert | google.protobuf.Int64Value If table contains at least that many active parts in single partition, artificially slow down insert into table. |
parts_to_throw_insert | google.protobuf.Int64Value If more than this number active parts in single partition, throw 'Too many parts ...' exception. |
inactive_parts_to_delay_insert | google.protobuf.Int64Value |
inactive_parts_to_throw_insert | google.protobuf.Int64Value |
max_replicated_merges_in_queue | google.protobuf.Int64Value How many tasks of merging and mutating parts are allowed simultaneously in ReplicatedMergeTree queue. |
number_of_free_entries_in_pool_to_lower_max_size_of_merge | google.protobuf.Int64Value If there is less than specified number of free entries in background pool (or replicated queue), start to lower maximum size of merge to process. |
max_bytes_to_merge_at_min_space_in_pool | google.protobuf.Int64Value Maximum in total size of parts to merge, when there are minimum free threads in background pool (or entries in replication queue). |
max_bytes_to_merge_at_max_space_in_pool | google.protobuf.Int64Value |
min_bytes_for_wide_part | google.protobuf.Int64Value Minimum number of bytes in a data part that can be stored in Wide format. More info see in ClickHouse documentation |
min_rows_for_wide_part | google.protobuf.Int64Value Minimum number of rows in a data part that can be stored in Wide format. More info see in ClickHouse documentation |
ttl_only_drop_parts | google.protobuf.BoolValue Enables or disables complete dropping of data parts where all rows are expired in MergeTree tables. More info see in ClickHouse documentation |
allow_remote_fs_zero_copy_replication | google.protobuf.BoolValue |
merge_with_ttl_timeout | google.protobuf.Int64Value |
merge_with_recompression_ttl_timeout | google.protobuf.Int64Value |
max_parts_in_total | google.protobuf.Int64Value |
max_number_of_merges_with_ttl_in_pool | google.protobuf.Int64Value |
cleanup_delay_period | google.protobuf.Int64Value |
number_of_free_entries_in_pool_to_execute_mutation | google.protobuf.Int64Value |
max_avg_part_size_for_too_many_parts | google.protobuf.Int64Value The 'too many parts' check according to 'parts_to_delay_insert' and 'parts_to_throw_insert' will be active only if the average part size (in the relevant partition) is not larger than the specified threshold. If it is larger than the specified threshold, the INSERTs will be neither delayed or rejected. This allows to have hundreds of terabytes in a single table on a single server if the parts are successfully merged to larger parts. This does not affect the thresholds on inactive parts or total parts. Default: 1 GiB Min version: 22.10 See in-depth description in ClickHouse GitHub |
min_age_to_force_merge_seconds | google.protobuf.Int64Value Merge parts if every part in the range is older than the value of min_age_to_force_merge_seconds. Default: 0 - disabled Min_version: 22.10 See in-depth description in ClickHouse documentation |
min_age_to_force_merge_on_partition_only | google.protobuf.BoolValue Whether min_age_to_force_merge_seconds should be applied only on the entire partition and not on subset. Default: false Min_version: 22.11 See in-depth description in ClickHouse documentation |
merge_selecting_sleep_ms | google.protobuf.Int64Value Sleep time for merge selecting when no part is selected. A lower setting triggers selecting tasks in background_schedule_pool frequently, which results in a large number of requests to ClickHouse Keeper in large-scale clusters. Default: 5000 Min_version: 21.10 See in-depth description in ClickHouse documentation |
Kafka
Field | Description |
---|---|
security_protocol | enum SecurityProtocol |
sasl_mechanism | enum SaslMechanism |
sasl_username | string |
sasl_password | string |
enable_ssl_certificate_verification | google.protobuf.BoolValue |
max_poll_interval_ms | google.protobuf.Int64Value The minimum value is 0. |
session_timeout_ms | google.protobuf.Int64Value The minimum value is 0. |
KafkaTopic
Field | Description |
---|---|
name | string Required. |
settings | Kafka Required. |
Rabbitmq
Field | Description |
---|---|
username | string RabbitMQ |
password | string RabbitMQ |
vhost | string RabbitMQ |
Compression
Field | Description |
---|---|
method | enum Method Compression method to use for the specified combination of min_part_size and min_part_size_ratio . |
min_part_size | int64 Minimum size of a part of a table. The minimum value is 1. |
min_part_size_ratio | double Minimum ratio of a part relative to the size of all the data in the table. |
level | google.protobuf.Int64Value The minimum value is 0. |
ExternalDictionary
Field | Description |
---|---|
name | string Required. Name of the external dictionary. |
structure | Structure Required. Set of attributes for the external dictionary. For in-depth description, see ClickHouse documentation |
layout | Layout Required. Layout for storing the dictionary in memory. For in-depth description, see ClickHouse documentation |
lifetime | oneof: fixed_lifetime or lifetime_range Setting for the period of time between dictionary updates. For details, see ClickHouse documentation |
fixed_lifetime | int64 Fixed interval between dictionary updates. |
lifetime_range | Range Range of intervals between dictionary updates for ClickHouse to choose from. |
source | oneof: http_source , mysql_source , clickhouse_source , mongodb_source or postgresql_source Description of the source for the external dictionary. |
http_source | HttpSource HTTP source for the dictionary. |
mysql_source | MysqlSource MySQL source for the dictionary. |
clickhouse_source | ClickhouseSource ClickHouse source for the dictionary. |
mongodb_source | MongodbSource MongoDB source for the dictionary. |
postgresql_source | PostgresqlSource PostgreSQL source for the dictionary. |
HttpSource
Field | Description |
---|---|
url | string Required. URL of the source dictionary available over HTTP. |
format | string Required. The data format. Valid values are all formats supported by ClickHouse SQL dialect. |
MysqlSource
Field | Description |
---|---|
db | string Required. Name of the MySQL database to connect to. |
table | string Required. Name of the database table to use as a ClickHouse dictionary. |
port | int64 Default port to use when connecting to a replica of the dictionary source. Acceptable values are 0 to 65535, inclusive. |
user | string Name of the default user for replicas of the dictionary source. |
password | string Password of the default user for replicas of the dictionary source. |
replicas[] | Replica List of MySQL replicas of the database used as dictionary source. The number of elements must be greater than 0. |
where | string Selection criteria for the data in the specified MySQL table. |
invalidate_query | string Query for checking the dictionary status, to pull only updated data. For more details, see ClickHouse documentation on dictionaries |
Replica
Field | Description |
---|---|
host | string Required. MySQL host of the replica. The maximum string length in characters is 253. |
priority | int64 Required. The priority of the replica that ClickHouse takes into account when connecting. Replica with the highest priority should have this field set to the lowest number. Value must be greater than 0. |
port | int64 Port to use when connecting to the replica. If a port is not specified for a replica, ClickHouse uses the port specified for the source. Acceptable values are 0 to 65535, inclusive. |
user | string Name of the MySQL database user. |
password | string Password of the MySQL database user. |
ClickhouseSource
Field | Description |
---|---|
db | string Required. Name of the ClickHouse database. |
table | string Required. Name of the table in the specified database to be used as the dictionary source. |
host | string Required. ClickHouse host of the specified database. The maximum string length in characters is 253. |
port | int64 Port to use when connecting to the host. Acceptable values are 0 to 65535, inclusive. |
user | string Required. Name of the ClickHouse database user. |
password | string Password of the ClickHouse database user. |
where | string Selection criteria for the data in the specified ClickHouse table. |
MongodbSource
Field | Description |
---|---|
db | string Required. Name of the MongoDB database. |
collection | string Required. Name of the collection in the specified database to be used as the dictionary source. |
host | string Required. MongoDB host of the specified database. The maximum string length in characters is 253. |
port | int64 Port to use when connecting to the host. Acceptable values are 0 to 65535, inclusive. |
user | string Required. Name of the MongoDB database user. |
password | string Password of the MongoDB database user. |
options | string |
PostgresqlSource
Field | Description |
---|---|
db | string Required. Name of the PostrgreSQL database. |
table | string Required. Name of the table in the specified database to be used as the dictionary source. |
hosts[] | string Name of the PostrgreSQL host The number of elements must be greater than 0. |
port | int64 Port to use when connecting to the host. Acceptable values are 0 to 65535, inclusive. |
user | string Required. Name of the PostrgreSQL database user. |
password | string Password of the PostrgreSQL database user. |
invalidate_query | string Query for checking the dictionary status, to pull only updated data. For more details, see ClickHouse documentation on dictionaries |
ssl_mode | enum SslMode Mode of SSL TCP/IP connection to the PostgreSQL host. For more details, see PostgreSQL documentation
|
Structure
Field | Description |
---|---|
id | Id Single numeric key column for the dictionary. |
key | Key Composite key for the dictionary, containing of one or more key columns. For details, see ClickHouse documentation |
range_min | Attribute Field holding the beginning of the range for dictionaries with RANGE_HASHED layout. For details, see ClickHouse documentation |
range_max | Attribute Field holding the end of the range for dictionaries with RANGE_HASHED layout. For details, see ClickHouse documentation |
attributes[] | Attribute Description of the fields available for database queries. For details, see ClickHouse documentation |
Attribute
Field | Description |
---|---|
name | string Required. Name of the column. |
type | string Required. Type of the column. |
null_value | string Default value for an element without data (for example, an empty string). |
expression | string Expression, describing the attribute, if applicable. |
hierarchical | bool Indication of hierarchy support. Default value: false . |
injective | bool Indication of injective mapping "id -> attribute". Default value: false . |
Id
Field | Description |
---|---|
name | string Required. Name of the numeric key. |
Key
Field | Description |
---|---|
attributes[] | Attribute Attributes of a complex key. The number of elements must be greater than 0. |
Layout
Field | Description |
---|---|
type | enum Type Required. Layout type for an external dictionary.
|
size_in_cells | int64 Number of cells in the cache. Rounded up to a power of two. Applicable only for CACHE and COMPLEX_KEY_CACHE layout types. |
Range
Field | Description |
---|---|
min | int64 Minimum dictionary lifetime. |
max | int64 Maximum dictionary lifetime. |
GraphiteRollup
Field | Description |
---|---|
name | string Required. Name for the specified combination of settings for Graphite rollup. |
patterns[] | Pattern Pattern to use for the rollup. The number of elements must be greater than 0. |
Pattern
Field | Description |
---|---|
regexp | string Pattern for metric names. |
function | string Required. Name of the aggregating function to apply to data of the age specified in retention . |
retention[] | Retention Age of data to use for thinning. The number of elements must be greater than 0. |
Retention
Field | Description |
---|---|
age | int64 Minimum age of the data in seconds. The minimum value is 0. |
precision | int64 Precision of determining the age of the data, in seconds. Value must be greater than 0. |
Resources
Field | Description |
---|---|
resource_preset_id | string ID of the preset for computational resources available to a host (CPU, memory etc.). All available presets are listed in the documentation |
disk_size | int64 Volume of the storage available to a host, in bytes. |
disk_type_id | string Type of the storage environment for the host. Possible values:
|
HostSpec
Field | Description |
---|---|
zone_id | string ID of the availability zone where the host resides. To get a list of available zones, use the yandex.cloud.compute.v1.ZoneService.List request. The maximum string length in characters is 50. |
type | Host.Type Required. Type of the host to be deployed. |
subnet_id | string ID of the subnet that the host should belong to. This subnet should be a part of the network that the cluster belongs to. The ID of the network is set in the Cluster.network_id field. The maximum string length in characters is 50. |
assign_public_ip | bool Whether the host should get a public IP address on creation. After a host has been created, this setting cannot be changed. To remove an assigned public IP, or to assign a public IP to a host without one, recreate the host with assign_public_ip set as needed. Possible values:
|
shard_name | string Name of the shard that the host is assigned to. The maximum string length in characters is 63. Value must match the regular expression [a-zA-Z0-9_-]* . |
Operation
Field | Description |
---|---|
id | string ID of the operation. |
description | string Description of the operation. 0-256 characters long. |
created_at | google.protobuf.Timestamp Creation timestamp. |
created_by | string ID of the user or service account who initiated the operation. |
modified_at | google.protobuf.Timestamp The time when the Operation resource was last modified. |
done | bool If the value is false , it means the operation is still in progress. If true , the operation is completed, and either error or response is available. |
metadata | google.protobuf.Any Service-specific metadata associated with the operation. It typically contains the ID of the target resource that the operation is performed on. Any method that returns a long-running operation should document the metadata type, if any. |
result | oneof: error or response The operation result. If done == false and there was no failure detected, neither error nor response is set. If done == false and there was a failure detected, error is set. If done == true , exactly one of error or response is set. |
error | google.rpc.Status The error result of the operation in case of failure or cancellation. |
response | google.protobuf.Any if operation finished successfully. |
RestoreClusterMetadata
Field | Description |
---|---|
cluster_id | string ID of the new ClickHouse cluster that is being created from a backup. |
backup_id | string ID of the backup that is being used for creating a cluster. |
Cluster
Field | Description |
---|---|
id | string ID of the ClickHouse cluster. This ID is assigned by MDB at creation time. |
folder_id | string ID of the folder that the ClickHouse cluster belongs to. |
created_at | google.protobuf.Timestamp Creation timestamp in RFC3339 |
name | string Name of the ClickHouse cluster. The name is unique within the folder. 1-63 characters long. |
description | string Description of the ClickHouse cluster. 0-256 characters long. |
labels | map<string,string> Custom labels for the ClickHouse cluster as key:value pairs. Maximum 64 per resource. |
environment | enum Environment Deployment environment of the ClickHouse cluster.
|
monitoring[] | Monitoring Description of monitoring systems relevant to the ClickHouse cluster. |
config | ClusterConfig Configuration of the ClickHouse cluster. |
network_id | string ID of the network that the cluster belongs to. |
health | enum Health Aggregated cluster health.
|
status | enum Status Current state of the cluster.
|
service_account_id | string ID of the service account used for access to Object Storage. |
maintenance_window | MaintenanceWindow Maintenance window for the cluster. |
planned_operation | MaintenanceOperation Planned maintenance operation to be started for the cluster within the nearest maintenance_window . |
security_group_ids[] | string User security groups |
deletion_protection | bool Deletion Protection inhibits deletion of the cluster |
Monitoring
Field | Description |
---|---|
name | string Name of the monitoring system. |
description | string Description of the monitoring system. |
link | string Link to the monitoring system charts for the ClickHouse cluster. |
ClusterConfig
Field | Description |
---|---|
version | string Version of the ClickHouse server software. |
clickhouse | Clickhouse Configuration and resource allocation for ClickHouse hosts. |
zookeeper | Zookeeper Configuration and resource allocation for ZooKeeper hosts. |
backup_window_start | google.type.TimeOfDay Time to start the daily backup, in the UTC timezone. |
access | Access Access policy for external services. |
cloud_storage | CloudStorage |
sql_database_management | google.protobuf.BoolValue Whether database management through SQL commands is enabled. |
sql_user_management | google.protobuf.BoolValue Whether user management through SQL commands is enabled. |
embedded_keeper | google.protobuf.BoolValue Whether cluster should use embedded Keeper instead of Zookeeper. |
Clickhouse
Field | Description |
---|---|
config | config.ClickhouseConfigSet Configuration settings of a ClickHouse server. |
resources | Resources Resources allocated to ClickHouse hosts. |
Zookeeper
Field | Description |
---|---|
resources | Resources Resources allocated to ZooKeeper hosts. |
ClickhouseConfigSet
Field | Description |
---|---|
effective_config | ClickhouseConfig Required. Effective settings for a ClickHouse cluster (a combination of settings defined in user_config and default_config ). |
user_config | ClickhouseConfig User-defined settings for a ClickHouse cluster. |
default_config | ClickhouseConfig Default configuration for a ClickHouse cluster. |
ClickhouseConfig
Field | Description |
---|---|
log_level | enum LogLevel Logging level for the ClickHouse cluster. Possible values: TRACE, DEBUG, INFORMATION, WARNING, ERROR. |
merge_tree | MergeTree Settings for the MergeTree engine. See description in ClickHouse documentation |
compression[] | Compression Compression settings for the ClickHouse cluster. See in-depth description in ClickHouse documentation |
dictionaries[] | ExternalDictionary Configuration of external dictionaries to be used by the ClickHouse cluster. See in-depth description in ClickHouse documentation |
graphite_rollup[] | GraphiteRollup Settings for thinning Graphite data. See in-depth description in ClickHouse documentation |
kafka | Kafka |
kafka_topics[] | KafkaTopic |
rabbitmq | Rabbitmq |
max_connections | google.protobuf.Int64Value Maximum number of inbound connections. The minimum value is 10. |
max_concurrent_queries | google.protobuf.Int64Value Maximum number of simultaneously processed requests. The minimum value is 10. |
keep_alive_timeout | google.protobuf.Int64Value Number of milliseconds that ClickHouse waits for incoming requests before closing the connection. |
uncompressed_cache_size | google.protobuf.Int64Value Cache size (in bytes) for uncompressed data used by MergeTree tables. |
mark_cache_size | google.protobuf.Int64Value Approximate size (in bytes) of the cache of "marks" used by MergeTree tables. Value must be greater than 0. |
max_table_size_to_drop | google.protobuf.Int64Value Maximum size of the table that can be deleted using a DROP query. |
max_partition_size_to_drop | google.protobuf.Int64Value Maximum size of the partition that can be deleted using a DROP query. |
builtin_dictionaries_reload_interval | google.protobuf.Int64Value The setting is deprecated and has no effect. |
timezone | string The server's time zone to be used in DateTime fields conversions. Specified as an IANA identifier. |
geobase_enabled | google.protobuf.BoolValue Enable or disable geobase. |
geobase_uri | string Address of the archive with the user geobase in Object Storage. |
query_log_retention_size | google.protobuf.Int64Value The maximum size that query_log can grow to before old data will be removed. If set to 0, automatic removal of query_log data based on size is disabled. |
query_log_retention_time | google.protobuf.Int64Value The maximum time that query_log records will be retained before removal. If set to 0, automatic removal of query_log data based on time is disabled. |
query_thread_log_enabled | google.protobuf.BoolValue Whether query_thread_log system table is enabled. |
query_thread_log_retention_size | google.protobuf.Int64Value The maximum size that query_thread_log can grow to before old data will be removed. If set to 0, automatic removal of query_thread_log data based on size is disabled. |
query_thread_log_retention_time | google.protobuf.Int64Value The maximum time that query_thread_log records will be retained before removal. If set to 0, automatic removal of query_thread_log data based on time is disabled. |
part_log_retention_size | google.protobuf.Int64Value The maximum size that part_log can grow to before old data will be removed. If set to 0, automatic removal of part_log data based on size is disabled. |
part_log_retention_time | google.protobuf.Int64Value The maximum time that part_log records will be retained before removal. If set to 0, automatic removal of part_log data based on time is disabled. |
metric_log_enabled | google.protobuf.BoolValue Whether metric_log system table is enabled. |
metric_log_retention_size | google.protobuf.Int64Value The maximum size that metric_log can grow to before old data will be removed. If set to 0, automatic removal of metric_log data based on size is disabled. |
metric_log_retention_time | google.protobuf.Int64Value The maximum time that metric_log records will be retained before removal. If set to 0, automatic removal of metric_log data based on time is disabled. |
trace_log_enabled | google.protobuf.BoolValue Whether trace_log system table is enabled. |
trace_log_retention_size | google.protobuf.Int64Value The maximum size that trace_log can grow to before old data will be removed. If set to 0, automatic removal of trace_log data based on size is disabled. |
trace_log_retention_time | google.protobuf.Int64Value The maximum time that trace_log records will be retained before removal. If set to 0, automatic removal of trace_log data based on time is disabled. |
text_log_enabled | google.protobuf.BoolValue Whether text_log system table is enabled. |
text_log_retention_size | google.protobuf.Int64Value The maximum size that text_log can grow to before old data will be removed. If set to 0, automatic removal of text_log data based on size is disabled. |
text_log_retention_time | google.protobuf.Int64Value The maximum time that text_log records will be retained before removal. If set to 0, automatic removal of text_log data based on time is disabled. |
text_log_level | enum LogLevel Logging level for text_log system table. Possible values: TRACE, DEBUG, INFORMATION, WARNING, ERROR. |
opentelemetry_span_log_enabled | google.protobuf.BoolValue Enable or disable opentelemetry_span_log system table. Default value: false. |
opentelemetry_span_log_retention_size | google.protobuf.Int64Value The maximum size that opentelemetry_span_log can grow to before old data will be removed. If set to 0 (default), automatic removal of opentelemetry_span_log data based on size is disabled. The minimum value is 0. |
opentelemetry_span_log_retention_time | google.protobuf.Int64Value The maximum time that opentelemetry_span_log records will be retained before removal. If set to 0, automatic removal of opentelemetry_span_log data based on time is disabled. The minimum value is 0. |
query_views_log_enabled | google.protobuf.BoolValue Enable or disable query_views_log system table. Default value: false. |
query_views_log_retention_size | google.protobuf.Int64Value The maximum size that query_views_log can grow to before old data will be removed. If set to 0 (default), automatic removal of query_views_log data based on size is disabled. The minimum value is 0. |
query_views_log_retention_time | google.protobuf.Int64Value The maximum time that query_views_log records will be retained before removal. If set to 0, automatic removal of query_views_log data based on time is disabled. The minimum value is 0. |
asynchronous_metric_log_enabled | google.protobuf.BoolValue Enable or disable asynchronous_metric_log system table. Default value: false. |
asynchronous_metric_log_retention_size | google.protobuf.Int64Value The maximum size that asynchronous_metric_log can grow to before old data will be removed. If set to 0 (default), automatic removal of asynchronous_metric_log data based on size is disabled. The minimum value is 0. |
asynchronous_metric_log_retention_time | google.protobuf.Int64Value The maximum time that asynchronous_metric_log records will be retained before removal. If set to 0, automatic removal of asynchronous_metric_log data based on time is disabled. The minimum value is 0. |
session_log_enabled | google.protobuf.BoolValue Enable or disable session_log system table. Default value: false. |
session_log_retention_size | google.protobuf.Int64Value The maximum size that session_log can grow to before old data will be removed. If set to 0 (default), automatic removal of session_log data based on size is disabled. The minimum value is 0. |
session_log_retention_time | google.protobuf.Int64Value The maximum time that session_log records will be retained before removal. If set to 0, automatic removal of session_log data based on time is disabled. The minimum value is 0. |
zookeeper_log_enabled | google.protobuf.BoolValue Enable or disable zookeeper_log system table. Default value: false. |
zookeeper_log_retention_size | google.protobuf.Int64Value The maximum size that zookeeper_log can grow to before old data will be removed. If set to 0 (default), automatic removal of zookeeper_log data based on size is disabled. The minimum value is 0. |
zookeeper_log_retention_time | google.protobuf.Int64Value The maximum time that zookeeper_log records will be retained before removal. If set to 0, automatic removal of zookeeper_log data based on time is disabled. The minimum value is 0. |
asynchronous_insert_log_enabled | google.protobuf.BoolValue Enable or disable asynchronous_insert_log system table. Default value: false. Minimal required ClickHouse version: 22.10. |
asynchronous_insert_log_retention_size | google.protobuf.Int64Value The maximum size that asynchronous_insert_log can grow to before old data will be removed. If set to 0 (default), automatic removal of asynchronous_insert_log data based on size is disabled. The minimum value is 0. |
asynchronous_insert_log_retention_time | google.protobuf.Int64Value The maximum time that asynchronous_insert_log records will be retained before removal. If set to 0, automatic removal of asynchronous_insert_log data based on time is disabled. The minimum value is 0. |
background_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_merges_mutations_concurrency_ratio | google.protobuf.Int64Value Sets a ratio between the number of threads and the number of background merges and mutations that can be executed concurrently. For example, if the ratio equals to 2 and background_pool_size is set to 16 then ClickHouse can execute 32 background merges concurrently. This is possible, because background operations could be suspended and postponed. This is needed to give small merges more execution priority. You can only increase this ratio at runtime. To lower it you have to restart the server. The same as for background_pool_size setting background_merges_mutations_concurrency_ratio could be applied from the default profile for backward compatibility. Default: 2 See in-depth description in ClickHouse documentation |
background_schedule_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_fetches_pool_size | google.protobuf.Int64Value Sets the number of threads performing background fetches for tables with ReplicatedMergeTree engines. Default value: 8. More info see in ClickHouse documentation |
background_move_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_distributed_schedule_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_buffer_flush_schedule_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_message_broker_schedule_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_common_pool_size | google.protobuf.Int64Value The maximum number of threads that will be used for performing a variety of operations (mostly garbage collection) for *MergeTree-engine tables in a background. Default: 8 See in-depth description in ClickHouse documentation |
default_database | google.protobuf.StringValue The default database. To get a list of cluster databases, see Yandex Managed ClickHouse documentation. |
total_memory_profiler_step | google.protobuf.Int64Value Sets the memory size (in bytes) for a stack trace at every peak allocation step. Default value: 4194304. More info see in ClickHouse documentation |
total_memory_tracker_sample_probability | google.protobuf.DoubleValue |
MergeTree
Field | Description |
---|---|
replicated_deduplication_window | google.protobuf.Int64Value Number of blocks of hashes to keep in ZooKeeper. |
replicated_deduplication_window_seconds | google.protobuf.Int64Value Period of time to keep blocks of hashes for. |
parts_to_delay_insert | google.protobuf.Int64Value If table contains at least that many active parts in single partition, artificially slow down insert into table. |
parts_to_throw_insert | google.protobuf.Int64Value If more than this number active parts in single partition, throw 'Too many parts ...' exception. |
inactive_parts_to_delay_insert | google.protobuf.Int64Value |
inactive_parts_to_throw_insert | google.protobuf.Int64Value |
max_replicated_merges_in_queue | google.protobuf.Int64Value How many tasks of merging and mutating parts are allowed simultaneously in ReplicatedMergeTree queue. |
number_of_free_entries_in_pool_to_lower_max_size_of_merge | google.protobuf.Int64Value If there is less than specified number of free entries in background pool (or replicated queue), start to lower maximum size of merge to process. |
max_bytes_to_merge_at_min_space_in_pool | google.protobuf.Int64Value Maximum in total size of parts to merge, when there are minimum free threads in background pool (or entries in replication queue). |
max_bytes_to_merge_at_max_space_in_pool | google.protobuf.Int64Value |
min_bytes_for_wide_part | google.protobuf.Int64Value Minimum number of bytes in a data part that can be stored in Wide format. More info see in ClickHouse documentation |
min_rows_for_wide_part | google.protobuf.Int64Value Minimum number of rows in a data part that can be stored in Wide format. More info see in ClickHouse documentation |
ttl_only_drop_parts | google.protobuf.BoolValue Enables or disables complete dropping of data parts where all rows are expired in MergeTree tables. More info see in ClickHouse documentation |
allow_remote_fs_zero_copy_replication | google.protobuf.BoolValue |
merge_with_ttl_timeout | google.protobuf.Int64Value |
merge_with_recompression_ttl_timeout | google.protobuf.Int64Value |
max_parts_in_total | google.protobuf.Int64Value |
max_number_of_merges_with_ttl_in_pool | google.protobuf.Int64Value |
cleanup_delay_period | google.protobuf.Int64Value |
number_of_free_entries_in_pool_to_execute_mutation | google.protobuf.Int64Value |
max_avg_part_size_for_too_many_parts | google.protobuf.Int64Value The 'too many parts' check according to 'parts_to_delay_insert' and 'parts_to_throw_insert' will be active only if the average part size (in the relevant partition) is not larger than the specified threshold. If it is larger than the specified threshold, the INSERTs will be neither delayed or rejected. This allows to have hundreds of terabytes in a single table on a single server if the parts are successfully merged to larger parts. This does not affect the thresholds on inactive parts or total parts. Default: 1 GiB Min version: 22.10 See in-depth description in ClickHouse GitHub |
min_age_to_force_merge_seconds | google.protobuf.Int64Value Merge parts if every part in the range is older than the value of min_age_to_force_merge_seconds. Default: 0 - disabled Min_version: 22.10 See in-depth description in ClickHouse documentation |
min_age_to_force_merge_on_partition_only | google.protobuf.BoolValue Whether min_age_to_force_merge_seconds should be applied only on the entire partition and not on subset. Default: false Min_version: 22.11 See in-depth description in ClickHouse documentation |
merge_selecting_sleep_ms | google.protobuf.Int64Value Sleep time for merge selecting when no part is selected. A lower setting triggers selecting tasks in background_schedule_pool frequently, which results in a large number of requests to ClickHouse Keeper in large-scale clusters. Default: 5000 Min_version: 21.10 See in-depth description in ClickHouse documentation |
Kafka
Field | Description |
---|---|
security_protocol | enum SecurityProtocol |
sasl_mechanism | enum SaslMechanism |
sasl_username | string |
sasl_password | string |
enable_ssl_certificate_verification | google.protobuf.BoolValue |
max_poll_interval_ms | google.protobuf.Int64Value The minimum value is 0. |
session_timeout_ms | google.protobuf.Int64Value The minimum value is 0. |
KafkaTopic
Field | Description |
---|---|
name | string Required. |
settings | Kafka Required. |
Rabbitmq
Field | Description |
---|---|
username | string RabbitMQ |
password | string RabbitMQ |
vhost | string RabbitMQ |
Compression
Field | Description |
---|---|
method | enum Method Compression method to use for the specified combination of min_part_size and min_part_size_ratio . |
min_part_size | int64 Minimum size of a part of a table. The minimum value is 1. |
min_part_size_ratio | double Minimum ratio of a part relative to the size of all the data in the table. |
level | google.protobuf.Int64Value The minimum value is 0. |
ExternalDictionary
Field | Description |
---|---|
name | string Required. Name of the external dictionary. |
structure | Structure Required. Set of attributes for the external dictionary. For in-depth description, see ClickHouse documentation |
layout | Layout Required. Layout for storing the dictionary in memory. For in-depth description, see ClickHouse documentation |
lifetime | oneof: fixed_lifetime or lifetime_range Setting for the period of time between dictionary updates. For details, see ClickHouse documentation |
fixed_lifetime | int64 Fixed interval between dictionary updates. |
lifetime_range | Range Range of intervals between dictionary updates for ClickHouse to choose from. |
source | oneof: http_source , mysql_source , clickhouse_source , mongodb_source or postgresql_source Description of the source for the external dictionary. |
http_source | HttpSource HTTP source for the dictionary. |
mysql_source | MysqlSource MySQL source for the dictionary. |
clickhouse_source | ClickhouseSource ClickHouse source for the dictionary. |
mongodb_source | MongodbSource MongoDB source for the dictionary. |
postgresql_source | PostgresqlSource PostgreSQL source for the dictionary. |
HttpSource
Field | Description |
---|---|
url | string Required. URL of the source dictionary available over HTTP. |
format | string Required. The data format. Valid values are all formats supported by ClickHouse SQL dialect. |
MysqlSource
Field | Description |
---|---|
db | string Required. Name of the MySQL database to connect to. |
table | string Required. Name of the database table to use as a ClickHouse dictionary. |
port | int64 Default port to use when connecting to a replica of the dictionary source. Acceptable values are 0 to 65535, inclusive. |
user | string Name of the default user for replicas of the dictionary source. |
password | string Password of the default user for replicas of the dictionary source. |
replicas[] | Replica List of MySQL replicas of the database used as dictionary source. The number of elements must be greater than 0. |
where | string Selection criteria for the data in the specified MySQL table. |
invalidate_query | string Query for checking the dictionary status, to pull only updated data. For more details, see ClickHouse documentation on dictionaries |
Replica
Field | Description |
---|---|
host | string Required. MySQL host of the replica. The maximum string length in characters is 253. |
priority | int64 Required. The priority of the replica that ClickHouse takes into account when connecting. Replica with the highest priority should have this field set to the lowest number. Value must be greater than 0. |
port | int64 Port to use when connecting to the replica. If a port is not specified for a replica, ClickHouse uses the port specified for the source. Acceptable values are 0 to 65535, inclusive. |
user | string Name of the MySQL database user. |
password | string Password of the MySQL database user. |
ClickhouseSource
Field | Description |
---|---|
db | string Required. Name of the ClickHouse database. |
table | string Required. Name of the table in the specified database to be used as the dictionary source. |
host | string Required. ClickHouse host of the specified database. The maximum string length in characters is 253. |
port | int64 Port to use when connecting to the host. Acceptable values are 0 to 65535, inclusive. |
user | string Required. Name of the ClickHouse database user. |
password | string Password of the ClickHouse database user. |
where | string Selection criteria for the data in the specified ClickHouse table. |
MongodbSource
Field | Description |
---|---|
db | string Required. Name of the MongoDB database. |
collection | string Required. Name of the collection in the specified database to be used as the dictionary source. |
host | string Required. MongoDB host of the specified database. The maximum string length in characters is 253. |
port | int64 Port to use when connecting to the host. Acceptable values are 0 to 65535, inclusive. |
user | string Required. Name of the MongoDB database user. |
password | string Password of the MongoDB database user. |
options | string |
PostgresqlSource
Field | Description |
---|---|
db | string Required. Name of the PostrgreSQL database. |
table | string Required. Name of the table in the specified database to be used as the dictionary source. |
hosts[] | string Name of the PostrgreSQL host The number of elements must be greater than 0. |
port | int64 Port to use when connecting to the host. Acceptable values are 0 to 65535, inclusive. |
user | string Required. Name of the PostrgreSQL database user. |
password | string Password of the PostrgreSQL database user. |
invalidate_query | string Query for checking the dictionary status, to pull only updated data. For more details, see ClickHouse documentation on dictionaries |
ssl_mode | enum SslMode Mode of SSL TCP/IP connection to the PostgreSQL host. For more details, see PostgreSQL documentation
|
Structure
Field | Description |
---|---|
id | Id Single numeric key column for the dictionary. |
key | Key Composite key for the dictionary, containing of one or more key columns. For details, see ClickHouse documentation |
range_min | Attribute Field holding the beginning of the range for dictionaries with RANGE_HASHED layout. For details, see ClickHouse documentation |
range_max | Attribute Field holding the end of the range for dictionaries with RANGE_HASHED layout. For details, see ClickHouse documentation |
attributes[] | Attribute Description of the fields available for database queries. For details, see ClickHouse documentation |
Attribute
Field | Description |
---|---|
name | string Required. Name of the column. |
type | string Required. Type of the column. |
null_value | string Default value for an element without data (for example, an empty string). |
expression | string Expression, describing the attribute, if applicable. |
hierarchical | bool Indication of hierarchy support. Default value: false . |
injective | bool Indication of injective mapping "id -> attribute". Default value: false . |
Id
Field | Description |
---|---|
name | string Required. Name of the numeric key. |
Key
Field | Description |
---|---|
attributes[] | Attribute Attributes of a complex key. The number of elements must be greater than 0. |
Layout
Field | Description |
---|---|
type | enum Type Required. Layout type for an external dictionary.
|
size_in_cells | int64 Number of cells in the cache. Rounded up to a power of two. Applicable only for CACHE and COMPLEX_KEY_CACHE layout types. |
Range
Field | Description |
---|---|
min | int64 Minimum dictionary lifetime. |
max | int64 Maximum dictionary lifetime. |
GraphiteRollup
Field | Description |
---|---|
name | string Required. Name for the specified combination of settings for Graphite rollup. |
patterns[] | Pattern Pattern to use for the rollup. The number of elements must be greater than 0. |
Pattern
Field | Description |
---|---|
regexp | string Pattern for metric names. |
function | string Required. Name of the aggregating function to apply to data of the age specified in retention . |
retention[] | Retention Age of data to use for thinning. The number of elements must be greater than 0. |
Retention
Field | Description |
---|---|
age | int64 Minimum age of the data in seconds. The minimum value is 0. |
precision | int64 Precision of determining the age of the data, in seconds. Value must be greater than 0. |
MaintenanceWindow
Field | Description |
---|---|
policy | oneof: anytime or weekly_maintenance_window The maintenance policy in effect. |
anytime | AnytimeMaintenanceWindow Maintenance operation can be scheduled anytime. |
weekly_maintenance_window | WeeklyMaintenanceWindow Maintenance operation can be scheduled on a weekly basis. |
AnytimeMaintenanceWindow
Empty.
WeeklyMaintenanceWindow
Field | Description |
---|---|
day | enum WeekDay Day of the week (in DDD format). |
hour | int64 Hour of the day in UTC (in HH format). Acceptable values are 1 to 24, inclusive. |
MaintenanceOperation
Field | Description |
---|---|
info | string Information about this maintenance operation. The maximum string length in characters is 256. |
delayed_until | google.protobuf.Timestamp Time until which this maintenance operation is delayed. |
RescheduleMaintenance
Reschedules planned maintenance operation.
rpc RescheduleMaintenance (RescheduleMaintenanceRequest) returns (operation.Operation)
Metadata and response of Operation:
Operation.metadata:RescheduleMaintenanceMetadata
Operation.response:Cluster
RescheduleMaintenanceRequest
Field | Description |
---|---|
cluster_id | string Required. ID of the ClickHouse cluster to reschedule the maintenance operation for. The maximum string length in characters is 50. |
reschedule_type | enum RescheduleType Required. The type of reschedule request.
|
delayed_until | google.protobuf.Timestamp The time until which this maintenance operation should be delayed. The value should be ahead of the first time when the maintenance operation has been scheduled for no more than two weeks. The value can also point to the past moment of time if reschedule_type.IMMEDIATE reschedule type is chosen. |
Operation
Field | Description |
---|---|
id | string ID of the operation. |
description | string Description of the operation. 0-256 characters long. |
created_at | google.protobuf.Timestamp Creation timestamp. |
created_by | string ID of the user or service account who initiated the operation. |
modified_at | google.protobuf.Timestamp The time when the Operation resource was last modified. |
done | bool If the value is false , it means the operation is still in progress. If true , the operation is completed, and either error or response is available. |
metadata | google.protobuf.Any Service-specific metadata associated with the operation. It typically contains the ID of the target resource that the operation is performed on. Any method that returns a long-running operation should document the metadata type, if any. |
result | oneof: error or response The operation result. If done == false and there was no failure detected, neither error nor response is set. If done == false and there was a failure detected, error is set. If done == true , exactly one of error or response is set. |
error | google.rpc.Status The error result of the operation in case of failure or cancellation. |
response | google.protobuf.Any if operation finished successfully. |
RescheduleMaintenanceMetadata
Field | Description |
---|---|
cluster_id | string Required. ID of the ClickHouse cluster. |
delayed_until | google.protobuf.Timestamp Required. The time until which this maintenance operation is to be delayed. |
Cluster
Field | Description |
---|---|
id | string ID of the ClickHouse cluster. This ID is assigned by MDB at creation time. |
folder_id | string ID of the folder that the ClickHouse cluster belongs to. |
created_at | google.protobuf.Timestamp Creation timestamp in RFC3339 |
name | string Name of the ClickHouse cluster. The name is unique within the folder. 1-63 characters long. |
description | string Description of the ClickHouse cluster. 0-256 characters long. |
labels | map<string,string> Custom labels for the ClickHouse cluster as key:value pairs. Maximum 64 per resource. |
environment | enum Environment Deployment environment of the ClickHouse cluster.
|
monitoring[] | Monitoring Description of monitoring systems relevant to the ClickHouse cluster. |
config | ClusterConfig Configuration of the ClickHouse cluster. |
network_id | string ID of the network that the cluster belongs to. |
health | enum Health Aggregated cluster health.
|
status | enum Status Current state of the cluster.
|
service_account_id | string ID of the service account used for access to Object Storage. |
maintenance_window | MaintenanceWindow Maintenance window for the cluster. |
planned_operation | MaintenanceOperation Planned maintenance operation to be started for the cluster within the nearest maintenance_window . |
security_group_ids[] | string User security groups |
deletion_protection | bool Deletion Protection inhibits deletion of the cluster |
Monitoring
Field | Description |
---|---|
name | string Name of the monitoring system. |
description | string Description of the monitoring system. |
link | string Link to the monitoring system charts for the ClickHouse cluster. |
ClusterConfig
Field | Description |
---|---|
version | string Version of the ClickHouse server software. |
clickhouse | Clickhouse Configuration and resource allocation for ClickHouse hosts. |
zookeeper | Zookeeper Configuration and resource allocation for ZooKeeper hosts. |
backup_window_start | google.type.TimeOfDay Time to start the daily backup, in the UTC timezone. |
access | Access Access policy for external services. |
cloud_storage | CloudStorage |
sql_database_management | google.protobuf.BoolValue Whether database management through SQL commands is enabled. |
sql_user_management | google.protobuf.BoolValue Whether user management through SQL commands is enabled. |
embedded_keeper | google.protobuf.BoolValue Whether cluster should use embedded Keeper instead of Zookeeper. |
Clickhouse
Field | Description |
---|---|
config | config.ClickhouseConfigSet Configuration settings of a ClickHouse server. |
resources | Resources Resources allocated to ClickHouse hosts. |
Zookeeper
Field | Description |
---|---|
resources | Resources Resources allocated to ZooKeeper hosts. |
Access
Field | Description |
---|---|
data_lens | bool Allow to export data from the cluster to DataLens. |
web_sql | bool Allow SQL queries to the cluster databases from the management console. See SQL queries in the management console for more details. |
metrika | bool Allow to import data from Yandex Metrica and AppMetrica to the cluster. See AppMetrica documentation |
serverless | bool Allow access to cluster for Serverless. |
data_transfer | bool Allow access for DataTransfer |
yandex_query | bool Allow access for Query |
CloudStorage
Field | Description |
---|---|
enabled | bool Whether to use Object Storage for storing ClickHouse data. |
move_factor | google.protobuf.DoubleValue Acceptable values are 0 to 1, inclusive. |
data_cache_enabled | google.protobuf.BoolValue |
data_cache_max_size | google.protobuf.Int64Value |
prefer_not_to_merge | google.protobuf.BoolValue |
ClickhouseConfigSet
Field | Description |
---|---|
effective_config | ClickhouseConfig Required. Effective settings for a ClickHouse cluster (a combination of settings defined in user_config and default_config ). |
user_config | ClickhouseConfig User-defined settings for a ClickHouse cluster. |
default_config | ClickhouseConfig Default configuration for a ClickHouse cluster. |
ClickhouseConfig
Field | Description |
---|---|
log_level | enum LogLevel Logging level for the ClickHouse cluster. Possible values: TRACE, DEBUG, INFORMATION, WARNING, ERROR. |
merge_tree | MergeTree Settings for the MergeTree engine. See description in ClickHouse documentation |
compression[] | Compression Compression settings for the ClickHouse cluster. See in-depth description in ClickHouse documentation |
dictionaries[] | ExternalDictionary Configuration of external dictionaries to be used by the ClickHouse cluster. See in-depth description in ClickHouse documentation |
graphite_rollup[] | GraphiteRollup Settings for thinning Graphite data. See in-depth description in ClickHouse documentation |
kafka | Kafka |
kafka_topics[] | KafkaTopic |
rabbitmq | Rabbitmq |
max_connections | google.protobuf.Int64Value Maximum number of inbound connections. The minimum value is 10. |
max_concurrent_queries | google.protobuf.Int64Value Maximum number of simultaneously processed requests. The minimum value is 10. |
keep_alive_timeout | google.protobuf.Int64Value Number of milliseconds that ClickHouse waits for incoming requests before closing the connection. |
uncompressed_cache_size | google.protobuf.Int64Value Cache size (in bytes) for uncompressed data used by MergeTree tables. |
mark_cache_size | google.protobuf.Int64Value Approximate size (in bytes) of the cache of "marks" used by MergeTree tables. Value must be greater than 0. |
max_table_size_to_drop | google.protobuf.Int64Value Maximum size of the table that can be deleted using a DROP query. |
max_partition_size_to_drop | google.protobuf.Int64Value Maximum size of the partition that can be deleted using a DROP query. |
builtin_dictionaries_reload_interval | google.protobuf.Int64Value The setting is deprecated and has no effect. |
timezone | string The server's time zone to be used in DateTime fields conversions. Specified as an IANA identifier. |
geobase_enabled | google.protobuf.BoolValue Enable or disable geobase. |
geobase_uri | string Address of the archive with the user geobase in Object Storage. |
query_log_retention_size | google.protobuf.Int64Value The maximum size that query_log can grow to before old data will be removed. If set to 0, automatic removal of query_log data based on size is disabled. |
query_log_retention_time | google.protobuf.Int64Value The maximum time that query_log records will be retained before removal. If set to 0, automatic removal of query_log data based on time is disabled. |
query_thread_log_enabled | google.protobuf.BoolValue Whether query_thread_log system table is enabled. |
query_thread_log_retention_size | google.protobuf.Int64Value The maximum size that query_thread_log can grow to before old data will be removed. If set to 0, automatic removal of query_thread_log data based on size is disabled. |
query_thread_log_retention_time | google.protobuf.Int64Value The maximum time that query_thread_log records will be retained before removal. If set to 0, automatic removal of query_thread_log data based on time is disabled. |
part_log_retention_size | google.protobuf.Int64Value The maximum size that part_log can grow to before old data will be removed. If set to 0, automatic removal of part_log data based on size is disabled. |
part_log_retention_time | google.protobuf.Int64Value The maximum time that part_log records will be retained before removal. If set to 0, automatic removal of part_log data based on time is disabled. |
metric_log_enabled | google.protobuf.BoolValue Whether metric_log system table is enabled. |
metric_log_retention_size | google.protobuf.Int64Value The maximum size that metric_log can grow to before old data will be removed. If set to 0, automatic removal of metric_log data based on size is disabled. |
metric_log_retention_time | google.protobuf.Int64Value The maximum time that metric_log records will be retained before removal. If set to 0, automatic removal of metric_log data based on time is disabled. |
trace_log_enabled | google.protobuf.BoolValue Whether trace_log system table is enabled. |
trace_log_retention_size | google.protobuf.Int64Value The maximum size that trace_log can grow to before old data will be removed. If set to 0, automatic removal of trace_log data based on size is disabled. |
trace_log_retention_time | google.protobuf.Int64Value The maximum time that trace_log records will be retained before removal. If set to 0, automatic removal of trace_log data based on time is disabled. |
text_log_enabled | google.protobuf.BoolValue Whether text_log system table is enabled. |
text_log_retention_size | google.protobuf.Int64Value The maximum size that text_log can grow to before old data will be removed. If set to 0, automatic removal of text_log data based on size is disabled. |
text_log_retention_time | google.protobuf.Int64Value The maximum time that text_log records will be retained before removal. If set to 0, automatic removal of text_log data based on time is disabled. |
text_log_level | enum LogLevel Logging level for text_log system table. Possible values: TRACE, DEBUG, INFORMATION, WARNING, ERROR. |
opentelemetry_span_log_enabled | google.protobuf.BoolValue Enable or disable opentelemetry_span_log system table. Default value: false. |
opentelemetry_span_log_retention_size | google.protobuf.Int64Value The maximum size that opentelemetry_span_log can grow to before old data will be removed. If set to 0 (default), automatic removal of opentelemetry_span_log data based on size is disabled. The minimum value is 0. |
opentelemetry_span_log_retention_time | google.protobuf.Int64Value The maximum time that opentelemetry_span_log records will be retained before removal. If set to 0, automatic removal of opentelemetry_span_log data based on time is disabled. The minimum value is 0. |
query_views_log_enabled | google.protobuf.BoolValue Enable or disable query_views_log system table. Default value: false. |
query_views_log_retention_size | google.protobuf.Int64Value The maximum size that query_views_log can grow to before old data will be removed. If set to 0 (default), automatic removal of query_views_log data based on size is disabled. The minimum value is 0. |
query_views_log_retention_time | google.protobuf.Int64Value The maximum time that query_views_log records will be retained before removal. If set to 0, automatic removal of query_views_log data based on time is disabled. The minimum value is 0. |
asynchronous_metric_log_enabled | google.protobuf.BoolValue Enable or disable asynchronous_metric_log system table. Default value: false. |
asynchronous_metric_log_retention_size | google.protobuf.Int64Value The maximum size that asynchronous_metric_log can grow to before old data will be removed. If set to 0 (default), automatic removal of asynchronous_metric_log data based on size is disabled. The minimum value is 0. |
asynchronous_metric_log_retention_time | google.protobuf.Int64Value The maximum time that asynchronous_metric_log records will be retained before removal. If set to 0, automatic removal of asynchronous_metric_log data based on time is disabled. The minimum value is 0. |
session_log_enabled | google.protobuf.BoolValue Enable or disable session_log system table. Default value: false. |
session_log_retention_size | google.protobuf.Int64Value The maximum size that session_log can grow to before old data will be removed. If set to 0 (default), automatic removal of session_log data based on size is disabled. The minimum value is 0. |
session_log_retention_time | google.protobuf.Int64Value The maximum time that session_log records will be retained before removal. If set to 0, automatic removal of session_log data based on time is disabled. The minimum value is 0. |
zookeeper_log_enabled | google.protobuf.BoolValue Enable or disable zookeeper_log system table. Default value: false. |
zookeeper_log_retention_size | google.protobuf.Int64Value The maximum size that zookeeper_log can grow to before old data will be removed. If set to 0 (default), automatic removal of zookeeper_log data based on size is disabled. The minimum value is 0. |
zookeeper_log_retention_time | google.protobuf.Int64Value The maximum time that zookeeper_log records will be retained before removal. If set to 0, automatic removal of zookeeper_log data based on time is disabled. The minimum value is 0. |
asynchronous_insert_log_enabled | google.protobuf.BoolValue Enable or disable asynchronous_insert_log system table. Default value: false. Minimal required ClickHouse version: 22.10. |
asynchronous_insert_log_retention_size | google.protobuf.Int64Value The maximum size that asynchronous_insert_log can grow to before old data will be removed. If set to 0 (default), automatic removal of asynchronous_insert_log data based on size is disabled. The minimum value is 0. |
asynchronous_insert_log_retention_time | google.protobuf.Int64Value The maximum time that asynchronous_insert_log records will be retained before removal. If set to 0, automatic removal of asynchronous_insert_log data based on time is disabled. The minimum value is 0. |
background_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_merges_mutations_concurrency_ratio | google.protobuf.Int64Value Sets a ratio between the number of threads and the number of background merges and mutations that can be executed concurrently. For example, if the ratio equals to 2 and background_pool_size is set to 16 then ClickHouse can execute 32 background merges concurrently. This is possible, because background operations could be suspended and postponed. This is needed to give small merges more execution priority. You can only increase this ratio at runtime. To lower it you have to restart the server. The same as for background_pool_size setting background_merges_mutations_concurrency_ratio could be applied from the default profile for backward compatibility. Default: 2 See in-depth description in ClickHouse documentation |
background_schedule_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_fetches_pool_size | google.protobuf.Int64Value Sets the number of threads performing background fetches for tables with ReplicatedMergeTree engines. Default value: 8. More info see in ClickHouse documentation |
background_move_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_distributed_schedule_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_buffer_flush_schedule_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_message_broker_schedule_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_common_pool_size | google.protobuf.Int64Value The maximum number of threads that will be used for performing a variety of operations (mostly garbage collection) for *MergeTree-engine tables in a background. Default: 8 See in-depth description in ClickHouse documentation |
default_database | google.protobuf.StringValue The default database. To get a list of cluster databases, see Yandex Managed ClickHouse documentation. |
total_memory_profiler_step | google.protobuf.Int64Value Sets the memory size (in bytes) for a stack trace at every peak allocation step. Default value: 4194304. More info see in ClickHouse documentation |
total_memory_tracker_sample_probability | google.protobuf.DoubleValue |
MergeTree
Field | Description |
---|---|
replicated_deduplication_window | google.protobuf.Int64Value Number of blocks of hashes to keep in ZooKeeper. |
replicated_deduplication_window_seconds | google.protobuf.Int64Value Period of time to keep blocks of hashes for. |
parts_to_delay_insert | google.protobuf.Int64Value If table contains at least that many active parts in single partition, artificially slow down insert into table. |
parts_to_throw_insert | google.protobuf.Int64Value If more than this number active parts in single partition, throw 'Too many parts ...' exception. |
inactive_parts_to_delay_insert | google.protobuf.Int64Value |
inactive_parts_to_throw_insert | google.protobuf.Int64Value |
max_replicated_merges_in_queue | google.protobuf.Int64Value How many tasks of merging and mutating parts are allowed simultaneously in ReplicatedMergeTree queue. |
number_of_free_entries_in_pool_to_lower_max_size_of_merge | google.protobuf.Int64Value If there is less than specified number of free entries in background pool (or replicated queue), start to lower maximum size of merge to process. |
max_bytes_to_merge_at_min_space_in_pool | google.protobuf.Int64Value Maximum in total size of parts to merge, when there are minimum free threads in background pool (or entries in replication queue). |
max_bytes_to_merge_at_max_space_in_pool | google.protobuf.Int64Value |
min_bytes_for_wide_part | google.protobuf.Int64Value Minimum number of bytes in a data part that can be stored in Wide format. More info see in ClickHouse documentation |
min_rows_for_wide_part | google.protobuf.Int64Value Minimum number of rows in a data part that can be stored in Wide format. More info see in ClickHouse documentation |
ttl_only_drop_parts | google.protobuf.BoolValue Enables or disables complete dropping of data parts where all rows are expired in MergeTree tables. More info see in ClickHouse documentation |
allow_remote_fs_zero_copy_replication | google.protobuf.BoolValue |
merge_with_ttl_timeout | google.protobuf.Int64Value |
merge_with_recompression_ttl_timeout | google.protobuf.Int64Value |
max_parts_in_total | google.protobuf.Int64Value |
max_number_of_merges_with_ttl_in_pool | google.protobuf.Int64Value |
cleanup_delay_period | google.protobuf.Int64Value |
number_of_free_entries_in_pool_to_execute_mutation | google.protobuf.Int64Value |
max_avg_part_size_for_too_many_parts | google.protobuf.Int64Value The 'too many parts' check according to 'parts_to_delay_insert' and 'parts_to_throw_insert' will be active only if the average part size (in the relevant partition) is not larger than the specified threshold. If it is larger than the specified threshold, the INSERTs will be neither delayed or rejected. This allows to have hundreds of terabytes in a single table on a single server if the parts are successfully merged to larger parts. This does not affect the thresholds on inactive parts or total parts. Default: 1 GiB Min version: 22.10 See in-depth description in ClickHouse GitHub |
min_age_to_force_merge_seconds | google.protobuf.Int64Value Merge parts if every part in the range is older than the value of min_age_to_force_merge_seconds. Default: 0 - disabled Min_version: 22.10 See in-depth description in ClickHouse documentation |
min_age_to_force_merge_on_partition_only | google.protobuf.BoolValue Whether min_age_to_force_merge_seconds should be applied only on the entire partition and not on subset. Default: false Min_version: 22.11 See in-depth description in ClickHouse documentation |
merge_selecting_sleep_ms | google.protobuf.Int64Value Sleep time for merge selecting when no part is selected. A lower setting triggers selecting tasks in background_schedule_pool frequently, which results in a large number of requests to ClickHouse Keeper in large-scale clusters. Default: 5000 Min_version: 21.10 See in-depth description in ClickHouse documentation |
Kafka
Field | Description |
---|---|
security_protocol | enum SecurityProtocol |
sasl_mechanism | enum SaslMechanism |
sasl_username | string |
sasl_password | string |
enable_ssl_certificate_verification | google.protobuf.BoolValue |
max_poll_interval_ms | google.protobuf.Int64Value The minimum value is 0. |
session_timeout_ms | google.protobuf.Int64Value The minimum value is 0. |
KafkaTopic
Field | Description |
---|---|
name | string Required. |
settings | Kafka Required. |
Rabbitmq
Field | Description |
---|---|
username | string RabbitMQ |
password | string RabbitMQ |
vhost | string RabbitMQ |
Compression
Field | Description |
---|---|
method | enum Method Compression method to use for the specified combination of min_part_size and min_part_size_ratio . |
min_part_size | int64 Minimum size of a part of a table. The minimum value is 1. |
min_part_size_ratio | double Minimum ratio of a part relative to the size of all the data in the table. |
level | google.protobuf.Int64Value The minimum value is 0. |
ExternalDictionary
Field | Description |
---|---|
name | string Required. Name of the external dictionary. |
structure | Structure Required. Set of attributes for the external dictionary. For in-depth description, see ClickHouse documentation |
layout | Layout Required. Layout for storing the dictionary in memory. For in-depth description, see ClickHouse documentation |
lifetime | oneof: fixed_lifetime or lifetime_range Setting for the period of time between dictionary updates. For details, see ClickHouse documentation |
fixed_lifetime | int64 Fixed interval between dictionary updates. |
lifetime_range | Range Range of intervals between dictionary updates for ClickHouse to choose from. |
source | oneof: http_source , mysql_source , clickhouse_source , mongodb_source or postgresql_source Description of the source for the external dictionary. |
http_source | HttpSource HTTP source for the dictionary. |
mysql_source | MysqlSource MySQL source for the dictionary. |
clickhouse_source | ClickhouseSource ClickHouse source for the dictionary. |
mongodb_source | MongodbSource MongoDB source for the dictionary. |
postgresql_source | PostgresqlSource PostgreSQL source for the dictionary. |
HttpSource
Field | Description |
---|---|
url | string Required. URL of the source dictionary available over HTTP. |
format | string Required. The data format. Valid values are all formats supported by ClickHouse SQL dialect. |
MysqlSource
Field | Description |
---|---|
db | string Required. Name of the MySQL database to connect to. |
table | string Required. Name of the database table to use as a ClickHouse dictionary. |
port | int64 Default port to use when connecting to a replica of the dictionary source. Acceptable values are 0 to 65535, inclusive. |
user | string Name of the default user for replicas of the dictionary source. |
password | string Password of the default user for replicas of the dictionary source. |
replicas[] | Replica List of MySQL replicas of the database used as dictionary source. The number of elements must be greater than 0. |
where | string Selection criteria for the data in the specified MySQL table. |
invalidate_query | string Query for checking the dictionary status, to pull only updated data. For more details, see ClickHouse documentation on dictionaries |
Replica
Field | Description |
---|---|
host | string Required. MySQL host of the replica. The maximum string length in characters is 253. |
priority | int64 Required. The priority of the replica that ClickHouse takes into account when connecting. Replica with the highest priority should have this field set to the lowest number. Value must be greater than 0. |
port | int64 Port to use when connecting to the replica. If a port is not specified for a replica, ClickHouse uses the port specified for the source. Acceptable values are 0 to 65535, inclusive. |
user | string Name of the MySQL database user. |
password | string Password of the MySQL database user. |
ClickhouseSource
Field | Description |
---|---|
db | string Required. Name of the ClickHouse database. |
table | string Required. Name of the table in the specified database to be used as the dictionary source. |
host | string Required. ClickHouse host of the specified database. The maximum string length in characters is 253. |
port | int64 Port to use when connecting to the host. Acceptable values are 0 to 65535, inclusive. |
user | string Required. Name of the ClickHouse database user. |
password | string Password of the ClickHouse database user. |
where | string Selection criteria for the data in the specified ClickHouse table. |
MongodbSource
Field | Description |
---|---|
db | string Required. Name of the MongoDB database. |
collection | string Required. Name of the collection in the specified database to be used as the dictionary source. |
host | string Required. MongoDB host of the specified database. The maximum string length in characters is 253. |
port | int64 Port to use when connecting to the host. Acceptable values are 0 to 65535, inclusive. |
user | string Required. Name of the MongoDB database user. |
password | string Password of the MongoDB database user. |
options | string |
PostgresqlSource
Field | Description |
---|---|
db | string Required. Name of the PostrgreSQL database. |
table | string Required. Name of the table in the specified database to be used as the dictionary source. |
hosts[] | string Name of the PostrgreSQL host The number of elements must be greater than 0. |
port | int64 Port to use when connecting to the host. Acceptable values are 0 to 65535, inclusive. |
user | string Required. Name of the PostrgreSQL database user. |
password | string Password of the PostrgreSQL database user. |
invalidate_query | string Query for checking the dictionary status, to pull only updated data. For more details, see ClickHouse documentation on dictionaries |
ssl_mode | enum SslMode Mode of SSL TCP/IP connection to the PostgreSQL host. For more details, see PostgreSQL documentation
|
Structure
Field | Description |
---|---|
id | Id Single numeric key column for the dictionary. |
key | Key Composite key for the dictionary, containing of one or more key columns. For details, see ClickHouse documentation |
range_min | Attribute Field holding the beginning of the range for dictionaries with RANGE_HASHED layout. For details, see ClickHouse documentation |
range_max | Attribute Field holding the end of the range for dictionaries with RANGE_HASHED layout. For details, see ClickHouse documentation |
attributes[] | Attribute Description of the fields available for database queries. For details, see ClickHouse documentation |
Attribute
Field | Description |
---|---|
name | string Required. Name of the column. |
type | string Required. Type of the column. |
null_value | string Default value for an element without data (for example, an empty string). |
expression | string Expression, describing the attribute, if applicable. |
hierarchical | bool Indication of hierarchy support. Default value: false . |
injective | bool Indication of injective mapping "id -> attribute". Default value: false . |
Id
Field | Description |
---|---|
name | string Required. Name of the numeric key. |
Key
Field | Description |
---|---|
attributes[] | Attribute Attributes of a complex key. The number of elements must be greater than 0. |
Layout
Field | Description |
---|---|
type | enum Type Required. Layout type for an external dictionary.
|
size_in_cells | int64 Number of cells in the cache. Rounded up to a power of two. Applicable only for CACHE and COMPLEX_KEY_CACHE layout types. |
Range
Field | Description |
---|---|
min | int64 Minimum dictionary lifetime. |
max | int64 Maximum dictionary lifetime. |
GraphiteRollup
Field | Description |
---|---|
name | string Required. Name for the specified combination of settings for Graphite rollup. |
patterns[] | Pattern Pattern to use for the rollup. The number of elements must be greater than 0. |
Pattern
Field | Description |
---|---|
regexp | string Pattern for metric names. |
function | string Required. Name of the aggregating function to apply to data of the age specified in retention . |
retention[] | Retention Age of data to use for thinning. The number of elements must be greater than 0. |
Retention
Field | Description |
---|---|
age | int64 Minimum age of the data in seconds. The minimum value is 0. |
precision | int64 Precision of determining the age of the data, in seconds. Value must be greater than 0. |
Resources
Field | Description |
---|---|
resource_preset_id | string ID of the preset for computational resources available to a host (CPU, memory etc.). All available presets are listed in the documentation |
disk_size | int64 Volume of the storage available to a host, in bytes. |
disk_type_id | string Type of the storage environment for the host. Possible values:
|
MaintenanceWindow
Field | Description |
---|---|
policy | oneof: anytime or weekly_maintenance_window The maintenance policy in effect. |
anytime | AnytimeMaintenanceWindow Maintenance operation can be scheduled anytime. |
weekly_maintenance_window | WeeklyMaintenanceWindow Maintenance operation can be scheduled on a weekly basis. |
AnytimeMaintenanceWindow
Empty.
WeeklyMaintenanceWindow
Field | Description |
---|---|
day | enum WeekDay Day of the week (in DDD format). |
hour | int64 Hour of the day in UTC (in HH format). Acceptable values are 1 to 24, inclusive. |
MaintenanceOperation
Field | Description |
---|---|
info | string Information about this maintenance operation. The maximum string length in characters is 256. |
delayed_until | google.protobuf.Timestamp Time until which this maintenance operation is delayed. |
ListLogs
Retrieves logs for the specified ClickHouse cluster.
rpc ListLogs (ListClusterLogsRequest) returns (ListClusterLogsResponse)
ListClusterLogsRequest
Field | Description |
---|---|
cluster_id | string Required. ID of the ClickHouse cluster to request logs for. To get the ClickHouse cluster ID, use a ClusterService.List request. The maximum string length in characters is 50. |
column_filter[] | string Columns from logs table to request. If no columns are specified, entire log records are returned. |
service_type | enum ServiceType Required. Type of the service to request logs about.
|
from_time | google.protobuf.Timestamp Start timestamp for the logs request, in RFC3339 |
to_time | google.protobuf.Timestamp End timestamp for the logs request, in RFC3339 |
page_size | int64 The maximum number of results per page to return. If the number of available results is larger than page_size , the service returns a ListClusterLogsResponse.next_page_token that can be used to get the next page of results in subsequent list requests. The maximum value is 1000. |
page_token | string Page token. To get the next page of results, set page_token to the ListClusterLogsResponse.next_page_token returned by the previous list request. The maximum string length in characters is 100. |
ListClusterLogsResponse
Field | Description |
---|---|
logs[] | LogRecord Requested log records. |
next_page_token | string This token allows you to get the next page of results for list requests. If the number of results is larger than ListClusterLogsRequest.page_size, use the next_page_token as the value for the ListClusterLogsRequest.page_token query parameter in the next list request. Each subsequent list request will have its own next_page_token to continue paging through the results. This value is interchangeable with the StreamLogRecord.next_record_token from StreamLogs method. |
LogRecord
Field | Description |
---|---|
timestamp | google.protobuf.Timestamp Log record timestamp in RFC3339 |
message | map<string,string> Contents of the log record. |
StreamLogs
Same as ListLogs but using server-side streaming. Also allows for tail -f
semantics.
rpc StreamLogs (StreamClusterLogsRequest) returns (stream StreamLogRecord)
StreamClusterLogsRequest
Field | Description |
---|---|
cluster_id | string Required. Required. ID of the ClickHouse cluster. The maximum string length in characters is 50. |
column_filter[] | string Columns from logs table to get in the response. |
service_type | enum ServiceType Required.
|
from_time | google.protobuf.Timestamp Start timestamp for the logs request. |
to_time | google.protobuf.Timestamp End timestamp for the logs request. If this field is not set, all existing logs will be sent and then the new ones as they appear. In essence it has tail -f semantics. |
record_token | string Record token. Set record_token to the StreamLogRecord.next_record_token returned by a previous StreamLogs request to start streaming from next log record. The maximum string length in characters is 100. |
filter | string A filter expression that filters resources listed in the response. The expression must specify:
|
StreamLogRecord
Field | Description |
---|---|
record | LogRecord One of the requested log records. |
next_record_token | string This token allows you to continue streaming logs starting from the exact same record. To continue streaming, specify value of [next_record_token [as value for the StreamClusterLogsRequest.record_token parameter in the next StreamLogs request. This value is interchangeable with the ListClusterLogsResponse.next_page_token from ListLogs method. |
LogRecord
Field | Description |
---|---|
timestamp | google.protobuf.Timestamp Log record timestamp in RFC3339 |
message | map<string,string> Contents of the log record. |
ListOperations
Retrieves the list of Operation resources for the specified cluster.
rpc ListOperations (ListClusterOperationsRequest) returns (ListClusterOperationsResponse)
ListClusterOperationsRequest
Field | Description |
---|---|
cluster_id | string Required. ID of the ClickHouse Cluster resource to list operations for. The maximum string length in characters is 50. |
page_size | int64 The maximum number of results per page to return. If the number of available results is larger than page_size , the service returns a ListClusterOperationsResponse.next_page_token that can be used to get the next page of results in subsequent list requests. The maximum value is 1000. |
page_token | string Page token. To get the next page of results, set page_token to the ListClusterOperationsResponse.next_page_token returned by the previous list request. The maximum string length in characters is 100. |
ListClusterOperationsResponse
Field | Description |
---|---|
operations[] | operation.Operation List of Operation resources for the specified ClickHouse cluster. |
next_page_token | string This token allows you to get the next page of results for list requests. If the number of results is larger than ListClusterOperationsRequest.page_size, use the next_page_token as the value for the ListClusterOperationsRequest.page_token query parameter in the next list request. Each subsequent list request will have its own next_page_token to continue paging through the results. |
Operation
Field | Description |
---|---|
id | string ID of the operation. |
description | string Description of the operation. 0-256 characters long. |
created_at | google.protobuf.Timestamp Creation timestamp. |
created_by | string ID of the user or service account who initiated the operation. |
modified_at | google.protobuf.Timestamp The time when the Operation resource was last modified. |
done | bool If the value is false , it means the operation is still in progress. If true , the operation is completed, and either error or response is available. |
metadata | google.protobuf.Any Service-specific metadata associated with the operation. It typically contains the ID of the target resource that the operation is performed on. Any method that returns a long-running operation should document the metadata type, if any. |
result | oneof: error or response The operation result. If done == false and there was no failure detected, neither error nor response is set. If done == false and there was a failure detected, error is set. If done == true , exactly one of error or response is set. |
error | google.rpc.Status The error result of the operation in case of failure or cancellation. |
response | google.protobuf.Any The normal response of the operation in case of success. If the original method returns no data on success, such as Delete, the response is google.protobuf.Empty |
ListBackups
Retrieves the list of available backups for the specified ClickHouse cluster.
rpc ListBackups (ListClusterBackupsRequest) returns (ListClusterBackupsResponse)
ListClusterBackupsRequest
Field | Description |
---|---|
cluster_id | string Required. ID of the ClickHouse cluster. To get the ClickHouse cluster ID, use a ClusterService.List request. The maximum string length in characters is 50. |
page_size | int64 The maximum number of results per page to return. If the number of available results is larger than page_size , the service returns a ListClusterBackupsResponse.next_page_token that can be used to get the next page of results in subsequent list requests. The maximum value is 1000. |
page_token | string Page token. To get the next page of results, set page_token to the ListClusterBackupsResponse.next_page_token returned by the previous list request. The maximum string length in characters is 100. |
ListClusterBackupsResponse
Field | Description |
---|---|
backups[] | Backup List of ClickHouse Backup resources. |
next_page_token | string This token allows you to get the next page of results for list requests. If the number of results is larger than ListClusterBackupsRequest.page_size, use the next_page_token as the value for the ListClusterBackupsRequest.page_token query parameter in the next list request. Each subsequent list request will have its own next_page_token to continue paging through the results. |
Backup
Field | Description |
---|---|
id | string ID of the backup. |
folder_id | string ID of the folder that the backup belongs to. |
created_at | google.protobuf.Timestamp Creation timestamp in RFC3339 |
source_cluster_id | string ID of the ClickHouse cluster that the backup was created for. |
source_shard_names[] | string Names of the shards included in the backup. |
started_at | google.protobuf.Timestamp Time when the backup operation was started. |
size | int64 Size of backup in bytes. |
type | enum BackupType How this backup was created (manual/automatic/etc...).
|
ListHosts
Retrieves a list of hosts for the specified cluster.
rpc ListHosts (ListClusterHostsRequest) returns (ListClusterHostsResponse)
ListClusterHostsRequest
Field | Description |
---|---|
cluster_id | string Required. ID of the ClickHouse cluster. To get the ClickHouse cluster ID use a ClusterService.List request. The maximum string length in characters is 50. |
page_size | int64 The maximum number of results per page to return. If the number of available results is larger than page_size , the service returns a ListClusterHostsResponse.next_page_token that can be used to get the next page of results in subsequent list requests. The maximum value is 1000. |
page_token | string Page token. To get the next page of results, set page_token to the ListClusterHostsResponse.next_page_token returned by the previous list request. The maximum string length in characters is 100. |
ListClusterHostsResponse
Field | Description |
---|---|
hosts[] | Host Requested list of hosts for the cluster. |
next_page_token | string This token allows you to get the next page of results for list requests. If the number of results is larger than ListClusterHostsRequest.page_size, use the next_page_token as the value for the ListClusterHostsRequest.page_token query parameter in the next list request. Each subsequent list request will have its own next_page_token to continue paging through the results. |
Host
Field | Description |
---|---|
name | string Name of the ClickHouse host. The host name is assigned by MDB at creation time, and cannot be changed. 1-63 characters long. The name is unique across all MDB hosts that exist on the platform, as it defines the FQDN of the host. |
cluster_id | string ID of the ClickHouse host. The ID is assigned by MDB at creation time. |
zone_id | string ID of the availability zone where the ClickHouse host resides. |
type | enum Type Type of the host.
|
resources | Resources Resources allocated to the ClickHouse host. |
health | enum Health Status code of the aggregated health of the host.
|
services[] | Service Services provided by the host. |
subnet_id | string ID of the subnet that the host belongs to. |
assign_public_ip | bool Flag showing public IP assignment status to this host. |
shard_name | string |
Resources
Field | Description |
---|---|
resource_preset_id | string ID of the preset for computational resources available to a host (CPU, memory etc.). All available presets are listed in the documentation |
disk_size | int64 Volume of the storage available to a host, in bytes. |
disk_type_id | string Type of the storage environment for the host. Possible values:
|
Service
Field | Description |
---|---|
type | enum Type Type of the service provided by the host.
|
health | enum Health Status code of server availability.
|
AddHosts
Creates new hosts for a cluster.
rpc AddHosts (AddClusterHostsRequest) returns (operation.Operation)
Metadata and response of Operation:
Operation.metadata:AddClusterHostsMetadata
Operation.response:google.protobuf.Empty
AddClusterHostsRequest
Field | Description |
---|---|
cluster_id | string Required. ID of the ClickHouse cluster to add hosts to. To get the ClickHouse cluster ID, use a ClusterService.List request. The maximum string length in characters is 50. |
host_specs[] | HostSpec Configurations for ClickHouse hosts that should be added to the cluster. The number of elements must be greater than 0. |
copy_schema | google.protobuf.BoolValue Whether to copy schema to new ClickHouse hosts from replicas. |
HostSpec
Field | Description |
---|---|
zone_id | string ID of the availability zone where the host resides. To get a list of available zones, use the yandex.cloud.compute.v1.ZoneService.List request. The maximum string length in characters is 50. |
type | Host.Type Required. Type of the host to be deployed. |
subnet_id | string ID of the subnet that the host should belong to. This subnet should be a part of the network that the cluster belongs to. The ID of the network is set in the Cluster.network_id field. The maximum string length in characters is 50. |
assign_public_ip | bool Whether the host should get a public IP address on creation. After a host has been created, this setting cannot be changed. To remove an assigned public IP, or to assign a public IP to a host without one, recreate the host with assign_public_ip set as needed. Possible values:
|
shard_name | string Name of the shard that the host is assigned to. The maximum string length in characters is 63. Value must match the regular expression [a-zA-Z0-9_-]* . |
Operation
Field | Description |
---|---|
id | string ID of the operation. |
description | string Description of the operation. 0-256 characters long. |
created_at | google.protobuf.Timestamp Creation timestamp. |
created_by | string ID of the user or service account who initiated the operation. |
modified_at | google.protobuf.Timestamp The time when the Operation resource was last modified. |
done | bool If the value is false , it means the operation is still in progress. If true , the operation is completed, and either error or response is available. |
metadata | google.protobuf.Any Service-specific metadata associated with the operation. It typically contains the ID of the target resource that the operation is performed on. Any method that returns a long-running operation should document the metadata type, if any. |
result | oneof: error or response The operation result. If done == false and there was no failure detected, neither error nor response is set. If done == false and there was a failure detected, error is set. If done == true , exactly one of error or response is set. |
error | google.rpc.Status The error result of the operation in case of failure or cancellation. |
response | google.protobuf.Any if operation finished successfully. |
AddClusterHostsMetadata
Field | Description |
---|---|
cluster_id | string ID of the ClickHouse cluster to which the hosts are being added. |
host_names[] | string Names of hosts that are being added to the cluster. |
UpdateHosts
Updates the specified hosts.
rpc UpdateHosts (UpdateClusterHostsRequest) returns (operation.Operation)
Metadata and response of Operation:
Operation.metadata:UpdateClusterHostsMetadata
Operation.response:google.protobuf.Empty
UpdateClusterHostsRequest
Field | Description |
---|---|
cluster_id | string Required. ID of the ClickHouse cluster to update hosts in. To get the ClickHouse cluster ID, use a ClusterService.List request. The maximum string length in characters is 50. |
update_host_specs[] | UpdateHostSpec New configurations to apply to hosts. The number of elements must be greater than 0. |
UpdateHostSpec
Field | Description |
---|---|
host_name | string Required. Name of the host to update. To get the ClickHouse host name, use a ClusterService.ListHosts request. |
update_mask | google.protobuf.FieldMask Field mask that specifies which fields of the ClickHouse host should be updated. |
assign_public_ip | google.protobuf.BoolValue Whether the host should get a public IP address on creation. |
Operation
Field | Description |
---|---|
id | string ID of the operation. |
description | string Description of the operation. 0-256 characters long. |
created_at | google.protobuf.Timestamp Creation timestamp. |
created_by | string ID of the user or service account who initiated the operation. |
modified_at | google.protobuf.Timestamp The time when the Operation resource was last modified. |
done | bool If the value is false , it means the operation is still in progress. If true , the operation is completed, and either error or response is available. |
metadata | google.protobuf.Any Service-specific metadata associated with the operation. It typically contains the ID of the target resource that the operation is performed on. Any method that returns a long-running operation should document the metadata type, if any. |
result | oneof: error or response The operation result. If done == false and there was no failure detected, neither error nor response is set. If done == false and there was a failure detected, error is set. If done == true , exactly one of error or response is set. |
error | google.rpc.Status The error result of the operation in case of failure or cancellation. |
response | google.protobuf.Any if operation finished successfully. |
UpdateClusterHostsMetadata
Field | Description |
---|---|
cluster_id | string ID of the ClickHouse cluster to modify hosts in. |
host_names[] | string Names of hosts that are being modified. |
DeleteHosts
Deletes the specified hosts for a cluster.
rpc DeleteHosts (DeleteClusterHostsRequest) returns (operation.Operation)
Metadata and response of Operation:
Operation.metadata:DeleteClusterHostsMetadata
Operation.response:google.protobuf.Empty
DeleteClusterHostsRequest
Field | Description |
---|---|
cluster_id | string Required. ID of the ClickHouse cluster to remove hosts from. To get the ClickHouse cluster ID, use a ClusterService.List request. The maximum string length in characters is 50. |
host_names[] | string Names of hosts to delete. The number of elements must be greater than 0. The maximum string length in characters for each value is 253. |
Operation
Field | Description |
---|---|
id | string ID of the operation. |
description | string Description of the operation. 0-256 characters long. |
created_at | google.protobuf.Timestamp Creation timestamp. |
created_by | string ID of the user or service account who initiated the operation. |
modified_at | google.protobuf.Timestamp The time when the Operation resource was last modified. |
done | bool If the value is false , it means the operation is still in progress. If true , the operation is completed, and either error or response is available. |
metadata | google.protobuf.Any Service-specific metadata associated with the operation. It typically contains the ID of the target resource that the operation is performed on. Any method that returns a long-running operation should document the metadata type, if any. |
result | oneof: error or response The operation result. If done == false and there was no failure detected, neither error nor response is set. If done == false and there was a failure detected, error is set. If done == true , exactly one of error or response is set. |
error | google.rpc.Status The error result of the operation in case of failure or cancellation. |
response | google.protobuf.Any if operation finished successfully. |
DeleteClusterHostsMetadata
Field | Description |
---|---|
cluster_id | string ID of the ClickHouse cluster to remove hosts from. |
host_names[] | string Names of hosts that are being deleted. |
RestartHosts
rpc RestartHosts (RestartClusterHostsRequest) returns (operation.Operation)
Metadata and response of Operation:
Operation.metadata:RestartClusterHostsMetadata
Operation.response:google.protobuf.Empty
RestartClusterHostsRequest
Field | Description |
---|---|
cluster_id | string Required. Required. ID of the Clickhouse cluster. The maximum string length in characters is 50. |
host_names[] | string Required. Name of the hosts to restart. The number of elements must be greater than 0. The maximum string length in characters for each value is 353. |
Operation
Field | Description |
---|---|
id | string ID of the operation. |
description | string Description of the operation. 0-256 characters long. |
created_at | google.protobuf.Timestamp Creation timestamp. |
created_by | string ID of the user or service account who initiated the operation. |
modified_at | google.protobuf.Timestamp The time when the Operation resource was last modified. |
done | bool If the value is false , it means the operation is still in progress. If true , the operation is completed, and either error or response is available. |
metadata | google.protobuf.Any Service-specific metadata associated with the operation. It typically contains the ID of the target resource that the operation is performed on. Any method that returns a long-running operation should document the metadata type, if any. |
result | oneof: error or response The operation result. If done == false and there was no failure detected, neither error nor response is set. If done == false and there was a failure detected, error is set. If done == true , exactly one of error or response is set. |
error | google.rpc.Status The error result of the operation in case of failure or cancellation. |
response | google.protobuf.Any if operation finished successfully. |
RestartClusterHostsMetadata
Field | Description |
---|---|
cluster_id | string Required. ID of the ClickHouse cluster. |
host_names[] | string Required. The name of restarting host. |
GetShard
Returns the specified shard.
rpc GetShard (GetClusterShardRequest) returns (Shard)
GetClusterShardRequest
Field | Description |
---|---|
cluster_id | string Required. ID of the cluster that the shard belongs to. To get the cluster ID, use a ClusterService.List request. To get the name of the database, use a ClusterService.List request. The maximum string length in characters is 50. |
shard_name | string Required. Name of the shard to request information about. To get the name of a shard, use a ClusterService.ListShards request. The maximum string length in characters is 63. Value must match the regular expression [a-zA-Z0-9_-]* . |
Shard
Field | Description |
---|---|
name | string Name of the shard. |
cluster_id | string ID of the cluster that the shard belongs to. |
config | ShardConfig Configuration of the shard. |
ShardConfig
Field | Description |
---|---|
clickhouse | Clickhouse ClickHouse configuration for a shard. |
Clickhouse
Field | Description |
---|---|
config | config.ClickhouseConfigSet ClickHouse settings for a shard. |
resources | Resources Computational resources for a shard. |
weight | google.protobuf.Int64Value Relative weight of a shard considered when writing data to the cluster. For details, see ClickHouse documentation |
ClickhouseConfigSet
Field | Description |
---|---|
effective_config | ClickhouseConfig Required. Effective settings for a ClickHouse cluster (a combination of settings defined in user_config and default_config ). |
user_config | ClickhouseConfig User-defined settings for a ClickHouse cluster. |
default_config | ClickhouseConfig Default configuration for a ClickHouse cluster. |
ClickhouseConfig
Field | Description |
---|---|
log_level | enum LogLevel Logging level for the ClickHouse cluster. Possible values: TRACE, DEBUG, INFORMATION, WARNING, ERROR. |
merge_tree | MergeTree Settings for the MergeTree engine. See description in ClickHouse documentation |
compression[] | Compression Compression settings for the ClickHouse cluster. See in-depth description in ClickHouse documentation |
dictionaries[] | ExternalDictionary Configuration of external dictionaries to be used by the ClickHouse cluster. See in-depth description in ClickHouse documentation |
graphite_rollup[] | GraphiteRollup Settings for thinning Graphite data. See in-depth description in ClickHouse documentation |
kafka | Kafka |
kafka_topics[] | KafkaTopic |
rabbitmq | Rabbitmq |
max_connections | google.protobuf.Int64Value Maximum number of inbound connections. The minimum value is 10. |
max_concurrent_queries | google.protobuf.Int64Value Maximum number of simultaneously processed requests. The minimum value is 10. |
keep_alive_timeout | google.protobuf.Int64Value Number of milliseconds that ClickHouse waits for incoming requests before closing the connection. |
uncompressed_cache_size | google.protobuf.Int64Value Cache size (in bytes) for uncompressed data used by MergeTree tables. |
mark_cache_size | google.protobuf.Int64Value Approximate size (in bytes) of the cache of "marks" used by MergeTree tables. Value must be greater than 0. |
max_table_size_to_drop | google.protobuf.Int64Value Maximum size of the table that can be deleted using a DROP query. |
max_partition_size_to_drop | google.protobuf.Int64Value Maximum size of the partition that can be deleted using a DROP query. |
builtin_dictionaries_reload_interval | google.protobuf.Int64Value The setting is deprecated and has no effect. |
timezone | string The server's time zone to be used in DateTime fields conversions. Specified as an IANA identifier. |
geobase_enabled | google.protobuf.BoolValue Enable or disable geobase. |
geobase_uri | string Address of the archive with the user geobase in Object Storage. |
query_log_retention_size | google.protobuf.Int64Value The maximum size that query_log can grow to before old data will be removed. If set to 0, automatic removal of query_log data based on size is disabled. |
query_log_retention_time | google.protobuf.Int64Value The maximum time that query_log records will be retained before removal. If set to 0, automatic removal of query_log data based on time is disabled. |
query_thread_log_enabled | google.protobuf.BoolValue Whether query_thread_log system table is enabled. |
query_thread_log_retention_size | google.protobuf.Int64Value The maximum size that query_thread_log can grow to before old data will be removed. If set to 0, automatic removal of query_thread_log data based on size is disabled. |
query_thread_log_retention_time | google.protobuf.Int64Value The maximum time that query_thread_log records will be retained before removal. If set to 0, automatic removal of query_thread_log data based on time is disabled. |
part_log_retention_size | google.protobuf.Int64Value The maximum size that part_log can grow to before old data will be removed. If set to 0, automatic removal of part_log data based on size is disabled. |
part_log_retention_time | google.protobuf.Int64Value The maximum time that part_log records will be retained before removal. If set to 0, automatic removal of part_log data based on time is disabled. |
metric_log_enabled | google.protobuf.BoolValue Whether metric_log system table is enabled. |
metric_log_retention_size | google.protobuf.Int64Value The maximum size that metric_log can grow to before old data will be removed. If set to 0, automatic removal of metric_log data based on size is disabled. |
metric_log_retention_time | google.protobuf.Int64Value The maximum time that metric_log records will be retained before removal. If set to 0, automatic removal of metric_log data based on time is disabled. |
trace_log_enabled | google.protobuf.BoolValue Whether trace_log system table is enabled. |
trace_log_retention_size | google.protobuf.Int64Value The maximum size that trace_log can grow to before old data will be removed. If set to 0, automatic removal of trace_log data based on size is disabled. |
trace_log_retention_time | google.protobuf.Int64Value The maximum time that trace_log records will be retained before removal. If set to 0, automatic removal of trace_log data based on time is disabled. |
text_log_enabled | google.protobuf.BoolValue Whether text_log system table is enabled. |
text_log_retention_size | google.protobuf.Int64Value The maximum size that text_log can grow to before old data will be removed. If set to 0, automatic removal of text_log data based on size is disabled. |
text_log_retention_time | google.protobuf.Int64Value The maximum time that text_log records will be retained before removal. If set to 0, automatic removal of text_log data based on time is disabled. |
text_log_level | enum LogLevel Logging level for text_log system table. Possible values: TRACE, DEBUG, INFORMATION, WARNING, ERROR. |
opentelemetry_span_log_enabled | google.protobuf.BoolValue Enable or disable opentelemetry_span_log system table. Default value: false. |
opentelemetry_span_log_retention_size | google.protobuf.Int64Value The maximum size that opentelemetry_span_log can grow to before old data will be removed. If set to 0 (default), automatic removal of opentelemetry_span_log data based on size is disabled. The minimum value is 0. |
opentelemetry_span_log_retention_time | google.protobuf.Int64Value The maximum time that opentelemetry_span_log records will be retained before removal. If set to 0, automatic removal of opentelemetry_span_log data based on time is disabled. The minimum value is 0. |
query_views_log_enabled | google.protobuf.BoolValue Enable or disable query_views_log system table. Default value: false. |
query_views_log_retention_size | google.protobuf.Int64Value The maximum size that query_views_log can grow to before old data will be removed. If set to 0 (default), automatic removal of query_views_log data based on size is disabled. The minimum value is 0. |
query_views_log_retention_time | google.protobuf.Int64Value The maximum time that query_views_log records will be retained before removal. If set to 0, automatic removal of query_views_log data based on time is disabled. The minimum value is 0. |
asynchronous_metric_log_enabled | google.protobuf.BoolValue Enable or disable asynchronous_metric_log system table. Default value: false. |
asynchronous_metric_log_retention_size | google.protobuf.Int64Value The maximum size that asynchronous_metric_log can grow to before old data will be removed. If set to 0 (default), automatic removal of asynchronous_metric_log data based on size is disabled. The minimum value is 0. |
asynchronous_metric_log_retention_time | google.protobuf.Int64Value The maximum time that asynchronous_metric_log records will be retained before removal. If set to 0, automatic removal of asynchronous_metric_log data based on time is disabled. The minimum value is 0. |
session_log_enabled | google.protobuf.BoolValue Enable or disable session_log system table. Default value: false. |
session_log_retention_size | google.protobuf.Int64Value The maximum size that session_log can grow to before old data will be removed. If set to 0 (default), automatic removal of session_log data based on size is disabled. The minimum value is 0. |
session_log_retention_time | google.protobuf.Int64Value The maximum time that session_log records will be retained before removal. If set to 0, automatic removal of session_log data based on time is disabled. The minimum value is 0. |
zookeeper_log_enabled | google.protobuf.BoolValue Enable or disable zookeeper_log system table. Default value: false. |
zookeeper_log_retention_size | google.protobuf.Int64Value The maximum size that zookeeper_log can grow to before old data will be removed. If set to 0 (default), automatic removal of zookeeper_log data based on size is disabled. The minimum value is 0. |
zookeeper_log_retention_time | google.protobuf.Int64Value The maximum time that zookeeper_log records will be retained before removal. If set to 0, automatic removal of zookeeper_log data based on time is disabled. The minimum value is 0. |
asynchronous_insert_log_enabled | google.protobuf.BoolValue Enable or disable asynchronous_insert_log system table. Default value: false. Minimal required ClickHouse version: 22.10. |
asynchronous_insert_log_retention_size | google.protobuf.Int64Value The maximum size that asynchronous_insert_log can grow to before old data will be removed. If set to 0 (default), automatic removal of asynchronous_insert_log data based on size is disabled. The minimum value is 0. |
asynchronous_insert_log_retention_time | google.protobuf.Int64Value The maximum time that asynchronous_insert_log records will be retained before removal. If set to 0, automatic removal of asynchronous_insert_log data based on time is disabled. The minimum value is 0. |
background_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_merges_mutations_concurrency_ratio | google.protobuf.Int64Value Sets a ratio between the number of threads and the number of background merges and mutations that can be executed concurrently. For example, if the ratio equals to 2 and background_pool_size is set to 16 then ClickHouse can execute 32 background merges concurrently. This is possible, because background operations could be suspended and postponed. This is needed to give small merges more execution priority. You can only increase this ratio at runtime. To lower it you have to restart the server. The same as for background_pool_size setting background_merges_mutations_concurrency_ratio could be applied from the default profile for backward compatibility. Default: 2 See in-depth description in ClickHouse documentation |
background_schedule_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_fetches_pool_size | google.protobuf.Int64Value Sets the number of threads performing background fetches for tables with ReplicatedMergeTree engines. Default value: 8. More info see in ClickHouse documentation |
background_move_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_distributed_schedule_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_buffer_flush_schedule_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_message_broker_schedule_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_common_pool_size | google.protobuf.Int64Value The maximum number of threads that will be used for performing a variety of operations (mostly garbage collection) for *MergeTree-engine tables in a background. Default: 8 See in-depth description in ClickHouse documentation |
default_database | google.protobuf.StringValue The default database. To get a list of cluster databases, see Yandex Managed ClickHouse documentation. |
total_memory_profiler_step | google.protobuf.Int64Value Sets the memory size (in bytes) for a stack trace at every peak allocation step. Default value: 4194304. More info see in ClickHouse documentation |
total_memory_tracker_sample_probability | google.protobuf.DoubleValue |
MergeTree
Field | Description |
---|---|
replicated_deduplication_window | google.protobuf.Int64Value Number of blocks of hashes to keep in ZooKeeper. |
replicated_deduplication_window_seconds | google.protobuf.Int64Value Period of time to keep blocks of hashes for. |
parts_to_delay_insert | google.protobuf.Int64Value If table contains at least that many active parts in single partition, artificially slow down insert into table. |
parts_to_throw_insert | google.protobuf.Int64Value If more than this number active parts in single partition, throw 'Too many parts ...' exception. |
inactive_parts_to_delay_insert | google.protobuf.Int64Value |
inactive_parts_to_throw_insert | google.protobuf.Int64Value |
max_replicated_merges_in_queue | google.protobuf.Int64Value How many tasks of merging and mutating parts are allowed simultaneously in ReplicatedMergeTree queue. |
number_of_free_entries_in_pool_to_lower_max_size_of_merge | google.protobuf.Int64Value If there is less than specified number of free entries in background pool (or replicated queue), start to lower maximum size of merge to process. |
max_bytes_to_merge_at_min_space_in_pool | google.protobuf.Int64Value Maximum in total size of parts to merge, when there are minimum free threads in background pool (or entries in replication queue). |
max_bytes_to_merge_at_max_space_in_pool | google.protobuf.Int64Value |
min_bytes_for_wide_part | google.protobuf.Int64Value Minimum number of bytes in a data part that can be stored in Wide format. More info see in ClickHouse documentation |
min_rows_for_wide_part | google.protobuf.Int64Value Minimum number of rows in a data part that can be stored in Wide format. More info see in ClickHouse documentation |
ttl_only_drop_parts | google.protobuf.BoolValue Enables or disables complete dropping of data parts where all rows are expired in MergeTree tables. More info see in ClickHouse documentation |
allow_remote_fs_zero_copy_replication | google.protobuf.BoolValue |
merge_with_ttl_timeout | google.protobuf.Int64Value |
merge_with_recompression_ttl_timeout | google.protobuf.Int64Value |
max_parts_in_total | google.protobuf.Int64Value |
max_number_of_merges_with_ttl_in_pool | google.protobuf.Int64Value |
cleanup_delay_period | google.protobuf.Int64Value |
number_of_free_entries_in_pool_to_execute_mutation | google.protobuf.Int64Value |
max_avg_part_size_for_too_many_parts | google.protobuf.Int64Value The 'too many parts' check according to 'parts_to_delay_insert' and 'parts_to_throw_insert' will be active only if the average part size (in the relevant partition) is not larger than the specified threshold. If it is larger than the specified threshold, the INSERTs will be neither delayed or rejected. This allows to have hundreds of terabytes in a single table on a single server if the parts are successfully merged to larger parts. This does not affect the thresholds on inactive parts or total parts. Default: 1 GiB Min version: 22.10 See in-depth description in ClickHouse GitHub |
min_age_to_force_merge_seconds | google.protobuf.Int64Value Merge parts if every part in the range is older than the value of min_age_to_force_merge_seconds. Default: 0 - disabled Min_version: 22.10 See in-depth description in ClickHouse documentation |
min_age_to_force_merge_on_partition_only | google.protobuf.BoolValue Whether min_age_to_force_merge_seconds should be applied only on the entire partition and not on subset. Default: false Min_version: 22.11 See in-depth description in ClickHouse documentation |
merge_selecting_sleep_ms | google.protobuf.Int64Value Sleep time for merge selecting when no part is selected. A lower setting triggers selecting tasks in background_schedule_pool frequently, which results in a large number of requests to ClickHouse Keeper in large-scale clusters. Default: 5000 Min_version: 21.10 See in-depth description in ClickHouse documentation |
Kafka
Field | Description |
---|---|
security_protocol | enum SecurityProtocol |
sasl_mechanism | enum SaslMechanism |
sasl_username | string |
sasl_password | string |
enable_ssl_certificate_verification | google.protobuf.BoolValue |
max_poll_interval_ms | google.protobuf.Int64Value The minimum value is 0. |
session_timeout_ms | google.protobuf.Int64Value The minimum value is 0. |
KafkaTopic
Field | Description |
---|---|
name | string Required. |
settings | Kafka Required. |
Rabbitmq
Field | Description |
---|---|
username | string RabbitMQ |
password | string RabbitMQ |
vhost | string RabbitMQ |
Compression
Field | Description |
---|---|
method | enum Method Compression method to use for the specified combination of min_part_size and min_part_size_ratio . |
min_part_size | int64 Minimum size of a part of a table. The minimum value is 1. |
min_part_size_ratio | double Minimum ratio of a part relative to the size of all the data in the table. |
level | google.protobuf.Int64Value The minimum value is 0. |
ExternalDictionary
Field | Description |
---|---|
name | string Required. Name of the external dictionary. |
structure | Structure Required. Set of attributes for the external dictionary. For in-depth description, see ClickHouse documentation |
layout | Layout Required. Layout for storing the dictionary in memory. For in-depth description, see ClickHouse documentation |
lifetime | oneof: fixed_lifetime or lifetime_range Setting for the period of time between dictionary updates. For details, see ClickHouse documentation |
fixed_lifetime | int64 Fixed interval between dictionary updates. |
lifetime_range | Range Range of intervals between dictionary updates for ClickHouse to choose from. |
source | oneof: http_source , mysql_source , clickhouse_source , mongodb_source or postgresql_source Description of the source for the external dictionary. |
http_source | HttpSource HTTP source for the dictionary. |
mysql_source | MysqlSource MySQL source for the dictionary. |
clickhouse_source | ClickhouseSource ClickHouse source for the dictionary. |
mongodb_source | MongodbSource MongoDB source for the dictionary. |
postgresql_source | PostgresqlSource PostgreSQL source for the dictionary. |
HttpSource
Field | Description |
---|---|
url | string Required. URL of the source dictionary available over HTTP. |
format | string Required. The data format. Valid values are all formats supported by ClickHouse SQL dialect. |
MysqlSource
Field | Description |
---|---|
db | string Required. Name of the MySQL database to connect to. |
table | string Required. Name of the database table to use as a ClickHouse dictionary. |
port | int64 Default port to use when connecting to a replica of the dictionary source. Acceptable values are 0 to 65535, inclusive. |
user | string Name of the default user for replicas of the dictionary source. |
password | string Password of the default user for replicas of the dictionary source. |
replicas[] | Replica List of MySQL replicas of the database used as dictionary source. The number of elements must be greater than 0. |
where | string Selection criteria for the data in the specified MySQL table. |
invalidate_query | string Query for checking the dictionary status, to pull only updated data. For more details, see ClickHouse documentation on dictionaries |
Replica
Field | Description |
---|---|
host | string Required. MySQL host of the replica. The maximum string length in characters is 253. |
priority | int64 Required. The priority of the replica that ClickHouse takes into account when connecting. Replica with the highest priority should have this field set to the lowest number. Value must be greater than 0. |
port | int64 Port to use when connecting to the replica. If a port is not specified for a replica, ClickHouse uses the port specified for the source. Acceptable values are 0 to 65535, inclusive. |
user | string Name of the MySQL database user. |
password | string Password of the MySQL database user. |
ClickhouseSource
Field | Description |
---|---|
db | string Required. Name of the ClickHouse database. |
table | string Required. Name of the table in the specified database to be used as the dictionary source. |
host | string Required. ClickHouse host of the specified database. The maximum string length in characters is 253. |
port | int64 Port to use when connecting to the host. Acceptable values are 0 to 65535, inclusive. |
user | string Required. Name of the ClickHouse database user. |
password | string Password of the ClickHouse database user. |
where | string Selection criteria for the data in the specified ClickHouse table. |
MongodbSource
Field | Description |
---|---|
db | string Required. Name of the MongoDB database. |
collection | string Required. Name of the collection in the specified database to be used as the dictionary source. |
host | string Required. MongoDB host of the specified database. The maximum string length in characters is 253. |
port | int64 Port to use when connecting to the host. Acceptable values are 0 to 65535, inclusive. |
user | string Required. Name of the MongoDB database user. |
password | string Password of the MongoDB database user. |
options | string |
PostgresqlSource
Field | Description |
---|---|
db | string Required. Name of the PostrgreSQL database. |
table | string Required. Name of the table in the specified database to be used as the dictionary source. |
hosts[] | string Name of the PostrgreSQL host The number of elements must be greater than 0. |
port | int64 Port to use when connecting to the host. Acceptable values are 0 to 65535, inclusive. |
user | string Required. Name of the PostrgreSQL database user. |
password | string Password of the PostrgreSQL database user. |
invalidate_query | string Query for checking the dictionary status, to pull only updated data. For more details, see ClickHouse documentation on dictionaries |
ssl_mode | enum SslMode Mode of SSL TCP/IP connection to the PostgreSQL host. For more details, see PostgreSQL documentation
|
Structure
Field | Description |
---|---|
id | Id Single numeric key column for the dictionary. |
key | Key Composite key for the dictionary, containing of one or more key columns. For details, see ClickHouse documentation |
range_min | Attribute Field holding the beginning of the range for dictionaries with RANGE_HASHED layout. For details, see ClickHouse documentation |
range_max | Attribute Field holding the end of the range for dictionaries with RANGE_HASHED layout. For details, see ClickHouse documentation |
attributes[] | Attribute Description of the fields available for database queries. For details, see ClickHouse documentation |
Attribute
Field | Description |
---|---|
name | string Required. Name of the column. |
type | string Required. Type of the column. |
null_value | string Default value for an element without data (for example, an empty string). |
expression | string Expression, describing the attribute, if applicable. |
hierarchical | bool Indication of hierarchy support. Default value: false . |
injective | bool Indication of injective mapping "id -> attribute". Default value: false . |
Id
Field | Description |
---|---|
name | string Required. Name of the numeric key. |
Key
Field | Description |
---|---|
attributes[] | Attribute Attributes of a complex key. The number of elements must be greater than 0. |
Layout
Field | Description |
---|---|
type | enum Type Required. Layout type for an external dictionary.
|
size_in_cells | int64 Number of cells in the cache. Rounded up to a power of two. Applicable only for CACHE and COMPLEX_KEY_CACHE layout types. |
Range
Field | Description |
---|---|
min | int64 Minimum dictionary lifetime. |
max | int64 Maximum dictionary lifetime. |
GraphiteRollup
Field | Description |
---|---|
name | string Required. Name for the specified combination of settings for Graphite rollup. |
patterns[] | Pattern Pattern to use for the rollup. The number of elements must be greater than 0. |
Pattern
Field | Description |
---|---|
regexp | string Pattern for metric names. |
function | string Required. Name of the aggregating function to apply to data of the age specified in retention . |
retention[] | Retention Age of data to use for thinning. The number of elements must be greater than 0. |
Retention
Field | Description |
---|---|
age | int64 Minimum age of the data in seconds. The minimum value is 0. |
precision | int64 Precision of determining the age of the data, in seconds. Value must be greater than 0. |
Resources
Field | Description |
---|---|
resource_preset_id | string ID of the preset for computational resources available to a host (CPU, memory etc.). All available presets are listed in the documentation |
disk_size | int64 Volume of the storage available to a host, in bytes. |
disk_type_id | string Type of the storage environment for the host. Possible values:
|
ListShards
Retrieves a list of shards that belong to the specified cluster.
rpc ListShards (ListClusterShardsRequest) returns (ListClusterShardsResponse)
ListClusterShardsRequest
Field | Description |
---|---|
cluster_id | string Required. ID of the ClickHouse cluster to list shards in. To get the cluster ID, use a ClusterService.List request. The maximum string length in characters is 50. |
page_size | int64 The maximum number of results per page to return. If the number of available results is larger than page_size , the service returns a ListClusterShardsResponse.next_page_token that can be used to get the next page of results in subsequent list requests. Acceptable values are 0 to 1000, inclusive. |
page_token | string Page token. to get the next page of results, set page_token to the ListClusterShardsResponse.next_page_token returned by the previous list request. The maximum string length in characters is 100. |
ListClusterShardsResponse
Field | Description |
---|---|
shards[] | Shard List of ClickHouse shards. |
next_page_token | string This token allows you to get the next page of results for list requests. If the number of results is larger than ListClusterShardsRequest.page_size, use the next_page_token as the value for the ListClusterShardsRequest.page_token parameter in the next list request. Each subsequent list request will have its own next_page_token to continue paging through the results. |
Shard
Field | Description |
---|---|
name | string Name of the shard. |
cluster_id | string ID of the cluster that the shard belongs to. |
config | ShardConfig Configuration of the shard. |
ShardConfig
Field | Description |
---|---|
clickhouse | Clickhouse ClickHouse configuration for a shard. |
Clickhouse
Field | Description |
---|---|
config | config.ClickhouseConfigSet ClickHouse settings for a shard. |
resources | Resources Computational resources for a shard. |
weight | google.protobuf.Int64Value Relative weight of a shard considered when writing data to the cluster. For details, see ClickHouse documentation |
ClickhouseConfigSet
Field | Description |
---|---|
effective_config | ClickhouseConfig Required. Effective settings for a ClickHouse cluster (a combination of settings defined in user_config and default_config ). |
user_config | ClickhouseConfig User-defined settings for a ClickHouse cluster. |
default_config | ClickhouseConfig Default configuration for a ClickHouse cluster. |
ClickhouseConfig
Field | Description |
---|---|
log_level | enum LogLevel Logging level for the ClickHouse cluster. Possible values: TRACE, DEBUG, INFORMATION, WARNING, ERROR. |
merge_tree | MergeTree Settings for the MergeTree engine. See description in ClickHouse documentation |
compression[] | Compression Compression settings for the ClickHouse cluster. See in-depth description in ClickHouse documentation |
dictionaries[] | ExternalDictionary Configuration of external dictionaries to be used by the ClickHouse cluster. See in-depth description in ClickHouse documentation |
graphite_rollup[] | GraphiteRollup Settings for thinning Graphite data. See in-depth description in ClickHouse documentation |
kafka | Kafka |
kafka_topics[] | KafkaTopic |
rabbitmq | Rabbitmq |
max_connections | google.protobuf.Int64Value Maximum number of inbound connections. The minimum value is 10. |
max_concurrent_queries | google.protobuf.Int64Value Maximum number of simultaneously processed requests. The minimum value is 10. |
keep_alive_timeout | google.protobuf.Int64Value Number of milliseconds that ClickHouse waits for incoming requests before closing the connection. |
uncompressed_cache_size | google.protobuf.Int64Value Cache size (in bytes) for uncompressed data used by MergeTree tables. |
mark_cache_size | google.protobuf.Int64Value Approximate size (in bytes) of the cache of "marks" used by MergeTree tables. Value must be greater than 0. |
max_table_size_to_drop | google.protobuf.Int64Value Maximum size of the table that can be deleted using a DROP query. |
max_partition_size_to_drop | google.protobuf.Int64Value Maximum size of the partition that can be deleted using a DROP query. |
builtin_dictionaries_reload_interval | google.protobuf.Int64Value The setting is deprecated and has no effect. |
timezone | string The server's time zone to be used in DateTime fields conversions. Specified as an IANA identifier. |
geobase_enabled | google.protobuf.BoolValue Enable or disable geobase. |
geobase_uri | string Address of the archive with the user geobase in Object Storage. |
query_log_retention_size | google.protobuf.Int64Value The maximum size that query_log can grow to before old data will be removed. If set to 0, automatic removal of query_log data based on size is disabled. |
query_log_retention_time | google.protobuf.Int64Value The maximum time that query_log records will be retained before removal. If set to 0, automatic removal of query_log data based on time is disabled. |
query_thread_log_enabled | google.protobuf.BoolValue Whether query_thread_log system table is enabled. |
query_thread_log_retention_size | google.protobuf.Int64Value The maximum size that query_thread_log can grow to before old data will be removed. If set to 0, automatic removal of query_thread_log data based on size is disabled. |
query_thread_log_retention_time | google.protobuf.Int64Value The maximum time that query_thread_log records will be retained before removal. If set to 0, automatic removal of query_thread_log data based on time is disabled. |
part_log_retention_size | google.protobuf.Int64Value The maximum size that part_log can grow to before old data will be removed. If set to 0, automatic removal of part_log data based on size is disabled. |
part_log_retention_time | google.protobuf.Int64Value The maximum time that part_log records will be retained before removal. If set to 0, automatic removal of part_log data based on time is disabled. |
metric_log_enabled | google.protobuf.BoolValue Whether metric_log system table is enabled. |
metric_log_retention_size | google.protobuf.Int64Value The maximum size that metric_log can grow to before old data will be removed. If set to 0, automatic removal of metric_log data based on size is disabled. |
metric_log_retention_time | google.protobuf.Int64Value The maximum time that metric_log records will be retained before removal. If set to 0, automatic removal of metric_log data based on time is disabled. |
trace_log_enabled | google.protobuf.BoolValue Whether trace_log system table is enabled. |
trace_log_retention_size | google.protobuf.Int64Value The maximum size that trace_log can grow to before old data will be removed. If set to 0, automatic removal of trace_log data based on size is disabled. |
trace_log_retention_time | google.protobuf.Int64Value The maximum time that trace_log records will be retained before removal. If set to 0, automatic removal of trace_log data based on time is disabled. |
text_log_enabled | google.protobuf.BoolValue Whether text_log system table is enabled. |
text_log_retention_size | google.protobuf.Int64Value The maximum size that text_log can grow to before old data will be removed. If set to 0, automatic removal of text_log data based on size is disabled. |
text_log_retention_time | google.protobuf.Int64Value The maximum time that text_log records will be retained before removal. If set to 0, automatic removal of text_log data based on time is disabled. |
text_log_level | enum LogLevel Logging level for text_log system table. Possible values: TRACE, DEBUG, INFORMATION, WARNING, ERROR. |
opentelemetry_span_log_enabled | google.protobuf.BoolValue Enable or disable opentelemetry_span_log system table. Default value: false. |
opentelemetry_span_log_retention_size | google.protobuf.Int64Value The maximum size that opentelemetry_span_log can grow to before old data will be removed. If set to 0 (default), automatic removal of opentelemetry_span_log data based on size is disabled. The minimum value is 0. |
opentelemetry_span_log_retention_time | google.protobuf.Int64Value The maximum time that opentelemetry_span_log records will be retained before removal. If set to 0, automatic removal of opentelemetry_span_log data based on time is disabled. The minimum value is 0. |
query_views_log_enabled | google.protobuf.BoolValue Enable or disable query_views_log system table. Default value: false. |
query_views_log_retention_size | google.protobuf.Int64Value The maximum size that query_views_log can grow to before old data will be removed. If set to 0 (default), automatic removal of query_views_log data based on size is disabled. The minimum value is 0. |
query_views_log_retention_time | google.protobuf.Int64Value The maximum time that query_views_log records will be retained before removal. If set to 0, automatic removal of query_views_log data based on time is disabled. The minimum value is 0. |
asynchronous_metric_log_enabled | google.protobuf.BoolValue Enable or disable asynchronous_metric_log system table. Default value: false. |
asynchronous_metric_log_retention_size | google.protobuf.Int64Value The maximum size that asynchronous_metric_log can grow to before old data will be removed. If set to 0 (default), automatic removal of asynchronous_metric_log data based on size is disabled. The minimum value is 0. |
asynchronous_metric_log_retention_time | google.protobuf.Int64Value The maximum time that asynchronous_metric_log records will be retained before removal. If set to 0, automatic removal of asynchronous_metric_log data based on time is disabled. The minimum value is 0. |
session_log_enabled | google.protobuf.BoolValue Enable or disable session_log system table. Default value: false. |
session_log_retention_size | google.protobuf.Int64Value The maximum size that session_log can grow to before old data will be removed. If set to 0 (default), automatic removal of session_log data based on size is disabled. The minimum value is 0. |
session_log_retention_time | google.protobuf.Int64Value The maximum time that session_log records will be retained before removal. If set to 0, automatic removal of session_log data based on time is disabled. The minimum value is 0. |
zookeeper_log_enabled | google.protobuf.BoolValue Enable or disable zookeeper_log system table. Default value: false. |
zookeeper_log_retention_size | google.protobuf.Int64Value The maximum size that zookeeper_log can grow to before old data will be removed. If set to 0 (default), automatic removal of zookeeper_log data based on size is disabled. The minimum value is 0. |
zookeeper_log_retention_time | google.protobuf.Int64Value The maximum time that zookeeper_log records will be retained before removal. If set to 0, automatic removal of zookeeper_log data based on time is disabled. The minimum value is 0. |
asynchronous_insert_log_enabled | google.protobuf.BoolValue Enable or disable asynchronous_insert_log system table. Default value: false. Minimal required ClickHouse version: 22.10. |
asynchronous_insert_log_retention_size | google.protobuf.Int64Value The maximum size that asynchronous_insert_log can grow to before old data will be removed. If set to 0 (default), automatic removal of asynchronous_insert_log data based on size is disabled. The minimum value is 0. |
asynchronous_insert_log_retention_time | google.protobuf.Int64Value The maximum time that asynchronous_insert_log records will be retained before removal. If set to 0, automatic removal of asynchronous_insert_log data based on time is disabled. The minimum value is 0. |
background_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_merges_mutations_concurrency_ratio | google.protobuf.Int64Value Sets a ratio between the number of threads and the number of background merges and mutations that can be executed concurrently. For example, if the ratio equals to 2 and background_pool_size is set to 16 then ClickHouse can execute 32 background merges concurrently. This is possible, because background operations could be suspended and postponed. This is needed to give small merges more execution priority. You can only increase this ratio at runtime. To lower it you have to restart the server. The same as for background_pool_size setting background_merges_mutations_concurrency_ratio could be applied from the default profile for backward compatibility. Default: 2 See in-depth description in ClickHouse documentation |
background_schedule_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_fetches_pool_size | google.protobuf.Int64Value Sets the number of threads performing background fetches for tables with ReplicatedMergeTree engines. Default value: 8. More info see in ClickHouse documentation |
background_move_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_distributed_schedule_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_buffer_flush_schedule_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_message_broker_schedule_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_common_pool_size | google.protobuf.Int64Value The maximum number of threads that will be used for performing a variety of operations (mostly garbage collection) for *MergeTree-engine tables in a background. Default: 8 See in-depth description in ClickHouse documentation |
default_database | google.protobuf.StringValue The default database. To get a list of cluster databases, see Yandex Managed ClickHouse documentation. |
total_memory_profiler_step | google.protobuf.Int64Value Sets the memory size (in bytes) for a stack trace at every peak allocation step. Default value: 4194304. More info see in ClickHouse documentation |
total_memory_tracker_sample_probability | google.protobuf.DoubleValue |
MergeTree
Field | Description |
---|---|
replicated_deduplication_window | google.protobuf.Int64Value Number of blocks of hashes to keep in ZooKeeper. |
replicated_deduplication_window_seconds | google.protobuf.Int64Value Period of time to keep blocks of hashes for. |
parts_to_delay_insert | google.protobuf.Int64Value If table contains at least that many active parts in single partition, artificially slow down insert into table. |
parts_to_throw_insert | google.protobuf.Int64Value If more than this number active parts in single partition, throw 'Too many parts ...' exception. |
inactive_parts_to_delay_insert | google.protobuf.Int64Value |
inactive_parts_to_throw_insert | google.protobuf.Int64Value |
max_replicated_merges_in_queue | google.protobuf.Int64Value How many tasks of merging and mutating parts are allowed simultaneously in ReplicatedMergeTree queue. |
number_of_free_entries_in_pool_to_lower_max_size_of_merge | google.protobuf.Int64Value If there is less than specified number of free entries in background pool (or replicated queue), start to lower maximum size of merge to process. |
max_bytes_to_merge_at_min_space_in_pool | google.protobuf.Int64Value Maximum in total size of parts to merge, when there are minimum free threads in background pool (or entries in replication queue). |
max_bytes_to_merge_at_max_space_in_pool | google.protobuf.Int64Value |
min_bytes_for_wide_part | google.protobuf.Int64Value Minimum number of bytes in a data part that can be stored in Wide format. More info see in ClickHouse documentation |
min_rows_for_wide_part | google.protobuf.Int64Value Minimum number of rows in a data part that can be stored in Wide format. More info see in ClickHouse documentation |
ttl_only_drop_parts | google.protobuf.BoolValue Enables or disables complete dropping of data parts where all rows are expired in MergeTree tables. More info see in ClickHouse documentation |
allow_remote_fs_zero_copy_replication | google.protobuf.BoolValue |
merge_with_ttl_timeout | google.protobuf.Int64Value |
merge_with_recompression_ttl_timeout | google.protobuf.Int64Value |
max_parts_in_total | google.protobuf.Int64Value |
max_number_of_merges_with_ttl_in_pool | google.protobuf.Int64Value |
cleanup_delay_period | google.protobuf.Int64Value |
number_of_free_entries_in_pool_to_execute_mutation | google.protobuf.Int64Value |
max_avg_part_size_for_too_many_parts | google.protobuf.Int64Value The 'too many parts' check according to 'parts_to_delay_insert' and 'parts_to_throw_insert' will be active only if the average part size (in the relevant partition) is not larger than the specified threshold. If it is larger than the specified threshold, the INSERTs will be neither delayed or rejected. This allows to have hundreds of terabytes in a single table on a single server if the parts are successfully merged to larger parts. This does not affect the thresholds on inactive parts or total parts. Default: 1 GiB Min version: 22.10 See in-depth description in ClickHouse GitHub |
min_age_to_force_merge_seconds | google.protobuf.Int64Value Merge parts if every part in the range is older than the value of min_age_to_force_merge_seconds. Default: 0 - disabled Min_version: 22.10 See in-depth description in ClickHouse documentation |
min_age_to_force_merge_on_partition_only | google.protobuf.BoolValue Whether min_age_to_force_merge_seconds should be applied only on the entire partition and not on subset. Default: false Min_version: 22.11 See in-depth description in ClickHouse documentation |
merge_selecting_sleep_ms | google.protobuf.Int64Value Sleep time for merge selecting when no part is selected. A lower setting triggers selecting tasks in background_schedule_pool frequently, which results in a large number of requests to ClickHouse Keeper in large-scale clusters. Default: 5000 Min_version: 21.10 See in-depth description in ClickHouse documentation |
Kafka
Field | Description |
---|---|
security_protocol | enum SecurityProtocol |
sasl_mechanism | enum SaslMechanism |
sasl_username | string |
sasl_password | string |
enable_ssl_certificate_verification | google.protobuf.BoolValue |
max_poll_interval_ms | google.protobuf.Int64Value The minimum value is 0. |
session_timeout_ms | google.protobuf.Int64Value The minimum value is 0. |
KafkaTopic
Field | Description |
---|---|
name | string Required. |
settings | Kafka Required. |
Rabbitmq
Field | Description |
---|---|
username | string RabbitMQ |
password | string RabbitMQ |
vhost | string RabbitMQ |
Compression
Field | Description |
---|---|
method | enum Method Compression method to use for the specified combination of min_part_size and min_part_size_ratio . |
min_part_size | int64 Minimum size of a part of a table. The minimum value is 1. |
min_part_size_ratio | double Minimum ratio of a part relative to the size of all the data in the table. |
level | google.protobuf.Int64Value The minimum value is 0. |
ExternalDictionary
Field | Description |
---|---|
name | string Required. Name of the external dictionary. |
structure | Structure Required. Set of attributes for the external dictionary. For in-depth description, see ClickHouse documentation |
layout | Layout Required. Layout for storing the dictionary in memory. For in-depth description, see ClickHouse documentation |
lifetime | oneof: fixed_lifetime or lifetime_range Setting for the period of time between dictionary updates. For details, see ClickHouse documentation |
fixed_lifetime | int64 Fixed interval between dictionary updates. |
lifetime_range | Range Range of intervals between dictionary updates for ClickHouse to choose from. |
source | oneof: http_source , mysql_source , clickhouse_source , mongodb_source or postgresql_source Description of the source for the external dictionary. |
http_source | HttpSource HTTP source for the dictionary. |
mysql_source | MysqlSource MySQL source for the dictionary. |
clickhouse_source | ClickhouseSource ClickHouse source for the dictionary. |
mongodb_source | MongodbSource MongoDB source for the dictionary. |
postgresql_source | PostgresqlSource PostgreSQL source for the dictionary. |
HttpSource
Field | Description |
---|---|
url | string Required. URL of the source dictionary available over HTTP. |
format | string Required. The data format. Valid values are all formats supported by ClickHouse SQL dialect. |
MysqlSource
Field | Description |
---|---|
db | string Required. Name of the MySQL database to connect to. |
table | string Required. Name of the database table to use as a ClickHouse dictionary. |
port | int64 Default port to use when connecting to a replica of the dictionary source. Acceptable values are 0 to 65535, inclusive. |
user | string Name of the default user for replicas of the dictionary source. |
password | string Password of the default user for replicas of the dictionary source. |
replicas[] | Replica List of MySQL replicas of the database used as dictionary source. The number of elements must be greater than 0. |
where | string Selection criteria for the data in the specified MySQL table. |
invalidate_query | string Query for checking the dictionary status, to pull only updated data. For more details, see ClickHouse documentation on dictionaries |
Replica
Field | Description |
---|---|
host | string Required. MySQL host of the replica. The maximum string length in characters is 253. |
priority | int64 Required. The priority of the replica that ClickHouse takes into account when connecting. Replica with the highest priority should have this field set to the lowest number. Value must be greater than 0. |
port | int64 Port to use when connecting to the replica. If a port is not specified for a replica, ClickHouse uses the port specified for the source. Acceptable values are 0 to 65535, inclusive. |
user | string Name of the MySQL database user. |
password | string Password of the MySQL database user. |
ClickhouseSource
Field | Description |
---|---|
db | string Required. Name of the ClickHouse database. |
table | string Required. Name of the table in the specified database to be used as the dictionary source. |
host | string Required. ClickHouse host of the specified database. The maximum string length in characters is 253. |
port | int64 Port to use when connecting to the host. Acceptable values are 0 to 65535, inclusive. |
user | string Required. Name of the ClickHouse database user. |
password | string Password of the ClickHouse database user. |
where | string Selection criteria for the data in the specified ClickHouse table. |
MongodbSource
Field | Description |
---|---|
db | string Required. Name of the MongoDB database. |
collection | string Required. Name of the collection in the specified database to be used as the dictionary source. |
host | string Required. MongoDB host of the specified database. The maximum string length in characters is 253. |
port | int64 Port to use when connecting to the host. Acceptable values are 0 to 65535, inclusive. |
user | string Required. Name of the MongoDB database user. |
password | string Password of the MongoDB database user. |
options | string |
PostgresqlSource
Field | Description |
---|---|
db | string Required. Name of the PostrgreSQL database. |
table | string Required. Name of the table in the specified database to be used as the dictionary source. |
hosts[] | string Name of the PostrgreSQL host The number of elements must be greater than 0. |
port | int64 Port to use when connecting to the host. Acceptable values are 0 to 65535, inclusive. |
user | string Required. Name of the PostrgreSQL database user. |
password | string Password of the PostrgreSQL database user. |
invalidate_query | string Query for checking the dictionary status, to pull only updated data. For more details, see ClickHouse documentation on dictionaries |
ssl_mode | enum SslMode Mode of SSL TCP/IP connection to the PostgreSQL host. For more details, see PostgreSQL documentation
|
Structure
Field | Description |
---|---|
id | Id Single numeric key column for the dictionary. |
key | Key Composite key for the dictionary, containing of one or more key columns. For details, see ClickHouse documentation |
range_min | Attribute Field holding the beginning of the range for dictionaries with RANGE_HASHED layout. For details, see ClickHouse documentation |
range_max | Attribute Field holding the end of the range for dictionaries with RANGE_HASHED layout. For details, see ClickHouse documentation |
attributes[] | Attribute Description of the fields available for database queries. For details, see ClickHouse documentation |
Attribute
Field | Description |
---|---|
name | string Required. Name of the column. |
type | string Required. Type of the column. |
null_value | string Default value for an element without data (for example, an empty string). |
expression | string Expression, describing the attribute, if applicable. |
hierarchical | bool Indication of hierarchy support. Default value: false . |
injective | bool Indication of injective mapping "id -> attribute". Default value: false . |
Id
Field | Description |
---|---|
name | string Required. Name of the numeric key. |
Key
Field | Description |
---|---|
attributes[] | Attribute Attributes of a complex key. The number of elements must be greater than 0. |
Layout
Field | Description |
---|---|
type | enum Type Required. Layout type for an external dictionary.
|
size_in_cells | int64 Number of cells in the cache. Rounded up to a power of two. Applicable only for CACHE and COMPLEX_KEY_CACHE layout types. |
Range
Field | Description |
---|---|
min | int64 Minimum dictionary lifetime. |
max | int64 Maximum dictionary lifetime. |
GraphiteRollup
Field | Description |
---|---|
name | string Required. Name for the specified combination of settings for Graphite rollup. |
patterns[] | Pattern Pattern to use for the rollup. The number of elements must be greater than 0. |
Pattern
Field | Description |
---|---|
regexp | string Pattern for metric names. |
function | string Required. Name of the aggregating function to apply to data of the age specified in retention . |
retention[] | Retention Age of data to use for thinning. The number of elements must be greater than 0. |
Retention
Field | Description |
---|---|
age | int64 Minimum age of the data in seconds. The minimum value is 0. |
precision | int64 Precision of determining the age of the data, in seconds. Value must be greater than 0. |
Resources
Field | Description |
---|---|
resource_preset_id | string ID of the preset for computational resources available to a host (CPU, memory etc.). All available presets are listed in the documentation |
disk_size | int64 Volume of the storage available to a host, in bytes. |
disk_type_id | string Type of the storage environment for the host. Possible values:
|
AddShard
Creates a new shard in the specified cluster.
rpc AddShard (AddClusterShardRequest) returns (operation.Operation)
Metadata and response of Operation:
Operation.metadata:AddClusterShardMetadata
Operation.response:Shard
AddClusterShardRequest
Field | Description |
---|---|
cluster_id | string Required. ID of the ClickHouse cluster to add a shard to. To get the ClickHouse cluster ID, use a ClusterService.List request. The maximum string length in characters is 50. |
shard_name | string Required. Name for the new shard. The maximum string length in characters is 63. Value must match the regular expression [a-zA-Z0-9_-]* . |
config_spec | ShardConfigSpec Configuration of the new shard. |
host_specs[] | HostSpec Configurations for ClickHouse hosts that should be created with the shard. The number of elements must be greater than 0. |
copy_schema | google.protobuf.BoolValue Whether to copy schema to hosts of the shard to be created. The schema is copied from hosts of an existing shard. |
ShardConfigSpec
Field | Description |
---|---|
clickhouse | Clickhouse ClickHouse configuration for a shard. |
Clickhouse
Field | Description |
---|---|
config | config.ClickhouseConfig ClickHouse settings for the shard. |
resources | Resources Computational resources for the shard. |
weight | google.protobuf.Int64Value Relative weight of the shard considered when writing data to the cluster. For details, see ClickHouse documentation |
ClickhouseConfig
Field | Description |
---|---|
log_level | enum LogLevel Logging level for the ClickHouse cluster. Possible values: TRACE, DEBUG, INFORMATION, WARNING, ERROR. |
merge_tree | MergeTree Settings for the MergeTree engine. See description in ClickHouse documentation |
compression[] | Compression Compression settings for the ClickHouse cluster. See in-depth description in ClickHouse documentation |
dictionaries[] | ExternalDictionary Configuration of external dictionaries to be used by the ClickHouse cluster. See in-depth description in ClickHouse documentation |
graphite_rollup[] | GraphiteRollup Settings for thinning Graphite data. See in-depth description in ClickHouse documentation |
kafka | Kafka |
kafka_topics[] | KafkaTopic |
rabbitmq | Rabbitmq |
max_connections | google.protobuf.Int64Value Maximum number of inbound connections. The minimum value is 10. |
max_concurrent_queries | google.protobuf.Int64Value Maximum number of simultaneously processed requests. The minimum value is 10. |
keep_alive_timeout | google.protobuf.Int64Value Number of milliseconds that ClickHouse waits for incoming requests before closing the connection. |
uncompressed_cache_size | google.protobuf.Int64Value Cache size (in bytes) for uncompressed data used by MergeTree tables. |
mark_cache_size | google.protobuf.Int64Value Approximate size (in bytes) of the cache of "marks" used by MergeTree tables. Value must be greater than 0. |
max_table_size_to_drop | google.protobuf.Int64Value Maximum size of the table that can be deleted using a DROP query. |
max_partition_size_to_drop | google.protobuf.Int64Value Maximum size of the partition that can be deleted using a DROP query. |
builtin_dictionaries_reload_interval | google.protobuf.Int64Value The setting is deprecated and has no effect. |
timezone | string The server's time zone to be used in DateTime fields conversions. Specified as an IANA identifier. |
geobase_enabled | google.protobuf.BoolValue Enable or disable geobase. |
geobase_uri | string Address of the archive with the user geobase in Object Storage. |
query_log_retention_size | google.protobuf.Int64Value The maximum size that query_log can grow to before old data will be removed. If set to 0, automatic removal of query_log data based on size is disabled. |
query_log_retention_time | google.protobuf.Int64Value The maximum time that query_log records will be retained before removal. If set to 0, automatic removal of query_log data based on time is disabled. |
query_thread_log_enabled | google.protobuf.BoolValue Whether query_thread_log system table is enabled. |
query_thread_log_retention_size | google.protobuf.Int64Value The maximum size that query_thread_log can grow to before old data will be removed. If set to 0, automatic removal of query_thread_log data based on size is disabled. |
query_thread_log_retention_time | google.protobuf.Int64Value The maximum time that query_thread_log records will be retained before removal. If set to 0, automatic removal of query_thread_log data based on time is disabled. |
part_log_retention_size | google.protobuf.Int64Value The maximum size that part_log can grow to before old data will be removed. If set to 0, automatic removal of part_log data based on size is disabled. |
part_log_retention_time | google.protobuf.Int64Value The maximum time that part_log records will be retained before removal. If set to 0, automatic removal of part_log data based on time is disabled. |
metric_log_enabled | google.protobuf.BoolValue Whether metric_log system table is enabled. |
metric_log_retention_size | google.protobuf.Int64Value The maximum size that metric_log can grow to before old data will be removed. If set to 0, automatic removal of metric_log data based on size is disabled. |
metric_log_retention_time | google.protobuf.Int64Value The maximum time that metric_log records will be retained before removal. If set to 0, automatic removal of metric_log data based on time is disabled. |
trace_log_enabled | google.protobuf.BoolValue Whether trace_log system table is enabled. |
trace_log_retention_size | google.protobuf.Int64Value The maximum size that trace_log can grow to before old data will be removed. If set to 0, automatic removal of trace_log data based on size is disabled. |
trace_log_retention_time | google.protobuf.Int64Value The maximum time that trace_log records will be retained before removal. If set to 0, automatic removal of trace_log data based on time is disabled. |
text_log_enabled | google.protobuf.BoolValue Whether text_log system table is enabled. |
text_log_retention_size | google.protobuf.Int64Value The maximum size that text_log can grow to before old data will be removed. If set to 0, automatic removal of text_log data based on size is disabled. |
text_log_retention_time | google.protobuf.Int64Value The maximum time that text_log records will be retained before removal. If set to 0, automatic removal of text_log data based on time is disabled. |
text_log_level | enum LogLevel Logging level for text_log system table. Possible values: TRACE, DEBUG, INFORMATION, WARNING, ERROR. |
opentelemetry_span_log_enabled | google.protobuf.BoolValue Enable or disable opentelemetry_span_log system table. Default value: false. |
opentelemetry_span_log_retention_size | google.protobuf.Int64Value The maximum size that opentelemetry_span_log can grow to before old data will be removed. If set to 0 (default), automatic removal of opentelemetry_span_log data based on size is disabled. The minimum value is 0. |
opentelemetry_span_log_retention_time | google.protobuf.Int64Value The maximum time that opentelemetry_span_log records will be retained before removal. If set to 0, automatic removal of opentelemetry_span_log data based on time is disabled. The minimum value is 0. |
query_views_log_enabled | google.protobuf.BoolValue Enable or disable query_views_log system table. Default value: false. |
query_views_log_retention_size | google.protobuf.Int64Value The maximum size that query_views_log can grow to before old data will be removed. If set to 0 (default), automatic removal of query_views_log data based on size is disabled. The minimum value is 0. |
query_views_log_retention_time | google.protobuf.Int64Value The maximum time that query_views_log records will be retained before removal. If set to 0, automatic removal of query_views_log data based on time is disabled. The minimum value is 0. |
asynchronous_metric_log_enabled | google.protobuf.BoolValue Enable or disable asynchronous_metric_log system table. Default value: false. |
asynchronous_metric_log_retention_size | google.protobuf.Int64Value The maximum size that asynchronous_metric_log can grow to before old data will be removed. If set to 0 (default), automatic removal of asynchronous_metric_log data based on size is disabled. The minimum value is 0. |
asynchronous_metric_log_retention_time | google.protobuf.Int64Value The maximum time that asynchronous_metric_log records will be retained before removal. If set to 0, automatic removal of asynchronous_metric_log data based on time is disabled. The minimum value is 0. |
session_log_enabled | google.protobuf.BoolValue Enable or disable session_log system table. Default value: false. |
session_log_retention_size | google.protobuf.Int64Value The maximum size that session_log can grow to before old data will be removed. If set to 0 (default), automatic removal of session_log data based on size is disabled. The minimum value is 0. |
session_log_retention_time | google.protobuf.Int64Value The maximum time that session_log records will be retained before removal. If set to 0, automatic removal of session_log data based on time is disabled. The minimum value is 0. |
zookeeper_log_enabled | google.protobuf.BoolValue Enable or disable zookeeper_log system table. Default value: false. |
zookeeper_log_retention_size | google.protobuf.Int64Value The maximum size that zookeeper_log can grow to before old data will be removed. If set to 0 (default), automatic removal of zookeeper_log data based on size is disabled. The minimum value is 0. |
zookeeper_log_retention_time | google.protobuf.Int64Value The maximum time that zookeeper_log records will be retained before removal. If set to 0, automatic removal of zookeeper_log data based on time is disabled. The minimum value is 0. |
asynchronous_insert_log_enabled | google.protobuf.BoolValue Enable or disable asynchronous_insert_log system table. Default value: false. Minimal required ClickHouse version: 22.10. |
asynchronous_insert_log_retention_size | google.protobuf.Int64Value The maximum size that asynchronous_insert_log can grow to before old data will be removed. If set to 0 (default), automatic removal of asynchronous_insert_log data based on size is disabled. The minimum value is 0. |
asynchronous_insert_log_retention_time | google.protobuf.Int64Value The maximum time that asynchronous_insert_log records will be retained before removal. If set to 0, automatic removal of asynchronous_insert_log data based on time is disabled. The minimum value is 0. |
background_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_merges_mutations_concurrency_ratio | google.protobuf.Int64Value Sets a ratio between the number of threads and the number of background merges and mutations that can be executed concurrently. For example, if the ratio equals to 2 and background_pool_size is set to 16 then ClickHouse can execute 32 background merges concurrently. This is possible, because background operations could be suspended and postponed. This is needed to give small merges more execution priority. You can only increase this ratio at runtime. To lower it you have to restart the server. The same as for background_pool_size setting background_merges_mutations_concurrency_ratio could be applied from the default profile for backward compatibility. Default: 2 See in-depth description in ClickHouse documentation |
background_schedule_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_fetches_pool_size | google.protobuf.Int64Value Sets the number of threads performing background fetches for tables with ReplicatedMergeTree engines. Default value: 8. More info see in ClickHouse documentation |
background_move_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_distributed_schedule_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_buffer_flush_schedule_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_message_broker_schedule_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_common_pool_size | google.protobuf.Int64Value The maximum number of threads that will be used for performing a variety of operations (mostly garbage collection) for *MergeTree-engine tables in a background. Default: 8 See in-depth description in ClickHouse documentation |
default_database | google.protobuf.StringValue The default database. To get a list of cluster databases, see Yandex Managed ClickHouse documentation. |
total_memory_profiler_step | google.protobuf.Int64Value Sets the memory size (in bytes) for a stack trace at every peak allocation step. Default value: 4194304. More info see in ClickHouse documentation |
total_memory_tracker_sample_probability | google.protobuf.DoubleValue |
MergeTree
Field | Description |
---|---|
replicated_deduplication_window | google.protobuf.Int64Value Number of blocks of hashes to keep in ZooKeeper. |
replicated_deduplication_window_seconds | google.protobuf.Int64Value Period of time to keep blocks of hashes for. |
parts_to_delay_insert | google.protobuf.Int64Value If table contains at least that many active parts in single partition, artificially slow down insert into table. |
parts_to_throw_insert | google.protobuf.Int64Value If more than this number active parts in single partition, throw 'Too many parts ...' exception. |
inactive_parts_to_delay_insert | google.protobuf.Int64Value |
inactive_parts_to_throw_insert | google.protobuf.Int64Value |
max_replicated_merges_in_queue | google.protobuf.Int64Value How many tasks of merging and mutating parts are allowed simultaneously in ReplicatedMergeTree queue. |
number_of_free_entries_in_pool_to_lower_max_size_of_merge | google.protobuf.Int64Value If there is less than specified number of free entries in background pool (or replicated queue), start to lower maximum size of merge to process. |
max_bytes_to_merge_at_min_space_in_pool | google.protobuf.Int64Value Maximum in total size of parts to merge, when there are minimum free threads in background pool (or entries in replication queue). |
max_bytes_to_merge_at_max_space_in_pool | google.protobuf.Int64Value |
min_bytes_for_wide_part | google.protobuf.Int64Value Minimum number of bytes in a data part that can be stored in Wide format. More info see in ClickHouse documentation |
min_rows_for_wide_part | google.protobuf.Int64Value Minimum number of rows in a data part that can be stored in Wide format. More info see in ClickHouse documentation |
ttl_only_drop_parts | google.protobuf.BoolValue Enables or disables complete dropping of data parts where all rows are expired in MergeTree tables. More info see in ClickHouse documentation |
allow_remote_fs_zero_copy_replication | google.protobuf.BoolValue |
merge_with_ttl_timeout | google.protobuf.Int64Value |
merge_with_recompression_ttl_timeout | google.protobuf.Int64Value |
max_parts_in_total | google.protobuf.Int64Value |
max_number_of_merges_with_ttl_in_pool | google.protobuf.Int64Value |
cleanup_delay_period | google.protobuf.Int64Value |
number_of_free_entries_in_pool_to_execute_mutation | google.protobuf.Int64Value |
max_avg_part_size_for_too_many_parts | google.protobuf.Int64Value The 'too many parts' check according to 'parts_to_delay_insert' and 'parts_to_throw_insert' will be active only if the average part size (in the relevant partition) is not larger than the specified threshold. If it is larger than the specified threshold, the INSERTs will be neither delayed or rejected. This allows to have hundreds of terabytes in a single table on a single server if the parts are successfully merged to larger parts. This does not affect the thresholds on inactive parts or total parts. Default: 1 GiB Min version: 22.10 See in-depth description in ClickHouse GitHub |
min_age_to_force_merge_seconds | google.protobuf.Int64Value Merge parts if every part in the range is older than the value of min_age_to_force_merge_seconds. Default: 0 - disabled Min_version: 22.10 See in-depth description in ClickHouse documentation |
min_age_to_force_merge_on_partition_only | google.protobuf.BoolValue Whether min_age_to_force_merge_seconds should be applied only on the entire partition and not on subset. Default: false Min_version: 22.11 See in-depth description in ClickHouse documentation |
merge_selecting_sleep_ms | google.protobuf.Int64Value Sleep time for merge selecting when no part is selected. A lower setting triggers selecting tasks in background_schedule_pool frequently, which results in a large number of requests to ClickHouse Keeper in large-scale clusters. Default: 5000 Min_version: 21.10 See in-depth description in ClickHouse documentation |
Kafka
Field | Description |
---|---|
security_protocol | enum SecurityProtocol |
sasl_mechanism | enum SaslMechanism |
sasl_username | string |
sasl_password | string |
enable_ssl_certificate_verification | google.protobuf.BoolValue |
max_poll_interval_ms | google.protobuf.Int64Value The minimum value is 0. |
session_timeout_ms | google.protobuf.Int64Value The minimum value is 0. |
KafkaTopic
Field | Description |
---|---|
name | string Required. |
settings | Kafka Required. |
Rabbitmq
Field | Description |
---|---|
username | string RabbitMQ |
password | string RabbitMQ |
vhost | string RabbitMQ |
Compression
Field | Description |
---|---|
method | enum Method Compression method to use for the specified combination of min_part_size and min_part_size_ratio . |
min_part_size | int64 Minimum size of a part of a table. The minimum value is 1. |
min_part_size_ratio | double Minimum ratio of a part relative to the size of all the data in the table. |
level | google.protobuf.Int64Value The minimum value is 0. |
ExternalDictionary
Field | Description |
---|---|
name | string Required. Name of the external dictionary. |
structure | Structure Required. Set of attributes for the external dictionary. For in-depth description, see ClickHouse documentation |
layout | Layout Required. Layout for storing the dictionary in memory. For in-depth description, see ClickHouse documentation |
lifetime | oneof: fixed_lifetime or lifetime_range Setting for the period of time between dictionary updates. For details, see ClickHouse documentation |
fixed_lifetime | int64 Fixed interval between dictionary updates. |
lifetime_range | Range Range of intervals between dictionary updates for ClickHouse to choose from. |
source | oneof: http_source , mysql_source , clickhouse_source , mongodb_source or postgresql_source Description of the source for the external dictionary. |
http_source | HttpSource HTTP source for the dictionary. |
mysql_source | MysqlSource MySQL source for the dictionary. |
clickhouse_source | ClickhouseSource ClickHouse source for the dictionary. |
mongodb_source | MongodbSource MongoDB source for the dictionary. |
postgresql_source | PostgresqlSource PostgreSQL source for the dictionary. |
HttpSource
Field | Description |
---|---|
url | string Required. URL of the source dictionary available over HTTP. |
format | string Required. The data format. Valid values are all formats supported by ClickHouse SQL dialect. |
MysqlSource
Field | Description |
---|---|
db | string Required. Name of the MySQL database to connect to. |
table | string Required. Name of the database table to use as a ClickHouse dictionary. |
port | int64 Default port to use when connecting to a replica of the dictionary source. Acceptable values are 0 to 65535, inclusive. |
user | string Name of the default user for replicas of the dictionary source. |
password | string Password of the default user for replicas of the dictionary source. |
replicas[] | Replica List of MySQL replicas of the database used as dictionary source. The number of elements must be greater than 0. |
where | string Selection criteria for the data in the specified MySQL table. |
invalidate_query | string Query for checking the dictionary status, to pull only updated data. For more details, see ClickHouse documentation on dictionaries |
Replica
Field | Description |
---|---|
host | string Required. MySQL host of the replica. The maximum string length in characters is 253. |
priority | int64 Required. The priority of the replica that ClickHouse takes into account when connecting. Replica with the highest priority should have this field set to the lowest number. Value must be greater than 0. |
port | int64 Port to use when connecting to the replica. If a port is not specified for a replica, ClickHouse uses the port specified for the source. Acceptable values are 0 to 65535, inclusive. |
user | string Name of the MySQL database user. |
password | string Password of the MySQL database user. |
ClickhouseSource
Field | Description |
---|---|
db | string Required. Name of the ClickHouse database. |
table | string Required. Name of the table in the specified database to be used as the dictionary source. |
host | string Required. ClickHouse host of the specified database. The maximum string length in characters is 253. |
port | int64 Port to use when connecting to the host. Acceptable values are 0 to 65535, inclusive. |
user | string Required. Name of the ClickHouse database user. |
password | string Password of the ClickHouse database user. |
where | string Selection criteria for the data in the specified ClickHouse table. |
MongodbSource
Field | Description |
---|---|
db | string Required. Name of the MongoDB database. |
collection | string Required. Name of the collection in the specified database to be used as the dictionary source. |
host | string Required. MongoDB host of the specified database. The maximum string length in characters is 253. |
port | int64 Port to use when connecting to the host. Acceptable values are 0 to 65535, inclusive. |
user | string Required. Name of the MongoDB database user. |
password | string Password of the MongoDB database user. |
options | string |
PostgresqlSource
Field | Description |
---|---|
db | string Required. Name of the PostrgreSQL database. |
table | string Required. Name of the table in the specified database to be used as the dictionary source. |
hosts[] | string Name of the PostrgreSQL host The number of elements must be greater than 0. |
port | int64 Port to use when connecting to the host. Acceptable values are 0 to 65535, inclusive. |
user | string Required. Name of the PostrgreSQL database user. |
password | string Password of the PostrgreSQL database user. |
invalidate_query | string Query for checking the dictionary status, to pull only updated data. For more details, see ClickHouse documentation on dictionaries |
ssl_mode | enum SslMode Mode of SSL TCP/IP connection to the PostgreSQL host. For more details, see PostgreSQL documentation
|
Structure
Field | Description |
---|---|
id | Id Single numeric key column for the dictionary. |
key | Key Composite key for the dictionary, containing of one or more key columns. For details, see ClickHouse documentation |
range_min | Attribute Field holding the beginning of the range for dictionaries with RANGE_HASHED layout. For details, see ClickHouse documentation |
range_max | Attribute Field holding the end of the range for dictionaries with RANGE_HASHED layout. For details, see ClickHouse documentation |
attributes[] | Attribute Description of the fields available for database queries. For details, see ClickHouse documentation |
Attribute
Field | Description |
---|---|
name | string Required. Name of the column. |
type | string Required. Type of the column. |
null_value | string Default value for an element without data (for example, an empty string). |
expression | string Expression, describing the attribute, if applicable. |
hierarchical | bool Indication of hierarchy support. Default value: false . |
injective | bool Indication of injective mapping "id -> attribute". Default value: false . |
Id
Field | Description |
---|---|
name | string Required. Name of the numeric key. |
Key
Field | Description |
---|---|
attributes[] | Attribute Attributes of a complex key. The number of elements must be greater than 0. |
Layout
Field | Description |
---|---|
type | enum Type Required. Layout type for an external dictionary.
|
size_in_cells | int64 Number of cells in the cache. Rounded up to a power of two. Applicable only for CACHE and COMPLEX_KEY_CACHE layout types. |
Range
Field | Description |
---|---|
min | int64 Minimum dictionary lifetime. |
max | int64 Maximum dictionary lifetime. |
GraphiteRollup
Field | Description |
---|---|
name | string Required. Name for the specified combination of settings for Graphite rollup. |
patterns[] | Pattern Pattern to use for the rollup. The number of elements must be greater than 0. |
Pattern
Field | Description |
---|---|
regexp | string Pattern for metric names. |
function | string Required. Name of the aggregating function to apply to data of the age specified in retention . |
retention[] | Retention Age of data to use for thinning. The number of elements must be greater than 0. |
Retention
Field | Description |
---|---|
age | int64 Minimum age of the data in seconds. The minimum value is 0. |
precision | int64 Precision of determining the age of the data, in seconds. Value must be greater than 0. |
Resources
Field | Description |
---|---|
resource_preset_id | string ID of the preset for computational resources available to a host (CPU, memory etc.). All available presets are listed in the documentation |
disk_size | int64 Volume of the storage available to a host, in bytes. |
disk_type_id | string Type of the storage environment for the host. Possible values:
|
HostSpec
Field | Description |
---|---|
zone_id | string ID of the availability zone where the host resides. To get a list of available zones, use the yandex.cloud.compute.v1.ZoneService.List request. The maximum string length in characters is 50. |
type | Host.Type Required. Type of the host to be deployed. |
subnet_id | string ID of the subnet that the host should belong to. This subnet should be a part of the network that the cluster belongs to. The ID of the network is set in the Cluster.network_id field. The maximum string length in characters is 50. |
assign_public_ip | bool Whether the host should get a public IP address on creation. After a host has been created, this setting cannot be changed. To remove an assigned public IP, or to assign a public IP to a host without one, recreate the host with assign_public_ip set as needed. Possible values:
|
shard_name | string Name of the shard that the host is assigned to. The maximum string length in characters is 63. Value must match the regular expression [a-zA-Z0-9_-]* . |
Operation
Field | Description |
---|---|
id | string ID of the operation. |
description | string Description of the operation. 0-256 characters long. |
created_at | google.protobuf.Timestamp Creation timestamp. |
created_by | string ID of the user or service account who initiated the operation. |
modified_at | google.protobuf.Timestamp The time when the Operation resource was last modified. |
done | bool If the value is false , it means the operation is still in progress. If true , the operation is completed, and either error or response is available. |
metadata | google.protobuf.Any Service-specific metadata associated with the operation. It typically contains the ID of the target resource that the operation is performed on. Any method that returns a long-running operation should document the metadata type, if any. |
result | oneof: error or response The operation result. If done == false and there was no failure detected, neither error nor response is set. If done == false and there was a failure detected, error is set. If done == true , exactly one of error or response is set. |
error | google.rpc.Status The error result of the operation in case of failure or cancellation. |
response | google.protobuf.Any if operation finished successfully. |
AddClusterShardMetadata
Field | Description |
---|---|
cluster_id | string ID of the cluster that a shard is being added to. |
shard_name | string Name of the shard being created. |
Shard
Field | Description |
---|---|
name | string Name of the shard. |
cluster_id | string ID of the cluster that the shard belongs to. |
config | ShardConfig Configuration of the shard. |
ShardConfig
Field | Description |
---|---|
clickhouse | Clickhouse ClickHouse configuration for a shard. |
Clickhouse
Field | Description |
---|---|
config | config.ClickhouseConfigSet ClickHouse settings for a shard. |
resources | Resources Computational resources for a shard. |
weight | google.protobuf.Int64Value Relative weight of a shard considered when writing data to the cluster. For details, see ClickHouse documentation |
ClickhouseConfigSet
Field | Description |
---|---|
effective_config | ClickhouseConfig Required. Effective settings for a ClickHouse cluster (a combination of settings defined in user_config and default_config ). |
user_config | ClickhouseConfig User-defined settings for a ClickHouse cluster. |
default_config | ClickhouseConfig Default configuration for a ClickHouse cluster. |
ClickhouseConfig
Field | Description |
---|---|
log_level | enum LogLevel Logging level for the ClickHouse cluster. Possible values: TRACE, DEBUG, INFORMATION, WARNING, ERROR. |
merge_tree | MergeTree Settings for the MergeTree engine. See description in ClickHouse documentation |
compression[] | Compression Compression settings for the ClickHouse cluster. See in-depth description in ClickHouse documentation |
dictionaries[] | ExternalDictionary Configuration of external dictionaries to be used by the ClickHouse cluster. See in-depth description in ClickHouse documentation |
graphite_rollup[] | GraphiteRollup Settings for thinning Graphite data. See in-depth description in ClickHouse documentation |
kafka | Kafka |
kafka_topics[] | KafkaTopic |
rabbitmq | Rabbitmq |
max_connections | google.protobuf.Int64Value Maximum number of inbound connections. The minimum value is 10. |
max_concurrent_queries | google.protobuf.Int64Value Maximum number of simultaneously processed requests. The minimum value is 10. |
keep_alive_timeout | google.protobuf.Int64Value Number of milliseconds that ClickHouse waits for incoming requests before closing the connection. |
uncompressed_cache_size | google.protobuf.Int64Value Cache size (in bytes) for uncompressed data used by MergeTree tables. |
mark_cache_size | google.protobuf.Int64Value Approximate size (in bytes) of the cache of "marks" used by MergeTree tables. Value must be greater than 0. |
max_table_size_to_drop | google.protobuf.Int64Value Maximum size of the table that can be deleted using a DROP query. |
max_partition_size_to_drop | google.protobuf.Int64Value Maximum size of the partition that can be deleted using a DROP query. |
builtin_dictionaries_reload_interval | google.protobuf.Int64Value The setting is deprecated and has no effect. |
timezone | string The server's time zone to be used in DateTime fields conversions. Specified as an IANA identifier. |
geobase_enabled | google.protobuf.BoolValue Enable or disable geobase. |
geobase_uri | string Address of the archive with the user geobase in Object Storage. |
query_log_retention_size | google.protobuf.Int64Value The maximum size that query_log can grow to before old data will be removed. If set to 0, automatic removal of query_log data based on size is disabled. |
query_log_retention_time | google.protobuf.Int64Value The maximum time that query_log records will be retained before removal. If set to 0, automatic removal of query_log data based on time is disabled. |
query_thread_log_enabled | google.protobuf.BoolValue Whether query_thread_log system table is enabled. |
query_thread_log_retention_size | google.protobuf.Int64Value The maximum size that query_thread_log can grow to before old data will be removed. If set to 0, automatic removal of query_thread_log data based on size is disabled. |
query_thread_log_retention_time | google.protobuf.Int64Value The maximum time that query_thread_log records will be retained before removal. If set to 0, automatic removal of query_thread_log data based on time is disabled. |
part_log_retention_size | google.protobuf.Int64Value The maximum size that part_log can grow to before old data will be removed. If set to 0, automatic removal of part_log data based on size is disabled. |
part_log_retention_time | google.protobuf.Int64Value The maximum time that part_log records will be retained before removal. If set to 0, automatic removal of part_log data based on time is disabled. |
metric_log_enabled | google.protobuf.BoolValue Whether metric_log system table is enabled. |
metric_log_retention_size | google.protobuf.Int64Value The maximum size that metric_log can grow to before old data will be removed. If set to 0, automatic removal of metric_log data based on size is disabled. |
metric_log_retention_time | google.protobuf.Int64Value The maximum time that metric_log records will be retained before removal. If set to 0, automatic removal of metric_log data based on time is disabled. |
trace_log_enabled | google.protobuf.BoolValue Whether trace_log system table is enabled. |
trace_log_retention_size | google.protobuf.Int64Value The maximum size that trace_log can grow to before old data will be removed. If set to 0, automatic removal of trace_log data based on size is disabled. |
trace_log_retention_time | google.protobuf.Int64Value The maximum time that trace_log records will be retained before removal. If set to 0, automatic removal of trace_log data based on time is disabled. |
text_log_enabled | google.protobuf.BoolValue Whether text_log system table is enabled. |
text_log_retention_size | google.protobuf.Int64Value The maximum size that text_log can grow to before old data will be removed. If set to 0, automatic removal of text_log data based on size is disabled. |
text_log_retention_time | google.protobuf.Int64Value The maximum time that text_log records will be retained before removal. If set to 0, automatic removal of text_log data based on time is disabled. |
text_log_level | enum LogLevel Logging level for text_log system table. Possible values: TRACE, DEBUG, INFORMATION, WARNING, ERROR. |
opentelemetry_span_log_enabled | google.protobuf.BoolValue Enable or disable opentelemetry_span_log system table. Default value: false. |
opentelemetry_span_log_retention_size | google.protobuf.Int64Value The maximum size that opentelemetry_span_log can grow to before old data will be removed. If set to 0 (default), automatic removal of opentelemetry_span_log data based on size is disabled. The minimum value is 0. |
opentelemetry_span_log_retention_time | google.protobuf.Int64Value The maximum time that opentelemetry_span_log records will be retained before removal. If set to 0, automatic removal of opentelemetry_span_log data based on time is disabled. The minimum value is 0. |
query_views_log_enabled | google.protobuf.BoolValue Enable or disable query_views_log system table. Default value: false. |
query_views_log_retention_size | google.protobuf.Int64Value The maximum size that query_views_log can grow to before old data will be removed. If set to 0 (default), automatic removal of query_views_log data based on size is disabled. The minimum value is 0. |
query_views_log_retention_time | google.protobuf.Int64Value The maximum time that query_views_log records will be retained before removal. If set to 0, automatic removal of query_views_log data based on time is disabled. The minimum value is 0. |
asynchronous_metric_log_enabled | google.protobuf.BoolValue Enable or disable asynchronous_metric_log system table. Default value: false. |
asynchronous_metric_log_retention_size | google.protobuf.Int64Value The maximum size that asynchronous_metric_log can grow to before old data will be removed. If set to 0 (default), automatic removal of asynchronous_metric_log data based on size is disabled. The minimum value is 0. |
asynchronous_metric_log_retention_time | google.protobuf.Int64Value The maximum time that asynchronous_metric_log records will be retained before removal. If set to 0, automatic removal of asynchronous_metric_log data based on time is disabled. The minimum value is 0. |
session_log_enabled | google.protobuf.BoolValue Enable or disable session_log system table. Default value: false. |
session_log_retention_size | google.protobuf.Int64Value The maximum size that session_log can grow to before old data will be removed. If set to 0 (default), automatic removal of session_log data based on size is disabled. The minimum value is 0. |
session_log_retention_time | google.protobuf.Int64Value The maximum time that session_log records will be retained before removal. If set to 0, automatic removal of session_log data based on time is disabled. The minimum value is 0. |
zookeeper_log_enabled | google.protobuf.BoolValue Enable or disable zookeeper_log system table. Default value: false. |
zookeeper_log_retention_size | google.protobuf.Int64Value The maximum size that zookeeper_log can grow to before old data will be removed. If set to 0 (default), automatic removal of zookeeper_log data based on size is disabled. The minimum value is 0. |
zookeeper_log_retention_time | google.protobuf.Int64Value The maximum time that zookeeper_log records will be retained before removal. If set to 0, automatic removal of zookeeper_log data based on time is disabled. The minimum value is 0. |
asynchronous_insert_log_enabled | google.protobuf.BoolValue Enable or disable asynchronous_insert_log system table. Default value: false. Minimal required ClickHouse version: 22.10. |
asynchronous_insert_log_retention_size | google.protobuf.Int64Value The maximum size that asynchronous_insert_log can grow to before old data will be removed. If set to 0 (default), automatic removal of asynchronous_insert_log data based on size is disabled. The minimum value is 0. |
asynchronous_insert_log_retention_time | google.protobuf.Int64Value The maximum time that asynchronous_insert_log records will be retained before removal. If set to 0, automatic removal of asynchronous_insert_log data based on time is disabled. The minimum value is 0. |
background_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_merges_mutations_concurrency_ratio | google.protobuf.Int64Value Sets a ratio between the number of threads and the number of background merges and mutations that can be executed concurrently. For example, if the ratio equals to 2 and background_pool_size is set to 16 then ClickHouse can execute 32 background merges concurrently. This is possible, because background operations could be suspended and postponed. This is needed to give small merges more execution priority. You can only increase this ratio at runtime. To lower it you have to restart the server. The same as for background_pool_size setting background_merges_mutations_concurrency_ratio could be applied from the default profile for backward compatibility. Default: 2 See in-depth description in ClickHouse documentation |
background_schedule_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_fetches_pool_size | google.protobuf.Int64Value Sets the number of threads performing background fetches for tables with ReplicatedMergeTree engines. Default value: 8. More info see in ClickHouse documentation |
background_move_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_distributed_schedule_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_buffer_flush_schedule_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_message_broker_schedule_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_common_pool_size | google.protobuf.Int64Value The maximum number of threads that will be used for performing a variety of operations (mostly garbage collection) for *MergeTree-engine tables in a background. Default: 8 See in-depth description in ClickHouse documentation |
default_database | google.protobuf.StringValue The default database. To get a list of cluster databases, see Yandex Managed ClickHouse documentation. |
total_memory_profiler_step | google.protobuf.Int64Value Sets the memory size (in bytes) for a stack trace at every peak allocation step. Default value: 4194304. More info see in ClickHouse documentation |
total_memory_tracker_sample_probability | google.protobuf.DoubleValue |
MergeTree
Field | Description |
---|---|
replicated_deduplication_window | google.protobuf.Int64Value Number of blocks of hashes to keep in ZooKeeper. |
replicated_deduplication_window_seconds | google.protobuf.Int64Value Period of time to keep blocks of hashes for. |
parts_to_delay_insert | google.protobuf.Int64Value If table contains at least that many active parts in single partition, artificially slow down insert into table. |
parts_to_throw_insert | google.protobuf.Int64Value If more than this number active parts in single partition, throw 'Too many parts ...' exception. |
inactive_parts_to_delay_insert | google.protobuf.Int64Value |
inactive_parts_to_throw_insert | google.protobuf.Int64Value |
max_replicated_merges_in_queue | google.protobuf.Int64Value How many tasks of merging and mutating parts are allowed simultaneously in ReplicatedMergeTree queue. |
number_of_free_entries_in_pool_to_lower_max_size_of_merge | google.protobuf.Int64Value If there is less than specified number of free entries in background pool (or replicated queue), start to lower maximum size of merge to process. |
max_bytes_to_merge_at_min_space_in_pool | google.protobuf.Int64Value Maximum in total size of parts to merge, when there are minimum free threads in background pool (or entries in replication queue). |
max_bytes_to_merge_at_max_space_in_pool | google.protobuf.Int64Value |
min_bytes_for_wide_part | google.protobuf.Int64Value Minimum number of bytes in a data part that can be stored in Wide format. More info see in ClickHouse documentation |
min_rows_for_wide_part | google.protobuf.Int64Value Minimum number of rows in a data part that can be stored in Wide format. More info see in ClickHouse documentation |
ttl_only_drop_parts | google.protobuf.BoolValue Enables or disables complete dropping of data parts where all rows are expired in MergeTree tables. More info see in ClickHouse documentation |
allow_remote_fs_zero_copy_replication | google.protobuf.BoolValue |
merge_with_ttl_timeout | google.protobuf.Int64Value |
merge_with_recompression_ttl_timeout | google.protobuf.Int64Value |
max_parts_in_total | google.protobuf.Int64Value |
max_number_of_merges_with_ttl_in_pool | google.protobuf.Int64Value |
cleanup_delay_period | google.protobuf.Int64Value |
number_of_free_entries_in_pool_to_execute_mutation | google.protobuf.Int64Value |
max_avg_part_size_for_too_many_parts | google.protobuf.Int64Value The 'too many parts' check according to 'parts_to_delay_insert' and 'parts_to_throw_insert' will be active only if the average part size (in the relevant partition) is not larger than the specified threshold. If it is larger than the specified threshold, the INSERTs will be neither delayed or rejected. This allows to have hundreds of terabytes in a single table on a single server if the parts are successfully merged to larger parts. This does not affect the thresholds on inactive parts or total parts. Default: 1 GiB Min version: 22.10 See in-depth description in ClickHouse GitHub |
min_age_to_force_merge_seconds | google.protobuf.Int64Value Merge parts if every part in the range is older than the value of min_age_to_force_merge_seconds. Default: 0 - disabled Min_version: 22.10 See in-depth description in ClickHouse documentation |
min_age_to_force_merge_on_partition_only | google.protobuf.BoolValue Whether min_age_to_force_merge_seconds should be applied only on the entire partition and not on subset. Default: false Min_version: 22.11 See in-depth description in ClickHouse documentation |
merge_selecting_sleep_ms | google.protobuf.Int64Value Sleep time for merge selecting when no part is selected. A lower setting triggers selecting tasks in background_schedule_pool frequently, which results in a large number of requests to ClickHouse Keeper in large-scale clusters. Default: 5000 Min_version: 21.10 See in-depth description in ClickHouse documentation |
Kafka
Field | Description |
---|---|
security_protocol | enum SecurityProtocol |
sasl_mechanism | enum SaslMechanism |
sasl_username | string |
sasl_password | string |
enable_ssl_certificate_verification | google.protobuf.BoolValue |
max_poll_interval_ms | google.protobuf.Int64Value The minimum value is 0. |
session_timeout_ms | google.protobuf.Int64Value The minimum value is 0. |
KafkaTopic
Field | Description |
---|---|
name | string Required. |
settings | Kafka Required. |
Rabbitmq
Field | Description |
---|---|
username | string RabbitMQ |
password | string RabbitMQ |
vhost | string RabbitMQ |
Compression
Field | Description |
---|---|
method | enum Method Compression method to use for the specified combination of min_part_size and min_part_size_ratio . |
min_part_size | int64 Minimum size of a part of a table. The minimum value is 1. |
min_part_size_ratio | double Minimum ratio of a part relative to the size of all the data in the table. |
level | google.protobuf.Int64Value The minimum value is 0. |
ExternalDictionary
Field | Description |
---|---|
name | string Required. Name of the external dictionary. |
structure | Structure Required. Set of attributes for the external dictionary. For in-depth description, see ClickHouse documentation |
layout | Layout Required. Layout for storing the dictionary in memory. For in-depth description, see ClickHouse documentation |
lifetime | oneof: fixed_lifetime or lifetime_range Setting for the period of time between dictionary updates. For details, see ClickHouse documentation |
fixed_lifetime | int64 Fixed interval between dictionary updates. |
lifetime_range | Range Range of intervals between dictionary updates for ClickHouse to choose from. |
source | oneof: http_source , mysql_source , clickhouse_source , mongodb_source or postgresql_source Description of the source for the external dictionary. |
http_source | HttpSource HTTP source for the dictionary. |
mysql_source | MysqlSource MySQL source for the dictionary. |
clickhouse_source | ClickhouseSource ClickHouse source for the dictionary. |
mongodb_source | MongodbSource MongoDB source for the dictionary. |
postgresql_source | PostgresqlSource PostgreSQL source for the dictionary. |
HttpSource
Field | Description |
---|---|
url | string Required. URL of the source dictionary available over HTTP. |
format | string Required. The data format. Valid values are all formats supported by ClickHouse SQL dialect. |
MysqlSource
Field | Description |
---|---|
db | string Required. Name of the MySQL database to connect to. |
table | string Required. Name of the database table to use as a ClickHouse dictionary. |
port | int64 Default port to use when connecting to a replica of the dictionary source. Acceptable values are 0 to 65535, inclusive. |
user | string Name of the default user for replicas of the dictionary source. |
password | string Password of the default user for replicas of the dictionary source. |
replicas[] | Replica List of MySQL replicas of the database used as dictionary source. The number of elements must be greater than 0. |
where | string Selection criteria for the data in the specified MySQL table. |
invalidate_query | string Query for checking the dictionary status, to pull only updated data. For more details, see ClickHouse documentation on dictionaries |
Replica
Field | Description |
---|---|
host | string Required. MySQL host of the replica. The maximum string length in characters is 253. |
priority | int64 Required. The priority of the replica that ClickHouse takes into account when connecting. Replica with the highest priority should have this field set to the lowest number. Value must be greater than 0. |
port | int64 Port to use when connecting to the replica. If a port is not specified for a replica, ClickHouse uses the port specified for the source. Acceptable values are 0 to 65535, inclusive. |
user | string Name of the MySQL database user. |
password | string Password of the MySQL database user. |
ClickhouseSource
Field | Description |
---|---|
db | string Required. Name of the ClickHouse database. |
table | string Required. Name of the table in the specified database to be used as the dictionary source. |
host | string Required. ClickHouse host of the specified database. The maximum string length in characters is 253. |
port | int64 Port to use when connecting to the host. Acceptable values are 0 to 65535, inclusive. |
user | string Required. Name of the ClickHouse database user. |
password | string Password of the ClickHouse database user. |
where | string Selection criteria for the data in the specified ClickHouse table. |
MongodbSource
Field | Description |
---|---|
db | string Required. Name of the MongoDB database. |
collection | string Required. Name of the collection in the specified database to be used as the dictionary source. |
host | string Required. MongoDB host of the specified database. The maximum string length in characters is 253. |
port | int64 Port to use when connecting to the host. Acceptable values are 0 to 65535, inclusive. |
user | string Required. Name of the MongoDB database user. |
password | string Password of the MongoDB database user. |
options | string |
PostgresqlSource
Field | Description |
---|---|
db | string Required. Name of the PostrgreSQL database. |
table | string Required. Name of the table in the specified database to be used as the dictionary source. |
hosts[] | string Name of the PostrgreSQL host The number of elements must be greater than 0. |
port | int64 Port to use when connecting to the host. Acceptable values are 0 to 65535, inclusive. |
user | string Required. Name of the PostrgreSQL database user. |
password | string Password of the PostrgreSQL database user. |
invalidate_query | string Query for checking the dictionary status, to pull only updated data. For more details, see ClickHouse documentation on dictionaries |
ssl_mode | enum SslMode Mode of SSL TCP/IP connection to the PostgreSQL host. For more details, see PostgreSQL documentation
|
Structure
Field | Description |
---|---|
id | Id Single numeric key column for the dictionary. |
key | Key Composite key for the dictionary, containing of one or more key columns. For details, see ClickHouse documentation |
range_min | Attribute Field holding the beginning of the range for dictionaries with RANGE_HASHED layout. For details, see ClickHouse documentation |
range_max | Attribute Field holding the end of the range for dictionaries with RANGE_HASHED layout. For details, see ClickHouse documentation |
attributes[] | Attribute Description of the fields available for database queries. For details, see ClickHouse documentation |
Attribute
Field | Description |
---|---|
name | string Required. Name of the column. |
type | string Required. Type of the column. |
null_value | string Default value for an element without data (for example, an empty string). |
expression | string Expression, describing the attribute, if applicable. |
hierarchical | bool Indication of hierarchy support. Default value: false . |
injective | bool Indication of injective mapping "id -> attribute". Default value: false . |
Id
Field | Description |
---|---|
name | string Required. Name of the numeric key. |
Key
Field | Description |
---|---|
attributes[] | Attribute Attributes of a complex key. The number of elements must be greater than 0. |
Layout
Field | Description |
---|---|
type | enum Type Required. Layout type for an external dictionary.
|
size_in_cells | int64 Number of cells in the cache. Rounded up to a power of two. Applicable only for CACHE and COMPLEX_KEY_CACHE layout types. |
Range
Field | Description |
---|---|
min | int64 Minimum dictionary lifetime. |
max | int64 Maximum dictionary lifetime. |
GraphiteRollup
Field | Description |
---|---|
name | string Required. Name for the specified combination of settings for Graphite rollup. |
patterns[] | Pattern Pattern to use for the rollup. The number of elements must be greater than 0. |
Pattern
Field | Description |
---|---|
regexp | string Pattern for metric names. |
function | string Required. Name of the aggregating function to apply to data of the age specified in retention . |
retention[] | Retention Age of data to use for thinning. The number of elements must be greater than 0. |
Retention
Field | Description |
---|---|
age | int64 Minimum age of the data in seconds. The minimum value is 0. |
precision | int64 Precision of determining the age of the data, in seconds. Value must be greater than 0. |
UpdateShard
Modifies the specified shard.
rpc UpdateShard (UpdateClusterShardRequest) returns (operation.Operation)
Metadata and response of Operation:
Operation.metadata:UpdateClusterShardMetadata
Operation.response:Shard
UpdateClusterShardRequest
Field | Description |
---|---|
cluster_id | string Required. ID of the ClickHouse cluster the shard belongs to. To get the cluster ID, use a ClusterService.List request. The maximum string length in characters is 50. |
shard_name | string Required. Name of the shard to be updated. To get the name of a shard, use a ClusterService.ListShards request. The maximum string length in characters is 63. Value must match the regular expression [a-zA-Z0-9_-]* . |
update_mask | google.protobuf.FieldMask Field mask that specifies which attributes of the ClickHouse shard should be updated. |
config_spec | ShardConfigSpec New configuration for the specified shard. |
ShardConfigSpec
Field | Description |
---|---|
clickhouse | Clickhouse ClickHouse configuration for a shard. |
Clickhouse
Field | Description |
---|---|
config | config.ClickhouseConfig ClickHouse settings for the shard. |
resources | Resources Computational resources for the shard. |
weight | google.protobuf.Int64Value Relative weight of the shard considered when writing data to the cluster. For details, see ClickHouse documentation |
ClickhouseConfig
Field | Description |
---|---|
log_level | enum LogLevel Logging level for the ClickHouse cluster. Possible values: TRACE, DEBUG, INFORMATION, WARNING, ERROR. |
merge_tree | MergeTree Settings for the MergeTree engine. See description in ClickHouse documentation |
compression[] | Compression Compression settings for the ClickHouse cluster. See in-depth description in ClickHouse documentation |
dictionaries[] | ExternalDictionary Configuration of external dictionaries to be used by the ClickHouse cluster. See in-depth description in ClickHouse documentation |
graphite_rollup[] | GraphiteRollup Settings for thinning Graphite data. See in-depth description in ClickHouse documentation |
kafka | Kafka |
kafka_topics[] | KafkaTopic |
rabbitmq | Rabbitmq |
max_connections | google.protobuf.Int64Value Maximum number of inbound connections. The minimum value is 10. |
max_concurrent_queries | google.protobuf.Int64Value Maximum number of simultaneously processed requests. The minimum value is 10. |
keep_alive_timeout | google.protobuf.Int64Value Number of milliseconds that ClickHouse waits for incoming requests before closing the connection. |
uncompressed_cache_size | google.protobuf.Int64Value Cache size (in bytes) for uncompressed data used by MergeTree tables. |
mark_cache_size | google.protobuf.Int64Value Approximate size (in bytes) of the cache of "marks" used by MergeTree tables. Value must be greater than 0. |
max_table_size_to_drop | google.protobuf.Int64Value Maximum size of the table that can be deleted using a DROP query. |
max_partition_size_to_drop | google.protobuf.Int64Value Maximum size of the partition that can be deleted using a DROP query. |
builtin_dictionaries_reload_interval | google.protobuf.Int64Value The setting is deprecated and has no effect. |
timezone | string The server's time zone to be used in DateTime fields conversions. Specified as an IANA identifier. |
geobase_enabled | google.protobuf.BoolValue Enable or disable geobase. |
geobase_uri | string Address of the archive with the user geobase in Object Storage. |
query_log_retention_size | google.protobuf.Int64Value The maximum size that query_log can grow to before old data will be removed. If set to 0, automatic removal of query_log data based on size is disabled. |
query_log_retention_time | google.protobuf.Int64Value The maximum time that query_log records will be retained before removal. If set to 0, automatic removal of query_log data based on time is disabled. |
query_thread_log_enabled | google.protobuf.BoolValue Whether query_thread_log system table is enabled. |
query_thread_log_retention_size | google.protobuf.Int64Value The maximum size that query_thread_log can grow to before old data will be removed. If set to 0, automatic removal of query_thread_log data based on size is disabled. |
query_thread_log_retention_time | google.protobuf.Int64Value The maximum time that query_thread_log records will be retained before removal. If set to 0, automatic removal of query_thread_log data based on time is disabled. |
part_log_retention_size | google.protobuf.Int64Value The maximum size that part_log can grow to before old data will be removed. If set to 0, automatic removal of part_log data based on size is disabled. |
part_log_retention_time | google.protobuf.Int64Value The maximum time that part_log records will be retained before removal. If set to 0, automatic removal of part_log data based on time is disabled. |
metric_log_enabled | google.protobuf.BoolValue Whether metric_log system table is enabled. |
metric_log_retention_size | google.protobuf.Int64Value The maximum size that metric_log can grow to before old data will be removed. If set to 0, automatic removal of metric_log data based on size is disabled. |
metric_log_retention_time | google.protobuf.Int64Value The maximum time that metric_log records will be retained before removal. If set to 0, automatic removal of metric_log data based on time is disabled. |
trace_log_enabled | google.protobuf.BoolValue Whether trace_log system table is enabled. |
trace_log_retention_size | google.protobuf.Int64Value The maximum size that trace_log can grow to before old data will be removed. If set to 0, automatic removal of trace_log data based on size is disabled. |
trace_log_retention_time | google.protobuf.Int64Value The maximum time that trace_log records will be retained before removal. If set to 0, automatic removal of trace_log data based on time is disabled. |
text_log_enabled | google.protobuf.BoolValue Whether text_log system table is enabled. |
text_log_retention_size | google.protobuf.Int64Value The maximum size that text_log can grow to before old data will be removed. If set to 0, automatic removal of text_log data based on size is disabled. |
text_log_retention_time | google.protobuf.Int64Value The maximum time that text_log records will be retained before removal. If set to 0, automatic removal of text_log data based on time is disabled. |
text_log_level | enum LogLevel Logging level for text_log system table. Possible values: TRACE, DEBUG, INFORMATION, WARNING, ERROR. |
opentelemetry_span_log_enabled | google.protobuf.BoolValue Enable or disable opentelemetry_span_log system table. Default value: false. |
opentelemetry_span_log_retention_size | google.protobuf.Int64Value The maximum size that opentelemetry_span_log can grow to before old data will be removed. If set to 0 (default), automatic removal of opentelemetry_span_log data based on size is disabled. The minimum value is 0. |
opentelemetry_span_log_retention_time | google.protobuf.Int64Value The maximum time that opentelemetry_span_log records will be retained before removal. If set to 0, automatic removal of opentelemetry_span_log data based on time is disabled. The minimum value is 0. |
query_views_log_enabled | google.protobuf.BoolValue Enable or disable query_views_log system table. Default value: false. |
query_views_log_retention_size | google.protobuf.Int64Value The maximum size that query_views_log can grow to before old data will be removed. If set to 0 (default), automatic removal of query_views_log data based on size is disabled. The minimum value is 0. |
query_views_log_retention_time | google.protobuf.Int64Value The maximum time that query_views_log records will be retained before removal. If set to 0, automatic removal of query_views_log data based on time is disabled. The minimum value is 0. |
asynchronous_metric_log_enabled | google.protobuf.BoolValue Enable or disable asynchronous_metric_log system table. Default value: false. |
asynchronous_metric_log_retention_size | google.protobuf.Int64Value The maximum size that asynchronous_metric_log can grow to before old data will be removed. If set to 0 (default), automatic removal of asynchronous_metric_log data based on size is disabled. The minimum value is 0. |
asynchronous_metric_log_retention_time | google.protobuf.Int64Value The maximum time that asynchronous_metric_log records will be retained before removal. If set to 0, automatic removal of asynchronous_metric_log data based on time is disabled. The minimum value is 0. |
session_log_enabled | google.protobuf.BoolValue Enable or disable session_log system table. Default value: false. |
session_log_retention_size | google.protobuf.Int64Value The maximum size that session_log can grow to before old data will be removed. If set to 0 (default), automatic removal of session_log data based on size is disabled. The minimum value is 0. |
session_log_retention_time | google.protobuf.Int64Value The maximum time that session_log records will be retained before removal. If set to 0, automatic removal of session_log data based on time is disabled. The minimum value is 0. |
zookeeper_log_enabled | google.protobuf.BoolValue Enable or disable zookeeper_log system table. Default value: false. |
zookeeper_log_retention_size | google.protobuf.Int64Value The maximum size that zookeeper_log can grow to before old data will be removed. If set to 0 (default), automatic removal of zookeeper_log data based on size is disabled. The minimum value is 0. |
zookeeper_log_retention_time | google.protobuf.Int64Value The maximum time that zookeeper_log records will be retained before removal. If set to 0, automatic removal of zookeeper_log data based on time is disabled. The minimum value is 0. |
asynchronous_insert_log_enabled | google.protobuf.BoolValue Enable or disable asynchronous_insert_log system table. Default value: false. Minimal required ClickHouse version: 22.10. |
asynchronous_insert_log_retention_size | google.protobuf.Int64Value The maximum size that asynchronous_insert_log can grow to before old data will be removed. If set to 0 (default), automatic removal of asynchronous_insert_log data based on size is disabled. The minimum value is 0. |
asynchronous_insert_log_retention_time | google.protobuf.Int64Value The maximum time that asynchronous_insert_log records will be retained before removal. If set to 0, automatic removal of asynchronous_insert_log data based on time is disabled. The minimum value is 0. |
background_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_merges_mutations_concurrency_ratio | google.protobuf.Int64Value Sets a ratio between the number of threads and the number of background merges and mutations that can be executed concurrently. For example, if the ratio equals to 2 and background_pool_size is set to 16 then ClickHouse can execute 32 background merges concurrently. This is possible, because background operations could be suspended and postponed. This is needed to give small merges more execution priority. You can only increase this ratio at runtime. To lower it you have to restart the server. The same as for background_pool_size setting background_merges_mutations_concurrency_ratio could be applied from the default profile for backward compatibility. Default: 2 See in-depth description in ClickHouse documentation |
background_schedule_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_fetches_pool_size | google.protobuf.Int64Value Sets the number of threads performing background fetches for tables with ReplicatedMergeTree engines. Default value: 8. More info see in ClickHouse documentation |
background_move_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_distributed_schedule_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_buffer_flush_schedule_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_message_broker_schedule_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_common_pool_size | google.protobuf.Int64Value The maximum number of threads that will be used for performing a variety of operations (mostly garbage collection) for *MergeTree-engine tables in a background. Default: 8 See in-depth description in ClickHouse documentation |
default_database | google.protobuf.StringValue The default database. To get a list of cluster databases, see Yandex Managed ClickHouse documentation. |
total_memory_profiler_step | google.protobuf.Int64Value Sets the memory size (in bytes) for a stack trace at every peak allocation step. Default value: 4194304. More info see in ClickHouse documentation |
total_memory_tracker_sample_probability | google.protobuf.DoubleValue |
MergeTree
Field | Description |
---|---|
replicated_deduplication_window | google.protobuf.Int64Value Number of blocks of hashes to keep in ZooKeeper. |
replicated_deduplication_window_seconds | google.protobuf.Int64Value Period of time to keep blocks of hashes for. |
parts_to_delay_insert | google.protobuf.Int64Value If table contains at least that many active parts in single partition, artificially slow down insert into table. |
parts_to_throw_insert | google.protobuf.Int64Value If more than this number active parts in single partition, throw 'Too many parts ...' exception. |
inactive_parts_to_delay_insert | google.protobuf.Int64Value |
inactive_parts_to_throw_insert | google.protobuf.Int64Value |
max_replicated_merges_in_queue | google.protobuf.Int64Value How many tasks of merging and mutating parts are allowed simultaneously in ReplicatedMergeTree queue. |
number_of_free_entries_in_pool_to_lower_max_size_of_merge | google.protobuf.Int64Value If there is less than specified number of free entries in background pool (or replicated queue), start to lower maximum size of merge to process. |
max_bytes_to_merge_at_min_space_in_pool | google.protobuf.Int64Value Maximum in total size of parts to merge, when there are minimum free threads in background pool (or entries in replication queue). |
max_bytes_to_merge_at_max_space_in_pool | google.protobuf.Int64Value |
min_bytes_for_wide_part | google.protobuf.Int64Value Minimum number of bytes in a data part that can be stored in Wide format. More info see in ClickHouse documentation |
min_rows_for_wide_part | google.protobuf.Int64Value Minimum number of rows in a data part that can be stored in Wide format. More info see in ClickHouse documentation |
ttl_only_drop_parts | google.protobuf.BoolValue Enables or disables complete dropping of data parts where all rows are expired in MergeTree tables. More info see in ClickHouse documentation |
allow_remote_fs_zero_copy_replication | google.protobuf.BoolValue |
merge_with_ttl_timeout | google.protobuf.Int64Value |
merge_with_recompression_ttl_timeout | google.protobuf.Int64Value |
max_parts_in_total | google.protobuf.Int64Value |
max_number_of_merges_with_ttl_in_pool | google.protobuf.Int64Value |
cleanup_delay_period | google.protobuf.Int64Value |
number_of_free_entries_in_pool_to_execute_mutation | google.protobuf.Int64Value |
max_avg_part_size_for_too_many_parts | google.protobuf.Int64Value The 'too many parts' check according to 'parts_to_delay_insert' and 'parts_to_throw_insert' will be active only if the average part size (in the relevant partition) is not larger than the specified threshold. If it is larger than the specified threshold, the INSERTs will be neither delayed or rejected. This allows to have hundreds of terabytes in a single table on a single server if the parts are successfully merged to larger parts. This does not affect the thresholds on inactive parts or total parts. Default: 1 GiB Min version: 22.10 See in-depth description in ClickHouse GitHub |
min_age_to_force_merge_seconds | google.protobuf.Int64Value Merge parts if every part in the range is older than the value of min_age_to_force_merge_seconds. Default: 0 - disabled Min_version: 22.10 See in-depth description in ClickHouse documentation |
min_age_to_force_merge_on_partition_only | google.protobuf.BoolValue Whether min_age_to_force_merge_seconds should be applied only on the entire partition and not on subset. Default: false Min_version: 22.11 See in-depth description in ClickHouse documentation |
merge_selecting_sleep_ms | google.protobuf.Int64Value Sleep time for merge selecting when no part is selected. A lower setting triggers selecting tasks in background_schedule_pool frequently, which results in a large number of requests to ClickHouse Keeper in large-scale clusters. Default: 5000 Min_version: 21.10 See in-depth description in ClickHouse documentation |
Kafka
Field | Description |
---|---|
security_protocol | enum SecurityProtocol |
sasl_mechanism | enum SaslMechanism |
sasl_username | string |
sasl_password | string |
enable_ssl_certificate_verification | google.protobuf.BoolValue |
max_poll_interval_ms | google.protobuf.Int64Value The minimum value is 0. |
session_timeout_ms | google.protobuf.Int64Value The minimum value is 0. |
KafkaTopic
Field | Description |
---|---|
name | string Required. |
settings | Kafka Required. |
Rabbitmq
Field | Description |
---|---|
username | string RabbitMQ |
password | string RabbitMQ |
vhost | string RabbitMQ |
Compression
Field | Description |
---|---|
method | enum Method Compression method to use for the specified combination of min_part_size and min_part_size_ratio . |
min_part_size | int64 Minimum size of a part of a table. The minimum value is 1. |
min_part_size_ratio | double Minimum ratio of a part relative to the size of all the data in the table. |
level | google.protobuf.Int64Value The minimum value is 0. |
ExternalDictionary
Field | Description |
---|---|
name | string Required. Name of the external dictionary. |
structure | Structure Required. Set of attributes for the external dictionary. For in-depth description, see ClickHouse documentation |
layout | Layout Required. Layout for storing the dictionary in memory. For in-depth description, see ClickHouse documentation |
lifetime | oneof: fixed_lifetime or lifetime_range Setting for the period of time between dictionary updates. For details, see ClickHouse documentation |
fixed_lifetime | int64 Fixed interval between dictionary updates. |
lifetime_range | Range Range of intervals between dictionary updates for ClickHouse to choose from. |
source | oneof: http_source , mysql_source , clickhouse_source , mongodb_source or postgresql_source Description of the source for the external dictionary. |
http_source | HttpSource HTTP source for the dictionary. |
mysql_source | MysqlSource MySQL source for the dictionary. |
clickhouse_source | ClickhouseSource ClickHouse source for the dictionary. |
mongodb_source | MongodbSource MongoDB source for the dictionary. |
postgresql_source | PostgresqlSource PostgreSQL source for the dictionary. |
HttpSource
Field | Description |
---|---|
url | string Required. URL of the source dictionary available over HTTP. |
format | string Required. The data format. Valid values are all formats supported by ClickHouse SQL dialect. |
MysqlSource
Field | Description |
---|---|
db | string Required. Name of the MySQL database to connect to. |
table | string Required. Name of the database table to use as a ClickHouse dictionary. |
port | int64 Default port to use when connecting to a replica of the dictionary source. Acceptable values are 0 to 65535, inclusive. |
user | string Name of the default user for replicas of the dictionary source. |
password | string Password of the default user for replicas of the dictionary source. |
replicas[] | Replica List of MySQL replicas of the database used as dictionary source. The number of elements must be greater than 0. |
where | string Selection criteria for the data in the specified MySQL table. |
invalidate_query | string Query for checking the dictionary status, to pull only updated data. For more details, see ClickHouse documentation on dictionaries |
Replica
Field | Description |
---|---|
host | string Required. MySQL host of the replica. The maximum string length in characters is 253. |
priority | int64 Required. The priority of the replica that ClickHouse takes into account when connecting. Replica with the highest priority should have this field set to the lowest number. Value must be greater than 0. |
port | int64 Port to use when connecting to the replica. If a port is not specified for a replica, ClickHouse uses the port specified for the source. Acceptable values are 0 to 65535, inclusive. |
user | string Name of the MySQL database user. |
password | string Password of the MySQL database user. |
ClickhouseSource
Field | Description |
---|---|
db | string Required. Name of the ClickHouse database. |
table | string Required. Name of the table in the specified database to be used as the dictionary source. |
host | string Required. ClickHouse host of the specified database. The maximum string length in characters is 253. |
port | int64 Port to use when connecting to the host. Acceptable values are 0 to 65535, inclusive. |
user | string Required. Name of the ClickHouse database user. |
password | string Password of the ClickHouse database user. |
where | string Selection criteria for the data in the specified ClickHouse table. |
MongodbSource
Field | Description |
---|---|
db | string Required. Name of the MongoDB database. |
collection | string Required. Name of the collection in the specified database to be used as the dictionary source. |
host | string Required. MongoDB host of the specified database. The maximum string length in characters is 253. |
port | int64 Port to use when connecting to the host. Acceptable values are 0 to 65535, inclusive. |
user | string Required. Name of the MongoDB database user. |
password | string Password of the MongoDB database user. |
options | string |
PostgresqlSource
Field | Description |
---|---|
db | string Required. Name of the PostrgreSQL database. |
table | string Required. Name of the table in the specified database to be used as the dictionary source. |
hosts[] | string Name of the PostrgreSQL host The number of elements must be greater than 0. |
port | int64 Port to use when connecting to the host. Acceptable values are 0 to 65535, inclusive. |
user | string Required. Name of the PostrgreSQL database user. |
password | string Password of the PostrgreSQL database user. |
invalidate_query | string Query for checking the dictionary status, to pull only updated data. For more details, see ClickHouse documentation on dictionaries |
ssl_mode | enum SslMode Mode of SSL TCP/IP connection to the PostgreSQL host. For more details, see PostgreSQL documentation
|
Structure
Field | Description |
---|---|
id | Id Single numeric key column for the dictionary. |
key | Key Composite key for the dictionary, containing of one or more key columns. For details, see ClickHouse documentation |
range_min | Attribute Field holding the beginning of the range for dictionaries with RANGE_HASHED layout. For details, see ClickHouse documentation |
range_max | Attribute Field holding the end of the range for dictionaries with RANGE_HASHED layout. For details, see ClickHouse documentation |
attributes[] | Attribute Description of the fields available for database queries. For details, see ClickHouse documentation |
Attribute
Field | Description |
---|---|
name | string Required. Name of the column. |
type | string Required. Type of the column. |
null_value | string Default value for an element without data (for example, an empty string). |
expression | string Expression, describing the attribute, if applicable. |
hierarchical | bool Indication of hierarchy support. Default value: false . |
injective | bool Indication of injective mapping "id -> attribute". Default value: false . |
Id
Field | Description |
---|---|
name | string Required. Name of the numeric key. |
Key
Field | Description |
---|---|
attributes[] | Attribute Attributes of a complex key. The number of elements must be greater than 0. |
Layout
Field | Description |
---|---|
type | enum Type Required. Layout type for an external dictionary.
|
size_in_cells | int64 Number of cells in the cache. Rounded up to a power of two. Applicable only for CACHE and COMPLEX_KEY_CACHE layout types. |
Range
Field | Description |
---|---|
min | int64 Minimum dictionary lifetime. |
max | int64 Maximum dictionary lifetime. |
GraphiteRollup
Field | Description |
---|---|
name | string Required. Name for the specified combination of settings for Graphite rollup. |
patterns[] | Pattern Pattern to use for the rollup. The number of elements must be greater than 0. |
Pattern
Field | Description |
---|---|
regexp | string Pattern for metric names. |
function | string Required. Name of the aggregating function to apply to data of the age specified in retention . |
retention[] | Retention Age of data to use for thinning. The number of elements must be greater than 0. |
Retention
Field | Description |
---|---|
age | int64 Minimum age of the data in seconds. The minimum value is 0. |
precision | int64 Precision of determining the age of the data, in seconds. Value must be greater than 0. |
Resources
Field | Description |
---|---|
resource_preset_id | string ID of the preset for computational resources available to a host (CPU, memory etc.). All available presets are listed in the documentation |
disk_size | int64 Volume of the storage available to a host, in bytes. |
disk_type_id | string Type of the storage environment for the host. Possible values:
|
Operation
Field | Description |
---|---|
id | string ID of the operation. |
description | string Description of the operation. 0-256 characters long. |
created_at | google.protobuf.Timestamp Creation timestamp. |
created_by | string ID of the user or service account who initiated the operation. |
modified_at | google.protobuf.Timestamp The time when the Operation resource was last modified. |
done | bool If the value is false , it means the operation is still in progress. If true , the operation is completed, and either error or response is available. |
metadata | google.protobuf.Any Service-specific metadata associated with the operation. It typically contains the ID of the target resource that the operation is performed on. Any method that returns a long-running operation should document the metadata type, if any. |
result | oneof: error or response The operation result. If done == false and there was no failure detected, neither error nor response is set. If done == false and there was a failure detected, error is set. If done == true , exactly one of error or response is set. |
error | google.rpc.Status The error result of the operation in case of failure or cancellation. |
response | google.protobuf.Any if operation finished successfully. |
UpdateClusterShardMetadata
Field | Description |
---|---|
cluster_id | string ID of the cluster that contains the shard being updated. |
shard_name | string Name of the shard being updated. |
Shard
Field | Description |
---|---|
name | string Name of the shard. |
cluster_id | string ID of the cluster that the shard belongs to. |
config | ShardConfig Configuration of the shard. |
ShardConfig
Field | Description |
---|---|
clickhouse | Clickhouse ClickHouse configuration for a shard. |
Clickhouse
Field | Description |
---|---|
config | config.ClickhouseConfigSet ClickHouse settings for a shard. |
resources | Resources Computational resources for a shard. |
weight | google.protobuf.Int64Value Relative weight of a shard considered when writing data to the cluster. For details, see ClickHouse documentation |
ClickhouseConfigSet
Field | Description |
---|---|
effective_config | ClickhouseConfig Required. Effective settings for a ClickHouse cluster (a combination of settings defined in user_config and default_config ). |
user_config | ClickhouseConfig User-defined settings for a ClickHouse cluster. |
default_config | ClickhouseConfig Default configuration for a ClickHouse cluster. |
ClickhouseConfig
Field | Description |
---|---|
log_level | enum LogLevel Logging level for the ClickHouse cluster. Possible values: TRACE, DEBUG, INFORMATION, WARNING, ERROR. |
merge_tree | MergeTree Settings for the MergeTree engine. See description in ClickHouse documentation |
compression[] | Compression Compression settings for the ClickHouse cluster. See in-depth description in ClickHouse documentation |
dictionaries[] | ExternalDictionary Configuration of external dictionaries to be used by the ClickHouse cluster. See in-depth description in ClickHouse documentation |
graphite_rollup[] | GraphiteRollup Settings for thinning Graphite data. See in-depth description in ClickHouse documentation |
kafka | Kafka |
kafka_topics[] | KafkaTopic |
rabbitmq | Rabbitmq |
max_connections | google.protobuf.Int64Value Maximum number of inbound connections. The minimum value is 10. |
max_concurrent_queries | google.protobuf.Int64Value Maximum number of simultaneously processed requests. The minimum value is 10. |
keep_alive_timeout | google.protobuf.Int64Value Number of milliseconds that ClickHouse waits for incoming requests before closing the connection. |
uncompressed_cache_size | google.protobuf.Int64Value Cache size (in bytes) for uncompressed data used by MergeTree tables. |
mark_cache_size | google.protobuf.Int64Value Approximate size (in bytes) of the cache of "marks" used by MergeTree tables. Value must be greater than 0. |
max_table_size_to_drop | google.protobuf.Int64Value Maximum size of the table that can be deleted using a DROP query. |
max_partition_size_to_drop | google.protobuf.Int64Value Maximum size of the partition that can be deleted using a DROP query. |
builtin_dictionaries_reload_interval | google.protobuf.Int64Value The setting is deprecated and has no effect. |
timezone | string The server's time zone to be used in DateTime fields conversions. Specified as an IANA identifier. |
geobase_enabled | google.protobuf.BoolValue Enable or disable geobase. |
geobase_uri | string Address of the archive with the user geobase in Object Storage. |
query_log_retention_size | google.protobuf.Int64Value The maximum size that query_log can grow to before old data will be removed. If set to 0, automatic removal of query_log data based on size is disabled. |
query_log_retention_time | google.protobuf.Int64Value The maximum time that query_log records will be retained before removal. If set to 0, automatic removal of query_log data based on time is disabled. |
query_thread_log_enabled | google.protobuf.BoolValue Whether query_thread_log system table is enabled. |
query_thread_log_retention_size | google.protobuf.Int64Value The maximum size that query_thread_log can grow to before old data will be removed. If set to 0, automatic removal of query_thread_log data based on size is disabled. |
query_thread_log_retention_time | google.protobuf.Int64Value The maximum time that query_thread_log records will be retained before removal. If set to 0, automatic removal of query_thread_log data based on time is disabled. |
part_log_retention_size | google.protobuf.Int64Value The maximum size that part_log can grow to before old data will be removed. If set to 0, automatic removal of part_log data based on size is disabled. |
part_log_retention_time | google.protobuf.Int64Value The maximum time that part_log records will be retained before removal. If set to 0, automatic removal of part_log data based on time is disabled. |
metric_log_enabled | google.protobuf.BoolValue Whether metric_log system table is enabled. |
metric_log_retention_size | google.protobuf.Int64Value The maximum size that metric_log can grow to before old data will be removed. If set to 0, automatic removal of metric_log data based on size is disabled. |
metric_log_retention_time | google.protobuf.Int64Value The maximum time that metric_log records will be retained before removal. If set to 0, automatic removal of metric_log data based on time is disabled. |
trace_log_enabled | google.protobuf.BoolValue Whether trace_log system table is enabled. |
trace_log_retention_size | google.protobuf.Int64Value The maximum size that trace_log can grow to before old data will be removed. If set to 0, automatic removal of trace_log data based on size is disabled. |
trace_log_retention_time | google.protobuf.Int64Value The maximum time that trace_log records will be retained before removal. If set to 0, automatic removal of trace_log data based on time is disabled. |
text_log_enabled | google.protobuf.BoolValue Whether text_log system table is enabled. |
text_log_retention_size | google.protobuf.Int64Value The maximum size that text_log can grow to before old data will be removed. If set to 0, automatic removal of text_log data based on size is disabled. |
text_log_retention_time | google.protobuf.Int64Value The maximum time that text_log records will be retained before removal. If set to 0, automatic removal of text_log data based on time is disabled. |
text_log_level | enum LogLevel Logging level for text_log system table. Possible values: TRACE, DEBUG, INFORMATION, WARNING, ERROR. |
opentelemetry_span_log_enabled | google.protobuf.BoolValue Enable or disable opentelemetry_span_log system table. Default value: false. |
opentelemetry_span_log_retention_size | google.protobuf.Int64Value The maximum size that opentelemetry_span_log can grow to before old data will be removed. If set to 0 (default), automatic removal of opentelemetry_span_log data based on size is disabled. The minimum value is 0. |
opentelemetry_span_log_retention_time | google.protobuf.Int64Value The maximum time that opentelemetry_span_log records will be retained before removal. If set to 0, automatic removal of opentelemetry_span_log data based on time is disabled. The minimum value is 0. |
query_views_log_enabled | google.protobuf.BoolValue Enable or disable query_views_log system table. Default value: false. |
query_views_log_retention_size | google.protobuf.Int64Value The maximum size that query_views_log can grow to before old data will be removed. If set to 0 (default), automatic removal of query_views_log data based on size is disabled. The minimum value is 0. |
query_views_log_retention_time | google.protobuf.Int64Value The maximum time that query_views_log records will be retained before removal. If set to 0, automatic removal of query_views_log data based on time is disabled. The minimum value is 0. |
asynchronous_metric_log_enabled | google.protobuf.BoolValue Enable or disable asynchronous_metric_log system table. Default value: false. |
asynchronous_metric_log_retention_size | google.protobuf.Int64Value The maximum size that asynchronous_metric_log can grow to before old data will be removed. If set to 0 (default), automatic removal of asynchronous_metric_log data based on size is disabled. The minimum value is 0. |
asynchronous_metric_log_retention_time | google.protobuf.Int64Value The maximum time that asynchronous_metric_log records will be retained before removal. If set to 0, automatic removal of asynchronous_metric_log data based on time is disabled. The minimum value is 0. |
session_log_enabled | google.protobuf.BoolValue Enable or disable session_log system table. Default value: false. |
session_log_retention_size | google.protobuf.Int64Value The maximum size that session_log can grow to before old data will be removed. If set to 0 (default), automatic removal of session_log data based on size is disabled. The minimum value is 0. |
session_log_retention_time | google.protobuf.Int64Value The maximum time that session_log records will be retained before removal. If set to 0, automatic removal of session_log data based on time is disabled. The minimum value is 0. |
zookeeper_log_enabled | google.protobuf.BoolValue Enable or disable zookeeper_log system table. Default value: false. |
zookeeper_log_retention_size | google.protobuf.Int64Value The maximum size that zookeeper_log can grow to before old data will be removed. If set to 0 (default), automatic removal of zookeeper_log data based on size is disabled. The minimum value is 0. |
zookeeper_log_retention_time | google.protobuf.Int64Value The maximum time that zookeeper_log records will be retained before removal. If set to 0, automatic removal of zookeeper_log data based on time is disabled. The minimum value is 0. |
asynchronous_insert_log_enabled | google.protobuf.BoolValue Enable or disable asynchronous_insert_log system table. Default value: false. Minimal required ClickHouse version: 22.10. |
asynchronous_insert_log_retention_size | google.protobuf.Int64Value The maximum size that asynchronous_insert_log can grow to before old data will be removed. If set to 0 (default), automatic removal of asynchronous_insert_log data based on size is disabled. The minimum value is 0. |
asynchronous_insert_log_retention_time | google.protobuf.Int64Value The maximum time that asynchronous_insert_log records will be retained before removal. If set to 0, automatic removal of asynchronous_insert_log data based on time is disabled. The minimum value is 0. |
background_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_merges_mutations_concurrency_ratio | google.protobuf.Int64Value Sets a ratio between the number of threads and the number of background merges and mutations that can be executed concurrently. For example, if the ratio equals to 2 and background_pool_size is set to 16 then ClickHouse can execute 32 background merges concurrently. This is possible, because background operations could be suspended and postponed. This is needed to give small merges more execution priority. You can only increase this ratio at runtime. To lower it you have to restart the server. The same as for background_pool_size setting background_merges_mutations_concurrency_ratio could be applied from the default profile for backward compatibility. Default: 2 See in-depth description in ClickHouse documentation |
background_schedule_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_fetches_pool_size | google.protobuf.Int64Value Sets the number of threads performing background fetches for tables with ReplicatedMergeTree engines. Default value: 8. More info see in ClickHouse documentation |
background_move_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_distributed_schedule_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_buffer_flush_schedule_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_message_broker_schedule_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_common_pool_size | google.protobuf.Int64Value The maximum number of threads that will be used for performing a variety of operations (mostly garbage collection) for *MergeTree-engine tables in a background. Default: 8 See in-depth description in ClickHouse documentation |
default_database | google.protobuf.StringValue The default database. To get a list of cluster databases, see Yandex Managed ClickHouse documentation. |
total_memory_profiler_step | google.protobuf.Int64Value Sets the memory size (in bytes) for a stack trace at every peak allocation step. Default value: 4194304. More info see in ClickHouse documentation |
total_memory_tracker_sample_probability | google.protobuf.DoubleValue |
MergeTree
Field | Description |
---|---|
replicated_deduplication_window | google.protobuf.Int64Value Number of blocks of hashes to keep in ZooKeeper. |
replicated_deduplication_window_seconds | google.protobuf.Int64Value Period of time to keep blocks of hashes for. |
parts_to_delay_insert | google.protobuf.Int64Value If table contains at least that many active parts in single partition, artificially slow down insert into table. |
parts_to_throw_insert | google.protobuf.Int64Value If more than this number active parts in single partition, throw 'Too many parts ...' exception. |
inactive_parts_to_delay_insert | google.protobuf.Int64Value |
inactive_parts_to_throw_insert | google.protobuf.Int64Value |
max_replicated_merges_in_queue | google.protobuf.Int64Value How many tasks of merging and mutating parts are allowed simultaneously in ReplicatedMergeTree queue. |
number_of_free_entries_in_pool_to_lower_max_size_of_merge | google.protobuf.Int64Value If there is less than specified number of free entries in background pool (or replicated queue), start to lower maximum size of merge to process. |
max_bytes_to_merge_at_min_space_in_pool | google.protobuf.Int64Value Maximum in total size of parts to merge, when there are minimum free threads in background pool (or entries in replication queue). |
max_bytes_to_merge_at_max_space_in_pool | google.protobuf.Int64Value |
min_bytes_for_wide_part | google.protobuf.Int64Value Minimum number of bytes in a data part that can be stored in Wide format. More info see in ClickHouse documentation |
min_rows_for_wide_part | google.protobuf.Int64Value Minimum number of rows in a data part that can be stored in Wide format. More info see in ClickHouse documentation |
ttl_only_drop_parts | google.protobuf.BoolValue Enables or disables complete dropping of data parts where all rows are expired in MergeTree tables. More info see in ClickHouse documentation |
allow_remote_fs_zero_copy_replication | google.protobuf.BoolValue |
merge_with_ttl_timeout | google.protobuf.Int64Value |
merge_with_recompression_ttl_timeout | google.protobuf.Int64Value |
max_parts_in_total | google.protobuf.Int64Value |
max_number_of_merges_with_ttl_in_pool | google.protobuf.Int64Value |
cleanup_delay_period | google.protobuf.Int64Value |
number_of_free_entries_in_pool_to_execute_mutation | google.protobuf.Int64Value |
max_avg_part_size_for_too_many_parts | google.protobuf.Int64Value The 'too many parts' check according to 'parts_to_delay_insert' and 'parts_to_throw_insert' will be active only if the average part size (in the relevant partition) is not larger than the specified threshold. If it is larger than the specified threshold, the INSERTs will be neither delayed or rejected. This allows to have hundreds of terabytes in a single table on a single server if the parts are successfully merged to larger parts. This does not affect the thresholds on inactive parts or total parts. Default: 1 GiB Min version: 22.10 See in-depth description in ClickHouse GitHub |
min_age_to_force_merge_seconds | google.protobuf.Int64Value Merge parts if every part in the range is older than the value of min_age_to_force_merge_seconds. Default: 0 - disabled Min_version: 22.10 See in-depth description in ClickHouse documentation |
min_age_to_force_merge_on_partition_only | google.protobuf.BoolValue Whether min_age_to_force_merge_seconds should be applied only on the entire partition and not on subset. Default: false Min_version: 22.11 See in-depth description in ClickHouse documentation |
merge_selecting_sleep_ms | google.protobuf.Int64Value Sleep time for merge selecting when no part is selected. A lower setting triggers selecting tasks in background_schedule_pool frequently, which results in a large number of requests to ClickHouse Keeper in large-scale clusters. Default: 5000 Min_version: 21.10 See in-depth description in ClickHouse documentation |
Kafka
Field | Description |
---|---|
security_protocol | enum SecurityProtocol |
sasl_mechanism | enum SaslMechanism |
sasl_username | string |
sasl_password | string |
enable_ssl_certificate_verification | google.protobuf.BoolValue |
max_poll_interval_ms | google.protobuf.Int64Value The minimum value is 0. |
session_timeout_ms | google.protobuf.Int64Value The minimum value is 0. |
KafkaTopic
Field | Description |
---|---|
name | string Required. |
settings | Kafka Required. |
Rabbitmq
Field | Description |
---|---|
username | string RabbitMQ |
password | string RabbitMQ |
vhost | string RabbitMQ |
Compression
Field | Description |
---|---|
method | enum Method Compression method to use for the specified combination of min_part_size and min_part_size_ratio . |
min_part_size | int64 Minimum size of a part of a table. The minimum value is 1. |
min_part_size_ratio | double Minimum ratio of a part relative to the size of all the data in the table. |
level | google.protobuf.Int64Value The minimum value is 0. |
ExternalDictionary
Field | Description |
---|---|
name | string Required. Name of the external dictionary. |
structure | Structure Required. Set of attributes for the external dictionary. For in-depth description, see ClickHouse documentation |
layout | Layout Required. Layout for storing the dictionary in memory. For in-depth description, see ClickHouse documentation |
lifetime | oneof: fixed_lifetime or lifetime_range Setting for the period of time between dictionary updates. For details, see ClickHouse documentation |
fixed_lifetime | int64 Fixed interval between dictionary updates. |
lifetime_range | Range Range of intervals between dictionary updates for ClickHouse to choose from. |
source | oneof: http_source , mysql_source , clickhouse_source , mongodb_source or postgresql_source Description of the source for the external dictionary. |
http_source | HttpSource HTTP source for the dictionary. |
mysql_source | MysqlSource MySQL source for the dictionary. |
clickhouse_source | ClickhouseSource ClickHouse source for the dictionary. |
mongodb_source | MongodbSource MongoDB source for the dictionary. |
postgresql_source | PostgresqlSource PostgreSQL source for the dictionary. |
HttpSource
Field | Description |
---|---|
url | string Required. URL of the source dictionary available over HTTP. |
format | string Required. The data format. Valid values are all formats supported by ClickHouse SQL dialect. |
MysqlSource
Field | Description |
---|---|
db | string Required. Name of the MySQL database to connect to. |
table | string Required. Name of the database table to use as a ClickHouse dictionary. |
port | int64 Default port to use when connecting to a replica of the dictionary source. Acceptable values are 0 to 65535, inclusive. |
user | string Name of the default user for replicas of the dictionary source. |
password | string Password of the default user for replicas of the dictionary source. |
replicas[] | Replica List of MySQL replicas of the database used as dictionary source. The number of elements must be greater than 0. |
where | string Selection criteria for the data in the specified MySQL table. |
invalidate_query | string Query for checking the dictionary status, to pull only updated data. For more details, see ClickHouse documentation on dictionaries |
Replica
Field | Description |
---|---|
host | string Required. MySQL host of the replica. The maximum string length in characters is 253. |
priority | int64 Required. The priority of the replica that ClickHouse takes into account when connecting. Replica with the highest priority should have this field set to the lowest number. Value must be greater than 0. |
port | int64 Port to use when connecting to the replica. If a port is not specified for a replica, ClickHouse uses the port specified for the source. Acceptable values are 0 to 65535, inclusive. |
user | string Name of the MySQL database user. |
password | string Password of the MySQL database user. |
ClickhouseSource
Field | Description |
---|---|
db | string Required. Name of the ClickHouse database. |
table | string Required. Name of the table in the specified database to be used as the dictionary source. |
host | string Required. ClickHouse host of the specified database. The maximum string length in characters is 253. |
port | int64 Port to use when connecting to the host. Acceptable values are 0 to 65535, inclusive. |
user | string Required. Name of the ClickHouse database user. |
password | string Password of the ClickHouse database user. |
where | string Selection criteria for the data in the specified ClickHouse table. |
MongodbSource
Field | Description |
---|---|
db | string Required. Name of the MongoDB database. |
collection | string Required. Name of the collection in the specified database to be used as the dictionary source. |
host | string Required. MongoDB host of the specified database. The maximum string length in characters is 253. |
port | int64 Port to use when connecting to the host. Acceptable values are 0 to 65535, inclusive. |
user | string Required. Name of the MongoDB database user. |
password | string Password of the MongoDB database user. |
options | string |
PostgresqlSource
Field | Description |
---|---|
db | string Required. Name of the PostrgreSQL database. |
table | string Required. Name of the table in the specified database to be used as the dictionary source. |
hosts[] | string Name of the PostrgreSQL host The number of elements must be greater than 0. |
port | int64 Port to use when connecting to the host. Acceptable values are 0 to 65535, inclusive. |
user | string Required. Name of the PostrgreSQL database user. |
password | string Password of the PostrgreSQL database user. |
invalidate_query | string Query for checking the dictionary status, to pull only updated data. For more details, see ClickHouse documentation on dictionaries |
ssl_mode | enum SslMode Mode of SSL TCP/IP connection to the PostgreSQL host. For more details, see PostgreSQL documentation
|
Structure
Field | Description |
---|---|
id | Id Single numeric key column for the dictionary. |
key | Key Composite key for the dictionary, containing of one or more key columns. For details, see ClickHouse documentation |
range_min | Attribute Field holding the beginning of the range for dictionaries with RANGE_HASHED layout. For details, see ClickHouse documentation |
range_max | Attribute Field holding the end of the range for dictionaries with RANGE_HASHED layout. For details, see ClickHouse documentation |
attributes[] | Attribute Description of the fields available for database queries. For details, see ClickHouse documentation |
Attribute
Field | Description |
---|---|
name | string Required. Name of the column. |
type | string Required. Type of the column. |
null_value | string Default value for an element without data (for example, an empty string). |
expression | string Expression, describing the attribute, if applicable. |
hierarchical | bool Indication of hierarchy support. Default value: false . |
injective | bool Indication of injective mapping "id -> attribute". Default value: false . |
Id
Field | Description |
---|---|
name | string Required. Name of the numeric key. |
Key
Field | Description |
---|---|
attributes[] | Attribute Attributes of a complex key. The number of elements must be greater than 0. |
Layout
Field | Description |
---|---|
type | enum Type Required. Layout type for an external dictionary.
|
size_in_cells | int64 Number of cells in the cache. Rounded up to a power of two. Applicable only for CACHE and COMPLEX_KEY_CACHE layout types. |
Range
Field | Description |
---|---|
min | int64 Minimum dictionary lifetime. |
max | int64 Maximum dictionary lifetime. |
GraphiteRollup
Field | Description |
---|---|
name | string Required. Name for the specified combination of settings for Graphite rollup. |
patterns[] | Pattern Pattern to use for the rollup. The number of elements must be greater than 0. |
Pattern
Field | Description |
---|---|
regexp | string Pattern for metric names. |
function | string Required. Name of the aggregating function to apply to data of the age specified in retention . |
retention[] | Retention Age of data to use for thinning. The number of elements must be greater than 0. |
Retention
Field | Description |
---|---|
age | int64 Minimum age of the data in seconds. The minimum value is 0. |
precision | int64 Precision of determining the age of the data, in seconds. Value must be greater than 0. |
DeleteShard
Deletes the specified shard.
rpc DeleteShard (DeleteClusterShardRequest) returns (operation.Operation)
Metadata and response of Operation:
Operation.metadata:DeleteClusterShardMetadata
Operation.response:google.protobuf.Empty
DeleteClusterShardRequest
Field | Description |
---|---|
cluster_id | string Required. ID of the ClickHouse cluster the shard belongs to. To get the cluster ID, use a ClusterService.List request. The maximum string length in characters is 50. |
shard_name | string Required. Name of the shard to be deleted. To get the name of a shard, use a ClusterService.ListShards request. The maximum string length in characters is 63. Value must match the regular expression [a-zA-Z0-9_-]* . |
Operation
Field | Description |
---|---|
id | string ID of the operation. |
description | string Description of the operation. 0-256 characters long. |
created_at | google.protobuf.Timestamp Creation timestamp. |
created_by | string ID of the user or service account who initiated the operation. |
modified_at | google.protobuf.Timestamp The time when the Operation resource was last modified. |
done | bool If the value is false , it means the operation is still in progress. If true , the operation is completed, and either error or response is available. |
metadata | google.protobuf.Any Service-specific metadata associated with the operation. It typically contains the ID of the target resource that the operation is performed on. Any method that returns a long-running operation should document the metadata type, if any. |
result | oneof: error or response The operation result. If done == false and there was no failure detected, neither error nor response is set. If done == false and there was a failure detected, error is set. If done == true , exactly one of error or response is set. |
error | google.rpc.Status The error result of the operation in case of failure or cancellation. |
response | google.protobuf.Any if operation finished successfully. |
DeleteClusterShardMetadata
Field | Description |
---|---|
cluster_id | string ID of the cluster that contains the shard being deleted. |
shard_name | string Name of the shard being deleted. |
GetShardGroup
Returns the specified shard group.
rpc GetShardGroup (GetClusterShardGroupRequest) returns (ShardGroup)
GetClusterShardGroupRequest
Field | Description |
---|---|
cluster_id | string Required. ID of the cluster that the shard group belongs to. To get the cluster ID, make a ClusterService.List request. The maximum string length in characters is 50. |
shard_group_name | string Required. Name of the shard group to request information about. To get the name of a shard group, make a ClusterService.ListShardGroups request. The maximum string length in characters is 63. Value must match the regular expression [a-zA-Z0-9_-]* . |
ShardGroup
Field | Description |
---|---|
name | string Name of the shard group. |
cluster_id | string ID of the ClickHouse cluster that the shard group belongs to. |
description | string Description of the shard group. 0-256 characters long. |
shard_names[] | string List of shard names contained in the shard group. |
ListShardGroups
Retrieves a list of shard groups that belong to specified cluster.
rpc ListShardGroups (ListClusterShardGroupsRequest) returns (ListClusterShardGroupsResponse)
ListClusterShardGroupsRequest
Field | Description |
---|---|
cluster_id | string Required. ID of the cluster that the shard group belongs to. To get the cluster ID, make a ClusterService.List request. The maximum string length in characters is 50. |
page_size | int64 The maximum number of results per page to return. If the number of available results is larger than page_size , the service returns a ListClusterShardGroupsResponse.next_page_token that can be used to get the next page of results in subsequent list requests. Acceptable values are 0 to 1000, inclusive. |
page_token | string Page token. To get the next page of results, set page_token to the ListClusterShardGroupsResponse.next_page_token returned by the previous list request. The maximum string length in characters is 100. |
ListClusterShardGroupsResponse
Field | Description |
---|---|
shard_groups[] | ShardGroup List of ClickHouse cluster's shard groups. |
next_page_token | string This token allows you to get the next page of results for list requests. If the number of results is larger than ListClusterShardGroupsRequest.page_size, use the next_page_token as the value for the ListClusterShardGroupsRequest.page_token parameter in the next list request. Each subsequent list request will have its own next_page_token to continue paging through the results. |
ShardGroup
Field | Description |
---|---|
name | string Name of the shard group. |
cluster_id | string ID of the ClickHouse cluster that the shard group belongs to. |
description | string Description of the shard group. 0-256 characters long. |
shard_names[] | string List of shard names contained in the shard group. |
CreateShardGroup
Creates a new shard group in the specified cluster.
rpc CreateShardGroup (CreateClusterShardGroupRequest) returns (operation.Operation)
Metadata and response of Operation:
Operation.metadata:CreateClusterShardGroupMetadata
Operation.response:ShardGroup
CreateClusterShardGroupRequest
Field | Description |
---|---|
cluster_id | string Required. ID of the ClickHouse cluster to add a shard group to. To get the cluster ID, make a ClusterService.List request. The maximum string length in characters is 50. |
shard_group_name | string Required. Name for the new shard group. The maximum string length in characters is 63. Value must match the regular expression [a-zA-Z0-9_-]* . |
description | string Description of the new shard group. 0-256 characters long. |
shard_names[] | string List of shard names that should be put into the new group. To get the list, make a ClusterService.ListShardGroups request. |
Operation
Field | Description |
---|---|
id | string ID of the operation. |
description | string Description of the operation. 0-256 characters long. |
created_at | google.protobuf.Timestamp Creation timestamp. |
created_by | string ID of the user or service account who initiated the operation. |
modified_at | google.protobuf.Timestamp The time when the Operation resource was last modified. |
done | bool If the value is false , it means the operation is still in progress. If true , the operation is completed, and either error or response is available. |
metadata | google.protobuf.Any Service-specific metadata associated with the operation. It typically contains the ID of the target resource that the operation is performed on. Any method that returns a long-running operation should document the metadata type, if any. |
result | oneof: error or response The operation result. If done == false and there was no failure detected, neither error nor response is set. If done == false and there was a failure detected, error is set. If done == true , exactly one of error or response is set. |
error | google.rpc.Status The error result of the operation in case of failure or cancellation. |
response | google.protobuf.Any if operation finished successfully. |
CreateClusterShardGroupMetadata
Field | Description |
---|---|
cluster_id | string ID of the cluster to add a shard group to. |
shard_group_name | string Name of the shard group that is being added. |
ShardGroup
Field | Description |
---|---|
name | string Name of the shard group. |
cluster_id | string ID of the ClickHouse cluster that the shard group belongs to. |
description | string Description of the shard group. 0-256 characters long. |
shard_names[] | string List of shard names contained in the shard group. |
UpdateShardGroup
Updates the specified shard group.
rpc UpdateShardGroup (UpdateClusterShardGroupRequest) returns (operation.Operation)
Metadata and response of Operation:
Operation.metadata:UpdateClusterShardGroupMetadata
Operation.response:ShardGroup
UpdateClusterShardGroupRequest
Field | Description |
---|---|
cluster_id | string Required. ID of the ClickHouse cluster that contains the shard group to update. To get the cluster ID, make a ClusterService.List request. The maximum string length in characters is 50. |
shard_group_name | string Required. Name of the shard group that should be updated. To get the name, make a ClusterService.ListShardGroups request. The maximum string length in characters is 63. Value must match the regular expression [a-zA-Z0-9_-]* . |
update_mask | google.protobuf.FieldMask |
description | string Updated description of the shard group. 0-256 characters long. |
shard_names[] | string Updated list of shard names that belongs to the shard group. |
Operation
Field | Description |
---|---|
id | string ID of the operation. |
description | string Description of the operation. 0-256 characters long. |
created_at | google.protobuf.Timestamp Creation timestamp. |
created_by | string ID of the user or service account who initiated the operation. |
modified_at | google.protobuf.Timestamp The time when the Operation resource was last modified. |
done | bool If the value is false , it means the operation is still in progress. If true , the operation is completed, and either error or response is available. |
metadata | google.protobuf.Any Service-specific metadata associated with the operation. It typically contains the ID of the target resource that the operation is performed on. Any method that returns a long-running operation should document the metadata type, if any. |
result | oneof: error or response The operation result. If done == false and there was no failure detected, neither error nor response is set. If done == false and there was a failure detected, error is set. If done == true , exactly one of error or response is set. |
error | google.rpc.Status The error result of the operation in case of failure or cancellation. |
response | google.protobuf.Any if operation finished successfully. |
UpdateClusterShardGroupMetadata
Field | Description |
---|---|
cluster_id | string ID of the cluster that contains the shard group being updated. |
shard_group_name | string Name of the shard group that is being updated. |
ShardGroup
Field | Description |
---|---|
name | string Name of the shard group. |
cluster_id | string ID of the ClickHouse cluster that the shard group belongs to. |
description | string Description of the shard group. 0-256 characters long. |
shard_names[] | string List of shard names contained in the shard group. |
DeleteShardGroup
Deletes the specified shard group.
rpc DeleteShardGroup (DeleteClusterShardGroupRequest) returns (operation.Operation)
Metadata and response of Operation:
Operation.metadata:DeleteClusterShardGroupMetadata
Operation.response:google.protobuf.Empty
DeleteClusterShardGroupRequest
Field | Description |
---|---|
cluster_id | string Required. ID of the ClickHouse cluster that contains the shard group to delete. To get the cluster ID, make a ClusterService.List request. The maximum string length in characters is 50. |
shard_group_name | string Required. Name of the shard group that should be deleted. To get the name, make a ClusterService.ListShardGroups request. The maximum string length in characters is 63. Value must match the regular expression [a-zA-Z0-9_-]* . |
Operation
Field | Description |
---|---|
id | string ID of the operation. |
description | string Description of the operation. 0-256 characters long. |
created_at | google.protobuf.Timestamp Creation timestamp. |
created_by | string ID of the user or service account who initiated the operation. |
modified_at | google.protobuf.Timestamp The time when the Operation resource was last modified. |
done | bool If the value is false , it means the operation is still in progress. If true , the operation is completed, and either error or response is available. |
metadata | google.protobuf.Any Service-specific metadata associated with the operation. It typically contains the ID of the target resource that the operation is performed on. Any method that returns a long-running operation should document the metadata type, if any. |
result | oneof: error or response The operation result. If done == false and there was no failure detected, neither error nor response is set. If done == false and there was a failure detected, error is set. If done == true , exactly one of error or response is set. |
error | google.rpc.Status The error result of the operation in case of failure or cancellation. |
response | google.protobuf.Any if operation finished successfully. |
DeleteClusterShardGroupMetadata
Field | Description |
---|---|
cluster_id | string ID of the cluster that contains the shard group being deleted. |
shard_group_name | string Name of the shard group that is being deleted. |
ListExternalDictionaries
Retrieves a list of external dictionaries that belong to specified cluster.
rpc ListExternalDictionaries (ListClusterExternalDictionariesRequest) returns (ListClusterExternalDictionariesResponse)
ListClusterExternalDictionariesRequest
Field | Description |
---|---|
cluster_id | string Required. ID of the cluster that the external dictionaries belong to. The maximum string length in characters is 50. |
page_size | int64 The maximum number of results per page to return. If the number of available results is larger than page_size , the service returns a [ListClusterExternalDictionaryResponse.next_page_token] that can be used to get the next page of results in subsequent list requests. Acceptable values are 0 to 1000, inclusive. |
page_token | string Page token. To get the next page of results, set page_token to the [ListClusterExternalDictionaryResponse.next_page_token] returned by a previous list request. The maximum string length in characters is 100. |
ListClusterExternalDictionariesResponse
Field | Description |
---|---|
external_dictionaries[] | config.ClickhouseConfig.ExternalDictionary List of ClickHouse Cluster external dictionaries. |
next_page_token | string This token allows you to get the next page of results for list requests. If the number of results is larger than [ListClusterExternalDictionaryRequest.page_size], use the next_page_token as the value for the [ListClusterExternalDictionaryRequest.page_token] parameter in the next list request. Each subsequent list request will have its own next_page_token to continue paging through the results. |
CreateExternalDictionary
Creates an external dictionary for the specified ClickHouse cluster.
rpc CreateExternalDictionary (CreateClusterExternalDictionaryRequest) returns (operation.Operation)
Metadata and response of Operation:
Operation.metadata:CreateClusterExternalDictionaryMetadata
Operation.response:Cluster
CreateClusterExternalDictionaryRequest
Field | Description |
---|---|
cluster_id | string Required. ID of the ClickHouse cluster to create the external dictionary for. To get the cluster ID, use a ClusterService.List request. The maximum string length in characters is 50. |
external_dictionary | config.ClickhouseConfig.ExternalDictionary Configuration of the external dictionary. |
Operation
Field | Description |
---|---|
id | string ID of the operation. |
description | string Description of the operation. 0-256 characters long. |
created_at | google.protobuf.Timestamp Creation timestamp. |
created_by | string ID of the user or service account who initiated the operation. |
modified_at | google.protobuf.Timestamp The time when the Operation resource was last modified. |
done | bool If the value is false , it means the operation is still in progress. If true , the operation is completed, and either error or response is available. |
metadata | google.protobuf.Any Service-specific metadata associated with the operation. It typically contains the ID of the target resource that the operation is performed on. Any method that returns a long-running operation should document the metadata type, if any. |
result | oneof: error or response The operation result. If done == false and there was no failure detected, neither error nor response is set. If done == false and there was a failure detected, error is set. If done == true , exactly one of error or response is set. |
error | google.rpc.Status The error result of the operation in case of failure or cancellation. |
response | google.protobuf.Any if operation finished successfully. |
CreateClusterExternalDictionaryMetadata
Field | Description |
---|---|
cluster_id | string ID of the cluster for which an external dictionary is being created. |
Cluster
Field | Description |
---|---|
id | string ID of the ClickHouse cluster. This ID is assigned by MDB at creation time. |
folder_id | string ID of the folder that the ClickHouse cluster belongs to. |
created_at | google.protobuf.Timestamp Creation timestamp in RFC3339 |
name | string Name of the ClickHouse cluster. The name is unique within the folder. 1-63 characters long. |
description | string Description of the ClickHouse cluster. 0-256 characters long. |
labels | map<string,string> Custom labels for the ClickHouse cluster as key:value pairs. Maximum 64 per resource. |
environment | enum Environment Deployment environment of the ClickHouse cluster.
|
monitoring[] | Monitoring Description of monitoring systems relevant to the ClickHouse cluster. |
config | ClusterConfig Configuration of the ClickHouse cluster. |
network_id | string ID of the network that the cluster belongs to. |
health | enum Health Aggregated cluster health.
|
status | enum Status Current state of the cluster.
|
service_account_id | string ID of the service account used for access to Object Storage. |
maintenance_window | MaintenanceWindow Maintenance window for the cluster. |
planned_operation | MaintenanceOperation Planned maintenance operation to be started for the cluster within the nearest maintenance_window . |
security_group_ids[] | string User security groups |
deletion_protection | bool Deletion Protection inhibits deletion of the cluster |
Monitoring
Field | Description |
---|---|
name | string Name of the monitoring system. |
description | string Description of the monitoring system. |
link | string Link to the monitoring system charts for the ClickHouse cluster. |
ClusterConfig
Field | Description |
---|---|
version | string Version of the ClickHouse server software. |
clickhouse | Clickhouse Configuration and resource allocation for ClickHouse hosts. |
zookeeper | Zookeeper Configuration and resource allocation for ZooKeeper hosts. |
backup_window_start | google.type.TimeOfDay Time to start the daily backup, in the UTC timezone. |
access | Access Access policy for external services. |
cloud_storage | CloudStorage |
sql_database_management | google.protobuf.BoolValue Whether database management through SQL commands is enabled. |
sql_user_management | google.protobuf.BoolValue Whether user management through SQL commands is enabled. |
embedded_keeper | google.protobuf.BoolValue Whether cluster should use embedded Keeper instead of Zookeeper. |
Clickhouse
Field | Description |
---|---|
config | config.ClickhouseConfigSet Configuration settings of a ClickHouse server. |
resources | Resources Resources allocated to ClickHouse hosts. |
Zookeeper
Field | Description |
---|---|
resources | Resources Resources allocated to ZooKeeper hosts. |
Access
Field | Description |
---|---|
data_lens | bool Allow to export data from the cluster to DataLens. |
web_sql | bool Allow SQL queries to the cluster databases from the management console. See SQL queries in the management console for more details. |
metrika | bool Allow to import data from Yandex Metrica and AppMetrica to the cluster. See AppMetrica documentation |
serverless | bool Allow access to cluster for Serverless. |
data_transfer | bool Allow access for DataTransfer |
yandex_query | bool Allow access for Query |
CloudStorage
Field | Description |
---|---|
enabled | bool Whether to use Object Storage for storing ClickHouse data. |
move_factor | google.protobuf.DoubleValue Acceptable values are 0 to 1, inclusive. |
data_cache_enabled | google.protobuf.BoolValue |
data_cache_max_size | google.protobuf.Int64Value |
prefer_not_to_merge | google.protobuf.BoolValue |
ClickhouseConfigSet
Field | Description |
---|---|
effective_config | ClickhouseConfig Required. Effective settings for a ClickHouse cluster (a combination of settings defined in user_config and default_config ). |
user_config | ClickhouseConfig User-defined settings for a ClickHouse cluster. |
default_config | ClickhouseConfig Default configuration for a ClickHouse cluster. |
ClickhouseConfig
Field | Description |
---|---|
log_level | enum LogLevel Logging level for the ClickHouse cluster. Possible values: TRACE, DEBUG, INFORMATION, WARNING, ERROR. |
merge_tree | MergeTree Settings for the MergeTree engine. See description in ClickHouse documentation |
compression[] | Compression Compression settings for the ClickHouse cluster. See in-depth description in ClickHouse documentation |
dictionaries[] | ExternalDictionary Configuration of external dictionaries to be used by the ClickHouse cluster. See in-depth description in ClickHouse documentation |
graphite_rollup[] | GraphiteRollup Settings for thinning Graphite data. See in-depth description in ClickHouse documentation |
kafka | Kafka |
kafka_topics[] | KafkaTopic |
rabbitmq | Rabbitmq |
max_connections | google.protobuf.Int64Value Maximum number of inbound connections. The minimum value is 10. |
max_concurrent_queries | google.protobuf.Int64Value Maximum number of simultaneously processed requests. The minimum value is 10. |
keep_alive_timeout | google.protobuf.Int64Value Number of milliseconds that ClickHouse waits for incoming requests before closing the connection. |
uncompressed_cache_size | google.protobuf.Int64Value Cache size (in bytes) for uncompressed data used by MergeTree tables. |
mark_cache_size | google.protobuf.Int64Value Approximate size (in bytes) of the cache of "marks" used by MergeTree tables. Value must be greater than 0. |
max_table_size_to_drop | google.protobuf.Int64Value Maximum size of the table that can be deleted using a DROP query. |
max_partition_size_to_drop | google.protobuf.Int64Value Maximum size of the partition that can be deleted using a DROP query. |
builtin_dictionaries_reload_interval | google.protobuf.Int64Value The setting is deprecated and has no effect. |
timezone | string The server's time zone to be used in DateTime fields conversions. Specified as an IANA identifier. |
geobase_enabled | google.protobuf.BoolValue Enable or disable geobase. |
geobase_uri | string Address of the archive with the user geobase in Object Storage. |
query_log_retention_size | google.protobuf.Int64Value The maximum size that query_log can grow to before old data will be removed. If set to 0, automatic removal of query_log data based on size is disabled. |
query_log_retention_time | google.protobuf.Int64Value The maximum time that query_log records will be retained before removal. If set to 0, automatic removal of query_log data based on time is disabled. |
query_thread_log_enabled | google.protobuf.BoolValue Whether query_thread_log system table is enabled. |
query_thread_log_retention_size | google.protobuf.Int64Value The maximum size that query_thread_log can grow to before old data will be removed. If set to 0, automatic removal of query_thread_log data based on size is disabled. |
query_thread_log_retention_time | google.protobuf.Int64Value The maximum time that query_thread_log records will be retained before removal. If set to 0, automatic removal of query_thread_log data based on time is disabled. |
part_log_retention_size | google.protobuf.Int64Value The maximum size that part_log can grow to before old data will be removed. If set to 0, automatic removal of part_log data based on size is disabled. |
part_log_retention_time | google.protobuf.Int64Value The maximum time that part_log records will be retained before removal. If set to 0, automatic removal of part_log data based on time is disabled. |
metric_log_enabled | google.protobuf.BoolValue Whether metric_log system table is enabled. |
metric_log_retention_size | google.protobuf.Int64Value The maximum size that metric_log can grow to before old data will be removed. If set to 0, automatic removal of metric_log data based on size is disabled. |
metric_log_retention_time | google.protobuf.Int64Value The maximum time that metric_log records will be retained before removal. If set to 0, automatic removal of metric_log data based on time is disabled. |
trace_log_enabled | google.protobuf.BoolValue Whether trace_log system table is enabled. |
trace_log_retention_size | google.protobuf.Int64Value The maximum size that trace_log can grow to before old data will be removed. If set to 0, automatic removal of trace_log data based on size is disabled. |
trace_log_retention_time | google.protobuf.Int64Value The maximum time that trace_log records will be retained before removal. If set to 0, automatic removal of trace_log data based on time is disabled. |
text_log_enabled | google.protobuf.BoolValue Whether text_log system table is enabled. |
text_log_retention_size | google.protobuf.Int64Value The maximum size that text_log can grow to before old data will be removed. If set to 0, automatic removal of text_log data based on size is disabled. |
text_log_retention_time | google.protobuf.Int64Value The maximum time that text_log records will be retained before removal. If set to 0, automatic removal of text_log data based on time is disabled. |
text_log_level | enum LogLevel Logging level for text_log system table. Possible values: TRACE, DEBUG, INFORMATION, WARNING, ERROR. |
opentelemetry_span_log_enabled | google.protobuf.BoolValue Enable or disable opentelemetry_span_log system table. Default value: false. |
opentelemetry_span_log_retention_size | google.protobuf.Int64Value The maximum size that opentelemetry_span_log can grow to before old data will be removed. If set to 0 (default), automatic removal of opentelemetry_span_log data based on size is disabled. The minimum value is 0. |
opentelemetry_span_log_retention_time | google.protobuf.Int64Value The maximum time that opentelemetry_span_log records will be retained before removal. If set to 0, automatic removal of opentelemetry_span_log data based on time is disabled. The minimum value is 0. |
query_views_log_enabled | google.protobuf.BoolValue Enable or disable query_views_log system table. Default value: false. |
query_views_log_retention_size | google.protobuf.Int64Value The maximum size that query_views_log can grow to before old data will be removed. If set to 0 (default), automatic removal of query_views_log data based on size is disabled. The minimum value is 0. |
query_views_log_retention_time | google.protobuf.Int64Value The maximum time that query_views_log records will be retained before removal. If set to 0, automatic removal of query_views_log data based on time is disabled. The minimum value is 0. |
asynchronous_metric_log_enabled | google.protobuf.BoolValue Enable or disable asynchronous_metric_log system table. Default value: false. |
asynchronous_metric_log_retention_size | google.protobuf.Int64Value The maximum size that asynchronous_metric_log can grow to before old data will be removed. If set to 0 (default), automatic removal of asynchronous_metric_log data based on size is disabled. The minimum value is 0. |
asynchronous_metric_log_retention_time | google.protobuf.Int64Value The maximum time that asynchronous_metric_log records will be retained before removal. If set to 0, automatic removal of asynchronous_metric_log data based on time is disabled. The minimum value is 0. |
session_log_enabled | google.protobuf.BoolValue Enable or disable session_log system table. Default value: false. |
session_log_retention_size | google.protobuf.Int64Value The maximum size that session_log can grow to before old data will be removed. If set to 0 (default), automatic removal of session_log data based on size is disabled. The minimum value is 0. |
session_log_retention_time | google.protobuf.Int64Value The maximum time that session_log records will be retained before removal. If set to 0, automatic removal of session_log data based on time is disabled. The minimum value is 0. |
zookeeper_log_enabled | google.protobuf.BoolValue Enable or disable zookeeper_log system table. Default value: false. |
zookeeper_log_retention_size | google.protobuf.Int64Value The maximum size that zookeeper_log can grow to before old data will be removed. If set to 0 (default), automatic removal of zookeeper_log data based on size is disabled. The minimum value is 0. |
zookeeper_log_retention_time | google.protobuf.Int64Value The maximum time that zookeeper_log records will be retained before removal. If set to 0, automatic removal of zookeeper_log data based on time is disabled. The minimum value is 0. |
asynchronous_insert_log_enabled | google.protobuf.BoolValue Enable or disable asynchronous_insert_log system table. Default value: false. Minimal required ClickHouse version: 22.10. |
asynchronous_insert_log_retention_size | google.protobuf.Int64Value The maximum size that asynchronous_insert_log can grow to before old data will be removed. If set to 0 (default), automatic removal of asynchronous_insert_log data based on size is disabled. The minimum value is 0. |
asynchronous_insert_log_retention_time | google.protobuf.Int64Value The maximum time that asynchronous_insert_log records will be retained before removal. If set to 0, automatic removal of asynchronous_insert_log data based on time is disabled. The minimum value is 0. |
background_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_merges_mutations_concurrency_ratio | google.protobuf.Int64Value Sets a ratio between the number of threads and the number of background merges and mutations that can be executed concurrently. For example, if the ratio equals to 2 and background_pool_size is set to 16 then ClickHouse can execute 32 background merges concurrently. This is possible, because background operations could be suspended and postponed. This is needed to give small merges more execution priority. You can only increase this ratio at runtime. To lower it you have to restart the server. The same as for background_pool_size setting background_merges_mutations_concurrency_ratio could be applied from the default profile for backward compatibility. Default: 2 See in-depth description in ClickHouse documentation |
background_schedule_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_fetches_pool_size | google.protobuf.Int64Value Sets the number of threads performing background fetches for tables with ReplicatedMergeTree engines. Default value: 8. More info see in ClickHouse documentation |
background_move_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_distributed_schedule_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_buffer_flush_schedule_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_message_broker_schedule_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_common_pool_size | google.protobuf.Int64Value The maximum number of threads that will be used for performing a variety of operations (mostly garbage collection) for *MergeTree-engine tables in a background. Default: 8 See in-depth description in ClickHouse documentation |
default_database | google.protobuf.StringValue The default database. To get a list of cluster databases, see Yandex Managed ClickHouse documentation. |
total_memory_profiler_step | google.protobuf.Int64Value Sets the memory size (in bytes) for a stack trace at every peak allocation step. Default value: 4194304. More info see in ClickHouse documentation |
total_memory_tracker_sample_probability | google.protobuf.DoubleValue |
MergeTree
Field | Description |
---|---|
replicated_deduplication_window | google.protobuf.Int64Value Number of blocks of hashes to keep in ZooKeeper. |
replicated_deduplication_window_seconds | google.protobuf.Int64Value Period of time to keep blocks of hashes for. |
parts_to_delay_insert | google.protobuf.Int64Value If table contains at least that many active parts in single partition, artificially slow down insert into table. |
parts_to_throw_insert | google.protobuf.Int64Value If more than this number active parts in single partition, throw 'Too many parts ...' exception. |
inactive_parts_to_delay_insert | google.protobuf.Int64Value |
inactive_parts_to_throw_insert | google.protobuf.Int64Value |
max_replicated_merges_in_queue | google.protobuf.Int64Value How many tasks of merging and mutating parts are allowed simultaneously in ReplicatedMergeTree queue. |
number_of_free_entries_in_pool_to_lower_max_size_of_merge | google.protobuf.Int64Value If there is less than specified number of free entries in background pool (or replicated queue), start to lower maximum size of merge to process. |
max_bytes_to_merge_at_min_space_in_pool | google.protobuf.Int64Value Maximum in total size of parts to merge, when there are minimum free threads in background pool (or entries in replication queue). |
max_bytes_to_merge_at_max_space_in_pool | google.protobuf.Int64Value |
min_bytes_for_wide_part | google.protobuf.Int64Value Minimum number of bytes in a data part that can be stored in Wide format. More info see in ClickHouse documentation |
min_rows_for_wide_part | google.protobuf.Int64Value Minimum number of rows in a data part that can be stored in Wide format. More info see in ClickHouse documentation |
ttl_only_drop_parts | google.protobuf.BoolValue Enables or disables complete dropping of data parts where all rows are expired in MergeTree tables. More info see in ClickHouse documentation |
allow_remote_fs_zero_copy_replication | google.protobuf.BoolValue |
merge_with_ttl_timeout | google.protobuf.Int64Value |
merge_with_recompression_ttl_timeout | google.protobuf.Int64Value |
max_parts_in_total | google.protobuf.Int64Value |
max_number_of_merges_with_ttl_in_pool | google.protobuf.Int64Value |
cleanup_delay_period | google.protobuf.Int64Value |
number_of_free_entries_in_pool_to_execute_mutation | google.protobuf.Int64Value |
max_avg_part_size_for_too_many_parts | google.protobuf.Int64Value The 'too many parts' check according to 'parts_to_delay_insert' and 'parts_to_throw_insert' will be active only if the average part size (in the relevant partition) is not larger than the specified threshold. If it is larger than the specified threshold, the INSERTs will be neither delayed or rejected. This allows to have hundreds of terabytes in a single table on a single server if the parts are successfully merged to larger parts. This does not affect the thresholds on inactive parts or total parts. Default: 1 GiB Min version: 22.10 See in-depth description in ClickHouse GitHub |
min_age_to_force_merge_seconds | google.protobuf.Int64Value Merge parts if every part in the range is older than the value of min_age_to_force_merge_seconds. Default: 0 - disabled Min_version: 22.10 See in-depth description in ClickHouse documentation |
min_age_to_force_merge_on_partition_only | google.protobuf.BoolValue Whether min_age_to_force_merge_seconds should be applied only on the entire partition and not on subset. Default: false Min_version: 22.11 See in-depth description in ClickHouse documentation |
merge_selecting_sleep_ms | google.protobuf.Int64Value Sleep time for merge selecting when no part is selected. A lower setting triggers selecting tasks in background_schedule_pool frequently, which results in a large number of requests to ClickHouse Keeper in large-scale clusters. Default: 5000 Min_version: 21.10 See in-depth description in ClickHouse documentation |
Kafka
Field | Description |
---|---|
security_protocol | enum SecurityProtocol |
sasl_mechanism | enum SaslMechanism |
sasl_username | string |
sasl_password | string |
enable_ssl_certificate_verification | google.protobuf.BoolValue |
max_poll_interval_ms | google.protobuf.Int64Value The minimum value is 0. |
session_timeout_ms | google.protobuf.Int64Value The minimum value is 0. |
KafkaTopic
Field | Description |
---|---|
name | string Required. |
settings | Kafka Required. |
Rabbitmq
Field | Description |
---|---|
username | string RabbitMQ |
password | string RabbitMQ |
vhost | string RabbitMQ |
Compression
Field | Description |
---|---|
method | enum Method Compression method to use for the specified combination of min_part_size and min_part_size_ratio . |
min_part_size | int64 Minimum size of a part of a table. The minimum value is 1. |
min_part_size_ratio | double Minimum ratio of a part relative to the size of all the data in the table. |
level | google.protobuf.Int64Value The minimum value is 0. |
ExternalDictionary
Field | Description |
---|---|
name | string Required. Name of the external dictionary. |
structure | Structure Required. Set of attributes for the external dictionary. For in-depth description, see ClickHouse documentation |
layout | Layout Required. Layout for storing the dictionary in memory. For in-depth description, see ClickHouse documentation |
lifetime | oneof: fixed_lifetime or lifetime_range Setting for the period of time between dictionary updates. For details, see ClickHouse documentation |
fixed_lifetime | int64 Fixed interval between dictionary updates. |
lifetime_range | Range Range of intervals between dictionary updates for ClickHouse to choose from. |
source | oneof: http_source , mysql_source , clickhouse_source , mongodb_source or postgresql_source Description of the source for the external dictionary. |
http_source | HttpSource HTTP source for the dictionary. |
mysql_source | MysqlSource MySQL source for the dictionary. |
clickhouse_source | ClickhouseSource ClickHouse source for the dictionary. |
mongodb_source | MongodbSource MongoDB source for the dictionary. |
postgresql_source | PostgresqlSource PostgreSQL source for the dictionary. |
HttpSource
Field | Description |
---|---|
url | string Required. URL of the source dictionary available over HTTP. |
format | string Required. The data format. Valid values are all formats supported by ClickHouse SQL dialect. |
MysqlSource
Field | Description |
---|---|
db | string Required. Name of the MySQL database to connect to. |
table | string Required. Name of the database table to use as a ClickHouse dictionary. |
port | int64 Default port to use when connecting to a replica of the dictionary source. Acceptable values are 0 to 65535, inclusive. |
user | string Name of the default user for replicas of the dictionary source. |
password | string Password of the default user for replicas of the dictionary source. |
replicas[] | Replica List of MySQL replicas of the database used as dictionary source. The number of elements must be greater than 0. |
where | string Selection criteria for the data in the specified MySQL table. |
invalidate_query | string Query for checking the dictionary status, to pull only updated data. For more details, see ClickHouse documentation on dictionaries |
Replica
Field | Description |
---|---|
host | string Required. MySQL host of the replica. The maximum string length in characters is 253. |
priority | int64 Required. The priority of the replica that ClickHouse takes into account when connecting. Replica with the highest priority should have this field set to the lowest number. Value must be greater than 0. |
port | int64 Port to use when connecting to the replica. If a port is not specified for a replica, ClickHouse uses the port specified for the source. Acceptable values are 0 to 65535, inclusive. |
user | string Name of the MySQL database user. |
password | string Password of the MySQL database user. |
ClickhouseSource
Field | Description |
---|---|
db | string Required. Name of the ClickHouse database. |
table | string Required. Name of the table in the specified database to be used as the dictionary source. |
host | string Required. ClickHouse host of the specified database. The maximum string length in characters is 253. |
port | int64 Port to use when connecting to the host. Acceptable values are 0 to 65535, inclusive. |
user | string Required. Name of the ClickHouse database user. |
password | string Password of the ClickHouse database user. |
where | string Selection criteria for the data in the specified ClickHouse table. |
MongodbSource
Field | Description |
---|---|
db | string Required. Name of the MongoDB database. |
collection | string Required. Name of the collection in the specified database to be used as the dictionary source. |
host | string Required. MongoDB host of the specified database. The maximum string length in characters is 253. |
port | int64 Port to use when connecting to the host. Acceptable values are 0 to 65535, inclusive. |
user | string Required. Name of the MongoDB database user. |
password | string Password of the MongoDB database user. |
options | string |
PostgresqlSource
Field | Description |
---|---|
db | string Required. Name of the PostrgreSQL database. |
table | string Required. Name of the table in the specified database to be used as the dictionary source. |
hosts[] | string Name of the PostrgreSQL host The number of elements must be greater than 0. |
port | int64 Port to use when connecting to the host. Acceptable values are 0 to 65535, inclusive. |
user | string Required. Name of the PostrgreSQL database user. |
password | string Password of the PostrgreSQL database user. |
invalidate_query | string Query for checking the dictionary status, to pull only updated data. For more details, see ClickHouse documentation on dictionaries |
ssl_mode | enum SslMode Mode of SSL TCP/IP connection to the PostgreSQL host. For more details, see PostgreSQL documentation
|
Structure
Field | Description |
---|---|
id | Id Single numeric key column for the dictionary. |
key | Key Composite key for the dictionary, containing of one or more key columns. For details, see ClickHouse documentation |
range_min | Attribute Field holding the beginning of the range for dictionaries with RANGE_HASHED layout. For details, see ClickHouse documentation |
range_max | Attribute Field holding the end of the range for dictionaries with RANGE_HASHED layout. For details, see ClickHouse documentation |
attributes[] | Attribute Description of the fields available for database queries. For details, see ClickHouse documentation |
Attribute
Field | Description |
---|---|
name | string Required. Name of the column. |
type | string Required. Type of the column. |
null_value | string Default value for an element without data (for example, an empty string). |
expression | string Expression, describing the attribute, if applicable. |
hierarchical | bool Indication of hierarchy support. Default value: false . |
injective | bool Indication of injective mapping "id -> attribute". Default value: false . |
Id
Field | Description |
---|---|
name | string Required. Name of the numeric key. |
Key
Field | Description |
---|---|
attributes[] | Attribute Attributes of a complex key. The number of elements must be greater than 0. |
Layout
Field | Description |
---|---|
type | enum Type Required. Layout type for an external dictionary.
|
size_in_cells | int64 Number of cells in the cache. Rounded up to a power of two. Applicable only for CACHE and COMPLEX_KEY_CACHE layout types. |
Range
Field | Description |
---|---|
min | int64 Minimum dictionary lifetime. |
max | int64 Maximum dictionary lifetime. |
GraphiteRollup
Field | Description |
---|---|
name | string Required. Name for the specified combination of settings for Graphite rollup. |
patterns[] | Pattern Pattern to use for the rollup. The number of elements must be greater than 0. |
Pattern
Field | Description |
---|---|
regexp | string Pattern for metric names. |
function | string Required. Name of the aggregating function to apply to data of the age specified in retention . |
retention[] | Retention Age of data to use for thinning. The number of elements must be greater than 0. |
Retention
Field | Description |
---|---|
age | int64 Minimum age of the data in seconds. The minimum value is 0. |
precision | int64 Precision of determining the age of the data, in seconds. Value must be greater than 0. |
Resources
Field | Description |
---|---|
resource_preset_id | string ID of the preset for computational resources available to a host (CPU, memory etc.). All available presets are listed in the documentation |
disk_size | int64 Volume of the storage available to a host, in bytes. |
disk_type_id | string Type of the storage environment for the host. Possible values:
|
MaintenanceWindow
Field | Description |
---|---|
policy | oneof: anytime or weekly_maintenance_window The maintenance policy in effect. |
anytime | AnytimeMaintenanceWindow Maintenance operation can be scheduled anytime. |
weekly_maintenance_window | WeeklyMaintenanceWindow Maintenance operation can be scheduled on a weekly basis. |
AnytimeMaintenanceWindow
Empty.
WeeklyMaintenanceWindow
Field | Description |
---|---|
day | enum WeekDay Day of the week (in DDD format). |
hour | int64 Hour of the day in UTC (in HH format). Acceptable values are 1 to 24, inclusive. |
MaintenanceOperation
Field | Description |
---|---|
info | string Information about this maintenance operation. The maximum string length in characters is 256. |
delayed_until | google.protobuf.Timestamp Time until which this maintenance operation is delayed. |
UpdateExternalDictionary
Updates an external dictionary for the specified ClickHouse cluster.
rpc UpdateExternalDictionary (UpdateClusterExternalDictionaryRequest) returns (operation.Operation)
Metadata and response of Operation:
Operation.metadata:UpdateClusterExternalDictionaryMetadata
Operation.response:Cluster
UpdateClusterExternalDictionaryRequest
Field | Description |
---|---|
cluster_id | string Required. ID of the ClickHouse cluster to update the external dictionary for. To get the cluster ID, use a List request. The maximum string length in characters is 50. |
external_dictionary | config.ClickhouseConfig.ExternalDictionary Configuration of the external dictionary. |
update_mask | google.protobuf.FieldMask Field mask that specifies which fields of the External Dictionary should be updated. |
Operation
Field | Description |
---|---|
id | string ID of the operation. |
description | string Description of the operation. 0-256 characters long. |
created_at | google.protobuf.Timestamp Creation timestamp. |
created_by | string ID of the user or service account who initiated the operation. |
modified_at | google.protobuf.Timestamp The time when the Operation resource was last modified. |
done | bool If the value is false , it means the operation is still in progress. If true , the operation is completed, and either error or response is available. |
metadata | google.protobuf.Any Service-specific metadata associated with the operation. It typically contains the ID of the target resource that the operation is performed on. Any method that returns a long-running operation should document the metadata type, if any. |
result | oneof: error or response The operation result. If done == false and there was no failure detected, neither error nor response is set. If done == false and there was a failure detected, error is set. If done == true , exactly one of error or response is set. |
error | google.rpc.Status The error result of the operation in case of failure or cancellation. |
response | google.protobuf.Any if operation finished successfully. |
UpdateClusterExternalDictionaryMetadata
Field | Description |
---|---|
cluster_id | string ID of the cluster for which an external dictionary is being updated. |
external_dictionary_name | string Name of the external dictionary. |
Cluster
Field | Description |
---|---|
id | string ID of the ClickHouse cluster. This ID is assigned by MDB at creation time. |
folder_id | string ID of the folder that the ClickHouse cluster belongs to. |
created_at | google.protobuf.Timestamp Creation timestamp in RFC3339 |
name | string Name of the ClickHouse cluster. The name is unique within the folder. 1-63 characters long. |
description | string Description of the ClickHouse cluster. 0-256 characters long. |
labels | map<string,string> Custom labels for the ClickHouse cluster as key:value pairs. Maximum 64 per resource. |
environment | enum Environment Deployment environment of the ClickHouse cluster.
|
monitoring[] | Monitoring Description of monitoring systems relevant to the ClickHouse cluster. |
config | ClusterConfig Configuration of the ClickHouse cluster. |
network_id | string ID of the network that the cluster belongs to. |
health | enum Health Aggregated cluster health.
|
status | enum Status Current state of the cluster.
|
service_account_id | string ID of the service account used for access to Object Storage. |
maintenance_window | MaintenanceWindow Maintenance window for the cluster. |
planned_operation | MaintenanceOperation Planned maintenance operation to be started for the cluster within the nearest maintenance_window . |
security_group_ids[] | string User security groups |
deletion_protection | bool Deletion Protection inhibits deletion of the cluster |
Monitoring
Field | Description |
---|---|
name | string Name of the monitoring system. |
description | string Description of the monitoring system. |
link | string Link to the monitoring system charts for the ClickHouse cluster. |
ClusterConfig
Field | Description |
---|---|
version | string Version of the ClickHouse server software. |
clickhouse | Clickhouse Configuration and resource allocation for ClickHouse hosts. |
zookeeper | Zookeeper Configuration and resource allocation for ZooKeeper hosts. |
backup_window_start | google.type.TimeOfDay Time to start the daily backup, in the UTC timezone. |
access | Access Access policy for external services. |
cloud_storage | CloudStorage |
sql_database_management | google.protobuf.BoolValue Whether database management through SQL commands is enabled. |
sql_user_management | google.protobuf.BoolValue Whether user management through SQL commands is enabled. |
embedded_keeper | google.protobuf.BoolValue Whether cluster should use embedded Keeper instead of Zookeeper. |
Clickhouse
Field | Description |
---|---|
config | config.ClickhouseConfigSet Configuration settings of a ClickHouse server. |
resources | Resources Resources allocated to ClickHouse hosts. |
Zookeeper
Field | Description |
---|---|
resources | Resources Resources allocated to ZooKeeper hosts. |
Access
Field | Description |
---|---|
data_lens | bool Allow to export data from the cluster to DataLens. |
web_sql | bool Allow SQL queries to the cluster databases from the management console. See SQL queries in the management console for more details. |
metrika | bool Allow to import data from Yandex Metrica and AppMetrica to the cluster. See AppMetrica documentation |
serverless | bool Allow access to cluster for Serverless. |
data_transfer | bool Allow access for DataTransfer |
yandex_query | bool Allow access for Query |
CloudStorage
Field | Description |
---|---|
enabled | bool Whether to use Object Storage for storing ClickHouse data. |
move_factor | google.protobuf.DoubleValue Acceptable values are 0 to 1, inclusive. |
data_cache_enabled | google.protobuf.BoolValue |
data_cache_max_size | google.protobuf.Int64Value |
prefer_not_to_merge | google.protobuf.BoolValue |
ClickhouseConfigSet
Field | Description |
---|---|
effective_config | ClickhouseConfig Required. Effective settings for a ClickHouse cluster (a combination of settings defined in user_config and default_config ). |
user_config | ClickhouseConfig User-defined settings for a ClickHouse cluster. |
default_config | ClickhouseConfig Default configuration for a ClickHouse cluster. |
ClickhouseConfig
Field | Description |
---|---|
log_level | enum LogLevel Logging level for the ClickHouse cluster. Possible values: TRACE, DEBUG, INFORMATION, WARNING, ERROR. |
merge_tree | MergeTree Settings for the MergeTree engine. See description in ClickHouse documentation |
compression[] | Compression Compression settings for the ClickHouse cluster. See in-depth description in ClickHouse documentation |
dictionaries[] | ExternalDictionary Configuration of external dictionaries to be used by the ClickHouse cluster. See in-depth description in ClickHouse documentation |
graphite_rollup[] | GraphiteRollup Settings for thinning Graphite data. See in-depth description in ClickHouse documentation |
kafka | Kafka |
kafka_topics[] | KafkaTopic |
rabbitmq | Rabbitmq |
max_connections | google.protobuf.Int64Value Maximum number of inbound connections. The minimum value is 10. |
max_concurrent_queries | google.protobuf.Int64Value Maximum number of simultaneously processed requests. The minimum value is 10. |
keep_alive_timeout | google.protobuf.Int64Value Number of milliseconds that ClickHouse waits for incoming requests before closing the connection. |
uncompressed_cache_size | google.protobuf.Int64Value Cache size (in bytes) for uncompressed data used by MergeTree tables. |
mark_cache_size | google.protobuf.Int64Value Approximate size (in bytes) of the cache of "marks" used by MergeTree tables. Value must be greater than 0. |
max_table_size_to_drop | google.protobuf.Int64Value Maximum size of the table that can be deleted using a DROP query. |
max_partition_size_to_drop | google.protobuf.Int64Value Maximum size of the partition that can be deleted using a DROP query. |
builtin_dictionaries_reload_interval | google.protobuf.Int64Value The setting is deprecated and has no effect. |
timezone | string The server's time zone to be used in DateTime fields conversions. Specified as an IANA identifier. |
geobase_enabled | google.protobuf.BoolValue Enable or disable geobase. |
geobase_uri | string Address of the archive with the user geobase in Object Storage. |
query_log_retention_size | google.protobuf.Int64Value The maximum size that query_log can grow to before old data will be removed. If set to 0, automatic removal of query_log data based on size is disabled. |
query_log_retention_time | google.protobuf.Int64Value The maximum time that query_log records will be retained before removal. If set to 0, automatic removal of query_log data based on time is disabled. |
query_thread_log_enabled | google.protobuf.BoolValue Whether query_thread_log system table is enabled. |
query_thread_log_retention_size | google.protobuf.Int64Value The maximum size that query_thread_log can grow to before old data will be removed. If set to 0, automatic removal of query_thread_log data based on size is disabled. |
query_thread_log_retention_time | google.protobuf.Int64Value The maximum time that query_thread_log records will be retained before removal. If set to 0, automatic removal of query_thread_log data based on time is disabled. |
part_log_retention_size | google.protobuf.Int64Value The maximum size that part_log can grow to before old data will be removed. If set to 0, automatic removal of part_log data based on size is disabled. |
part_log_retention_time | google.protobuf.Int64Value The maximum time that part_log records will be retained before removal. If set to 0, automatic removal of part_log data based on time is disabled. |
metric_log_enabled | google.protobuf.BoolValue Whether metric_log system table is enabled. |
metric_log_retention_size | google.protobuf.Int64Value The maximum size that metric_log can grow to before old data will be removed. If set to 0, automatic removal of metric_log data based on size is disabled. |
metric_log_retention_time | google.protobuf.Int64Value The maximum time that metric_log records will be retained before removal. If set to 0, automatic removal of metric_log data based on time is disabled. |
trace_log_enabled | google.protobuf.BoolValue Whether trace_log system table is enabled. |
trace_log_retention_size | google.protobuf.Int64Value The maximum size that trace_log can grow to before old data will be removed. If set to 0, automatic removal of trace_log data based on size is disabled. |
trace_log_retention_time | google.protobuf.Int64Value The maximum time that trace_log records will be retained before removal. If set to 0, automatic removal of trace_log data based on time is disabled. |
text_log_enabled | google.protobuf.BoolValue Whether text_log system table is enabled. |
text_log_retention_size | google.protobuf.Int64Value The maximum size that text_log can grow to before old data will be removed. If set to 0, automatic removal of text_log data based on size is disabled. |
text_log_retention_time | google.protobuf.Int64Value The maximum time that text_log records will be retained before removal. If set to 0, automatic removal of text_log data based on time is disabled. |
text_log_level | enum LogLevel Logging level for text_log system table. Possible values: TRACE, DEBUG, INFORMATION, WARNING, ERROR. |
opentelemetry_span_log_enabled | google.protobuf.BoolValue Enable or disable opentelemetry_span_log system table. Default value: false. |
opentelemetry_span_log_retention_size | google.protobuf.Int64Value The maximum size that opentelemetry_span_log can grow to before old data will be removed. If set to 0 (default), automatic removal of opentelemetry_span_log data based on size is disabled. The minimum value is 0. |
opentelemetry_span_log_retention_time | google.protobuf.Int64Value The maximum time that opentelemetry_span_log records will be retained before removal. If set to 0, automatic removal of opentelemetry_span_log data based on time is disabled. The minimum value is 0. |
query_views_log_enabled | google.protobuf.BoolValue Enable or disable query_views_log system table. Default value: false. |
query_views_log_retention_size | google.protobuf.Int64Value The maximum size that query_views_log can grow to before old data will be removed. If set to 0 (default), automatic removal of query_views_log data based on size is disabled. The minimum value is 0. |
query_views_log_retention_time | google.protobuf.Int64Value The maximum time that query_views_log records will be retained before removal. If set to 0, automatic removal of query_views_log data based on time is disabled. The minimum value is 0. |
asynchronous_metric_log_enabled | google.protobuf.BoolValue Enable or disable asynchronous_metric_log system table. Default value: false. |
asynchronous_metric_log_retention_size | google.protobuf.Int64Value The maximum size that asynchronous_metric_log can grow to before old data will be removed. If set to 0 (default), automatic removal of asynchronous_metric_log data based on size is disabled. The minimum value is 0. |
asynchronous_metric_log_retention_time | google.protobuf.Int64Value The maximum time that asynchronous_metric_log records will be retained before removal. If set to 0, automatic removal of asynchronous_metric_log data based on time is disabled. The minimum value is 0. |
session_log_enabled | google.protobuf.BoolValue Enable or disable session_log system table. Default value: false. |
session_log_retention_size | google.protobuf.Int64Value The maximum size that session_log can grow to before old data will be removed. If set to 0 (default), automatic removal of session_log data based on size is disabled. The minimum value is 0. |
session_log_retention_time | google.protobuf.Int64Value The maximum time that session_log records will be retained before removal. If set to 0, automatic removal of session_log data based on time is disabled. The minimum value is 0. |
zookeeper_log_enabled | google.protobuf.BoolValue Enable or disable zookeeper_log system table. Default value: false. |
zookeeper_log_retention_size | google.protobuf.Int64Value The maximum size that zookeeper_log can grow to before old data will be removed. If set to 0 (default), automatic removal of zookeeper_log data based on size is disabled. The minimum value is 0. |
zookeeper_log_retention_time | google.protobuf.Int64Value The maximum time that zookeeper_log records will be retained before removal. If set to 0, automatic removal of zookeeper_log data based on time is disabled. The minimum value is 0. |
asynchronous_insert_log_enabled | google.protobuf.BoolValue Enable or disable asynchronous_insert_log system table. Default value: false. Minimal required ClickHouse version: 22.10. |
asynchronous_insert_log_retention_size | google.protobuf.Int64Value The maximum size that asynchronous_insert_log can grow to before old data will be removed. If set to 0 (default), automatic removal of asynchronous_insert_log data based on size is disabled. The minimum value is 0. |
asynchronous_insert_log_retention_time | google.protobuf.Int64Value The maximum time that asynchronous_insert_log records will be retained before removal. If set to 0, automatic removal of asynchronous_insert_log data based on time is disabled. The minimum value is 0. |
background_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_merges_mutations_concurrency_ratio | google.protobuf.Int64Value Sets a ratio between the number of threads and the number of background merges and mutations that can be executed concurrently. For example, if the ratio equals to 2 and background_pool_size is set to 16 then ClickHouse can execute 32 background merges concurrently. This is possible, because background operations could be suspended and postponed. This is needed to give small merges more execution priority. You can only increase this ratio at runtime. To lower it you have to restart the server. The same as for background_pool_size setting background_merges_mutations_concurrency_ratio could be applied from the default profile for backward compatibility. Default: 2 See in-depth description in ClickHouse documentation |
background_schedule_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_fetches_pool_size | google.protobuf.Int64Value Sets the number of threads performing background fetches for tables with ReplicatedMergeTree engines. Default value: 8. More info see in ClickHouse documentation |
background_move_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_distributed_schedule_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_buffer_flush_schedule_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_message_broker_schedule_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_common_pool_size | google.protobuf.Int64Value The maximum number of threads that will be used for performing a variety of operations (mostly garbage collection) for *MergeTree-engine tables in a background. Default: 8 See in-depth description in ClickHouse documentation |
default_database | google.protobuf.StringValue The default database. To get a list of cluster databases, see Yandex Managed ClickHouse documentation. |
total_memory_profiler_step | google.protobuf.Int64Value Sets the memory size (in bytes) for a stack trace at every peak allocation step. Default value: 4194304. More info see in ClickHouse documentation |
total_memory_tracker_sample_probability | google.protobuf.DoubleValue |
MergeTree
Field | Description |
---|---|
replicated_deduplication_window | google.protobuf.Int64Value Number of blocks of hashes to keep in ZooKeeper. |
replicated_deduplication_window_seconds | google.protobuf.Int64Value Period of time to keep blocks of hashes for. |
parts_to_delay_insert | google.protobuf.Int64Value If table contains at least that many active parts in single partition, artificially slow down insert into table. |
parts_to_throw_insert | google.protobuf.Int64Value If more than this number active parts in single partition, throw 'Too many parts ...' exception. |
inactive_parts_to_delay_insert | google.protobuf.Int64Value |
inactive_parts_to_throw_insert | google.protobuf.Int64Value |
max_replicated_merges_in_queue | google.protobuf.Int64Value How many tasks of merging and mutating parts are allowed simultaneously in ReplicatedMergeTree queue. |
number_of_free_entries_in_pool_to_lower_max_size_of_merge | google.protobuf.Int64Value If there is less than specified number of free entries in background pool (or replicated queue), start to lower maximum size of merge to process. |
max_bytes_to_merge_at_min_space_in_pool | google.protobuf.Int64Value Maximum in total size of parts to merge, when there are minimum free threads in background pool (or entries in replication queue). |
max_bytes_to_merge_at_max_space_in_pool | google.protobuf.Int64Value |
min_bytes_for_wide_part | google.protobuf.Int64Value Minimum number of bytes in a data part that can be stored in Wide format. More info see in ClickHouse documentation |
min_rows_for_wide_part | google.protobuf.Int64Value Minimum number of rows in a data part that can be stored in Wide format. More info see in ClickHouse documentation |
ttl_only_drop_parts | google.protobuf.BoolValue Enables or disables complete dropping of data parts where all rows are expired in MergeTree tables. More info see in ClickHouse documentation |
allow_remote_fs_zero_copy_replication | google.protobuf.BoolValue |
merge_with_ttl_timeout | google.protobuf.Int64Value |
merge_with_recompression_ttl_timeout | google.protobuf.Int64Value |
max_parts_in_total | google.protobuf.Int64Value |
max_number_of_merges_with_ttl_in_pool | google.protobuf.Int64Value |
cleanup_delay_period | google.protobuf.Int64Value |
number_of_free_entries_in_pool_to_execute_mutation | google.protobuf.Int64Value |
max_avg_part_size_for_too_many_parts | google.protobuf.Int64Value The 'too many parts' check according to 'parts_to_delay_insert' and 'parts_to_throw_insert' will be active only if the average part size (in the relevant partition) is not larger than the specified threshold. If it is larger than the specified threshold, the INSERTs will be neither delayed or rejected. This allows to have hundreds of terabytes in a single table on a single server if the parts are successfully merged to larger parts. This does not affect the thresholds on inactive parts or total parts. Default: 1 GiB Min version: 22.10 See in-depth description in ClickHouse GitHub |
min_age_to_force_merge_seconds | google.protobuf.Int64Value Merge parts if every part in the range is older than the value of min_age_to_force_merge_seconds. Default: 0 - disabled Min_version: 22.10 See in-depth description in ClickHouse documentation |
min_age_to_force_merge_on_partition_only | google.protobuf.BoolValue Whether min_age_to_force_merge_seconds should be applied only on the entire partition and not on subset. Default: false Min_version: 22.11 See in-depth description in ClickHouse documentation |
merge_selecting_sleep_ms | google.protobuf.Int64Value Sleep time for merge selecting when no part is selected. A lower setting triggers selecting tasks in background_schedule_pool frequently, which results in a large number of requests to ClickHouse Keeper in large-scale clusters. Default: 5000 Min_version: 21.10 See in-depth description in ClickHouse documentation |
Kafka
Field | Description |
---|---|
security_protocol | enum SecurityProtocol |
sasl_mechanism | enum SaslMechanism |
sasl_username | string |
sasl_password | string |
enable_ssl_certificate_verification | google.protobuf.BoolValue |
max_poll_interval_ms | google.protobuf.Int64Value The minimum value is 0. |
session_timeout_ms | google.protobuf.Int64Value The minimum value is 0. |
KafkaTopic
Field | Description |
---|---|
name | string Required. |
settings | Kafka Required. |
Rabbitmq
Field | Description |
---|---|
username | string RabbitMQ |
password | string RabbitMQ |
vhost | string RabbitMQ |
Compression
Field | Description |
---|---|
method | enum Method Compression method to use for the specified combination of min_part_size and min_part_size_ratio . |
min_part_size | int64 Minimum size of a part of a table. The minimum value is 1. |
min_part_size_ratio | double Minimum ratio of a part relative to the size of all the data in the table. |
level | google.protobuf.Int64Value The minimum value is 0. |
ExternalDictionary
Field | Description |
---|---|
name | string Required. Name of the external dictionary. |
structure | Structure Required. Set of attributes for the external dictionary. For in-depth description, see ClickHouse documentation |
layout | Layout Required. Layout for storing the dictionary in memory. For in-depth description, see ClickHouse documentation |
lifetime | oneof: fixed_lifetime or lifetime_range Setting for the period of time between dictionary updates. For details, see ClickHouse documentation |
fixed_lifetime | int64 Fixed interval between dictionary updates. |
lifetime_range | Range Range of intervals between dictionary updates for ClickHouse to choose from. |
source | oneof: http_source , mysql_source , clickhouse_source , mongodb_source or postgresql_source Description of the source for the external dictionary. |
http_source | HttpSource HTTP source for the dictionary. |
mysql_source | MysqlSource MySQL source for the dictionary. |
clickhouse_source | ClickhouseSource ClickHouse source for the dictionary. |
mongodb_source | MongodbSource MongoDB source for the dictionary. |
postgresql_source | PostgresqlSource PostgreSQL source for the dictionary. |
HttpSource
Field | Description |
---|---|
url | string Required. URL of the source dictionary available over HTTP. |
format | string Required. The data format. Valid values are all formats supported by ClickHouse SQL dialect. |
MysqlSource
Field | Description |
---|---|
db | string Required. Name of the MySQL database to connect to. |
table | string Required. Name of the database table to use as a ClickHouse dictionary. |
port | int64 Default port to use when connecting to a replica of the dictionary source. Acceptable values are 0 to 65535, inclusive. |
user | string Name of the default user for replicas of the dictionary source. |
password | string Password of the default user for replicas of the dictionary source. |
replicas[] | Replica List of MySQL replicas of the database used as dictionary source. The number of elements must be greater than 0. |
where | string Selection criteria for the data in the specified MySQL table. |
invalidate_query | string Query for checking the dictionary status, to pull only updated data. For more details, see ClickHouse documentation on dictionaries |
Replica
Field | Description |
---|---|
host | string Required. MySQL host of the replica. The maximum string length in characters is 253. |
priority | int64 Required. The priority of the replica that ClickHouse takes into account when connecting. Replica with the highest priority should have this field set to the lowest number. Value must be greater than 0. |
port | int64 Port to use when connecting to the replica. If a port is not specified for a replica, ClickHouse uses the port specified for the source. Acceptable values are 0 to 65535, inclusive. |
user | string Name of the MySQL database user. |
password | string Password of the MySQL database user. |
ClickhouseSource
Field | Description |
---|---|
db | string Required. Name of the ClickHouse database. |
table | string Required. Name of the table in the specified database to be used as the dictionary source. |
host | string Required. ClickHouse host of the specified database. The maximum string length in characters is 253. |
port | int64 Port to use when connecting to the host. Acceptable values are 0 to 65535, inclusive. |
user | string Required. Name of the ClickHouse database user. |
password | string Password of the ClickHouse database user. |
where | string Selection criteria for the data in the specified ClickHouse table. |
MongodbSource
Field | Description |
---|---|
db | string Required. Name of the MongoDB database. |
collection | string Required. Name of the collection in the specified database to be used as the dictionary source. |
host | string Required. MongoDB host of the specified database. The maximum string length in characters is 253. |
port | int64 Port to use when connecting to the host. Acceptable values are 0 to 65535, inclusive. |
user | string Required. Name of the MongoDB database user. |
password | string Password of the MongoDB database user. |
options | string |
PostgresqlSource
Field | Description |
---|---|
db | string Required. Name of the PostrgreSQL database. |
table | string Required. Name of the table in the specified database to be used as the dictionary source. |
hosts[] | string Name of the PostrgreSQL host The number of elements must be greater than 0. |
port | int64 Port to use when connecting to the host. Acceptable values are 0 to 65535, inclusive. |
user | string Required. Name of the PostrgreSQL database user. |
password | string Password of the PostrgreSQL database user. |
invalidate_query | string Query for checking the dictionary status, to pull only updated data. For more details, see ClickHouse documentation on dictionaries |
ssl_mode | enum SslMode Mode of SSL TCP/IP connection to the PostgreSQL host. For more details, see PostgreSQL documentation
|
Structure
Field | Description |
---|---|
id | Id Single numeric key column for the dictionary. |
key | Key Composite key for the dictionary, containing of one or more key columns. For details, see ClickHouse documentation |
range_min | Attribute Field holding the beginning of the range for dictionaries with RANGE_HASHED layout. For details, see ClickHouse documentation |
range_max | Attribute Field holding the end of the range for dictionaries with RANGE_HASHED layout. For details, see ClickHouse documentation |
attributes[] | Attribute Description of the fields available for database queries. For details, see ClickHouse documentation |
Attribute
Field | Description |
---|---|
name | string Required. Name of the column. |
type | string Required. Type of the column. |
null_value | string Default value for an element without data (for example, an empty string). |
expression | string Expression, describing the attribute, if applicable. |
hierarchical | bool Indication of hierarchy support. Default value: false . |
injective | bool Indication of injective mapping "id -> attribute". Default value: false . |
Id
Field | Description |
---|---|
name | string Required. Name of the numeric key. |
Key
Field | Description |
---|---|
attributes[] | Attribute Attributes of a complex key. The number of elements must be greater than 0. |
Layout
Field | Description |
---|---|
type | enum Type Required. Layout type for an external dictionary.
|
size_in_cells | int64 Number of cells in the cache. Rounded up to a power of two. Applicable only for CACHE and COMPLEX_KEY_CACHE layout types. |
Range
Field | Description |
---|---|
min | int64 Minimum dictionary lifetime. |
max | int64 Maximum dictionary lifetime. |
GraphiteRollup
Field | Description |
---|---|
name | string Required. Name for the specified combination of settings for Graphite rollup. |
patterns[] | Pattern Pattern to use for the rollup. The number of elements must be greater than 0. |
Pattern
Field | Description |
---|---|
regexp | string Pattern for metric names. |
function | string Required. Name of the aggregating function to apply to data of the age specified in retention . |
retention[] | Retention Age of data to use for thinning. The number of elements must be greater than 0. |
Retention
Field | Description |
---|---|
age | int64 Minimum age of the data in seconds. The minimum value is 0. |
precision | int64 Precision of determining the age of the data, in seconds. Value must be greater than 0. |
Resources
Field | Description |
---|---|
resource_preset_id | string ID of the preset for computational resources available to a host (CPU, memory etc.). All available presets are listed in the documentation |
disk_size | int64 Volume of the storage available to a host, in bytes. |
disk_type_id | string Type of the storage environment for the host. Possible values:
|
MaintenanceWindow
Field | Description |
---|---|
policy | oneof: anytime or weekly_maintenance_window The maintenance policy in effect. |
anytime | AnytimeMaintenanceWindow Maintenance operation can be scheduled anytime. |
weekly_maintenance_window | WeeklyMaintenanceWindow Maintenance operation can be scheduled on a weekly basis. |
AnytimeMaintenanceWindow
Empty.
WeeklyMaintenanceWindow
Field | Description |
---|---|
day | enum WeekDay Day of the week (in DDD format). |
hour | int64 Hour of the day in UTC (in HH format). Acceptable values are 1 to 24, inclusive. |
MaintenanceOperation
Field | Description |
---|---|
info | string Information about this maintenance operation. The maximum string length in characters is 256. |
delayed_until | google.protobuf.Timestamp Time until which this maintenance operation is delayed. |
DeleteExternalDictionary
Deletes the specified external dictionary.
rpc DeleteExternalDictionary (DeleteClusterExternalDictionaryRequest) returns (operation.Operation)
Metadata and response of Operation:
Operation.metadata:DeleteClusterExternalDictionaryMetadata
Operation.response:Cluster
DeleteClusterExternalDictionaryRequest
Field | Description |
---|---|
cluster_id | string Required. ID of the ClickHouse cluster to delete the external dictionary from. To get the cluster ID, use a ClusterService.List request. The maximum string length in characters is 50. |
external_dictionary_name | string Name of the external dictionary to delete. |
Operation
Field | Description |
---|---|
id | string ID of the operation. |
description | string Description of the operation. 0-256 characters long. |
created_at | google.protobuf.Timestamp Creation timestamp. |
created_by | string ID of the user or service account who initiated the operation. |
modified_at | google.protobuf.Timestamp The time when the Operation resource was last modified. |
done | bool If the value is false , it means the operation is still in progress. If true , the operation is completed, and either error or response is available. |
metadata | google.protobuf.Any Service-specific metadata associated with the operation. It typically contains the ID of the target resource that the operation is performed on. Any method that returns a long-running operation should document the metadata type, if any. |
result | oneof: error or response The operation result. If done == false and there was no failure detected, neither error nor response is set. If done == false and there was a failure detected, error is set. If done == true , exactly one of error or response is set. |
error | google.rpc.Status The error result of the operation in case of failure or cancellation. |
response | google.protobuf.Any if operation finished successfully. |
DeleteClusterExternalDictionaryMetadata
Field | Description |
---|---|
cluster_id | string ID of the cluster where an external dictionary is being deleted. |
Cluster
Field | Description |
---|---|
id | string ID of the ClickHouse cluster. This ID is assigned by MDB at creation time. |
folder_id | string ID of the folder that the ClickHouse cluster belongs to. |
created_at | google.protobuf.Timestamp Creation timestamp in RFC3339 |
name | string Name of the ClickHouse cluster. The name is unique within the folder. 1-63 characters long. |
description | string Description of the ClickHouse cluster. 0-256 characters long. |
labels | map<string,string> Custom labels for the ClickHouse cluster as key:value pairs. Maximum 64 per resource. |
environment | enum Environment Deployment environment of the ClickHouse cluster.
|
monitoring[] | Monitoring Description of monitoring systems relevant to the ClickHouse cluster. |
config | ClusterConfig Configuration of the ClickHouse cluster. |
network_id | string ID of the network that the cluster belongs to. |
health | enum Health Aggregated cluster health.
|
status | enum Status Current state of the cluster.
|
service_account_id | string ID of the service account used for access to Object Storage. |
maintenance_window | MaintenanceWindow Maintenance window for the cluster. |
planned_operation | MaintenanceOperation Planned maintenance operation to be started for the cluster within the nearest maintenance_window . |
security_group_ids[] | string User security groups |
deletion_protection | bool Deletion Protection inhibits deletion of the cluster |
Monitoring
Field | Description |
---|---|
name | string Name of the monitoring system. |
description | string Description of the monitoring system. |
link | string Link to the monitoring system charts for the ClickHouse cluster. |
ClusterConfig
Field | Description |
---|---|
version | string Version of the ClickHouse server software. |
clickhouse | Clickhouse Configuration and resource allocation for ClickHouse hosts. |
zookeeper | Zookeeper Configuration and resource allocation for ZooKeeper hosts. |
backup_window_start | google.type.TimeOfDay Time to start the daily backup, in the UTC timezone. |
access | Access Access policy for external services. |
cloud_storage | CloudStorage |
sql_database_management | google.protobuf.BoolValue Whether database management through SQL commands is enabled. |
sql_user_management | google.protobuf.BoolValue Whether user management through SQL commands is enabled. |
embedded_keeper | google.protobuf.BoolValue Whether cluster should use embedded Keeper instead of Zookeeper. |
Clickhouse
Field | Description |
---|---|
config | config.ClickhouseConfigSet Configuration settings of a ClickHouse server. |
resources | Resources Resources allocated to ClickHouse hosts. |
Zookeeper
Field | Description |
---|---|
resources | Resources Resources allocated to ZooKeeper hosts. |
Access
Field | Description |
---|---|
data_lens | bool Allow to export data from the cluster to DataLens. |
web_sql | bool Allow SQL queries to the cluster databases from the management console. See SQL queries in the management console for more details. |
metrika | bool Allow to import data from Yandex Metrica and AppMetrica to the cluster. See AppMetrica documentation |
serverless | bool Allow access to cluster for Serverless. |
data_transfer | bool Allow access for DataTransfer |
yandex_query | bool Allow access for Query |
CloudStorage
Field | Description |
---|---|
enabled | bool Whether to use Object Storage for storing ClickHouse data. |
move_factor | google.protobuf.DoubleValue Acceptable values are 0 to 1, inclusive. |
data_cache_enabled | google.protobuf.BoolValue |
data_cache_max_size | google.protobuf.Int64Value |
prefer_not_to_merge | google.protobuf.BoolValue |
ClickhouseConfigSet
Field | Description |
---|---|
effective_config | ClickhouseConfig Required. Effective settings for a ClickHouse cluster (a combination of settings defined in user_config and default_config ). |
user_config | ClickhouseConfig User-defined settings for a ClickHouse cluster. |
default_config | ClickhouseConfig Default configuration for a ClickHouse cluster. |
ClickhouseConfig
Field | Description |
---|---|
log_level | enum LogLevel Logging level for the ClickHouse cluster. Possible values: TRACE, DEBUG, INFORMATION, WARNING, ERROR. |
merge_tree | MergeTree Settings for the MergeTree engine. See description in ClickHouse documentation |
compression[] | Compression Compression settings for the ClickHouse cluster. See in-depth description in ClickHouse documentation |
dictionaries[] | ExternalDictionary Configuration of external dictionaries to be used by the ClickHouse cluster. See in-depth description in ClickHouse documentation |
graphite_rollup[] | GraphiteRollup Settings for thinning Graphite data. See in-depth description in ClickHouse documentation |
kafka | Kafka |
kafka_topics[] | KafkaTopic |
rabbitmq | Rabbitmq |
max_connections | google.protobuf.Int64Value Maximum number of inbound connections. The minimum value is 10. |
max_concurrent_queries | google.protobuf.Int64Value Maximum number of simultaneously processed requests. The minimum value is 10. |
keep_alive_timeout | google.protobuf.Int64Value Number of milliseconds that ClickHouse waits for incoming requests before closing the connection. |
uncompressed_cache_size | google.protobuf.Int64Value Cache size (in bytes) for uncompressed data used by MergeTree tables. |
mark_cache_size | google.protobuf.Int64Value Approximate size (in bytes) of the cache of "marks" used by MergeTree tables. Value must be greater than 0. |
max_table_size_to_drop | google.protobuf.Int64Value Maximum size of the table that can be deleted using a DROP query. |
max_partition_size_to_drop | google.protobuf.Int64Value Maximum size of the partition that can be deleted using a DROP query. |
builtin_dictionaries_reload_interval | google.protobuf.Int64Value The setting is deprecated and has no effect. |
timezone | string The server's time zone to be used in DateTime fields conversions. Specified as an IANA identifier. |
geobase_enabled | google.protobuf.BoolValue Enable or disable geobase. |
geobase_uri | string Address of the archive with the user geobase in Object Storage. |
query_log_retention_size | google.protobuf.Int64Value The maximum size that query_log can grow to before old data will be removed. If set to 0, automatic removal of query_log data based on size is disabled. |
query_log_retention_time | google.protobuf.Int64Value The maximum time that query_log records will be retained before removal. If set to 0, automatic removal of query_log data based on time is disabled. |
query_thread_log_enabled | google.protobuf.BoolValue Whether query_thread_log system table is enabled. |
query_thread_log_retention_size | google.protobuf.Int64Value The maximum size that query_thread_log can grow to before old data will be removed. If set to 0, automatic removal of query_thread_log data based on size is disabled. |
query_thread_log_retention_time | google.protobuf.Int64Value The maximum time that query_thread_log records will be retained before removal. If set to 0, automatic removal of query_thread_log data based on time is disabled. |
part_log_retention_size | google.protobuf.Int64Value The maximum size that part_log can grow to before old data will be removed. If set to 0, automatic removal of part_log data based on size is disabled. |
part_log_retention_time | google.protobuf.Int64Value The maximum time that part_log records will be retained before removal. If set to 0, automatic removal of part_log data based on time is disabled. |
metric_log_enabled | google.protobuf.BoolValue Whether metric_log system table is enabled. |
metric_log_retention_size | google.protobuf.Int64Value The maximum size that metric_log can grow to before old data will be removed. If set to 0, automatic removal of metric_log data based on size is disabled. |
metric_log_retention_time | google.protobuf.Int64Value The maximum time that metric_log records will be retained before removal. If set to 0, automatic removal of metric_log data based on time is disabled. |
trace_log_enabled | google.protobuf.BoolValue Whether trace_log system table is enabled. |
trace_log_retention_size | google.protobuf.Int64Value The maximum size that trace_log can grow to before old data will be removed. If set to 0, automatic removal of trace_log data based on size is disabled. |
trace_log_retention_time | google.protobuf.Int64Value The maximum time that trace_log records will be retained before removal. If set to 0, automatic removal of trace_log data based on time is disabled. |
text_log_enabled | google.protobuf.BoolValue Whether text_log system table is enabled. |
text_log_retention_size | google.protobuf.Int64Value The maximum size that text_log can grow to before old data will be removed. If set to 0, automatic removal of text_log data based on size is disabled. |
text_log_retention_time | google.protobuf.Int64Value The maximum time that text_log records will be retained before removal. If set to 0, automatic removal of text_log data based on time is disabled. |
text_log_level | enum LogLevel Logging level for text_log system table. Possible values: TRACE, DEBUG, INFORMATION, WARNING, ERROR. |
opentelemetry_span_log_enabled | google.protobuf.BoolValue Enable or disable opentelemetry_span_log system table. Default value: false. |
opentelemetry_span_log_retention_size | google.protobuf.Int64Value The maximum size that opentelemetry_span_log can grow to before old data will be removed. If set to 0 (default), automatic removal of opentelemetry_span_log data based on size is disabled. The minimum value is 0. |
opentelemetry_span_log_retention_time | google.protobuf.Int64Value The maximum time that opentelemetry_span_log records will be retained before removal. If set to 0, automatic removal of opentelemetry_span_log data based on time is disabled. The minimum value is 0. |
query_views_log_enabled | google.protobuf.BoolValue Enable or disable query_views_log system table. Default value: false. |
query_views_log_retention_size | google.protobuf.Int64Value The maximum size that query_views_log can grow to before old data will be removed. If set to 0 (default), automatic removal of query_views_log data based on size is disabled. The minimum value is 0. |
query_views_log_retention_time | google.protobuf.Int64Value The maximum time that query_views_log records will be retained before removal. If set to 0, automatic removal of query_views_log data based on time is disabled. The minimum value is 0. |
asynchronous_metric_log_enabled | google.protobuf.BoolValue Enable or disable asynchronous_metric_log system table. Default value: false. |
asynchronous_metric_log_retention_size | google.protobuf.Int64Value The maximum size that asynchronous_metric_log can grow to before old data will be removed. If set to 0 (default), automatic removal of asynchronous_metric_log data based on size is disabled. The minimum value is 0. |
asynchronous_metric_log_retention_time | google.protobuf.Int64Value The maximum time that asynchronous_metric_log records will be retained before removal. If set to 0, automatic removal of asynchronous_metric_log data based on time is disabled. The minimum value is 0. |
session_log_enabled | google.protobuf.BoolValue Enable or disable session_log system table. Default value: false. |
session_log_retention_size | google.protobuf.Int64Value The maximum size that session_log can grow to before old data will be removed. If set to 0 (default), automatic removal of session_log data based on size is disabled. The minimum value is 0. |
session_log_retention_time | google.protobuf.Int64Value The maximum time that session_log records will be retained before removal. If set to 0, automatic removal of session_log data based on time is disabled. The minimum value is 0. |
zookeeper_log_enabled | google.protobuf.BoolValue Enable or disable zookeeper_log system table. Default value: false. |
zookeeper_log_retention_size | google.protobuf.Int64Value The maximum size that zookeeper_log can grow to before old data will be removed. If set to 0 (default), automatic removal of zookeeper_log data based on size is disabled. The minimum value is 0. |
zookeeper_log_retention_time | google.protobuf.Int64Value The maximum time that zookeeper_log records will be retained before removal. If set to 0, automatic removal of zookeeper_log data based on time is disabled. The minimum value is 0. |
asynchronous_insert_log_enabled | google.protobuf.BoolValue Enable or disable asynchronous_insert_log system table. Default value: false. Minimal required ClickHouse version: 22.10. |
asynchronous_insert_log_retention_size | google.protobuf.Int64Value The maximum size that asynchronous_insert_log can grow to before old data will be removed. If set to 0 (default), automatic removal of asynchronous_insert_log data based on size is disabled. The minimum value is 0. |
asynchronous_insert_log_retention_time | google.protobuf.Int64Value The maximum time that asynchronous_insert_log records will be retained before removal. If set to 0, automatic removal of asynchronous_insert_log data based on time is disabled. The minimum value is 0. |
background_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_merges_mutations_concurrency_ratio | google.protobuf.Int64Value Sets a ratio between the number of threads and the number of background merges and mutations that can be executed concurrently. For example, if the ratio equals to 2 and background_pool_size is set to 16 then ClickHouse can execute 32 background merges concurrently. This is possible, because background operations could be suspended and postponed. This is needed to give small merges more execution priority. You can only increase this ratio at runtime. To lower it you have to restart the server. The same as for background_pool_size setting background_merges_mutations_concurrency_ratio could be applied from the default profile for backward compatibility. Default: 2 See in-depth description in ClickHouse documentation |
background_schedule_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_fetches_pool_size | google.protobuf.Int64Value Sets the number of threads performing background fetches for tables with ReplicatedMergeTree engines. Default value: 8. More info see in ClickHouse documentation |
background_move_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_distributed_schedule_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_buffer_flush_schedule_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_message_broker_schedule_pool_size | google.protobuf.Int64Value Value must be greater than 0. |
background_common_pool_size | google.protobuf.Int64Value The maximum number of threads that will be used for performing a variety of operations (mostly garbage collection) for *MergeTree-engine tables in a background. Default: 8 See in-depth description in ClickHouse documentation |
default_database | google.protobuf.StringValue The default database. To get a list of cluster databases, see Yandex Managed ClickHouse documentation. |
total_memory_profiler_step | google.protobuf.Int64Value Sets the memory size (in bytes) for a stack trace at every peak allocation step. Default value: 4194304. More info see in ClickHouse documentation |
total_memory_tracker_sample_probability | google.protobuf.DoubleValue |
MergeTree
Field | Description |
---|---|
replicated_deduplication_window | google.protobuf.Int64Value Number of blocks of hashes to keep in ZooKeeper. |
replicated_deduplication_window_seconds | google.protobuf.Int64Value Period of time to keep blocks of hashes for. |
parts_to_delay_insert | google.protobuf.Int64Value If table contains at least that many active parts in single partition, artificially slow down insert into table. |
parts_to_throw_insert | google.protobuf.Int64Value If more than this number active parts in single partition, throw 'Too many parts ...' exception. |
inactive_parts_to_delay_insert | google.protobuf.Int64Value |
inactive_parts_to_throw_insert | google.protobuf.Int64Value |
max_replicated_merges_in_queue | google.protobuf.Int64Value How many tasks of merging and mutating parts are allowed simultaneously in ReplicatedMergeTree queue. |
number_of_free_entries_in_pool_to_lower_max_size_of_merge | google.protobuf.Int64Value If there is less than specified number of free entries in background pool (or replicated queue), start to lower maximum size of merge to process. |
max_bytes_to_merge_at_min_space_in_pool | google.protobuf.Int64Value Maximum in total size of parts to merge, when there are minimum free threads in background pool (or entries in replication queue). |
max_bytes_to_merge_at_max_space_in_pool | google.protobuf.Int64Value |
min_bytes_for_wide_part | google.protobuf.Int64Value Minimum number of bytes in a data part that can be stored in Wide format. More info see in ClickHouse documentation |
min_rows_for_wide_part | google.protobuf.Int64Value Minimum number of rows in a data part that can be stored in Wide format. More info see in ClickHouse documentation |
ttl_only_drop_parts | google.protobuf.BoolValue Enables or disables complete dropping of data parts where all rows are expired in MergeTree tables. More info see in ClickHouse documentation |
allow_remote_fs_zero_copy_replication | google.protobuf.BoolValue |
merge_with_ttl_timeout | google.protobuf.Int64Value |
merge_with_recompression_ttl_timeout | google.protobuf.Int64Value |
max_parts_in_total | google.protobuf.Int64Value |
max_number_of_merges_with_ttl_in_pool | google.protobuf.Int64Value |
cleanup_delay_period | google.protobuf.Int64Value |
number_of_free_entries_in_pool_to_execute_mutation | google.protobuf.Int64Value |
max_avg_part_size_for_too_many_parts | google.protobuf.Int64Value The 'too many parts' check according to 'parts_to_delay_insert' and 'parts_to_throw_insert' will be active only if the average part size (in the relevant partition) is not larger than the specified threshold. If it is larger than the specified threshold, the INSERTs will be neither delayed or rejected. This allows to have hundreds of terabytes in a single table on a single server if the parts are successfully merged to larger parts. This does not affect the thresholds on inactive parts or total parts. Default: 1 GiB Min version: 22.10 See in-depth description in ClickHouse GitHub |
min_age_to_force_merge_seconds | google.protobuf.Int64Value Merge parts if every part in the range is older than the value of min_age_to_force_merge_seconds. Default: 0 - disabled Min_version: 22.10 See in-depth description in ClickHouse documentation |
min_age_to_force_merge_on_partition_only | google.protobuf.BoolValue Whether min_age_to_force_merge_seconds should be applied only on the entire partition and not on subset. Default: false Min_version: 22.11 See in-depth description in ClickHouse documentation |
merge_selecting_sleep_ms | google.protobuf.Int64Value Sleep time for merge selecting when no part is selected. A lower setting triggers selecting tasks in background_schedule_pool frequently, which results in a large number of requests to ClickHouse Keeper in large-scale clusters. Default: 5000 Min_version: 21.10 See in-depth description in ClickHouse documentation |
Kafka
Field | Description |
---|---|
security_protocol | enum SecurityProtocol |
sasl_mechanism | enum SaslMechanism |
sasl_username | string |
sasl_password | string |
enable_ssl_certificate_verification | google.protobuf.BoolValue |
max_poll_interval_ms | google.protobuf.Int64Value The minimum value is 0. |
session_timeout_ms | google.protobuf.Int64Value The minimum value is 0. |
KafkaTopic
Field | Description |
---|---|
name | string Required. |
settings | Kafka Required. |
Rabbitmq
Field | Description |
---|---|
username | string RabbitMQ |
password | string RabbitMQ |
vhost | string RabbitMQ |
Compression
Field | Description |
---|---|
method | enum Method Compression method to use for the specified combination of min_part_size and min_part_size_ratio . |
min_part_size | int64 Minimum size of a part of a table. The minimum value is 1. |
min_part_size_ratio | double Minimum ratio of a part relative to the size of all the data in the table. |
level | google.protobuf.Int64Value The minimum value is 0. |
ExternalDictionary
Field | Description |
---|---|
name | string Required. Name of the external dictionary. |
structure | Structure Required. Set of attributes for the external dictionary. For in-depth description, see ClickHouse documentation |
layout | Layout Required. Layout for storing the dictionary in memory. For in-depth description, see ClickHouse documentation |
lifetime | oneof: fixed_lifetime or lifetime_range Setting for the period of time between dictionary updates. For details, see ClickHouse documentation |
fixed_lifetime | int64 Fixed interval between dictionary updates. |
lifetime_range | Range Range of intervals between dictionary updates for ClickHouse to choose from. |
source | oneof: http_source , mysql_source , clickhouse_source , mongodb_source or postgresql_source Description of the source for the external dictionary. |
http_source | HttpSource HTTP source for the dictionary. |
mysql_source | MysqlSource MySQL source for the dictionary. |
clickhouse_source | ClickhouseSource ClickHouse source for the dictionary. |
mongodb_source | MongodbSource MongoDB source for the dictionary. |
postgresql_source | PostgresqlSource PostgreSQL source for the dictionary. |
HttpSource
Field | Description |
---|---|
url | string Required. URL of the source dictionary available over HTTP. |
format | string Required. The data format. Valid values are all formats supported by ClickHouse SQL dialect. |
MysqlSource
Field | Description |
---|---|
db | string Required. Name of the MySQL database to connect to. |
table | string Required. Name of the database table to use as a ClickHouse dictionary. |
port | int64 Default port to use when connecting to a replica of the dictionary source. Acceptable values are 0 to 65535, inclusive. |
user | string Name of the default user for replicas of the dictionary source. |
password | string Password of the default user for replicas of the dictionary source. |
replicas[] | Replica List of MySQL replicas of the database used as dictionary source. The number of elements must be greater than 0. |
where | string Selection criteria for the data in the specified MySQL table. |
invalidate_query | string Query for checking the dictionary status, to pull only updated data. For more details, see ClickHouse documentation on dictionaries |
Replica
Field | Description |
---|---|
host | string Required. MySQL host of the replica. The maximum string length in characters is 253. |
priority | int64 Required. The priority of the replica that ClickHouse takes into account when connecting. Replica with the highest priority should have this field set to the lowest number. Value must be greater than 0. |
port | int64 Port to use when connecting to the replica. If a port is not specified for a replica, ClickHouse uses the port specified for the source. Acceptable values are 0 to 65535, inclusive. |
user | string Name of the MySQL database user. |
password | string Password of the MySQL database user. |
ClickhouseSource
Field | Description |
---|---|
db | string Required. Name of the ClickHouse database. |
table | string Required. Name of the table in the specified database to be used as the dictionary source. |
host | string Required. ClickHouse host of the specified database. The maximum string length in characters is 253. |
port | int64 Port to use when connecting to the host. Acceptable values are 0 to 65535, inclusive. |
user | string Required. Name of the ClickHouse database user. |
password | string Password of the ClickHouse database user. |
where | string Selection criteria for the data in the specified ClickHouse table. |
MongodbSource
Field | Description |
---|---|
db | string Required. Name of the MongoDB database. |
collection | string Required. Name of the collection in the specified database to be used as the dictionary source. |
host | string Required. MongoDB host of the specified database. The maximum string length in characters is 253. |
port | int64 Port to use when connecting to the host. Acceptable values are 0 to 65535, inclusive. |
user | string Required. Name of the MongoDB database user. |
password | string Password of the MongoDB database user. |
options | string |
PostgresqlSource
Field | Description |
---|---|
db | string Required. Name of the PostrgreSQL database. |
table | string Required. Name of the table in the specified database to be used as the dictionary source. |
hosts[] | string Name of the PostrgreSQL host The number of elements must be greater than 0. |
port | int64 Port to use when connecting to the host. Acceptable values are 0 to 65535, inclusive. |
user | string Required. Name of the PostrgreSQL database user. |
password | string Password of the PostrgreSQL database user. |
invalidate_query | string Query for checking the dictionary status, to pull only updated data. For more details, see ClickHouse documentation on dictionaries |
ssl_mode | enum SslMode Mode of SSL TCP/IP connection to the PostgreSQL host. For more details, see PostgreSQL documentation
|
Structure
Field | Description |
---|---|
id | Id Single numeric key column for the dictionary. |
key | Key Composite key for the dictionary, containing of one or more key columns. For details, see ClickHouse documentation |
range_min | Attribute Field holding the beginning of the range for dictionaries with RANGE_HASHED layout. For details, see ClickHouse documentation |
range_max | Attribute Field holding the end of the range for dictionaries with RANGE_HASHED layout. For details, see ClickHouse documentation |
attributes[] | Attribute Description of the fields available for database queries. For details, see ClickHouse documentation |
Attribute
Field | Description |
---|---|
name | string Required. Name of the column. |
type | string Required. Type of the column. |
null_value | string Default value for an element without data (for example, an empty string). |
expression | string Expression, describing the attribute, if applicable. |
hierarchical | bool Indication of hierarchy support. Default value: false . |
injective | bool Indication of injective mapping "id -> attribute". Default value: false . |
Id
Field | Description |
---|---|
name | string Required. Name of the numeric key. |
Key
Field | Description |
---|---|
attributes[] | Attribute Attributes of a complex key. The number of elements must be greater than 0. |
Layout
Field | Description |
---|---|
type | enum Type Required. Layout type for an external dictionary.
|
size_in_cells | int64 Number of cells in the cache. Rounded up to a power of two. Applicable only for CACHE and COMPLEX_KEY_CACHE layout types. |
Range
Field | Description |
---|---|
min | int64 Minimum dictionary lifetime. |
max | int64 Maximum dictionary lifetime. |
GraphiteRollup
Field | Description |
---|---|
name | string Required. Name for the specified combination of settings for Graphite rollup. |
patterns[] | Pattern Pattern to use for the rollup. The number of elements must be greater than 0. |
Pattern
Field | Description |
---|---|
regexp | string Pattern for metric names. |
function | string Required. Name of the aggregating function to apply to data of the age specified in retention . |
retention[] | Retention Age of data to use for thinning. The number of elements must be greater than 0. |
Retention
Field | Description |
---|---|
age | int64 Minimum age of the data in seconds. The minimum value is 0. |
precision | int64 Precision of determining the age of the data, in seconds. Value must be greater than 0. |
Resources
Field | Description |
---|---|
resource_preset_id | string ID of the preset for computational resources available to a host (CPU, memory etc.). All available presets are listed in the documentation |
disk_size | int64 Volume of the storage available to a host, in bytes. |
disk_type_id | string Type of the storage environment for the host. Possible values:
|
MaintenanceWindow
Field | Description |
---|---|
policy | oneof: anytime or weekly_maintenance_window The maintenance policy in effect. |
anytime | AnytimeMaintenanceWindow Maintenance operation can be scheduled anytime. |
weekly_maintenance_window | WeeklyMaintenanceWindow Maintenance operation can be scheduled on a weekly basis. |
AnytimeMaintenanceWindow
Empty.
WeeklyMaintenanceWindow
Field | Description |
---|---|
day | enum WeekDay Day of the week (in DDD format). |
hour | int64 Hour of the day in UTC (in HH format). Acceptable values are 1 to 24, inclusive. |
MaintenanceOperation
Field | Description |
---|---|
info | string Information about this maintenance operation. The maximum string length in characters is 256. |
delayed_until | google.protobuf.Timestamp Time until which this maintenance operation is delayed. |