Connecting to a database in a ClickHouse cluster
In Yandex.Cloud, you can only connect to a DB cluster from a VM that is in the same subnet as the cluster.
A ClickHouse cluster can be accessed using the command-line client (port 9440) or HTTP interface (port 8443). All connections to DB clusters are encrypted.
Getting an SSL certificate
To use an encrypted connection, you should prepare an SSL certificate, for example, like this:
sudo mkdir -p /usr/local/share/ca-certificates/Yandex && \
sudo wget "https://storage.yandexcloud.net/cloud-certs/CA.pem" -O /usr/local/share/ca-certificates/Yandex/YandexInternalRootCA.crt && \
sudo chmod 655 /usr/local/share/ca-certificates/Yandex/YandexInternalRootCA.crt
Sample connection strings
Examples were tested in the following environment:
- Virtual machine in Yandex.Cloud running Ubuntu 20.04 LTS.
- Bash:
5.0.16
. - Python:
3.8.2
; pip3:20.0.2
. - PHP:
7.4.3
. - OpenJDK:
11.0.8
; Maven:3.6.3
. - Node.JS:
10.19.0
, npm:6.14.4
. - Go:
1.13.8
. - Ruby:
2.7.0p0
. - unixODBC:
2.3.6
.
You can only connect to a ClickHouse cluster using an SSL certificate. Before connecting to clusters, prepare a certificate.
In the examples below, it is assumed that the YandexInternalRootCA.crt
certificate is located in the /usr/local/share/ca-certificates/Yandex/
directory.
To view an example of the command with the host FQDN filled in, open the cluster page in the management console and click Connect.
Before connecting:
-
Add the ClickHouse DEB repository:
sudo apt update && sudo apt install -y apt-transport-https ca-certificates dirmngr && \ sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv E0C56BD4 && \ echo "deb https://repo.clickhouse.tech/deb/stable/ main/" | sudo tee \ /etc/apt/sources.list.d/clickhouse.list
-
Install the dependencies:
sudo apt update && sudo apt install -y clickhouse-client
-
Download the configuration file for
clickhouse-client
:mkdir -p ~/.clickhouse-client && wget "https://storage.yandexcloud.net/mdb/clickhouse-client.conf.example" -O ~/.clickhouse-client/config.xml
Connecting via SSL:
clickhouse-client --host <FQDN of any ClickHouse host> \
--secure \
--user <username> \
--database <database name> \
--port 9440 \
--ask-password
After running the command, enter the user password to complete the connection procedure.
After connecting to the DBMS, run the command SELECT version();
.
Connecting via SSL:
curl --cacert /usr/local/share/ca-certificates/Yandex/YandexInternalRootCA.crt \
-H "X-ClickHouse-User: <DB user name>" \
-H "X-ClickHouse-Key: <DB user password>" \
'https://<FQDN of any ClickHouse host>:8443/?database=<DB name>&query=SELECT%20version()'
Before connecting, install the dependencies:
sudo apt update && sudo apt install -y python3 python3-pip && \
pip3 install requests
Code example for connecting via SSL:
connect.py
import requests
url = 'https://{host}:8443/?database={db}&query={query}'.format(
host='<FQDN of any ClickHouse host>',
db='<DB name>',
query='SELECT version()')
auth = {
'X-ClickHouse-User': '<DB username>',
'X-ClickHouse-Key': '<DB user password>',
}
cacert = '/usr/local/share/ca-certificates/Yandex/YandexInternalRootCA.crt'
rs = requests.get(url, headers=auth, verify=cacert)
rs.raise_for_status()
print(rs.text)
Connecting:
python3 connect.py
Before connecting:
-
Install the dependencies:
sudo apt update && apt install -y php
-
Make sure that the
allow_url_fopen
parameter is set toOn
in the PHP settings:cat /etc/php/7.4/apache2/php.ini | grep "allow_url_fopen"
Otherwise, set this value and restart Apache:
sudo sed -i 's/\(^allow_url_fopen = \).*/\1On/' /etc/php/7.4/apache2/php.ini && \ sudo systemctl restart apache2
Code example for connecting via SSL:
connect.php
<?php
$host = '<FQDN of any ClickHouse host>';
$db = '<DB name>';
$query = 'SELECT version()';
$auth = [
'X-ClickHouse-User: <DB username>',
'X-ClickHouse-Key: <DB user password>',
];
$ssl = [
'cafile' => '/usr/local/share/ca-certificates/Yandex/YandexInternalRootCA.crt',
'verify_peer' => true,
];
$context = stream_context_create([
'http' => [
'method' => 'GET',
'protocol_version' => 1.1,
'header' => $auth
],
'ssl' => $ssl
]);
$url = sprintf('https://%s:8443/?database=%s&query=%s', $host, $db, urlencode($query));
$rs = file_get_contents($url, false, $context);
print_r($rs);
?>
Connecting:
php connect.php
Before connecting:
-
Install the dependencies:
sudo apt update && sudo apt install -y default-jdk maven
-
Create a folder for the Maven project:
cd ~/ && mkdir -p project/src/java/com/example && cd project/
-
Create a configuration file for Maven:
pom.xml<?xml version="1.0" encoding="UTF-8"?> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd"> <modelVersion>4.0.0</modelVersion> <groupId>com.example</groupId> <artifactId>app</artifactId> <packaging>jar</packaging> <version>0.1.0</version> <properties> <maven.compiler.source>1.8</maven.compiler.source> <maven.compiler.target>1.8</maven.compiler.target> </properties> <dependencies> <dependency> <groupId>ru.yandex.clickhouse</groupId> <artifactId>clickhouse-jdbc</artifactId> <version>0.2.4</version> </dependency> <dependency> <groupId>org.slf4j</groupId> <artifactId>slf4j-simple</artifactId> <version>1.7.30</version> </dependency> </dependencies> <build> <finalName>${project.artifactId}-${project.version}</finalName> <sourceDirectory>src</sourceDirectory> <resources> <resource> <directory>src</directory> </resource> </resources> <plugins> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-assembly-plugin</artifactId> <executions> <execution> <goals> <goal>attached</goal> </goals> <phase>package</phase> <configuration> <descriptorRefs> <descriptorRef>jar-with-dependencies</descriptorRef> </descriptorRefs> <archive> <manifest> <mainClass>com.example.App</mainClass> </manifest> </archive> </configuration> </execution> </executions> </plugin> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-jar-plugin</artifactId> <version>3.1.0</version> <configuration> <archive> <manifest> <mainClass>com.example.App</mainClass> </manifest> </archive> </configuration> </plugin> </plugins> </build> </project>
Up-to-date versions of dependencies for Maven:
Code example for connecting via SSL:
src/java/com/example/App.java
package com.example;
import java.sql.*;
public class App {
public static void main(String[] args) {
String DB_HOST = "<FQDN of any ClickHouse host>";
String DB_NAME = "<DB name>";
String DB_USER = "<DB username>";
String DB_PASS = "<DB user password>";
String CACERT = "/usr/local/share/ca-certificates/Yandex/YandexInternalRootCA.crt";
String DB_URL = String.format("jdbc:clickhouse://%s:8443/%s?ssl=1&sslmode=strict&sslrootcert=%s", DB_HOST, DB_NAME, CACERT);
try {
Class.forName("ru.yandex.clickhouse.ClickHouseDriver");
Connection conn = DriverManager.getConnection(DB_URL, DB_USER, DB_PASS);
ResultSet rs = conn.createStatement().executeQuery("SELECT version()");
if(rs.next()) {System.out.println(rs.getString(1));}
conn.close();
}
catch(Exception ex) {ex.printStackTrace();}
}
}
Connecting:
mvn clean package && \
java -jar target/app-0.1.0-jar-with-dependencies.jar
Before connecting, install the dependencies:
sudo apt update && sudo apt install -y nodejs npm && \
npm install querystring
Code example for connecting via SSL:
app.js
"use strict"
const https = require('https');
const querystring = require('querystring');
const fs = require('fs');
const DB_HOST = "<FQDN of any ClickHouse host>";
const DB_NAME = "<DB name>";
const DB_USER = "<DB username>";
const DB_PASS = "<DB user password>";
const CACERT = "/usr/local/share/ca-certificates/Yandex/YandexInternalRootCA.crt";
const options = {
'method': 'GET',
'ca': fs.readFileSync(CACERT),
'path': '/?' + querystring.stringify({
'database': DB_NAME,
'query': 'SELECT version()',
}),
'port': 8443,
'hostname': DB_HOST,
'headers': {
'X-ClickHouse-User': DB_USER,
'X-ClickHouse-Key': DB_PASS,
},
};
const rs = https.request(options, (res) => {
res.setEncoding('utf8');
res.on('data', (chunk) => {
console.log(chunk);
});
});
rs.end();
Connecting:
node app.js
Before connecting, install the dependencies:
sudo apt update && sudo apt install -y golang git
Code example for connecting via SSL:
connect.go
package main
import (
"fmt"
"net/http"
"io/ioutil"
"crypto/x509"
"crypto/tls"
)
func main() {
const DB_HOST = "<FQDN of any ClickHouse host>"
const DB_NAME = "<DB name>"
const DB_USER = "<DB username>"
const DB_PASS = "<DB user password>"
const CACERT = "/usr/local/share/ca-certificates/Yandex/YandexInternalRootCA.crt"
caCert, err := ioutil.ReadFile(CACERT)
if err != nil {
panic(err)
}
caCertPool := x509.NewCertPool()
caCertPool.AppendCertsFromPEM(caCert)
conn := &http.Client{
Transport: &http.Transport{
TLSClientConfig: &tls.Config{
RootCAs: caCertPool,
},
},
}
req, _ := http.NewRequest("GET", fmt.Sprintf("https://%s:8443/", DB_HOST), nil)
query := req.URL.Query()
query.Add("database", DB_NAME)
query.Add("query", "SELECT version()")
req.URL.RawQuery = query.Encode()
req.Header.Add("X-ClickHouse-User", DB_USER)
req.Header.Add("X-ClickHouse-Key", DB_PASS)
resp, err := conn.Do(req)
if err != nil {
panic(err)
}
defer resp.Body.Close()
data, _ := ioutil.ReadAll(resp.Body)
fmt.Println(string(data))
}
Connecting:
go run connect.go
Before connecting, install the dependencies:
sudo apt update && sudo apt install -y ruby
Code example for connecting via SSL:
connect.rb
require "net/http"
require "openssl"
require "uri"
DB_HOST = "<FQDN of any ClickHouse host>"
DB_NAME = "<DB name>"
DB_USER = "<DB username>"
DB_PASS = "<DB user password>"
QUERYSTRING = { :database => DB_NAME, :query => "SELECT version()" }
uri = URI("https://" + DB_HOST + "/")
uri.port = 8443
uri.query = URI.encode_www_form(QUERYSTRING)
req = Net::HTTP::Get.new(uri)
req.add_field("X-ClickHouse-User", DB_USER)
req.add_field("X-ClickHouse-Key", DB_PASS)
conn = Net::HTTP.new(uri.host, uri.port)
conn.ca_file = "/usr/local/share/ca-certificates/Yandex/YandexInternalRootCA.crt"
conn.use_ssl = true
conn.verify_mode = OpenSSL::SSL::VERIFY_PEER
rs = conn.request(req)
puts rs.body
Connecting:
ruby connect.rb
Before connecting:
-
Install the dependencies:
sudo apt update && sudo apt install -y git unixodbc build-essential cmake \ libpoco-dev libssl-dev libicu-dev unixodbc-dev && \ cd ~/ && git clone https://github.com/ClickHouse/clickhouse-odbc.git && \ cd clickhouse-odbc/ && git submodule update --init
-
Build the driver from the source files:
cd ~/clickhouse-odbc/ && mkdir build && cd build && \ cmake -DCMAKE_BUILD_TYPE=RelWithDebInfo .. && \ cmake --build . --config RelWithDebInfo
-
After the build process is complete, copy the driver files to the
/usr/local/lib64/
directory:sudo mkdir -p /usr/local/lib64 && sudo cp driver/*.so /usr/local/lib64/
-
Register the ClickHouse ODBC driver by adding the following lines to the
odbcinst.ini
file (sample file):/etc/odbcinst.ini[ODBC Drivers] ClickHouse ODBC Driver (ANSI) = Installed ClickHouse ODBC Driver (Unicode) = Installed [ClickHouse ODBC Driver (ANSI)] Description = ODBC Driver (ANSI) for ClickHouse Driver = /usr/local/lib64/libclickhouseodbc.so Setup = /usr/local/lib64/libclickhouseodbc.so UsageCount = 1 [ClickHouse ODBC Driver (Unicode)] Description = ODBC Driver (Unicode) for ClickHouse Driver = /usr/local/lib64/libclickhouseodbcw.so Setup = /usr/local/lib64/libclickhouseodbcw.so UsageCount = 1
Example of settings in the 'odbc.ini' file for connecting via SSL:
/etc/odbc.ini
[ClickHouse]
Driver = ClickHouse ODBC Driver (Unicode)
Server = <FQDN of any ClickHouse host>
Database = <DB name>
UID = <DB username>
PWD = <DB user password>
Port = 8443
Proto = https
SSLMode = allow
CertificateFile = /usr/local/share/ca-certificates/Yandex/YandexInternalRootCA.crt
Connecting:
isql -v ClickHouse
After connecting to the DBMS, run the command SELECT version();
.
If the connection to the cluster and the test query are successful, the ClickHouse version is output.
Selecting an available host automatically
If you don't want to manually connect to another host in case the current one becomes unavailable, use an address like this:
-
c-<cluster ID>.rw.mdb.yandexcloud.net
to connect to the cluster master host. -
<shard name>.c-<cluster ID>.rw.mdb.yandexcloud.net
to connect to the shard master host.
If the host that this address points to becomes unavailable, there may be a slight delay before the address starts pointing to another available host.