This Terraform project sets up a ClickHouse cluster with 3 nodes in Google Cloud Platform:

• 2 nodes running both ClickHouse data server and ClickHouse Keeper
• 1 node running only ClickHouse Keeper for quorum (no data)

• Google Cloud Platform account with a project
• Terraform installed locally
• Google Cloud SDK installed
• SSH key pair
• Change default password in modules/clickhouse/user_admin.xml with ( echo -n 'your_new_strong_password' | sha256sum | tr -d ' -' )

1. Set up your Google Cloud credentials:

export GOOGLE_APPLICATION_CREDENTIALS="$HOME/.googlecloud.json"

2. Update the private key path in the terraform.tfvars file:

private_key_path = "~/.ssh/id_ed25519"  # Path to your private SSH key

To deploy the ClickHouse cluster:

1. Navigate to the dev environment:

cd environments/dev

2. Initialize Terraform:

terraform init

3. Apply the Terraform plan:

terraform apply

4. After deployment completes (approx. 5-10 minutes), you'll get outputs with the ClickHouse endpoints:

clickhouse_endpoints = {
  "chnode1" = "x.x.x.x:8123"
  "chnode2" = "y.y.y.y:8123"
  "chnode3" = "z.z.z.z:8123"
}

clickhouse_clients = {
  "chnode1" = "x.x.x.x:9000"
  "chnode2" = "y.y.y.y:9000"
  "chnode3" = "z.z.z.z:9000"
}

You can connect to your ClickHouse cluster using the clickhouse-client:

clickhouse-client --host=x.x.x.x --port=9000

Or via HTTP:

curl "http://x.x.x.x:8123/?query=SELECT%20*%20FROM%20db1.table1_dist"

The cluster is configured with:

• 2 shards (one on each data node)
• 1 replica per shard
• A distributed table to query data across all shards

Each node has the following configuration files:

1. network-and-logging.xml - Basic network and logging configuration
2. enable-keeper.xml - ClickHouse Keeper configuration with server ID
3. macros.xml - Defines shard and replica macros (data nodes only)
4. remote-servers.xml - Cluster configuration
5. use-keeper.xml - Configuration to use ClickHouse Keeper for coordination

To destroy the infrastructure:

terraform destroy