一、背景
一是为了学习容器,另外也是帮助一些同学解决机器资源紧缺,能够在一台是宿主机部署一套clickhouse集群服务。
本章内容适合开发测试环境使用,生产环境还需要更多细节的处理工作,只能作为参考。
二、部署准备
1、机器准备
172.25.16.108
2、创建容器网络
docker network create ckcluster
3、镜像准备
# 查看仓库镜像列表
curl -XGET https://nexus.***.com:8445/v2/_catalog | json_reformat
# 查看clickhouse镜像信息
curl -XGET https://nexus.***.com:8445/v2/yandex/clickhouse-server/tags/list | json_reformat
# 查看zookeeper镜像信息
curl -XGET https://nexus.***.com:8445/v2/zookeeper/tags/list | json_reformat
三、zookeeper单机安装(1节点)
1、下载镜像
docker pull nexus.***.com:8445/zookeeper:3.8.0
2、容器启动
docker run --restart always -d \
--name zookeeper \
--ulimit nofile=262144:262144 \
--hostname zookeeper \
--network ckcluster \
-p 2181:2181 \
nexus.***.com:8445/zookeeper:3.8.0
四、clickhouse集群安装(3分片1副本)
1、创建目录
cd /home
mkdir ckcluster
cd cluster
mkdir ck1
mkdir ck2
mkdir ck3
2、创建节点配置
# 启动ck1
docker run -d --name clickhouse-server-1 --ulimit nofile=262144:262144 -v /home/ckcluster/ck1/:/var/lib/clickhouse -p 18123:8123 nexus.***.com:8445/yandex/clickhouse-server:latest
# 拷贝容器内容的配置到/data/clickhouse/目录下
docker cp clickhouse-server-1:/etc/clickhouse-server/ /home/ckcluster/ck1/
# 启动ck2
docker run -d --name clickhouse-server-2 --ulimit nofile=262144:262144 -v /home/ckcluster/ck2/:/var/lib/clickhouse -p 28123:8123 nexus.***.com:8445/yandex/clickhouse-server:latest
# 拷贝容器内容的配置到/data/clickhouse/目录下
docker cp clickhouse-server-2:/etc/clickhouse-server/ /home/ckcluster/ck2/
# 启动ck3
docker run -d --name clickhouse-server-3 --ulimit nofile=262144:262144 -v /home/ckcluster/ck3/:/var/lib/clickhouse -p 38123:8123 nexus.***.com:8445/yandex/clickhouse-server:latest
# 拷贝容器内容的配置到/data/clickhouse/目录下
docker cp clickhouse-server-3:/etc/clickhouse-server/ /home/ckcluster/ck3/
3、清除临时容器
docker stop **
docker rm **
4、修改config.xml配置
vim /etc/clickhouse-server/config.xml
<include_from>/etc/clickhouse-server/config.d/metrika.xml</include_from> <!--添加该参数-->
5、增加服务节点配置文件【metrika.xml】
cd /home/ckcluster/ck1/
cd clickhouse-server
vim config.d/metrika.xml
### 在metrika.xml中我们配置后期使用的clickhouse集群中创建分布式表时使用3个分片,每个分片有1个副本,配置如下:
<yandex>
<remote_servers>
<clickhouse_cluster_3shards_1replicas>
<shard>
<internal_replication>true</internal_replication>
<replica>
<host>clickhouse-server01</host>
<port>19000</port>
</replica>
</shard>
<shard>
<internal_replication>true</internal_replication>
<replica>
<host>clickhouse-server02</host>
<port>29000</port>
</replica>
</shard>
<shard>
<internal_replication>true</internal_replication>
<replica>
<host>clickhouse-server03</host>
<port>39000</port>
</replica>
</shard>
</clickhouse_cluster_3shards_1replicas>
</remote_servers>
<zookeeper>
<node index="1">
<host>zookeeper</host>
<port>2181</port>
</node>
</zookeeper>
<macros>
<shard>01</shard>
<replica>node1</replica>
</macros>
<networks>
<ip>0.0.0.0/0</ip>
</networks>
<clickhouse_compression>
<case>
<min_part_size>10000000000</min_part_size>
<min_part_size_ratio>0.01</min_part_size_ratio>
<method>lz4</method>
</case>
</clickhouse_compression>
</yandex>
注意:需要在每台clickhouse节点上配置metrika.xml文件,并且修改每个节点的 macros配置名称。
<!-- node2节点修改metrika.xml中的宏变量如下:-->
<macros>
<shard>02</shard>
<replica>node2</replica>
</macros>
<!-- node3节点修改metrika.xml中的宏变量如下: -->
<macros>
<shard>03</shard>
<replica>node3</replica>
</macros>
6、启动服务
节点1
# clickhouse-server01
docker run --restart always -d \
--name clickhouse-server01 \
--ulimit nofile=262144:262144 \
--hostname clickhouse-server01 \
--network ckcluster \
-v /home/ckcluster/ck1/data/:/var/lib/clickhouse/ \
-v /home/ckcluster/ck1/clickhouse-server/:/etc/clickhouse-server/ \
-v /home/ckcluster/ck1/logs/:/var/log/clickhouse-server/ \
-p 19000:9000 -p 18123:8123 -p 19009:9009 \
nexus.***.com:8445/yandex/clickhouse-server:latest
节点2
# clickhouse-server02
docker run --restart always -d \
--name clickhouse-server02 \
--ulimit nofile=262144:262144 \
--hostname clickhouse-server02 \
--network ckcluster \
-v /home/ckcluster/ck2/data/:/var/lib/clickhouse/ \
-v /home/ckcluster/ck2/clickhouse-server/:/etc/clickhouse-server/ \
-v /home/ckcluster/ck2/logs/:/var/log/clickhouse-server/ \
-p 29000:9000 -p 28123:8123 -p 29009:9009 \
nexus.***.com:8445/yandex/clickhouse-server:latest
节点3
# clickhouse-server03
docker run --restart always -d \
--name clickhouse-server03 \
--ulimit nofile=262144:262144 \
--hostname clickhouse-server03 \
--network ckcluster \
-v /home/ckcluster/ck3/data/:/var/lib/clickhouse/ \
-v /home/ckcluster/ck3/clickhouse-server/:/etc/clickhouse-server/ \
-v /home/ckcluster/ck3/logs/:/var/log/clickhouse-server/ \
-p 39000:9000 -p 38123:8123 -p 39009:9009 \
nexus.***.com:8445/yandex/clickhouse-server:latest
7、验证
可以通过客户端工具DBeaver访问。文章来源:https://www.toymoban.com/news/detail-658101.html
参考文章
Docker搭建Clickhouse集群_docker clickhouse_这学习真难的博客-CSDN博客文章来源地址https://www.toymoban.com/news/detail-658101.html
到了这里,关于OLAP系列:三、clickhouse Docker集群部署指南(3分片1副本模式)的文章就介绍完了。如果您还想了解更多内容,请在右上角搜索TOY模板网以前的文章或继续浏览下面的相关文章,希望大家以后多多支持TOY模板网!