docker搭建clickhouse集群【笔记】

确保已经搭建好zookeeper集群

可参考博客方式二搭建

1.创建挂载目录

mkdir -p $HOME/data/docker/clickhouse1/{data,logs}

2.先运行clickhouse服务作为临时服务

docker run -d --name=ch1 --ulimit nofile=262144:262144 yandex/clickhouse-server

3.复制配置文件到挂载目录

docker cp ch1:/etc/clickhouse-server/ $HOME/data/docker/clickhouse1/

4.停止并删除启动文件

docker stop ch1 && docker rm ch1

5.配置登录密码

  • 准备一个sha256加密的密码
# 例如密码 123456
echo -n 123456 | openssl dgst -sha256
# 输出结果
# 8d969eef6ecad3c29a3a629280e686cf0c3f5d5a86aff3ca12020c923adc6c92
# 或者使用随机密码(二选一)
PASSWORD=$(base64 < /dev/urandom | head -c8); 
echo "$PASSWORD"; 
# 输出
# EZgaVTP1
echo -n "$PASSWORD" | openssl dgst -sha256
#输出结果
# fcce245f93608febf6d495fb585717a46db3f20c0976a68c76eb0a7f6d3620a9
  • 配置
    修改挂载目录中的user.xml文件(路径在config下,主要是配置账号密码)
    docker搭建clickhouse集群【笔记】
  • 修改config.xml(主要是设置时区、监听设置,解决ipv6可能不能用问题)
     <!-- 引入外部配置,为集群配置做准备,可加在头部位置(yandex标签下,和logger标签同级) -->
	<include_from>/etc/clickhouse/cluster.xml</include_from>
	<!-- 时区设置 -->
    <timezone>Asia/Shanghai</timezone>
    <!-- ip支持设置 -->
    <listen_host>0.0.0.0</listen_host>

6. 复制clickhouse1文件夹,并分别重命名为clickhouse2、clickhouse3

cp -r clickhouse1 clickhouse2
cp -r clickhouse1 clickhouse3

7. 进入/etc/文件下,新建metrika.xml文件

输入以下内容

<!--所有实例均使用这个集群配置,不用个性化 -->
<yandex>
    <!-- 集群配置 -->
    <clickhouse_remote_servers>
        <cluster_shard>
            <!-- 数据分片1  -->
            <shard>
                <internal_replication>true</internal_replication>
                <replica>
                    <host>ch2</host>
                    <port>19104</port>
                    <user>default</user>
                    <password>123456</password>
                </replica>
                <replica>
                    <host>ch3</host>
                    <port>19107</port>
                    <user>default</user>
                    <password>123456</password>
                </replica>
            </shard>

            <!-- 数据分片2  -->
            <shard>
                <internal_replication>true</internal_replication>
                <replica>
                    <host>ch1</host>
                    <port>19101</port>
                    <user>default</user>
                    <password>123456</password>
                </replica>
                <replica>
                    <host>ch3</host>
                    <port>19107</port>
                    <user>default</user>
                    <password>123456</password>
                </replica>
            </shard>

            <!-- 数据分片3  -->
            <shard>
                <internal_replication>true</internal_replication>
                <replica>
                    <host>ch1</host>
                    <port>19101</port>
                    <user>default</user>
                    <password>123456</password>
                </replica>
                <replica>
                    <host>ch2</host>
                    <port>19104</port>
                    <user>default</user>
                    <password>123456</password>
                </replica>
            </shard>
        </cluster_shard>
    </clickhouse_remote_servers>

    <!-- zookeeper_servers所有实例配置都一样, 这里zookeeper根据你自己情况更改-->
    <zookeeper-servers>
        <node index="1">
            <host>100.0.0.117</host>
            <port>2181</port>
        </node>
        <node index="2">
            <host>100.0.0.117</host>
            <port>2182</port>
        </node>
        <node index="3">
            <host>100.0.0.117</host>
            <port>2183</port>
        </node>
    </zookeeper-servers>

    <!-- marcos每个实例配置不一样,统一使用 分片5, 副本5 -->
    <macros>
        <layer>05</layer>
        <shard>05</shard>
    </macros>

    <networks>
        <ip>::/0</ip>
    </networks>

    <!-- 数据压缩算法  -->
    <clickhouse_compression>
        <case>
            <min_part_size>10000000000</min_part_size>
            <min_part_size_ratio>0.01</min_part_size_ratio>
            <method>lz4</method>
        </case>
    </clickhouse_compression>
</yandex>

8. 新建docker-compose.yml文件

输入以下内容

version: '3.1'
services:
  ch1:
    image: yandex/clickhouse-server
    restart: always
    container_name: ch1
    ports:
      - 19101:9000
      - 19102:8123
      - 19103:9009
    volumes:
      - /etc/localtime:/etc/localtime
      - $HOME/data/docker/clickhouse1/data:/var/lib/clickhouse
      - $HOME/data/docker/clickhouse1/config/:/etc/clickhouse-server/
      - $HOME/data/docker/clickhouse1/logs:/var/log/clickhouse-server/
    networks:
      default:
        ipv4_address: 100.0.0.120
  ch2:
    image: yandex/clickhouse-server
    restart: always
    container_name: ch2
    ports:
      - 19104:9000
      - 19105:8123
      - 19106:9009
    volumes:
      - /etc/localtime:/etc/localtime
      - $HOME/data/docker/clickhouse2/data:/var/lib/clickhouse
      - $HOME/data/docker/clickhouse2/config/:/etc/clickhouse-server/
      - $HOME/data/docker/clickhouse2/logs:/var/log/clickhouse-server/
    networks:
      default:
        ipv4_address: 100.0.0.121
  ch3:
    image: yandex/clickhouse-server
    restart: always
    container_name: ch3
    ports:
      - 19107:9000
      - 19108:8123
      - 19109:9009
    volumes:
      - /etc/localtime:/etc/localtime
      - $HOME/data/docker/clickhouse3/data:/var/lib/clickhouse
      - $HOME/data/docker/clickhouse3/config/:/etc/clickhouse-server/
      - $HOME/data/docker/clickhouse3/logs:/var/log/clickhouse-server/
    networks:
      default:
        ipv4_address: 100.0.0.122

networks:
  default:
    external:
      name: network1

9.启动容器

进入到docker-compose.yml文件所在的目录,执行指令

docker-compose up -d

10.用连接工具测试

jdbc:clickhouse://x.x.x.x:19102/default?socket_timeout=300000

上一篇:【clickhouse】阿里clickhouse 随便查询一条数据都报错 read time out


下一篇:clickhouse表引擎和去重