docker 安装 kafka集群 (SASL)

2024-04-05 15:02:44

1. 下载安装包

从官网或者从镜像服务站下载,本次下载的kafka版本为 2.12-2.3.0

kafka 添加jaas配置文件件
mkdir -p $kafka_dir/.conf; vi $kafka_dir/.conf/kafka_server_jaas.conf;
- kafka_server_jaas.conf 内容
  KafkaServer { org.apache.kafka.common.security.plain.PlainLoginModule required username="admin" password="admin-sec" user_admin="admin-sec" user_producer="prod-sec" user_consumer="cons-sec"; };
Dockerfile 文件编写
FROM centos:7 MAINTAINER ozh ADD ./kafka_2.12.tgz /opt/ RUN mkdir -p /tmp/kafka-logs && \ mkdir -p /data && \ mkdir -p /conf && \ mv /opt/kafka_2.12 /kafka && \ yum install -y java-1.8.0-openjdk* ENV PATH /kafka/bin:$PATH CMD /usr/sbin/init
打包
docker build -t kafka-2.12:0.0.1 .
查看镜像包 docker images |grep kafka-2.12

本次安装了3个节点,分别为kafka1,kafka2,kafka3

目录结构
- kafka1
  - conf
  - server.properties (配置文件)
- tmp
  - kafka-logs (数据存储目录)
server.properties 内容
roker.id=1 port=9092 num.network.threads=2 num.io.threads=8 socket.send.buffer.bytes=1048576 socket.receive.buffer.bytes=1048576 socket.request.max.bytes=104857600 log.dirs=/tmp/kafka-logs num.partitions=2 log.retention.hours=168 log.segment.bytes=536870912 log.retention.check.interval.ms=60000 log.cleaner.enable=false zookeeper.connect=zook1:2181,zook2:2181,zook3:2181 zookeeper.connection.timeout.ms=60000 #listeners=PLAINTEXT://0.0.0.0:9092 #SSL config security.inter.broker.protocol=SASL_PLAINTEXT sasl.enabled.mechanisms=PLAIN sasl.mechanism.inter.broker.protocol=PLAIN authorizer.class.name=kafka.security.auth.SimpleAclAuthorizer allow.everyone.if.no.acl.found=true listeners=SASL_PLAINTEXT://0.0.0.0:9092
roker.id: 服务ID,跟据服务进行调整
zookeeper.connect :zookeeper连接池

为方便操作, 写了一个shell脚本来执行
ip_addr=111; port=9092; host_port=9092; for id inseq 1 3;do docker run -d -ti \ -p ${host_port}:${port} \ -v $PWD/kafka${id}/conf:/conf \ -v $PWD/kafka${id}/tmp/kafka-logs:/tmp/kafka-logs \ --restart=always \ --add-host zook1:172.172.66.101 \ --add-host zook2:172.172.66.102 \ --add-host zook3:172.172.66.103 \ --name kafka-${id} \ --net redis-net \ --ip 172.172.66.${ip_addr} \ kafka-2.12:0.0.1 kafka-server-start_sasl.sh /conf/server.properties let ip_addr+=1; let host_port+=1; done
–net redis-net : 这是这前在docker上创建的虚拟网络,方便内部通讯

docker pull dockerkafka/kafka-manager
docker run -it -d \ --env ZK_HOSTS=zook1:2181,zook2:2181,zook3:2181 \ --restart=always \ -p 39000:9000 \ --name kafka-manager \ --net redis-net \ --ip 172.172.66.100 \ dockerkafka/kafka-manager
登陆 host:39000添加 Clusters