kafka-manager配置安装及Kerberos(CDH)认证

kafka-manager配置安装Kerberos(CDH)认证

 

1、安装包(已编译)

   Kafka-manager安装包版本为1.3.3.22, 低版本可能不支持kerberbos 认证。

2、解压安装包及配置

   #unzip -d kafka-manager kafka-manager-1.3.3.22.zip

   #cd /user/kafka-manager

   bin  conf  lib  logs  README.md  script  share

3、认证配置

(1) 修改 conf/application.conf文件中zk的地址以及启用Kafka-Manager使用账号登录和消费者配置

##zkhosts

kafka-manager.zkhosts="dcdl-test-datanode1.essence.com:2181,dcdl-test-datanode2.essence.com:2181,dcdl-test-namenode1.essence.com:2181/kafka2"  #配置连kafka zk

kafka-manager.zkhosts=${?ZK_HOSTS}

pinned-dispatcher.type="PinnedDispatcher"

pinned-dispatcher.executor="thread-pool-executor"

application.features=["KMClusterManagerFeature","KMTopicManagerFeature","KMPreferredReplicaElectionFeature","KMReassignPartitionsFeature"]

akka {

  loggers = ["akka.event.slf4j.Slf4jLogger"]

  loglevel = "INFO"

}

akka.logger-startup-timeout = 60s    

basicAuthentication.enabled=true  #开启登录kafka-manager验证

basicAuthentication.enabled=${?KAFKA_MANAGER_AUTH_ENABLED}

basicAuthentication.username="admin"

basicAuthentication.username=${?KAFKA_MANAGER_USERNAME}

basicAuthentication.password="admin"

basicAuthentication.password=${?KAFKA_MANAGER_PASSWORD}

basicAuthentication.realm="Kafka-Manager"

basicAuthentication.excluded=["/api/health"] # ping the health of your instance without authentification

 

kafka-manager.consumer.properties.file=/user/kafka-manager/conf/consumer.properties  #配置消费者文件

#kafka-manager.consumer.properties.file=${?CONSUMER_PROPERTIES_FILE}

 

(2) 修改conf/consumer.properties内容如下:

 

security.protocol=SASL_PLAINTEXT

key.deserializer=org.apache.kafka.common.serialization.ByteArrayDeserializer

value.deserializer=org.apache.kafka.common.serialization.ByteArrayDeserializer

sasl.mechanism=GSSAPI

sasl.kerberos.service.name=kafka

 

 

(3) 配置kerberbos 认证文件,创建conf/manager_jaas.conf

# 其中Clinet段用于连接zookeeper认证,KafkaClient用于连接kafka服务器认证

Client{

   com.sun.security.auth.module.Krb5LoginModule required

   useKeyTab=true

   keyTab="/home/kafka/kafka.keytab"

   principal="kafka@ESSENCE.COM";

};

KafkaClient{

   com.sun.security.auth.module.Krb5LoginModule required

   useKeyTab=true

   keyTab="/home/kafka/kafka.keytab"

   principal="kafka@ESSENCE.COM";

};

 

4、创建kafka-manager 启动脚本, /user/kafka-manager/script/start.sh 脚本内容如下:

#bin/bash

echo ‘-----‘$(date +%F%t%T)> manager.out

export ZK_HOSTS="dcdl-test-datanode1.essence.com:2181,dcdl-test-datanode2.essence.com:2181,

dcdl-test-namenode1.essence.com:2181/kafka2"

# kafka-manager HOME

MANAGER_HOME=/user/kafka-manager

KAFKA_MANAGER=$MANAGER_HOME/bin/kafka-manager

APP_HOME=-Dapplication.home=$MANAGER_HOME

HTTP_PORT=-Dhttp.port=8090  

# SASL

JAAS_CONF=-Djava.security.auth.login.config=$MANAGER_HOME/conf/manager_jaas.conf

KRB5_CONF=-Djava.security.krb5.conf=$MANAGER_HOME/conf/krb5.conf

 

nohup  $KAFKA_MANAGER $JAAS_CONF $KRB5_CONF $APP_HOME $HTTP_PORT >manager.out 2>&1 &

 

echo "$!"

echo "$!" >mpid

tailf manager.out

 

5、添加kafka-manager停止脚本,/user/kafka-manager/script/stop.sh 内容如下:

echo ‘----------------------------------‘$(date +%F%t%T)> manager.out

ps -ef |grep kafka-manager | grep -v grep |awk ‘{print $2}‘| xargs kill

rm -rf /user/kafka-manager/RUNNING_PID

tailf manager.out

 

6、kafka-manager 监控界面介绍; 登录地址:http://10.2.98.128:8090

(1) 添加kafka集群,kerberbos认证参数设置

 kafka-manager配置安装及Kerberos(CDH)认证

 

 kafka-manager配置安装及Kerberos(CDH)认证

 

 

 

 

(2) CDH环境修改broker_java_opts和mirror_maker_java_opts两个配置项,开放外部访问jmx_port

 kafka-manager配置安装及Kerberos(CDH)认证

 

 

把这两项中的-Dcom.sun.management.jmxremote.local.only参数改为false,并且删除掉-Djava.rmi.server.hostname和-Dcom.sun.management.jmxremote.host这两个配置参数,然后重启kafka,jmx即可对外访问

 

(3) kafka集群添加保存完成后,进入监控界面\

监控kafka topic的offset,

 kafka-manager配置安装及Kerberos(CDH)认证

 

 

 

kafka-manager配置安装及Kerberos(CDH)认证

上一篇:XSS


下一篇:使用iView中的Modal组件点击确定使其不消失的方法