一、采集（使用rsyslog）

客户端使用rsyslog8.19.0做的收集，直接centos安装rpm包，安装详细见：

http://www.rsyslog.com/rhelcentos-rpms/

将yum源配置好后：

1、yum install rsyslog -y
2、yum install rsyslog-kafka -y

二、配置文件示例

module(load="imfile")
module(load="omkafka")
$PreserveFQDN on
main_queue(
queue.workerthreads="10" # threads to work on the queue
queue.dequeueBatchSize="1000" # max number of messages to process at once
queue.size="50000" # max queue size
)

tomcat

$template api-tomcat,"%hostname% ◊ %msg%"
ruleset(name="api-tomcat") {
action(
broker=["1.1.1.1:9092,1.1.1.2:9092,1.1.1.3:9092"]
type="omkafka"
topic="api-tomcat"
template="api-tomcat"
partitions.auto="on"
)
}
input(type="imfile"
File="/www/webcode/api/logs/info.log"
Tag="m.w"
ruleset="api-tomcat"
startmsg.regex="^[[:digit:]]{4}-[[:digit:]]{2}-[[:digit:]]{2}"
escapeLF="on"
freshStartTail="on" #start tailf
reopenOnTruncate="on" #Truncate reopen
)

三、测试

添加 kafka配置vhosts

rsyslogd -f api-tomcat.conf -dn