用了这个logstash扩展
https://github.com/PeterPaulH/logstash-influxdb/blob/master/src/influxdb.rb
把这个文件放到 logstash-1.4.2/lib/logstash/outputs
看一下午logstash的文档，终于解决了自己的需求
用python描述就是这样的
开发要求统计日志中各种类型的数量
while True:
line = f.readline()
try:
if ‘"type":"text","receiver_id"‘ in line:
type = ‘directmessage‘
elif ‘"subtype":"unfollow"‘ in line:
type = ‘unfollow‘
elif ‘"subtype":"follow"‘ in line:
type = ‘follow‘
elif ‘"subtype":"status"‘ in line:
type = ‘weibo‘
elif ‘"subtype":"comment"‘ in line:
type = ‘comment‘
else:
type = None
if type:
data = [
{"name":"pingpong_processor",
"columns" : ["type"],
"points" : [[type]]
}
]
logstash配置文件如下
input {
stdin {}
}
filter {
if ‘"type":"text"‘ in [message] {
mutate {
add_field => { "type" => "directmessage" }
remove_field => [ "message", "search" , "@version" ]
}
} else if ‘"subtype":"unfollow"‘ in [message] {
mutate {
add_field => { "type" => "unfollow" }
remove_field => [ "message", "search" , "@version" ]
}
} else if ‘"subtype":"follow"‘ in [message] {
mutate {
add_field => { "type" => "follow" }
remove_field => [ "message", "search" , "@version" ]
}
} else if ‘"subtype":"status"‘ in [message] {
mutate {
add_field => { "type" => "weibo" }
remove_field => [ "message", "search" , "@version" ]
}
} else if ‘"subtype":"comment"‘ in [message] {
mutate {
add_field => { "type" => "comment" }
remove_field => [ "message", "search" , "@version" ]
}
} else {
drop {}
}
}
output {
influxdb {
host => "10.75.28.180"
port => 4444
name => ["pingpong_processor"]
columns => ["type", "host"]
points => [
"%{type}", "c",
"%{host}", "c"
]
}
stdout {}
}
别忘记把influxdb的配置也修改下，因为默认upd协议是没有打开的
# Configure the udp api [input_plugins.udp] enabled = true port = 4444 database = "pingpong_processor"
我用的influxdb版本是 influxdb-0.8.2-1.x86_64,用/etc/init.d/influxdb这个启动报错，无奈手动启动的
/usr/bin/influxdb -pidfile /tmp/influxdb.pid -config config.toml
看一下udp端口是否启动了
netstat -anup|grep influxdb udp 0 0 :::4444 :::* 27512/influxdb
完工