今天做地图定位展示，展示的是ApacheWeb服务器的访问日志文件中的来源IP。但是中间出现了报错环节，说是索引不能匹配到geo_point类型，实在是不懂这是在说什么，后来在网站找了方法就解决了。主要报错如下：
报错信息：
No Compatible Fields: The "logstash_apachelogs" index pattern does not contain any of the following field types: geo_point”
觉得很是奇怪，再来看看我的配置文件
input {
redis {
host => "172.16.0.54"
port =>
db =>
password => ""
data_type => "list"
key => "apache_filter_index"
codec => json {
charset => "UTF-8"
}
add_field => {"[@metadata][myfulltotal]" => "apacheaccess_log"}
}
}
filter {
if [@metadata][myfulltotal] == "apacheaccess_log" {
mutate {
gsub => ["message","\\x","\\\x"]
}
if ( 'method":"HEAD' in [message] ) {
drop{}
}
json {
source => "message"
add_field => {"[@metadata][direct_ip]" => "%{direct_ip}"}
remove_field => "message"
remove_field => "prospector"
remove_field => "beat"
remove_field => "host"
remove_field => "input"
remove_field => "source"
remove_field => "offset"
remove_field => "fields"
remove_field => "@version"
}
date {
match => ["timestamp","yyyy-MM-dd HH:mm:ss Z"]
}
mutate {
split => ["client_ip",","]
}
mutate {
replace => { "client_ip" => "%{client_ip[0]}"}
}
mutate {
convert => ["body_bytes_sent","integer"]
convert => ["total_bytes_sent","integer"]
}
if [client_ip] == "-" {
if [@metadata][direct_ip] not in ["%{direct_ip}","-"]{
mutate {
replace => { "client_ip" => "%{direct_ip}" }
}
} else {
drop {}
}
}
geoip {
source => "client_ip"
target => ["geoip"]
add_field => ["[geoip][coordinates]", "%{[geoip][longitude]}"]
add_field => ["[geoip][coordinates]", "%{[geoip][latitude]}"]
}
mutate {
convert => ["[geoip][coordinates]","float"]
}
mutate {
remove_field => ["direct_ip"]
remove_field => ["timestamp"]
}
}
}
output {
if [@metadata][myfulltotal] == "apacheaccess_log" {
elasticsearch {
hosts => ["172.16.0.51:9200"]
index => "logstash_apachelogs"
}
}
}
看上去都是没有问题的，只要是按照这个格式来写，都没有什么错误。只是在报错信息中看出geo的location类型不是geo_point类型的，那我们通过GET命令查看一下mapping
[root@ELK-chaofeng07 httpd]# curl -XGET http://172.16.0.51:9200/logstash_apachelogs/_mapping/
{"logstash-apachelogs":{"mappings":{"_default_":{"dynamic_templates":,{"string_fields":{"match":"*","m,"@version":{"type":"keyword"},"geoip":{"dynamic":"true","properties":{"ip":{"type":"ip"},"latitude":{"type":"half_float"},"location":{"type":"geo_point"},"longitude":{"type":"half_float"}}}}},"doc":{"dynamic_templates":[{"message_field":{"path_match":"message","match_mapping_type":"string","mapping":{"norms":false,"type":"text"}}},{"string_fields":{"match":"*","match_mapping_type":"string","mapping":{"fields":{"keyword":{"ignore_above":,"type":"keyword"}},"norms":false,"type":"text"}}}],"properties":{"@timestamp":{"type":"date"},"@version":{"type":"keyword"},"body_bytes_sent":{"type":"long"},"client_ip":{"type":"text","norms":false,"fields":{"keyword":{"type":"keyword",},"continent_code":{"type":"text","norms":false,"fields":{"keyword":{"type":"keyword","ignore_above":}}},"coordinates":{"type":"float"},"country_code2":{"type":"text","norms":false,"fields":{"keyword":{"type":"keyword","ignore_above":}}},"country_code3":{"type":"text","norms":false,"fields":{"keyword":{"type":"keyword","ignore_above":}}},"country_name":{"type":"text","norms":false,"fields":{"keyword":{"type":"keyword","ignore_above":}}},"ip":{"type":"ip"},"latitude":{"type":"half_float"},"location":{"type":"float"},"longitude":{"type":"half_float"},"region_code":}}}}}}}
看的出来我们此时的location是float类型的。所以如何将它变成geo_point类型是我们的解决目标方法
我们分析一下原因：
索引格式为logstash_apachelogs日志文件由logstash输出到Elasticsearch；在 elasticsearch 中，所有的数据都有一个类型，什么样的类型，就可以在其上做一些对应类型的特殊操作。geo信息中的location字段是经纬度，我们需要使用经纬度来定位地理位置；在 elasticsearch 中，对于经纬度来说，要想使用 elasticsearch 提供的地理位置查询相关的功能，就需要构造一个结构，并且将其类型属性设置为geo_point。
解决方法：
Elasticsearch支持给索引预定义设置和mapping，其实ES中已经有一个默认预定义的模板，我们只要使用预定的模板即可。我们要想使用预定义的模板，那么索引名必须匹配 logstash-* 的索引才会应用这个模板，由于我们在logstash中使用的是logstash_*的索引方式，因此并没有匹配默认模板，所以只需要修改一下索引名即可。然后我们就可以看到map不再报这个错误了。
所以说这是个相当不起眼的错误，但是引发了这个大的问题，搞的让人很懵比，所以接下来我们得好好研究一下mapping是什么，