1、生产环境有部分报错
信息如下：
2021-10-15T15:49:28,932][WARN ][logstash.filters.json ][main][7e17a3dc7e2c6df08ed7012ca6bfe17e3277d05d745f2c5bf55d2b01b151e25b] Error parsing json {:source=>"message", :raw=>"查询用户详情接口", :exception=>#<LogStash::Json::ParserError: Invalid UTF-8 start byte 0x9f
at [Source: (byte[])"查询用户详情接口"; line: 1, column: 3]>}
[2021-10-15T15:49:28,946][ERROR][logstash.outputs.elasticsearch][main][862f1cf74fcefb4312e0b0aa2e9fdf074e2e77c675e5c24e9b7a04d1054f1947] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"%{aName}-log_2021_10", :routing=>nil, :_type=>"_doc"}, #<LogStash::Event:0x6527baeb>], :response=>{"index"=>{"_index"=>"%{aName}-log_2021_10", "_type"=>"_doc", "_id"=>nil, "status"=>400, "error"=>{"type"=>"invalid_index_name_exception", "reason"=>"Invalid index name [%{aName}-log_2021_10], must be lowercase", "index_uuid"=>"_na_", "index"=>"%{aName}-log_2021_10"}}}}
[2021-10-15T15:49:29,446][WARN ][logstash.filters.json ][main][52e7421840cd9e07ac4081f9b005972562be1954f89c4f4bc5e30c5c19ebae78] Error parsing json {:source=>"message", :raw=>"spAuthAttributes ==========> {\"chainCodes\":\"60d9acb97314488689b899f4495e0857,3d01cb64635b46d4955606e6b356af70,c3283b04c2ad48a09d770b5a96b96a4c,850ff333c6664138b38344a75545ce8e\",\"customLoginStyle\":\"default\",\"samlEntityID\":\"portal\",\"securityLevel\":\"1\",\"spCode\":\"portal\",\"spLocalLogoutUrl\":\"\",\"spName\":\"portal\",\"spOsType\":0,\"tokenValidPeriod\":36000,\"url\":\"https://i.gt.cn\",\"userAttr\":\"\"}", :exception=>#<LogStash::Json::ParserError: Unrecognized token 'spAuthAttributes': was expecting ('true', 'false' or 'null')
at [Source: (byte[])"spAuthAttributes ==========> {"chainCodes":"60d9acb97314488689b899f4495e0857,3d01cb64635b46d4955606e6b356af70,c3283b04c2ad48a09d770b5a96b96a4c,850ff333c6664138b38344a75545ce8e","customLoginStyle":"default","samlEntityID":"portal","securityLevel":"1","spCode":"portal","spLocalLogoutUrl":"","spName":"portal","spOsType":0,"tokenValidPeriod":36000,"url":"https://i.gt.cn","userAttr":""}"; line: 1, column: 18]>}
[2021-10-15T15:49:29,447][WARN ][logstash.filters.json ][main][52e7421840cd9e07ac4081f9b005972562be1954f89c4f4bc5e30c5c19ebae78] Error parsing json {:source=>"message", :raw=>"redis执行时间：{}1", :exception=>#<LogStash::Json::ParserError: Unrecognized token 'redis执行时间': was expecting ('true', 'false' or 'null')
at [Source: (byte[])"redis执行时间：{}1"; line: 1, column: 21]>}
2、处理方法
这是在处理过程中有日志没有按照要进行打印，我们需要新开一个logstash把异常的日志打印出来；
配置如下：
[root@k8s-master1 conf.d]# more aName1.conf
input{
kafka{
bootstrap_servers => "10.152.17.50:9092"
group_id => "gt-scs-log-new"
topics => "gt-scs-log"
consumer_threads => 1
decorate_events => true
auto_offset_reset => "latest"
type => "gt-scs"
}
}
filter {
if [type] == "gt-scs" {
mutate {
add_field => { "types" => "%{type}"}
}
json {
source => "message"
}
json {
source => "message"
}
}
}
output {
if [tags] {
stdout {
codec => "rubydebug"
}
}
}