运维日志排查记录

前言

记录一些排查常见日志的命令,方法wiki,欢迎补充(Markdown 语法)。

常用命令

1. 查找关键词并统计行数

cat 2015_7_25_test_access.log | grep "sqlmap" | wc -l

1. 删除含有匹配字符的行

sed -i '/Indy Library/d' 2015_7_25_test_access.log

1. 查找所有日志中的关键词

find ./ -name "*.log" |xargs grep "sqlmap" |wc -l

1. 获取特殊行(如id)并且排序统计

cat cszl988.log | awk '{print $1}' | awk -F : '{print $2}' | sort -u | wc -l

1. 正则匹配内容(如提取ip)

grep -E -o "([0-9]{1,3}[\.]){3}[0-9]{1,3}"

1. 去重并统计数量

tail 3.log | awk '{print $7}' | sort | uniq -c

1. 批量提取(全流量中)数据包并且过滤数据

#!/bin/bash

 for file in ` ls $1 `

 do

 parse_pcap -vvb $file | grep -v "Host:" | grep -v "Cookie:" | grep -v "User-Agent:" | grep -v "Accept:" | grep -v "Accept:" | grep -v "Accept-Language:" | grep -v "Accept-Encoding:" | grep -v "Connection:" | grep -v "Content-Type:" | grep -v "Content-Length" | grep -v "Server"

 done

1. url 解码

cat luban.log | grep sqlmap | awk '{print $7}' | xargs python -c 'import sys, urllib; print urllib.unquote(sys.argv[1])'

1. 欢迎补充....

示范:xxxx站注入日志排查

• 查看所有sqlmap注入记录条数

[root@pentest temp]# cat luban.log | grep sqlmap | wc -l

1241

• 预览几条url

cat luban.log | grep sqlmap | awk '{print $7}' | more

/news.php?id=771%28.%28%22%29.%27%29%29%27&fid=168

/news.php?id=771%27IddP%3C%27%22%3EvCBw&fid=168

/news.php?id=771%29%20AND%201148%3D8887%20AND%20%288975%3D8975&fid=168

/news.php?id=771%29%20AND%208790%3D8790%20AND%20%287928%3D7928&fid=168

/news.php?id=771%20AND%204294%3D9647&fid=168

/news.php?id=771%20AND%208790%3D8790&fid=168

/news.php?id=771%27%29%20AND%205983%3D7073%20AND%20%28%27UwRr%27%3D%27UwRr&fid=168

/news.php?id=771%27%29%20AND%208790%3D8790%20AND%20%28%27hwaT%27%3D%27hwaT&fid=168

/news.php?id=771%27%20AND%206578%3D7565%20AND%20%27EoTZ%27%3D%27EoTZ&fid=168

/news.php?id=771%27%20AND%208790%3D8790%20AND%20%27lBdL%27%3D%27lBdL&fid=168

/news.php?id=771%25%27%20AND%205177%3D1107%20AND%20%27%25%27%3D%27&fid=168

/news.php?id=771%25%27%20AND%208790%3D8790%20AND%20%27%25%27%3D%27&fid=168

• 方便查看 urldecode

cat luban.log | grep sqlmap | awk '{print $7}' | xargs python -c 'import sys, urllib; print urllib.unquote(sys.argv[1])'

/news.php?id=771&fid=168

/news.php?id=771&fid=168 AND ASCII(SUBSTRING((SELECT DISTINCT(COALESCE(CAST(schemaname AS CHARACTER(10000)),(CHR(32)))) FROM pg_tables OFFSET 1 LIMIT 1)::text FROM 3 FOR 1))>

97

/news.php?id=771&fid=168 UNION ALL SELECT NULL,(CHR(113)||CHR(122)||CHR(106)||CHR(120)||CHR(113))||(CHR(103)||CHR(75)||CHR(78)||CHR(87)||CHR(76)||CHR(74)||CHR(110)||CHR(1

15)||CHR(100)||CHR(85))||(CHR(113)||CHR(122)||CHR(120)||CHR(113)||CHR(113)),NULL,NULL,NULL,NULL,NULL,NULL,NULL UNION ALL SELECT NULL,(CHR(113)||CHR(122)||CHR(106)||CHR(120)||CHR(113))||(CHR(113)||CHR(71)||C

HR(74)||CHR(82)||CHR(101)||CHR(120)||CHR(69)||CHR(112)||CHR(117)||CHR(79))||(CHR(113)||CHR(122)||CHR(120)||CHR(113)||CHR(113)),NULL,NULL,NULL,NULL,NULL,NULL,NULL--