1.背景
某天,突然收到报警,某非法用户登录gpu64机器,联系业务方确认,回复此员工已经离职,运维开始介入排查
2.现象
[root@gpu64 ~]# who
cuiysh pts/0 2021-02-04 09:30 (10.108.162.49)
renzh pts/1 2020-10-12 13:54 (10.108.162.49)
renzh pts/4 2020-10-12 11:43 (10.108.162.49)
root pts/22 2021-02-04 12:31 (10.168.1.241)
dengbf pts/46 2019-12-28 08:24 (10.105.132.91)
#dengbf用户为非法用户
[root@gpu64 ~]# w #使用此命令查无此人
12:32:11 up 499 days, 19:38, 6 users, load average: 18.39, 18.91, 18.78
USER TTY FROM LOGIN@ IDLE JCPU PCPU WHAT
cuiysh pts/0 10.108.162.49 09:30 1:45m 0.05s 0.00s tmux a
renzh pts/1 10.108.162.49 12Oct20 107days 0.22s 0.22s -bash
renzh pts/4 10.108.162.49 12Oct20 92days 0.38s 0.38s -bash
root c 10.168.1.241 12:31 3.00s 0.03s 0.00s w
其他命令
ps -aux | grep dengbf #查不到此人的进程
[root@gpu64 ]# strace who
……
stat("/dev/pts/46", 0x7ffe1eee67f0) = -1 ENOENT (No such file or directory)
stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=528, ...}) = 0
write(1, "dengbf pts/46 2019-12-28"..., 55dengbf pts/46 2019-12-28 08:24 (10.105.132.91)
……
[root@gpu64 ]# ls /dev/pts/46 #不存在此终端
[root@gpu64 ]# who -a |grep dengbf
dengbf ? pts/46 2019-12-28 08:24 ? 137996 (10.105.132.91)
[root@gpu64.corp.yodao.com ]# ps aux |grep 137996
3.解决流程
3.1 踢出用户
法一:pkill -kill -t pts/46
法二:fuser -k /dev/pts/46
who查看此用户还存在
其他查询命令
3.2 查看连接
netstat -an |grep 10.105.132.91
ss -an|grep 10.105.132.91
发现均不存在
3.3 升级who命令
whereis who
rpm -qf /usr/bin/who
yum update coreutils-8.22-23.el7.x86_64
发现升级失败,查询原因是对/目录没有写权限,猜测这应该是出现此报警的原因,重新挂载/分区,重新升级,重新查看
mount -o remount,rw /
yum update coreutils-8.22-23.el7.x86_64
然后who查看仍然存在
3.4 查看who帮助
man who #发现如下
If FILE is not specified, use /var/run/utmp. /var/log/wtmp as FILE is common. If ARG1 ARG2 given, -m presumed: ‘am
i’ or ‘mom likes’ are usual.
发现who读取的此文件/var/run/utmp. /var/log/wtmp
3.5 查看who文件
由于此文件无法直接编辑
strings /var/run/utmp |grep deng
strings /var/log/wtmp |grep deng
man /var/run/utmp #直接乱码
发现存在此用户信息,通过github查看有存在修改这两个文件的代码
3.6 修改who文件
cp -a /var/log/wtmp{,_20210204}
cp -a /var/run/utmp{,_20210204}
wget https://github.com/no-hope/wtmped/archive/master.zip
unzip master.zip
cd wtmped-master/
ls
cd wtmp_editor/
ls
make
./wtmped --help
./wtmped --user=dengbf /var/run/utmp
./wtmped --user=dengbf /var/log/wtmp
who
后续查看一切正常