CentOS 7上最新的kernel-debuginfo包,是kernel-debuginfo-4.x.x-x.el7,而当前内核是kernel-3.10.0。因此,如果安装或者升级到了最新的kernel-debuginfo包,会导致类似SystemTap这样需要内核头文件和调试符号的工具执行出错。这是CentOS 7的bug。
以SystemTap为例,其报错大致如下
[root@pusf ~]# rpm -qa | grep kernel-debuginfo
kernel-debuginfo-4.9.31-203.el7.centos.x86_64
kernel-debuginfo-common-x86_64-4.9.31-203.el7.centos.x86_64[root@pusf ~]
[root@pusf ~]# stap -v -e 'probe vfs.read {printf("read performed\n"); exit()}'
Pass 1: parsed user script and 119 library scripts using 117224virt/33688res/3120shr/30824data kb, in 240usr/10sys/292real ms.
semantic error: while resolving probe point: identifier 'kernel' at /usr/share/systemtap/tapset/linux/vfs.stp:882:18
source: probe vfs.read = kernel.function("vfs_read")
^
semantic error: missing x86_64 kernel/module debuginfo [man warning::debuginfo] under '/lib/modules/3.10.0-514.26.2.el7.x86_64/build'
semantic error: while resolving probe point: identifier 'vfs' at <input>:1:7
source: probe vfs.read {printf("read performed\n"); exit()}
^
semantic error: no match
Pass 2: analyzed script: 0 probes, 0 functions, 0 embeds, 0 globals using 120568virt/37092res/5108shr/32068data kb, in 100usr/140sys/443real ms.
Missing separate debuginfos, use: debuginfo-install kernel-3.10.0-514.26.2.el7.x86_64
Pass 2: analysis failed. [man error::pass2]
[root@pusf ~]#
因此,出现这种情况时,需要卸载kernel-debuginfo-4.x.x-x.el7和kernel-debuginfo-common-4.x.x-x.el7的包,重新按照当前内核版本安装kernel-debuginfo即可
rpm -qa | grep -E '^kernel-' | grep -v 3.10.0 | xargs yum -y remove
debuginfo-install -y kernel-$(uname -r)
再测试下SystemTap的脚本,会发现问题已经解决了
[root@pusf ~]# rpm -qa | grep kernel-debuginfo
kernel-debuginfo-common-x86_64-3.10.0-514.26.2.el7.x86_64
kernel-debuginfo-3.10.0-514.26.2.el7.x86_64
[root@pusf ~]# stap -v -e 'probe vfs.read {printf("read performed\n"); exit()}'
Pass 1: parsed user script and 119 library scripts using 117224virt/33684res/3120shr/30824data kb, in 230usr/10sys/293real ms.
Pass 2: analyzed script: 1 probe, 1 function, 4 embeds, 0 globals using 248216virt/165932res/4336shr/161816data kb, in 1260usr/370sys/1809real ms.
Pass 3: translated to C into "/tmp/stapI5iwL4/stap_1aa47f863c3f13e51da3e80cc92942be_1682_src.c" using 248216virt/166236res/4640shr/161816data kb, in 20usr/40sys/57real ms.
Pass 4: compiled C into "stap_1aa47f863c3f13e51da3e80cc92942be_1682.ko" in 5550usr/1240sys/7187real ms.
Pass 5: starting run.
read performed
Pass 5: run completed in 0usr/60sys/378real ms.
[root@pusf ~]#
在CentOS官方修正bug前,可以在/etc/yum.conf中加入如下配置,先排除问题包
exclude=kernel-debuginfo*
这样,升级时不会再次安装了问题包。当然,内核升级时,需要额外调整下配置。
顺便说下,在Ubuntu上使用SystemTap,需要额外配置和步骤,请参考SystemTap on Ubuntu。