awk的基本用法
awk [options] 'program' var=value file…
awk [options] -f programfile var=value file…
awk [options] 'BEGIN{action;… }pattern{action;… }END{action;… }' file ...
AWK 是一种处理文本文件的语言，是一个强大的文本分析工具。
之所以叫 AWK 是因为其取了三位创始人 Alfred Aho，Peter Weinberger, 和 Brian Kernighan 的 Family Name 的首字符。
#sed 和grep 逐行处理
#逐行打印
[root04:33 AMcentos8 ~]#awk '{print "hello,awk"}'
ssss
hello,awk
# 等待你标准输入，你输入一行他就处理一行
hello,awk
ss
hello,awk
[root04:35 AMcentos8 ~]#seq 5 | awk '{print "hello,awk"}'
hello,awk
hello,awk
hello,awk
hello,awk
hello,awk
#数值运算不要加双引号
[root04:36 AMcentos8 ~]#seq 3 | awk '{print 2*3}'
6
6
6
[root04:37 AMcentos8 ~]#awk -F: '{print "wang"}' /etc/issue
wang
wang
wang #文件多少行，就打多少个wang
[root04:37 AMcentos8 ~]#awk -F: '{print}' /etc/issue
\S
Kernel \r on an \m
[root04:40 AMcentos8 ~]#awk -F: '{print $1"\t"$3}' /etc/passwd
内置变量
-F 等价于 FS=
默认以空格作为分隔符
-F 以什么作为分隔符 等价于 -v FS=
[root04:50 AMcentos8 ~]#awk -v FS=":" '{print $1FS$3 }' /etc/passwd |head -3
root:0
bin:1
daemon:2
[root04:50 AMcentos8 ~]#awk -v FS=":" '{print $1":"$3 }' /etc/passwd |head -3
root:0
bin:1 用FS代替 ":"
daemon:2
引用shell中的变量
[root04:55 AMcentos8 ~]#S=:;awk -v FS=$S '{print $1FS$3}' /etc/passwd |head -3
root:0
bin:1
daemon:2
OFS输出分隔符
[root04:59 AMcentos8 ~]#awk -v FS=":" -v OFS="======" '{print $1,$3 }' /etc/passwd |head -3
root======0 #默认是空格作为分隔符，加了OFS修改默认的分隔符
bin======1
daemon======2
或者
[root05:00 AMcentos8 ~]#S=:;awk -v FS=$S '{print $1"======="$3}' /etc/passwd |head -3
root=======0
bin=======1
daemon=======2
RS 记录的分隔符
[root05:02 AMcentos8 /]#cat test
a,b,c;12,
34,56,78;AA,
BB
#默认换行是行的分隔符，RS可以指定别的符号是分隔符
[root05:02 AMcentos8 /]#awk -F"," -v RS=";" '{print $1,$3}' test
a c #指定了;是每一条记录的分隔符
12 56
AA
ORS 定义记录和记录之间的输出分隔符，默认是换行
[root05:12 AMcentos8 /]#awk -F"," -v RS=";" -v ORS="+++" '{print $1,$3}' test
a c+++12 56+++AA +++[root05:13 AMcentos8 /]#
NF每一记录总共的字段数
[root05:16 AMcentos8 /]#cat test
a,b,c;12,
34,56,78;AA,
BB
[root05:16 AMcentos8 /]#awk -F"," -v RS=";" '{print NF}' test
3
4
2
===========================================================
$NF 每一条记录的最后一列
[root05:18 AMcentos8 /]#awk -F: '{print $NF}' /etc/passwd |head -3
/bin/bash
/sbin/nologin
/sbin/nologin
$(NF-1) 倒数第二列
NR 打印每一条记录的编号
[root05:23 AMcentos8 /]#awk -F: '{print NR,$0 }' /etc/passwd |head -3
1 root:x:0:0:root:/root:/bin/bash
2 bin:x:1:1:bin:/bin:/sbin/nologin
3 daemon:x:2:2:daemon:/sbin:/sbin/nologin
#挑出指定的行来打
[root05:23 AMcentos8 /]#awk "NR==2" /etc/passwd
bin:x:1:1:bin:/bin:/sbin/nologin
FNR 分开打印文件的行号
[root05:27 AMcentos8 /]#awk -F: '{print FNR,$0 }' /etc/redhat-release /etc/issue
1 CentOS Linux release 8.2.2004 (Core)
1 \S
2 Kernel \r on an \m
3
FILENAE
显示文件名
[root05:30 AMcentos8 /]#awk -F: '{print FILENAME,FNR,$0 }' /etc/redhat-release /etc/issue
/etc/redhat-release 1 CentOS Linux release 8.2.2004 (Core)
/etc/issue 1 \S
/etc/issue 2 Kernel \r on an \m
/etc/issue 3
ARGV ARGC
[root05:32 AMcentos8 /]#awk -F: '{print ARGC}' /etc/redhat-release /etc/issue
3
3
3
3
[root05:32 AMcentos8 /]#awk -F: '{print ARGV[0],ARGV[1],ARGV[2],ARGV[3]}' /etc/redhat-release /etc/issue
awk /etc/redhat-release /etc/issue
awk /etc/redhat-release /etc/issue # ARGV[3]为空值
awk /etc/redhat-release /etc/issue 三个参数
awk /etc/redhat-release /etc/issue
ARGC 命令行参数个数
ARGV 数组，命令行所有给的的参数
[15:11:23root@localhost ~]#awk 'BEGIN{print ARGC}' /etc/fstab /etc/issue
3
[15:12:19root@localhost ~]#awk 'BEGIN{print ARGV[1]}' /etc/fstab /etc/issue
/etc/fstab
[15:12:43root@localhost ~]#awk 'BEGIN{print ARGV[0]}' /etc/fstab /etc/issue
awk
自定义变量：
-v 定义变量
变量名依然区分字符大小写，此时的/etc/fstab 文件是被拿来做遍历的
[15:13:48root@localhost ~]#awk -v test="hello gwak" '{print test}' /etc/fstab
hello gwak
hello gwak
hello gwak
hello gwak
hello gwak
hello gwak
hello gwak
hello gwak
hello gwak
hello gwak
hello gwak
[15:18:00root@localhost ~]#awk -v test="hello gwak" 'BEGIN{print test}' /etc/fstab
hello gwak
或者在program中直接定义：
[15:18:13root@localhost ~]#awk 'BEGIN{test="hello gwak";print test}' /etc/fstab
hello gwak
#变量是先赋值后使用
[root05:43 AMcentos8 /]#awk -F: '{sex="male";print $1,sex,age;age=18}' /etc/passwd
root male
bin male 18
daemon male 18
adm male 18
lp male 18
sync male 18
-f 选项
[root05:47 AMcentos8 /]#cat awktest
{print script,$1,$2}
[root05:47 AMcentos8 /]#awk -F: -f awktest script="awk" /etc/passwd |head -3
awk root x
awk bin x
awk daemon x
printf
%d %i 十进制有符号整数
%u 十进制无符号整数
%f 浮点数
%s 字符串
例如：
[16:04:24root@localhost ~]#awk -F: '{printf "%s",$1}' /etc/passwd
rootbindaemonadmlpsyncshutdownhaltmailoperatorgamesftpnobodyavahi-autoipdsystemd-bus-proxysystemd-networkdbuspolkitdtsspostfixsshdntprsyncrpcrpcusernfsnobodyapachemysqlwanghuauser3user4user5testuser7user8user9user6user1tomcentosgentookkkkerkker-zabbixsrvnginxsaslauth[16:11:08root@localhost ~]#
[16:12:48root@localhost ~]#awk -F: '{printf "%s\n",$1}' /etc/passwd
加 \n 自动换行
[16:14:43root@localhost ~]#awk -F: '{printf "USERNAME:%s\n",$1}' /etc/passwd
USERNAME:root
USERNAME:bin
USERNAME:daemon
USERNAME:adm
USERNAME:lp
USERNAME:sync
USERNAME:shutdown
[root05:51 AMcentos8 /]#awk -F: '{printf "USERNAME:%-20s UID:%d\n",$1,$3}' /etc/passwd
USERNAME:root UID:0
USERNAME:bin UID:1 #负数左对齐 正数右对齐
USERNAME:daemon UID:2
USERNAME:adm UID:3
USERNAME:lp UID:4
USERNAME:sync UID:5
[root05:56 AMcentos8 /]#awk -F: 'BEGIN{print "----------------------"}{printf "USERNAME:%-10s| UID:%d\n---------------------\n",$1,$3}' /etc/passwd
----------------------
USERNAME:root | UID:0
---------------------
USERNAME:bin | UID:1
---------------------
USERNAME:daemon | UID:2
---------------------
USERNAME:adm | UID:3
---------------------
[root05:59 AMcentos8 /]#awk -F: 'BEGIN{print "----------------------"}{printf "USERNAME:%-10s| UID:%.2f\n---------------------\n",$1,$3}' /etc/passwd
----------------------
USERNAME:root | UID:0.00
---------------------
USERNAME:bin | UID:1.00
---------------------
USERNAME:daemon | UID:2.00
---------------------
%c 字符的AIISC码
%p 指针的值
%e %E科学计数法数值显示
%% %自身
#[.#] 第一个数值控制显示的宽度 ，第二个#表示小数点后的精度
%3.f
例如：默认右对齐
[16:28:22root@localhost ~]#awk -F: '{printf "USERNAME:%s, UID:%d\n",$1,$3}' /etc/passwd
USERNAME:root, UID:0
USERNAME:bin, UID:1
USERNAME:daemon, UID:2
USERNAME:adm, UID:3
USERNAME:lp, UID:4
USERNAME:sync, UID:5
USERNAME:shutdown, UID:6
USERNAME:halt, UID:7
[16:25:07root@localhost ~]#awk -F: '{printf "USERNAME:%20s, UID:%d\n",$1,$3}' /etc/passwd
USERNAME: root, UID:0
USERNAME: bin, UID:1
USERNAME: daemon, UID:2
- 左对齐
[16:26:56root@localhost ~]#awk -F: '{printf "USERNAME:%-20s, UID:%d\n",$1,$3}' /etc/passwd
USERNAME:root , UID:0
USERNAME:bin , UID:1
USERNAME:daemon , UID:2
USERNAME:adm , UID:3
USERNAME:lp , UID:4
USERNAME:sync , UID:5
USERNAME:shutdown , UID:6
+ 显示数值符号
取非
[root06:25 AMcentos8 ~]#awk -v n=0 '!n++{print $0}' /etc/passwd
root:x:0:0:root:/root:/bin/bash
[root06:25 AMcentos8 ~]#awk -v n=1 '!n++{print $0}' /etc/passwd
[root06:26 AMcentos8 ~]#awk -v n=-1 '!n++{print $0}' /etc/passwd
bin:x:1:1:bin:/bin:/sbin/nologin
[root06:26 AMcentos8 ~]#awk -v n=-1 '!++n{print $0}' /etc/passwd
root:x:0:0:root:/root:/bin/bash
打印第几行
[root06:27 AMcentos8 ~]#awk NR==2 /etc/issue
Kernel \r on an \m
[root06:29 AMcentos8 ~]#awk NR!=2 /etc/issue
\S
取奇偶
[root06:32 AMcentos8 ~]#seq 10 |awk NR%2!=0
1
3
5
7
9
[root06:32 AMcentos8 ~]#seq 10 |awk NR%2==0
2
4
6
8
10
[root06:32 AMcentos8 ~]#seq 10 |awk NR%2==1
1
3
5
7
9
[root07:31 AMcentos8 ~]#seq 10 |awk 'i=!i'
1
3 i的初始值为空 i取反得到1 第三次i=!i i的值为1 第四次判断i为真
5
7
9
[root07:32 AMcentos8 ~]#seq 10 |awk '!(i=!i)'
2
4
6
8
10
[root07:36 AMcentos8 ~]#seq 10 |awk -v i=1 'i=!i'
2
4
6
8
10
[root07:37 AMcentos8 ~]#seq 10 |awk -v i=0 'i=!i'
1
3
5
7
9
[root07:37 AMcentos8 ~]#seq 10 |awk '{i=!i;print i}'
1
0
1
0
1
0
1
0
1
0
正则表达式匹配
[root06:35 AMcentos8 ~]#awk -F: '$0 !~ /root/' /etc/passwd |head -3
bin:x:1:1:bin:/bin:/sbin/nologin
daemon:x:2:2:daemon:/sbin:/sbin/nologin
adm:x:3:4:adm:/var/adm:/sbin/nologin
[root06:35 AMcentos8 ~]#awk -F: '$0 ~ /root/' /etc/passwd
root:x:0:0:root:/root:/bin/bash
operator:x:11:0:operator:/root:/sbin/nologin
[root06:35 AMcentos8 ~]#awk -F: '$0 ~ /root/{print $1}' /etc/passwd
root
operator
[root06:48 AMcentos8 ~]#awk '/^UUID/' /etc/fstab
UUID=55d00b68-d8ef-4c35-aa11-b8c11b91e867 / xfs defaults 0 0
UUID=37e18d75-7231-4d84-8536-18e1b3a891f6 /boot ext4 defaults 1 2
UUID=fdcd162f-2ea2-4088-9e86-6a911336304a /data xfs defaults 0 0
UUID=d81ef34a-c3dd-4483-b830-4cf516cbec7c swap swap defaults 0 0
不写默认打印$0
!取反
[root06:49 AMcentos8 ~]#awk '!/^UUID/' /etc/fstab
#
# /etc/fstab
# Created by anaconda on Wed Apr 21 23:09:37 2021
#
# Accessible filesystems, by reference, are maintained under '/dev/disk/'.
# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info.
#
# After editing this file, run 'systemctl daemon-reload' to update systemd
# units generated from this file.
#
条件表达式
[root06:42 AMcentos8 ~]#awk -F: '$3>=0 && $3<=1000 {print $1,$3}' /etc/passwd
[root07:06 AMcentos8 ~]#awk -F: '$NF~ /bash$/{print $1,$NF}' /etc/passwd
root /bin/bash
wang /bin/bash
li /bin/bash
zhang /bin/bash
[root07:06 AMcentos8 ~]#awk -F: '$NF=="/bin/bash"{print $1,$NF}' /etc/passwd
root /bin/bash
wang /bin/bash
li /bin/bash
zhang /bin/bash
[root07:12 AMcentos8 ~]#awk -F: '$NF="/bin/AAAAA"{print $1,$NF}' /etc/passwd |head -3
root /bin/AAAAA
bin /bin/AAAAA
daemon /bin/AAAAA
[root07:12 AMcentos8 ~]#seq 10 |awk 'NR>=3 && NR<=10'
3
4
5
6
7
8
9
10
[root07:15 AMcentos8 ~]#seq 10 |sed -n 3,10p
3
4
5
6
7
8
9
10
模式匹配
[root07:16 AMcentos8 ~]#awk '/^b/,/^f/' /etc/passwd 省略了print $0
bin:x:1:1:bin:/bin:/sbin/nologin
daemon:x:2:2:daemon:/sbin:/sbin/nologin
adm:x:3:4:adm:/var/adm:/sbin/nologin
lp:x:4:7:lp:/var/spool/lpd:/sbin/nologin
sync:x:5:0:sync:/sbin:/bin/sync
shutdown:x:6:0:shutdown:/sbin:/sbin/shutdown
halt:x:7:0:halt:/sbin:/sbin/halt
mail:x:8:12:mail:/var/spool/mail:/sbin/nologin
operator:x:11:0:operator:/root:/sbin/nologin
games:x:12:100:games:/usr/games:/sbin/nologin
ftp:x:14:50:FTP User:/var/ftp:/sbin/nologin