shell 删除重复文件脚本

摘自 《Linux Shell脚本攻略》一书,例子在109页,原理在110页,原理讲解的很好哦!

需要了解awk命令、xargs,脚本中所用的命令在Linux Shell脚本攻略一书中都有讲解。

#!/bin/bash

ls -lS --time-style=long-iso | awk '
         BEGIN {getline; getline;
                     name1=$8; size=$5
                 }
          {
              name2=$8;
              if (size==$5)
              {
                  "md5sum "name1 | getline; csum1=$1;
                  "md5sum "name2 | getline; csum2=$1;
                  if (csum1 == csum2 )
                  {
                      print name1 ; print name2
                  }
              };
              size=$5 ;name1=name2;
          }' | sort -u > duplicate_files
cat duplicate_files | xargs -I {} md5sum {} | sort | uniq -w 32 | awk '{ print "^"$2"$" }' | sort -u > duplicate_sample
echo Removing..
comm duplicate_files duplicate_sample  -2 -3 | tee /dev/stderr | xargs rm
echo Removed duplicates files successfully.

上一篇:ubuntu 下面手动安装jdk


下一篇:在 Linux 中查找和删除重复文件