unison + inotify 实现文件实时双向同步部署步骤
一. Unison简介
Unison是Windows、Linux以及其他Unix平台下都可以使用的文件同步工具，它能使两个文件夹（本地或网络上的）保持内容的一致。Unison拥有与其它一些同步工具或文件系统的相同的特性，但也有自身的特点：
1. 跨平台使用；
2. 对内核和用户权限没有特别要求；
3. Unison是双向的，它能自动处理两分拷贝中更新没有冲突的部分，有冲突的部分将会显示出来让用户选择更新策略；
4. 只要是能连通的两台主机，就可以运行unison，可以直接使用socket连接或安全的ssh连接方式，对带宽的要求不高，使用类似rsync的压缩传输协议。
二. 编译安装Unison
unison各种版本下载地址：
http://www.seas.upenn.edu/~bcpierce/unison//download.html
unison编译器下载地址：
http://caml.inria.fr/pub/distrib
从以上地址可以下载各种平台,各种版本的unison,有基于源码安装的,有二进制的,我下载的是二进制的,可以直接使用.这里介绍源码安装:
1.实验环境：
txzlkhdweb1 192.168.16.1 同步目录 /root/test
txzlkhdweb2 192.168.16.2 同步目录 /root/test
2.创建共享文件目录
txzlkhdweb1 :
[root@txzlkhdweb1 /]# mkdir /root/test
txzlkhdweb2:
[root@txzlkhdweb2 ~]# mkdir /root/test
3.安装Objective Caml compiler
Linux下从源码包编译安装，需要一个叫做Objective Caml compiler的工具，版本至少3.0.7，可以从这里下载：http://caml.inria.fr/
[tomcat@txzlkhdweb1 ~]$ cd /usr/local/src/
[tomcat@txzlkhdweb1 ~]$ wget http://caml.inria.fr/pub/distrib/ocaml-3.12/ocaml-3.12.1.tar.gz
--2013-05-03 20:23:02-- http://caml.inria.fr/pub/distrib/ocaml-3.12/ocaml-3.12.1.tar.gz
正在解析主机 caml.inria.fr... 128.93.11.35
Connecting to caml.inria.fr|128.93.11.35|:80... 已连接。
已发出 HTTP 请求，正在等待回应... 200 OK
长度：3660473 (3.5M) [application/x-gzip]
Saving to: `ocaml-3.12.1.tar.gz'
100%[======================================>] 3,660,473 150K/s in 44s
2013-05-03 20:23:49 (81.5 KB/s) - `ocaml-3.12.1.tar.gz' saved [3660473/3660473]
[root@txzlkhdweb1 src]# tar -zxf ocaml-3.12.1.tar.gz
[root@txzlkhdweb1 src]# cd ocaml-3.12.1
[root@txzlkhdweb1 ocaml-3.12.1]# ./configure
Configuring for a x86_64-unknown-linux-gnu ...
gcc found
........................................
options for linking ...... -lX11
The "labltk" library: not supported
** Objective Caml configuration completed successfully **
[root@txzlkhdweb1 ocaml-3.12.1]# make world opt
make coldstart
make[1]: Entering directory `/usr/local/src/ocaml-3.12.1'
cd byterun; make all
make[2]: Entering directory `/usr/local/src/ocaml-3.12.1/byterun'
sed -n -e '/^ /s/ \([A-Z]\)/ \&\&lbl_\1/gp' \
-e '/^}/q' instruct.h > jumptbl.h
......................................................
../ocamlcompopt.sh -nostdlib -c -g -annot -warn-error A -w L -w R -w Z -I ../otherlibs/unix -I ocamlbuild -I stdlib -o ocamlbuild/ocamlbuild_unix_plugin.cmx ocamlbuild/ocamlbuild_unix_plugin.ml
../ocamlcompopt.sh -nostdlib -a -I stdlib ocamlbuild/ocamlbuild_pack.cmx ocamlbuild/ocamlbuild_plugin.cmx ocamlbuild/ocamlbuild_executor.cmx ocamlbuild/ocamlbuild_unix_plugin.cmx -o ocamlbuild/ocamlbuildlib.cmxa
../ocamlcompopt.sh -nostdlib -a -I stdlib ocamlbuild/ocamlbuild_pack.cmx ocamlbuild/ocamlbuild_plugin.cmx -o ocamlbuild/ocamlbuildlightlib.cmxa
make[1]: Leaving directory `/usr/local/src/ocaml-3.12.1'
[root@txzlkhdweb1 ocaml-3.12.1]# make install
if test -d /usr/local/bin; then : ; else mkdir -p /usr/local/bin; fi
if test -d /usr/local/lib/ocaml; then : ; else mkdir -p /usr/local/lib/ocaml; fi
if test -d /usr/local/lib/ocaml/stublibs; then : ; else mkdir -p /usr/local/lib/ocaml/stublibs; fi
..................................
install /usr/local/lib/ocaml/ocamlbuild/ocamlbuild_executor.o
install /usr/local/lib/ocaml/ocamlbuild/ocamlbuild.cmo
install /usr/local/man/man1/ocamlbuild.1
4.编译安装Unison
[root@txzlkhdweb1 src]# wget http://www.seas.upenn.edu/~bcpierce/unison//download/releases/stable/unison-2.40.102.tar.gz
--2013-05-03 22:29:49-- http://www.seas.upenn.edu/~bcpierce/unison//download/releases/stable/unison-2.40.102.tar.gz
正在解析主机 www.seas.upenn.edu... 158.130.68.91, 2607:f470:8:64:5ea5::9
Connecting to www.seas.upenn.edu|158.130.68.91|:80... 已连接。
已发出 HTTP 请求，正在等待回应... 200 OK
长度：2694761 (2.6M) [application/x-gzip]
Saving to: `unison-2.40.102.tar.gz'
100%[======================================================>] 2,694,761 113K/s in 34s
2013-05-03 22:30:26 (76.8 KB/s) - `unison-2.40.102.tar.gz' saved [2694761/2694761]
[root@txzlkhdweb1 ocaml-3.12.1]# make install
if test -d /usr/local/bin; then : ; else mkdir -p /usr/local/bin; fi
if test -d /usr/local/lib/ocaml; then : ; else mkdir -p /usr/local/lib/ocaml; fi
............................
install /usr/local/lib/ocaml/ocamlbuild/ocamlbuild.cmo
install /usr/local/man/man1/ocamlbuild.1
[root@txzlkhdweb1 src]# cd unison-2.40.102
[root@txzlkhdweb1 unison-2.40.102]# make UISTYLE=text
ocamlc -o mkProjectInfo unix.cma str.cma mkProjectInfo.ml
./mkProjectInfo > Makefile.ProjectInfo
UISTYLE = text
Building for Unix
NATIVE = true
THREADS = false
STATIC = false
OSTYPE =
OSARCH =
ocamlopt: ubase/rx.mli ---> ubase/rx.cmi
ocamlopt -I lwt -I ubase -I system -I system/generic -I lwt/generic -c /usr/local/src/unison-2.40.102/ubase/rx.mli
.........................................
make[1]: Entering directory `/usr/local/src/unison-2.40.102'
if [ -f `which etags` ]; then \
etags *.mli */*.mli *.ml */*.ml */*.m *.c */*.c *.txt \
; fi
make[1]: Leaving directory `/usr/local/src/unison-2.40.102'
[root@txzlkhdweb1 unison-2.40.102]# make install
mv /root/bin//unison /tmp/unison-15209
mv: 无法 stat “/root/bin//unison”: 没有那个文件或目录
make: [doinstall] 错误 1 (忽略)
cp unison /root/bin/
cp: 无法创建一般文件“/root/bin/”: 是一个目录
make: *** [doinstall] 错误 1
#出现错误的原因在与Unison默认是将文件Copy到/root/bin目录，但Linux默认是没有该目录的，因此我们需要将生成的可执行文件unison复制到系统的PATH目录。
[root@txzlkhdweb1 unison-2.40.102]# cp unison /usr/local/bin
5.通过SSH登陆到远程主机txzlkhdweb2 ，再将unison复制到slave的PATH目录
[root@txzlkhdweb1 unison-2.40.102]# scp unison root@192.168.16.2:/usr/local/bin/unison
The authenticity of host '192.168.16.2 (192.168.16.2)' can't be established.
RSA key fingerprint is 7d:9a:b3:1b:c8:a4:85:de:dc:59:ac:ce:a3:8b:a7:41.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '192.168.16.2' (RSA) to the list of known hosts.
root@192.168.16.2s password:
unison 100% 1473KB 1.4MB/s 00:00
三．配置双机ssh信任
由于unison在远程同步文件夹要登陆远程服务器,因此要配置两机互相信任
本例假设本地机为：192.168.16.1(txzlkhdweb1)
远程机：192.168.16.2(txzlkhdweb2)
1. 在两台机器上创建 RSA密钥
以下操作要在本地机和远程机上都执行一遍
（1）以 root 用户登录
（2）在 root 用户的 主目录内创建.ssh 目录并设置正确的权限
[root@txzlkhdweb1 ~]# mkdir ~/.ssh
[root@txzlkhdweb1 ~]# chmod 700 ~/.ssh
[root@txzlkhdweb1 ~]# ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/root/.ssh/id_rsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /root/.ssh/id_rsa.
Your public key has been saved in /root/.ssh/id_rsa.pub.
The key fingerprint is:
20:4e:83:71:d4:12:e1:b1:a3:d0:fe:b1:8e:9e:ed:a8 root@txzlkhdweb1
[root@txzlkhdweb1 ~]# ssh 192.168.16.1cat /root/.ssh/id_rsa.pub >> authorized_keys
The authenticity of host '192.168.16.1(192.168.16.1)' can't be established.
RSA key fingerprint is 7d:9a:b3:1b:c8:a4:85:de:dc:59:ac:ce:a3:8b:a7:41.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '192.168.16.1(RSA) to the list of known hosts.
root@192.168.16.1's password:
[root@txzlkhdweb1 ~]# ssh 192.168.16.2cat /root/.ssh/id_rsa.pub >> authorized_keys
root@192.168.16.2s password:
[root@txzlkhdweb1 ~]# scp authorized_keys 192.168.16.2:/root/.ssh/
root@192.168.16.2's password:
authorized_keys 100% 785 0.8KB/s 00:00
#重启两台服务器的SSH服务
[root@txzlkhdweb1 ~]# /etc/init.d/sshd restart
[root@txzlkhdweb2~]# /etc/init.d/sshd restart
四．unison的使用
1.unison远程使用
使用方法：
# unison <本地目录> ssh://remotehostname(IP)/<远程目录的绝对路径>
例如:
[root@txzlkhdweb1 ~]# /usr/local/bin/unison /root/test ssh://root@192.168.16.1//root/test
Contacting server...
Connected [//txzlkhdweb1//root/test -> //txzlkhdweb2//root/test]
Looking for changes
Waiting for changes from server
Reconciling changes
Nothing to do: replicas have not changed since last sync.
表示将本机的目录/txzlkhdweb1/root/test和远端主机的/txzlkhdweb2/root/test进行同步。一般的，需要两台机能ssh连接。
注意 在主机和目录间又多加了一个 "/"
2.修改参数说明
[root@txzlkhdweb1 .unison]# vim ~/.unison/default.prf
#!/bin/bash
# this script is created by yourname.
# Unison preferences file
#root = /root/test
#root = ssh://root@192.168.16.2//root/test
#force =
#ignore =
batch = true
#repeat = 1
#retry = 3
owner = true
group = true
perms = -1
fastcheck = false
rsync = false
sshargs = -C
xferbycopying = true
log = true
logfile = /root/.unison/unison.log
如果服务器端 unison 可执行文件不在默认目录下，甚至没有 unison 命令（需要你编译一个上传到服务器），则需要使用 -servercmd 参数告诉要执行的服务器 unison 命令位置。
使用 -testserver 参数，则成功链接即退出，也不会去执行目录的比较等后续操作。
• -servercmd xxx
告诉 unison， 服务器端的 unison 命令是什么。参见上面的示例。
• -auto
接受缺省的动作，然后等待用户确认是否执行。
• -batch
batch mode, 全自动模式，接受缺省动作，并执行。
• -ignore xxx
增加 xxx 到忽略列表中
• -ignorecase [true|false|default]
是否忽略文件名大小写
• -follow xxx
是否支持对符号连接指向内容的同步
• owner = true (保持同步过来的文件属主)
• group = true (保持同步过来的文件组信息)
• perms = -1 (保持同步过来的文件读写权限)
• repeat = 1 (间隔1秒后,开始新的一次同步检查)
• retry = 3 (失败重试)
• sshargs = -C (使用ssh的压缩传输方式)
• xferbycopying = true
• -immutable xxx
不变目录，扫描时可以忽略
• -silent
安静模式
• -times
同步修改时间
• -path xxx 参数
只同步 -path 参数指定的子目录以及文件，而非整个目录。-path 可以多次出现,例如
unison /home/username ssh://remotehost//home/username \
-path shared \
-path pub \
-path .netscape/bookmarks.html
五．安装inotify 软件
1.查看当前系统是否支持inotify
[root@txzlkhdweb1 test]# ls -l /proc/sys/fs/inotify
总计 0
-rw-r--r-- 1 root root 0 05-04 00:56 max_queued_events
-rw-r--r-- 1 root root 0 05-04 00:56 max_user_instances
-rw-r--r-- 1 root root 0 05-04 00:56 max_user_watches
2.下载inotify 源码包
[root@txzlkhdweb1 ~]# wget https://github.com/downloads/rvoicilas/inotify-tools/inotify-tools-3.14.tar.gz
正在解析主机 github.com... 192.30.252.129
Connecting to github.com|192.30.252.129|:443... 已连接。
WARNING: cannot verify github.com's certificate, issued by `/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert High Assurance EV CA-1':
Issued certificate not yet valid.
已发出 HTTP 请求，正在等待回应... 302 Found
位置：http://cloud.github.com/downloads/rvoicilas/inotify-tools/inotify-tools-3.14.tar.gz [跟随至新的 URL]
--2013-05-04 01:23:55-- http://cloud.github.com/downloads/rvoicilas/inotify-tools/inotify-tools-3.14.tar.gz
正在解析主机 cloud.github.com... 54.240.164.110, 54.230.113.195, 54.240.164.29, ...
Connecting to cloud.github.com|54.240.164.110|:80... 已连接。
已发出 HTTP 请求，正在等待回应... 200 OK
长度：358772 (350K) [null]
Saving to: `inotify-tools-3.14.tar.gz'
100%[=================================================>] 358,772 356K/s in 1.0s
2013-05-04 01:23:56 (356 KB/s) - `inotify-tools-3.14.tar.gz' saved [358772/358772]
3.编译安装inotify
[root@txzlkhdweb1 ~]# tar zxf inotify-tools-3.14.tar.gz
[root@txzlkhdweb1 ~]# cd inotify-tools-3.14
[root@txzlkhdweb1 inotify-tools-3.14]# make && make install
make all-recursive
make[1]: Entering directory `/root/inotify-tools-3.14'
.......
make[2]: Nothing to be done for `install-data-am'.
make[2]: Leaving directory `/root/inotify-tools-3.14'
make[1]: Leaving directory `/root/inotify-tools-3.14'
4.编写inotify 实时监控脚本
[root@txzlkhdweb1 inotify-tools-3.14]# cat /proc/sys/fs/inotify/max_user_watches
5000000000
[root@txzlkhdweb1 inotify-tools-3.14]# cat /proc/sys/fs/inotify/max_queued_events
327679
#事件监控脚本：
[root@txzlkhdweb1 inotify-tools-3.14]# vim /usr/local/inotify/unison.sh
#!/bin/bash
PATH=/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/bin:/usr/local/sbin
LOCAL_DIR="/root/test/"
REMOTE_DIR="/root/test/"
REMOTE_USER="root"
INOTIFY="/usr/local/inotify-tools-3.14/bin/inotifywait"
UNISION="/usr/local/bin/unison"
#目前iplist.txt 文件中就一个地址 192.168.16.2
IP_LIST_FILE="/usr/local/inotify/iplist.txt"
ATTRIBUTES="modify,delete,create,attrib"
${INOTIFY} -mrq --timefmt '%d/%m/%y %H:%M' --format '%T %w%f' -e ${ATTRIBUTES} ${LOCAL_DIR} | while read files
do
for IPADDR in `sed '/^$/d' ${IP_LIST_FILE}`
do
${UNISION} common "${LOCAL_DIR}" "ssh://${REMOTE_USER}@${IPADDR}/${REMOTE_DIR}"
done
Done
[root@txzlkhdweb1 inotify-tools-3.14]# chmod +x /usr/local/inotify/unison.sh
[root@txzlkhdweb1 inotify-tools-3.14]# cd /usr/local/inotify
5.在txzlkhdweb2 上安装同样的inotify工具，步骤同上，此处省略
6.编写同步脚本
[root@txzlkhdweb2 inotify-tools-3.14]# cat /proc/sys/fs/inotify/max_user_watches
5000000000
[root@txzlkhdweb2 inotify-tools-3.14]# cat /proc/sys/fs/inotify/max_queued_events
327679
#事件监控脚本：
[root@txzlkhdweb2 inotify-tools-3.14]# vim /usr/local/inotify/unison.sh
#!/bin/bash
PATH=/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/bin:/usr/local/sbin
LOCAL_DIR="/root/test/"
REMOTE_DIR="/root/test/"
REMOTE_USER="root"
INOTIFY="/usr/local/inotify-tools-3.14/bin/inotifywait"
UNISION="/usr/local/bin/unison"
#目前iplist.txt 文件中就一个地址 192.168.16.1
IP_LIST_FILE="/usr/local/inotify/iplist.txt"
ATTRIBUTES="modify,delete,create,attrib"
${INOTIFY} -mrq --timefmt '%d/%m/%y %H:%M' --format '%T %w%f' -e ${ATTRIBUTES} ${LOCAL_DIR} | while read files
do
for IPADDR in `sed '/^$/d' ${IP_LIST_FILE}`
do
${UNISION} common "${LOCAL_DIR}" "ssh://${REMOTE_USER}@${IPADDR}/${REMOTE_DIR}"
done
Done
[root@txzlkhdweb2 inotify-tools-3.14]# chmod +x /usr/local/inotify/unison.sh
[root@txzlkhdweb2 inotify-tools-3.14]# cd /usr/local/inotify
[root@txzlkhdweb2 inotify-tools-3.14]# nohup ./unison.sh &
7. 执行inotify 监控脚本
[root@txzlkhdweb1 inotify-tools-3.14]# nohup ./unison.sh &
[root@txzlkhdweb2 inotify-tools-3.14]# nohup ./unison.sh &
8. 查看命令脚本执行程序是否在执行
[root@txzlkhdweb1 ~]# ps -ef | grep inotify
root 31194 31193 0 May18 pts/0 00:00:00 /usr/local/inotify-tools-3.14/bin/inotifywait -mrq --timefmt %d/%m/%y %H:%M --format %T %w%f -e modify,delete,create,attrib /root/test/
root 31469 24614 0 01:36 pts/0 00:00:00 grep inotify
9. 手工创建文件测试
[root@txzlkhdweb1 ~]# touch /root/test/f1.txt
[root@txzlkhdweb1 ~]# Contacting server...
Contacting server...
Connected [//txzlkhdweb1 //root/test -> //txzlkhdweb2//root/test]
Connected [//txzlkhdweb1 //root/test -> //txzlkhdweb2 //root/test]
Looking for changes
Waiting for changes from server
Reconciling changes
new file ----> f1.txt
local : new file modified on 2013-05-19 at 0:22:58 size 0 --rw-r--r-- user=0 group=0
slave : absent
Propagating updates
UNISON 2.40.102 started propagating changes at 00:22:59.08 on 19 May 2013
[BGN] Copying f1.txt from /root/test to //txzlkhdweb2//root/test
Looking for changes
Waiting for changes from server
Shortcut: copied /root/test/f1.txt from local file /root/test/www.prf
[END] Copying f1.txt
UNISON 2.40.102 finished propagating changes at 00:22:59.09 on 19 May 2013
Saving synchronizer state
Synchronization complete at 00:22:59 (1 item transferred, 0 skipped, 0 failed)
Contacting server...
Connected [//txzlkhdweb1//root/test -> //txzlkhdweb2//root/test]
Looking for changes
Waiting for changes from server
Reconciling changes
new file ----> f1.txt
local : new file modified on 2013-05-19 at 0:22:58 size 0 --rw-r--r-- user=0 group=0
slave : absent
Propagating updates
UNISON 2.40.102 started propagating changes at 00:23:00.09 on 19 May 2013
[BGN] Copying f1.txt from /root/test to //txzlkhdweb2//root/test
Shortcut: copied /root/test/f1.txt from local file /root/test/www.prf
Failed: Destination updated during synchronization
The file f1.txt has been created
100% 00:00 ETAFailed [f1.txt]: Destination updated during synchronization
The file f1.txt has been created
UNISON 2.40.102 finished propagating changes at 00:23:00.09 on 19 May 2013
Saving synchronizer state
Synchronization incomplete at 00:23:00 (0 items transferred, 0 skipped, 1 failed)
failed: f1.txt
#这里同步多次问题暂时不知道如何解决，希望高手指点
Contacting server...
Connected [//txzlkhdweb1//root/test -> //txzlkhdweb2//root/test]
Reconciling changes
Nothing to do: replicas have not changed since last sync.
Looking for changes
Waiting for changes from server
Reconciling changes
Nothing to do: replicas have been changed only in identical ways since last sync.
参考网址：