现在云已经是一个趋势毋庸置疑,但是怎样在有限的时间内把数据弄上云端是个需要考虑的问题。
如果能增量复制,整个时间窗口可以不需要太关注,因为切换的时间窗口比较短。
但是如果是全量迁移,或者增量迁移前的全量迁移,则需要考虑时间窗口。
一般来说,从线下到线上,网络带宽都比较低,这个我在以前把香港机房的数据迁移到国内机房时深有同感,速度之慢难以忍受。
下面是一套加速方案:
下面是一个例子:
首先在本地服务器建立远程数据库服务器隧道 :
ssh -C -L 6666:127.0.0.1:1921 postgres@remote_ip
在另外一个本地的Shell里面执行如下命令,备份远程数据库的数据到本地:
postgres@db-172-16-3-33-> date +%F%T;psql -h 127.0.0.1 -p 6666 -U postgres db_remote -c "copy schema.tbl_log to stdout" >./tbl_log.dmp;date +%F%T
2011-06-2717:16:50
2011-06-2717:17:08
耗时18秒
不走隧道的话
postgres@db-172-16-3-33-> date +%F%T;psql -h remote_ip -p 1921 -U schema db_remote -c "copy schema.tbl_log to stdout" >./tbl_log2.dmp;date +%F%T
2011-06-2717:20:06
2011-06-2717:22:29
耗时143秒
速度提升非常惊人.
参考:
man ssh
-L [bind_address:]port:host:hostport
Specifies that the given port on the local (client) host is to be forwarded to the given host and port on
the remote side. This works by allocating a socket to listen to port on the local side, optionally bound
to the specified bind_address. Whenever a connection is made to this port, the connection is forwarded
over the secure channel, and a connection is made to host port hostport from the remote machine. Port
forwardings can also be specified in the configuration file. IPv6 addresses can be specified with an
alternative syntax: [bind_address/]port/host/hostport or by enclosing the address in square brackets.
Only the superuser can forward privileged ports. By default, the local port is bound in accordance with
the GatewayPorts setting. However, an explicit bind_address may be used to bind the connection to a spe-
cific address. The bind_address of “localhost” indicates that the listening port be bound for local use
only, while an empty address or ‘*’ indicates that the port should be available from all interfaces.
-C Requests compression of all data (including stdin, stdout, stderr, and data for forwarded X11 and TCP
connections). The compression algorithm is the same used by gzip(1), and the “level” can be controlled
by the CompressionLevel option for protocol version 1. Compression is desirable on modem lines and other
slow connections, but will only slow down things on fast networks. The default value can be set on a
host-by-host basis in the configuration files; see the Compression option.