1 安装教程
官网安装教程:https://www.percona.com/doc/percona-xtrabackup/2.4/installation/apt_repo.html
-----------------------------------------------------------
安装2.3版本
root@ubuntu190:/opt/xtrabackup# wget https://repo.percona.com/apt/percona-release_0.1-6.$(lsb_release -sc)_all.deb
root@ubuntu190:/opt/xtrabackup# sudo dpkg -i percona-release_0.1-6.$(lsb_release -sc)_all.deb
此时检查是否自动生成配置文件
root@ubuntu190:/opt/xtrabackup# cat /etc/apt/sources.list.d/percona-release.list
该文件有内容则再执行
root@ubuntu190:/opt/xtrabackup# sudo apt-get update
root@ubuntu190:/opt/xtrabackup# sudo apt-get install percona-xtrabackup -y
报错
Reading package lists... Done
Building dependency tree
Reading state information... Done
You might want to run ‘apt-get -f install‘ to correct these:
The following packages have unmet dependencies:
percona-xtrabackup-80 : Depends: libgcrypt20 (>= 1.7.0) but 1.6.5-2ubuntu0.3 is to be installed
Depends: libssl1.1 (>= 1.1.0) but it is not installable
Depends: libstdc++6 (>= 6) but 5.4.0-6ubuntu1~16.04.11 is to be installed
Conflicts: percona-xtrabackup but 2.3.10-1.xenial is to be installed
percona-xtrabackup-dbg-80 : Depends: percona-xtrabackup-80 (= 8.0.11-1.xenial) but 8.0.4-1.stretch is to be installed
E: Unmet dependencies. Try ‘apt-get -f install‘ with no packages (or specify a solution).
解决
root@ubuntu190:/opt/xtrabackup# apt-get -f install -y
再次安装
root@ubuntu190:/opt/xtrabackup# sudo apt-get install percona-xtrabackup -y
查看版本
root@ubuntu190:/opt/xtrabackup# xtrabackup -version
xtrabackup version 2.3.10 based on MySQL server 5.6.24 Linux (x86_64) (revision id: bd0d4403f36)
You have new mail in /var/mail/root
此时表示安装成功。
-----------------------------------------------------------
补充(安装2.4版本):
我在后面使用2.3.10版本的xtrabackup执行备份时,提示我以下错误
root@ubuntu190:~# sudo innobackupex --defaults-file=/etc/mysql/my.cnf --user=backup --password=yayun --socket=/var/run/mysqld/mysqld.sock /data/backup/
200527 14:21:47 innobackupex: Starting the backup operation
IMPORTANT: Please check that the backup run completes successfully.
At the end of a successful backup run innobackupex
prints "completed OK!".
200527 14:21:48 version_check Connecting to MySQL server with DSN ‘dbi:mysql:;mysql_read_default_group=xtrabackup;mysql_socket=/var/run/mysqld/mysqld.sock‘ as ‘backup‘ (using password: YES).
200527 14:21:48 version_check Connected to MySQL server
200527 14:21:48 version_check Executing a version check against the server...
200527 14:21:48 version_check Done.
200527 14:21:48 Connecting to MySQL server host: localhost, user: backup, password: set, port: not set, socket: /var/run/mysqld/mysqld.sock
Error: Unsupported server version: ‘5.7.27-0ubuntu0.16.04.1-log‘. Please report a bug at https://bugs.launchpad.net/percona-xtrabackup
root@ubuntu190:~#
原因是xtrabackup版本太老,2.3版本与mysql5.7不匹配,最好重新安装2.4版。如下:
root@ubuntu190:/opt/xtrabackup# wget https://repo.percona.com/apt/percona-release_latest.$(lsb_release -sc)_all.deb
root@ubuntu190:/opt/xtrabackup# sudo dpkg -i percona-release_latest.$(lsb_release -sc)_all.deb
root@ubuntu190:/opt/xtrabackup# cat /etc/apt/sources.list.d/percona-release.list
cat: /etc/apt/sources.list.d/percona-release.list: No such file or directory
此时发现该文件中无任何内容,只能手动加入源
root@ubuntu190:/opt/xtrabackup# nano /etc/apt/sources.list.d/percona-release.list
加入以下内容
#
# Percona releases, stable
#
deb http://repo.percona.com/apt xenial main
deb-src http://repo.percona.com/apt xenial main
#
# Testing & pre-release packages
#
#deb http://repo.percona.com/apt xenial testing
#deb-src http://repo.percona.com/apt xenial testing
#
# Experimental packages, use with caution!
#
#deb http://repo.percona.com/apt xenial experimental
#deb-src http://repo.percona.com/apt xenial experimental
卸载2.3版本
root@ubuntu190:/opt/xtrabackup# sudo apt-get remove percona-xtrabackup -y
安装2.4版本
root@ubuntu190:/opt/xtrabackup# sudo apt-get update
root@ubuntu190:/opt/xtrabackup# sudo apt-get install percona-xtrabackup-24 -y
查看版本
root@ubuntu190:/opt/xtrabackup# xtrabackup -version
xtrabackup: recognized server arguments: --datadir=/var/lib/mysql --tmpdir=/tmp --server-id=1 --log_bin=/var/log/mysql/mysql_bin.log
xtrabackup version 2.4.20 based on MySQL server 5.7.26 Linux (x86_64) (revision id: c8b4056)
root@ubuntu190:/opt/xtrabackup#
2 自动化备份和恢复脚本
必须先清楚xtrabackup的原理和备份流程才能看懂以下代码。
import os import time from automatic_backup_and_recovery_mysql.tools.Ssh import Ssh class AutoBakAndRecMysql(): ‘‘‘ 目的:自动备份和恢复远程目标服务器上的数据库集群的数据 备份策略: 该脚本每天(24h为一个循环周期)执行一次,一共备份"bakTotalTimes"次,第1次全备份,剩下"bakTotalTimes-1"次都是增量备份,每天完成"bakTotalTimes"次备份后, 将"bakTotalTimes-1"份增量备份目录(按照备份时间升序排列)依次恢复到当天的全备份目录, 得到一个总的全备份目录,该目录即为当天所有数据的备份。然后将其放入另一目录下备用,该目录最多放置”totalBakDirNum“个总的全备份目录。 运行方式:此脚本运行在堡垒机192.168.1.190上,通过ssh远程操控目标服务器 ‘‘‘ def __init__(self, ip, sshUsername, sshPasswd, sshPort, mysqlUsername, mysqlPasswd, mysqlPort, backupDir="/data/backup/", fullBackupDir="/data/backup/full/", incrementalBackupDir="/data/backup/incremental/", totalBackupDir="/data/backup/last/", totalFullBakDirNum=4, totalIncBakDirNum=10, totalBakDirNum=5, bakInterval=2, bakTotalTimes=6, mysqlSocketFile="/var/run/mysqld/mysqld.sock", mysqlCnfFile="/etc/mysql/my.cnf", mysqlDataDir="/var/lib/mysql"): self.ip = ip # ssh连接信息 self.sshUsername = sshUsername self.sshPasswd = sshPasswd self.sshPort = sshPort # 数据库连接信息 self.mysqlUsername = mysqlUsername self.mysqlPasswd = mysqlPasswd self.mysqlPort = mysqlPort # 数据库配置信息 self.mysqlDataDir = mysqlDataDir # 数据库数据存储目录 self.mysqlSocketFile = mysqlSocketFile # 数据库主配置文件 self.mysqlCnfFile = mysqlCnfFile # 数据库sock文件 # 备份 self.backupDir = backupDir # 备份主目录 self.fullBackupDir = fullBackupDir # 全量备份目录 self.incrementalBackupDir = incrementalBackupDir # 增量备份目录 self.totalBackupDir = totalBackupDir # 将每天得到的总的全备份目录放入该目录中 self.totalFullBakDirNum = totalFullBakDirNum # 保留几个全量备份目录 self.totalIncBakDirNum = totalIncBakDirNum # 保留几个增量备份目录 self.totalBakDirNum = totalBakDirNum # 保留几个总的全备份目录 self.bakInterval = bakInterval # 备份时间间隔,单位分钟 self.bakTotalTimes = bakTotalTimes # 总备份次数 def getSsh(self): ‘‘‘ 获取ssh操作对象 :return: ‘‘‘ ssh = Ssh(ip=self.ip, username=self.sshUsername, password=self.sshPasswd, sshPort=self.sshPort) return ssh def judgeMysqlVersion(self): ‘‘‘ 判断数据库版本,要求版本>=5.7,返回True表示数据库满足要求,前提:通过apt方式安装的mysql :return: ‘‘‘ cmd = "mysql -V | awk -F‘,‘ ‘{print $1}‘ | awk ‘{print $5}‘ |awk -F‘.‘ ‘{print $1,$2}‘" res = self.getSsh().runCommandToString(cmd=cmd) res = str(res, encoding="utf-8") res = res.strip("\n").split() res = "".join(res) res = int(res) # 将版本大小转换成整型数字 if res >= 57: print("%s 数据库版本>=5.7,满足要求,可以使用xtrabackup2.4" % self.ip) return True else: print("%s 数据库版本<=5.7,不满足要求,无法使用xtrabackup2.4" % self.ip) return False def isMysqlRun(self): ‘‘‘ 检测数据库是否在运行,返回True表示运行。前提:通过apt方式安装的mysql :return: ‘‘‘ cmd = "service mysql status |grep ‘Active‘" res = self.getSsh().runCommandToString(cmd) res = str(res, encoding="utf-8").strip("\n").split(":") isActive = res[1].strip().startswith("active") return isActive def countDown(self): ‘‘‘ 指定时间的倒计时 :return: ‘‘‘ for i in range(self.bakInterval, 0, -1): msg = u"\r系统将在 " + str(i) + " 分钟后进行下一次备份" print(msg, end="") time.sleep(60) # 休眠60秒 end_msg = "开始备份" + " " * (len(msg) - len("开始备份")) # 如果单纯只用“开始备份”4个字,无法完全覆盖之前的内容 print(u"\r" + end_msg) def installXtrabackup(self): ‘‘‘ 在Ubuntu上安装xtrabackup2.4版 :return: ‘‘‘ checkXtrabackup = self.getSsh().runCommandToStringList(" xtrabackup -version") if checkXtrabackup[-1].find("2.4") != -1: print("%s xtrabackup已安装" % self.ip) else: self.getSsh().runCommandToString( "wget https://repo.percona.com/apt/percona-release_latest.$(lsb_release -sc)_all.deb") self.getSsh().runCommandToString("sudo dpkg -i percona-release_latest.$(lsb_release -sc)_all.deb") res = self.getSsh().runCommandToStringList("cat /etc/apt/sources.list.d/percona-release.list") if "No such file or directory" in res[0]: print("%s 更新xtrabackup源文件失败" % self.ip) # 此时手动加入源 rootFilePath = os.path.dirname(__file__) + "/tools/percona-release.list" # 堡垒机上的文件 targetDir = "/etc/apt/sources.list.d/" # 目标服务器上的目录 os.system("apt install sshpass -y") os.system("sshpass -p %s scp -r %s %s@%s:%s" % ( self.sshPasswd, rootFilePath, self.sshUsername, self.ip, targetDir)) # 复制源文件到目标服务器 res2 = self.getSsh().runCommandToStringList("cat /etc/apt/sources.list.d/percona-release.list") if len(res2) <= 1 and "No such file or directory" in res[0]: print("%s 创建xtrabackup源文件失败" % self.ip) else: print("%s 创建xtrabackup源文件成功" % self.ip) self.getSsh().runCommandToString( "sudo apt-get update && sudo apt-get install percona-xtrabackup-24 -y") res3 = self.getSsh().runCommandToStringList(" xtrabackup -version") if res3[-1].find("2.4") != -1: # 检测xtrabackup版本 print("%s xtrabackup安装成功" % self.ip) else: print("%s xtrabackup安装失败" % self.ip) def limitDirNums(self, flag): ‘‘‘ 限制指定目录的子目录数量,按照创建时间降序排列,只保留时间最新的指定个数的子目录 :param flag: 标志位,为0则限制全量备份目录,为1则限制增量备份目录,为2则限制总的全量备份目录 :return: ‘‘‘ if flag == 0: cmd = """find %s -mindepth 1 -maxdepth 1 -type d -printf "%%P\n" | sort -nr | head -%s""" % ( self.fullBackupDir, self.totalFullBakDirNum) res = self.getSsh().runCommandToStringList(cmd) # 需要保留的目录 allDirPath = self.getSsh().runCommandToStringList("ls %s" % self.fullBackupDir) dirDelete = [path for path in allDirPath if path not in res] # 需要被删除的目录 dirRootDelete = [self.fullBackupDir + path + "/" for path in dirDelete] [self.getSsh().runCommandToString("rm -r %s" % path) for path in dirRootDelete] print("%s 上的全量备份目录 %s 的子目录数量被限制为 %s 个" % (self.ip, self.fullBackupDir, self.totalFullBakDirNum)) elif flag == 1: cmd2 = """find %s -mindepth 1 -maxdepth 1 -type d -printf "%%P\n" | sort -nr | head -%s""" % ( self.incrementalBackupDir, self.totalIncBakDirNum) res2 = self.getSsh().runCommandToStringList(cmd2) # 需要保留的目录 allDirPath2 = self.getSsh().runCommandToStringList("ls %s" % self.incrementalBackupDir) dirDelete2 = [path for path in allDirPath2 if path not in res2] # 需要被删除的目录 dirRootDelete2 = [self.incrementalBackupDir + path + "/" for path in dirDelete2] [self.getSsh().runCommandToString("rm -r %s" % path) for path in dirRootDelete2] print("%s 上的增量备份目录 %s 的子目录数量被限制为 %s 个" % (self.ip, self.incrementalBackupDir, self.totalIncBakDirNum)) elif flag == 2: cmd3 = """find %s -mindepth 1 -maxdepth 1 -type d -printf "%%P\n" | sort -nr | head -%s""" % ( self.totalBackupDir, self.totalBakDirNum) res3 = self.getSsh().runCommandToStringList(cmd3) # 需要保留的目录 allDirPath3 = self.getSsh().runCommandToStringList("ls %s" % self.totalBackupDir) dirDelete3 = [path for path in allDirPath3 if path not in res3] # 需要被删除的目录 dirRootDelete3 = [self.totalBackupDir + path + "/" for path in dirDelete3] [self.getSsh().runCommandToString("rm -r %s" % path) for path in dirRootDelete3] print("%s 上的总的全量备份目录 %s 的子目录数量被限制为 %s 个" % (self.ip, self.totalBackupDir, self.totalBakDirNum)) def singleIncrementalBak(self, baseBakDir): ‘‘‘ 单次自动增量备份,得到一个增量备份目录 :param baseBakDir: 备份基础目录 :return: ‘‘‘ res = self.getSsh().runCommandToString( " innobackupex --defaults-file=%s --user=%s --password=%s --host=%s --port=%s --socket=%s --incremental %s --incremental-basedir=%s --parallel=2" % ( self.mysqlCnfFile, self.mysqlUsername, self.mysqlPasswd, self.ip, self.mysqlPort, self.mysqlSocketFile, self.incrementalBackupDir, baseBakDir)) res = str(res, encoding="utf-8") if "completed OK" in res: # 增量备份成功 newestIncBakDir = self.getSsh().runCommandToString( "find %s -mindepth 1 -maxdepth 1 -type d -printf ‘%%P\n‘ | sort -nr | head -1" % self.incrementalBackupDir) # 最新备份目录 newestIncBakDir = str(newestIncBakDir, encoding="utf-8").strip("\n") newestBakRootDir = self.incrementalBackupDir + newestIncBakDir # 增量备份目录全路径 print("%s 剩余备份次数:%s,本次增量备份成功,目录为:%s" % (self.ip, self.bakTotalTimes - 1, newestBakRootDir)) self.bakTotalTimes -= 1 return newestBakRootDir def autoFullBak(self): ‘‘‘ 自动全量备份 :return: ‘‘‘ # 先检测数据库是否运行 isRun = self.isMysqlRun() if isRun: res = self.getSsh().runCommandToString("mkdir -p %s" % self.fullBackupDir) print("%s 全量备份目录 %s 创建成功" % (self.ip, self.fullBackupDir)) res = self.getSsh().runCommandToString( "innobackupex --defaults-file=%s --user=%s --password=%s --host=%s --port=%s --socket=%s %s" % ( self.mysqlCnfFile, self.mysqlUsername, self.mysqlPasswd, self.ip, self.mysqlPort, self.mysqlSocketFile, self.fullBackupDir)) res = str(res, encoding="utf-8") if "completed OK" in res: # 备份成功 newestFullBakDir = self.getSsh().runCommandToString( "find %s -mindepth 1 -maxdepth 1 -type d -printf ‘%%P\n‘ | sort -nr | head -1" % self.fullBackupDir) # 最新备份目录 newestFullBakDir = str(newestFullBakDir, encoding="utf-8").strip("\n") newestFullBakRootDir = self.fullBackupDir + newestFullBakDir # 最新全量备份目录的全路径 # print("%s 全量备份成功,最新备份目录为:%s" % (self.ip, newestFullBakRootDir)) return newestFullBakRootDir else: print("%s 全量备份失败" % (self.ip)) return None else: print("%s 数据库未运行" % self.ip) return None def autoIncrementalBak(self): ‘‘‘ 总的自动增量备份,得到单个全量备份目录和多个增量备份目录 :return: ‘‘‘ bakDirList = list() # 存放一次循环中的全量备份目录和所有增量备份目录 res = self.getSsh().runCommandToString("mkdir -p %s" % self.incrementalBackupDir) print("%s 增量备份目录 %s 创建成功" % (self.ip, self.incrementalBackupDir)) while self.bakTotalTimes: newestFullBakDir = self.autoFullBak() # 进行首次增量备份之前都先要来一次全量备份,因首次增量备份的基础是全量备份 if newestFullBakDir is not None: # 全量备份成功时 print("%s 剩余备份次数:%s,本次全量备份成功,目录为:%s" % (self.ip, self.bakTotalTimes - 1, newestFullBakDir)) bakDirList.append(newestFullBakDir) self.bakTotalTimes -= 1 self.countDown() # 第1次增量备份 newestBakRootDir1 = self.singleIncrementalBak(newestFullBakDir) bakDirList.append(newestBakRootDir1) self.countDown() # 第2次增量备份 newestBakRootDir2 = self.singleIncrementalBak(newestBakRootDir1) bakDirList.append(newestBakRootDir2) self.countDown() # 第3次增量备份 newestBakRootDir3 = self.singleIncrementalBak(newestBakRootDir2) bakDirList.append(newestBakRootDir3) self.countDown() # 第4次增量备份 newestBakRootDir4 = self.singleIncrementalBak(newestBakRootDir3) bakDirList.append(newestBakRootDir4) self.countDown() # 第5次增量备份 newestBakRootDir5 = self.singleIncrementalBak(newestBakRootDir4) bakDirList.append(newestBakRootDir5) return bakDirList else: print("%s 全量备份失败,所以无法进行增量备份" % self.ip) return None def getLastFullBakDir(self): ‘‘‘ 得到最新的总的全量备份目录 :return: ‘‘‘ res = self.getSsh().runCommandToString("mkdir -p %s" % self.totalBackupDir) print("%s 存放总的全量备份目录的目录 %s 创建成功" % (self.ip, self.totalBackupDir)) bakDirList = self.autoIncrementalBak() if bakDirList is not None: fullBak = bakDirList[0] # 全量备份目录 incBakList = bakDirList[1:-1] # 增量备份目录列表,按照时间先后升序排列,不包含最后一个增量备份目录 incBakLast = bakDirList[-1] # 最后一个增量备份目录,需要去掉--redo-only参数 res2 = self.getSsh().runCommandToString(" innobackupex --apply-log --redo-only %s" % fullBak) res2 = str(res2, encoding="utf-8") if "completed OK" in res2: print("%s 恢复全量备份目录 %s 成功" % (self.ip, fullBak)) else: print("%s 恢复全量备份目录 %s 失败" % (self.ip, fullBak)) for path in incBakList: result = self.getSsh().runCommandToString( " innobackupex --apply-log --redo-only %s --incremental-dir=%s" % (fullBak, path)) result = str(result, encoding="utf-8") if "completed OK" in result: print("%s 将增量备份目录 %s 应用到全量备份目录 %s 成功" % (self.ip, path, fullBak)) else: print("%s 将增量备份目录 %s 应用到全量备份目录 %s 失败" % (self.ip, path, fullBak)) res3 = self.getSsh().runCommandToString( " innobackupex --apply-log %s --incremental-dir=%s" % (fullBak, path)) res3 = str(res3, encoding="utf-8") if "completed OK" in res3: print("%s 将最后一个增量备份目录 %s 应用到全量备份目录 %s 成功" % (self.ip, incBakLast, fullBak)) else: print("%s 将最后一个增量备份目录 %s 应用到全量备份目录 %s 失败" % (self.ip, incBakLast, fullBak)) res4 = self.getSsh().runCommandToString(" innobackupex --apply-log %s" % fullBak) res4 = str(res4, encoding="utf-8") if "completed OK" in res4: print("%s 将总的全量备份目录 %s 再进行一次apply操作,用以回滚未提交的数据 成功" % (self.ip, fullBak)) else: print("%s 将总的全量备份目录 %s 再进行一次apply操作,用以回滚未提交的数据 失败" % (self.ip, fullBak)) self.getSsh().runCommandToString( "cp -r %s %s" % (fullBak, self.totalBackupDir)) # 得到总的全量备份目录之后,将该目录复制到指定目录备用 newestLastFullBakDir = self.getSsh().runCommandToString( "find %s -mindepth 1 -maxdepth 1 -type d -printf ‘%%P\n‘ | sort -nr | head -1" % self.totalBackupDir) # 最新的总的全量备份目录 newestLastFullBakDir = str(newestLastFullBakDir, encoding="utf-8").strip("\n") newestLastFullBakDir = self.totalBackupDir + newestLastFullBakDir # 最新的总的全量备份目录全路径 return newestLastFullBakDir def autoRecMysql(self): ‘‘‘ 自动恢复数据库数据 :return: ‘‘‘ newestLastFullBakDir = self.getLastFullBakDir() dirSize = self.getSsh().runCommandToString("du -sh %s | awk ‘{print $1}‘ " % newestLastFullBakDir) dirSize = str(dirSize, encoding="utf-8") print("%s 上最新的总的全量备份目录为 %s 目录大小:%s" % (self.ip, newestLastFullBakDir, dirSize)) choice = input("%s 是否恢复数据库数据?(y/n)" % self.ip) if str(choice).lower() == "y": self.getSsh().runCommandToString("service mysql stop") isMysqlRun = self.isMysqlRun() if not isMysqlRun: # 数据库停止服务,恢复数据之前必须关闭数据库 print("关闭 %s mysql 成功,开始恢复数据" % self.ip) now = time.strftime("%Y-%m-%d-%H-%M-%S", time.localtime(time.time())) # 当前时间,格式2020-06-02-10-44-43 self.getSsh().runCommandToString(" mv %s %s-%s" % (self.mysqlDataDir, self.mysqlDataDir, now)) self.getSsh().runCommandToString(" mkdir %s" % self.mysqlDataDir) self.getSsh().runCommandToString(" innobackupex --defaults-file=%s --copy-back --rsync %s" % ( self.mysqlCnfFile, newestLastFullBakDir)) # 恢复数据 self.getSsh().runCommandToString("chown -R mysql.mysql %s" % self.mysqlDataDir) # 文件和目录的所属组改为mysql self.getSsh().runCommandToString("service mysql start") isMysqlRun2 = self.isMysqlRun() if isMysqlRun2: print("%s 数据库恢复成功" % self.ip) else: print("%s 数据库恢复失败" % self.ip) else: print("关闭 %s mysql 失败,无法恢复数据" % self.ip) elif str(choice).lower() == "n": print("%s 已放弃恢复数据库数据" % self.ip) def Main(self): ‘‘‘ 数据库数据备份和恢复主方法 :return: ‘‘‘ if self.judgeMysqlVersion(): self.installXtrabackup() self.autoRecMysql() # 限制备份相关目录的子目录数量 self.limitDirNums(flag=0) self.limitDirNums(flag=1) self.limitDirNums(flag=2) else: pass if __name__ == ‘__main__‘: autoBakAndRecMysql = AutoBakAndRecMysql(ip="192.168.1.190", sshUsername="root", sshPasswd="**", sshPort=22, mysqlUsername="root", mysqlPasswd="**", mysqlPort="3306", ) # autoBakAndRecMysql.judgeMysqlVersion() # autoBakAndRecMysql.isMysqlRun() # autoBakAndRecMysql.installXtrabackup() # autoBakAndRecMysql.autoFullBak() # autoBakAndRecMysql.limitDirNums(flag=1) # autoBakAndRecMysql.autoIncrementalBak() # autoBakAndRecMysql.countDown() # autoBakAndRecMysql.getLastFullBakDir() autoBakAndRecMysql.Main()
上面代码中用到的工具类如下:
import os import sys import time class Ssh(): ‘‘‘ ssh远程连接工具 ‘‘‘ def __init__(self, ip, username, password, sshPort): ‘‘‘ ssh远程连接信息,即目标服务器的ssh登录信息 :param ip: :param username: :param password: :param sshPort: :return: ‘‘‘ self.ip = ip self.username = username self.passwprd = password self.sshPort = sshPort def runCommandToString(self, cmd): ‘‘‘ 远程执行单条命令,输入参数为需要执行的单条命令,输出为结果的字符串形式 ‘‘‘ # 检测虚拟机上是否安装了模块paramiko try: import paramiko except ImportError: try: command1 = "apt-get update && apt install python-pip && pip install --upgrade pip" command2 = "pip install paramiko " os.system(command1) time.sleep(5) os.system(command2) print("paramiko install Fail") except: print("Can not install paramiko, Aborted!") sys.exit(1) finally: import paramiko try: ssh = paramiko.SSHClient() ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy()) ssh.connect(hostname=self.ip, username=self.username, password=self.passwprd, port=self.sshPort, timeout=10, allow_agent=False, look_for_keys=False) std_in, std_out, std_err = ssh.exec_command(cmd) res, err = std_out.read(), std_err.read() # 获取命令结果,字符串形式 if res: # if isinstance(res,bytes): # res=str(res,encoding="utf-8").strip() # result = res result=res elif err: result = err else: # 如果执行的命令无返回值,则手动将返回结果置为空字符串 result = " " # print("%s执行命令成功,但返回值为空:%s \n" %(self.ip ,str(cmd)) ,end="") ssh.close() # print("%s执行命令成功:%s \n" %(self.ip ,str(cmd)) ,end="") return result # 字符串形式 except Exception as e: print(str(e)) print("%s执行命令异常:%s \n" % (self.ip, str(cmd)), end="") def runCommandToStringList(self, cmd): ‘‘‘ 远程执行单条命令,输入参数为需要执行的单条命令,输出为结果的字符串列表形式 ‘‘‘ # 检测虚拟机上是否安装了模块paramiko try: import paramiko except ImportError: try: command1 = "apt-get update && apt install python-pip && pip install --upgrade pip" command2 = "pip install paramiko " os.system(command1) time.sleep(5) os.system(command2) print("paramiko install Fail") except: print("Can not install paramiko, Aborted!") sys.exit(1) finally: import paramiko try: ssh = paramiko.SSHClient() ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy()) ssh.connect(hostname=self.ip, username=self.username, password=self.passwprd, port=self.sshPort, timeout=10, allow_agent=False, look_for_keys=False) std_in, std_out, std_err = ssh.exec_command(cmd) res, err = std_out.readlines(), std_err.readlines() # 获取命令结果,字符串形式 if res: result = res result = [r.strip() for r in result] elif err: result = err result = [r.strip() for r in result] else: # 如果执行的命令无返回值,则手动将返回结果置为空列表 result = [] # print("%s执行命令成功,但返回值为空列表:%s \n" %(self.ip ,str(cmd)) ,end="") ssh.close() # print("%s执行命令成功:%s \n" %(self.ip ,str(cmd)) ,end="") return result # 字符串形式 except Exception as e: print(str(e)) print("%s执行命令异常:%s \n" % (self.ip, str(cmd)), end="") if __name__ == ‘__main__‘: ssh=Ssh(ip=‘192.168.1.160‘,username=‘root‘,sshPort=22,password=‘NT60ts1689‘) res=ssh.runCommandToStringList(‘date‘) print(res)
文件“percona-release.list”的内容如下:
#
# Percona releases, stable
#
deb http://repo.percona.com/apt xenial main
deb-src http://repo.percona.com/apt xenial main
#
# Testing & pre-release packages
#
#deb http://repo.percona.com/apt xenial testing
#deb-src http://repo.percona.com/apt xenial testing
#
# Experimental packages, use with caution!
#
#deb http://repo.percona.com/apt xenial experimental
#deb-src http://repo.percona.com/apt xenial experimental
3 运行效果
ssh://root@192.168.1.190:22/usr/bin/python3.5 -u /opt/pycharm_to_ubuntu/automatic_backup_and_recovery_mysql/auto_backup_and_recovery.py /usr/local/lib/python3.5/dist-packages/paramiko/kex_ecdh_nist.py:39: CryptographyDeprecationWarning: encode_point has been deprecated on EllipticCurvePublicNumbers and will be removed in a future version. Please use EllipticCurvePublicKey.public_bytes to obtain both compressed and uncompressed point encoding. m.add_string(self.Q_C.public_numbers().encode_point()) /usr/local/lib/python3.5/dist-packages/paramiko/kex_ecdh_nist.py:96: CryptographyDeprecationWarning: Support for unsafe construction of public numbers from encoded data will be removed in a future version. Please use EllipticCurvePublicKey.from_encoded_point self.curve, Q_S_bytes /usr/local/lib/python3.5/dist-packages/paramiko/kex_ecdh_nist.py:111: CryptographyDeprecationWarning: encode_point has been deprecated on EllipticCurvePublicNumbers and will be removed in a future version. Please use EllipticCurvePublicKey.public_bytes to obtain both compressed and uncompressed point encoding. hm.add_string(self.Q_C.public_numbers().encode_point()) 192.168.1.190 数据库版本>=5.7,满足要求,可以使用xtrabackup2.4 192.168.1.190 xtrabackup已安装 192.168.1.190 存放总的全量备份目录的目录 /data/backup/last/ 创建成功 192.168.1.190 增量备份目录 /data/backup/incremental/ 创建成功 192.168.1.190 全量备份目录 /data/backup/full/ 创建成功 192.168.1.190 剩余备份次数:5,本次全量备份成功,目录为:/data/backup/full/2020-06-02_13-54-17 开始备份 192.168.1.190 剩余备份次数:4,本次增量备份成功,目录为:/data/backup/incremental/2020-06-02_13-56-21 开始备份 192.168.1.190 剩余备份次数:3,本次增量备份成功,目录为:/data/backup/incremental/2020-06-02_13-58-23 开始备份 192.168.1.190 剩余备份次数:2,本次增量备份成功,目录为:/data/backup/incremental/2020-06-02_14-00-26 开始备份 192.168.1.190 剩余备份次数:1,本次增量备份成功,目录为:/data/backup/incremental/2020-06-02_14-02-29 开始备份 192.168.1.190 剩余备份次数:0,本次增量备份成功,目录为:/data/backup/incremental/2020-06-02_14-04-32 192.168.1.190 恢复全量备份目录 /data/backup/full/2020-06-02_13-54-17 成功 192.168.1.190 将增量备份目录 /data/backup/incremental/2020-06-02_13-56-21 应用到全量备份目录 /data/backup/full/2020-06-02_13-54-17 成功 192.168.1.190 将增量备份目录 /data/backup/incremental/2020-06-02_13-58-23 应用到全量备份目录 /data/backup/full/2020-06-02_13-54-17 成功 192.168.1.190 将增量备份目录 /data/backup/incremental/2020-06-02_14-00-26 应用到全量备份目录 /data/backup/full/2020-06-02_13-54-17 成功 192.168.1.190 将增量备份目录 /data/backup/incremental/2020-06-02_14-02-29 应用到全量备份目录 /data/backup/full/2020-06-02_13-54-17 成功 192.168.1.190 将最后一个增量备份目录 /data/backup/incremental/2020-06-02_14-04-32 应用到全量备份目录 /data/backup/full/2020-06-02_13-54-17 成功 192.168.1.190 将总的全量备份目录 /data/backup/full/2020-06-02_13-54-17 再进行一次apply操作,用以回滚未提交的数据 成功 192.168.1.190 上最新的总的全量备份目录为 /data/backup/last/2020-06-02_13-54-17 目录大小:554M 192.168.1.190 是否恢复数据库数据?(y/n)y y 关闭 192.168.1.190 mysql 成功,开始恢复数据 192.168.1.190 数据库恢复成功 192.168.1.190 上的全量备份目录 /data/backup/full/ 的子目录数量被限制为 4 个 192.168.1.190 上的增量备份目录 /data/backup/incremental/ 的子目录数量被限制为 10 个 192.168.1.190 上的总的全量备份目录 /data/backup/last/ 的子目录数量被限制为 5 个 Process finished with exit code 0
python脚本:在Ubuntu16系统上基于xtrabackup2.4和mysql5.7实现数据库数据的自动化备份和恢复,亲测有效!