最完善的自动化运维脚本(打包、复制、上传、备份、启动、删除)
1、需求背景:为了快速部署测试,故有如下脚本
2、windows下maven 打包(单个、多个)和copy脚本：
现有开发工具都提供了很方便的打包快捷键,但是如果是项目模块较多,一个个打包麻烦,这时候我们就需要一个自动打包的脚本
打包脚本如下:
参数说明:
D:\codejiedian\workcodejiedian201105\Cloud-Power-Gateway 项目路径
D:\MAVEN\apache-maven-3.6.1\conf\riest-study-settings.xml maven 配置文件位置
clean install package -Dmaven.test.skip=true 调用maven的 clean install package 跳过测试
脚本如下:
@echo OFF
:START
:TUNNEL
cd /d D:\codejiedian\workcodejiedian201105\Cloud-Power-Gateway
mvn --settings D:\MAVEN\apache-maven-3.6.1\conf\riest-study-settings.xml clean install package -Dmaven.test.skip=true
PAUSE
goto TUNNEL
如果有很多个模块怎么处理?
-
写多个上述的打包脚本，每一个是一个单独的脚本
-
然后写一个总的bat脚本来启动多个打包脚本
参数说明: D:\packagebatWirhIdea\workcodejiedian201105\power\maven-Cloud-Power-WebBase.bat 单个打包脚本未知 ...... 表示可以写多个 @echo OFF call D:\packagebatWirhIdea\workcodejiedian201105\power\maven-Cloud-Power-WebBase.bat call D:\packagebatWirhIdea\workcodejiedian201105\power\maven-Cloud-Power-Gateway.bat ...... -
将所有打包好的包复制到一个指定的文件夹(就是一个copy脚本)
参数说明: D:\mavenjar\com\riest\power\Cloud-Power-Activiti\1.0.0\Cloud-Power-Activiti-1.0.0.jar jar包位置 E:\JarManager\fushuntest\201217-01 目的地 ...... 表示可以写多个 @echo off copy D:\mavenjar\com\riest\power\Cloud-Power-Activiti\1.0.0\Cloud-Power-Activiti-1.0.0.jar E:\JarManager\fushuntest\201217-01 copy D:\mavenjar\com\riest\power\Cloud-Power-Gateway\1.0.0\Cloud-Power-Gateway-1.0.0.jar E:\JarManager\fushuntest\201217-01 ...... TIMEOUT /T 5
3、将所有j准备好jar上传至服务器
我们这采用pcsp ,这个很简单,下载安装就行,也可用自带的ssh
打开windows命令行窗口,推荐使用powershell
命令如下:
参数说明
--password:服务器密码
--path:要上传的文件路径
--ip:服务器ip
--serverpath对应的服务器ip
pscp -r -P 22 -pw password path root@ip:serverpath
4、Liunx 启动jar脚本
需求：调用该脚本后,如果之前有对应的jar启动,将器kill,然后启动新的
脚本如下(我们以同时启动两个jar为例子)
#!/bin/sh
# jar 名称，根据实际情况来
export Jenkins_01=Jenkins-1.0.0-01.jar
export Jenkins_02=Jenkins-1.0.0-02.jar
# 启动参数,根据实际情况配置
export Jenkins_01_xms=-Xms256m -Xmx256m -Xmn128m -Duser.timezone=GMT+8
export Jenkins_02_xms=-Xms256m -Xmx256m -Xmn128m -Duser.timezone=GMT+8
# 启动端口
export Jenkins_01_port=9090
export Jenkins_02_port=9091
case "$1" in
start)
## 启动Jenkins9090
echo "--------Jenkins9090 开始启动--------------"
nohup java -jar $Jenkins_01 $Jenkins_01_xms &
Jenkins_01_pid=`lsof -i:$Jenkins_01_port|grep "LISTEN"|awk ‘{print $2}‘`
until [ -n "$Jenkins_01_pid" ]
do
Jenkins_01_pid=`lsof -i:$Jenkins_01_port|grep "LISTEN"|awk ‘{print $2}‘`
done
echo "Jenkins_01 pid is $Jenkins_01_pid"
sleep 10
echo "--------Jenkins9090 启动成功--------------"
## 启动Jenkins9091
echo "--------Jenkins9091 开始启动--------------"
nohup java -jar $Jenkins_02 $Jenkins_02_xms &
Jenkins_02_pid=`lsof -i:$Jenkins_02_port|grep "LISTEN"|awk ‘{print $2}‘`
until [ -n "$Jenkins_02_pid" ]
do
Jenkins_02_pid=`lsof -i:$Jenkins_02_port|grep "LISTEN"|awk ‘{print $2}‘`
done
echo "Jenkins_02 pid is $Jenkins_02_pid"
sleep 10
echo "--------Jenkins9091 启动成功--------------"
echo "===startAll success==="
;;
stop)
P_ID=`ps -ef | grep -w $Jenkins_01 | grep -v "grep" | awk ‘{print $2}‘`
if [ "$P_ID" == "" ]; then
echo "===Jenkins-01 process not exists or stop success"
else
kill -9 $P_ID
echo "Jenkins-01 killed success"
fi
P_ID=`ps -ef | grep -w $Jenkins_02 | grep -v "grep" | awk ‘{print $2}‘`
if [ "$P_ID" == "" ]; then
echo "===Jenkins-02 process not exists or stop success"
else
kill -9 $P_ID
echo "Jenkins-02 killed success"
fi
echo "===stop success==="
;;
restart)
$0 stop
sleep 10
$0 start
echo "===restart success==="
;;
esac
exit 0
5、Liunx 下备份脚本
我们在启动新版本的jar时需要备份之前的jar
备份命令如下
#!/bin/bash
#获得当前的时间
ctime=$(date +%Y%m%d%H%M%S)
#创建文件夹
mkdir -p /usr/local/backup/${ctime}
#将需要备份的数据copy
cp -r /mydata/mongodb/* /usr/local/backup/${ctime}
# 压缩备份 这里有两个问题
#1、给压缩文件起名字的时候不能用前面的ctime,暂不清楚原因
#2、压缩文件会将/usr/local/backup全部备份,后续解决
tar -zcPvf /usr/local/backup/backup$(date +%Y%m%d%H%M%S).tar.gz /usr/local/backup/${ctime}
#删除copy文件
rm -rf /usr/local/backup/${ctime}
6、聚合脚本：让前面所有的脚本达到一键启动
- 详见测试
7、本地写代码打包上传备份测试
建2个springboot项目,写一个controller,发送get请求，返回一个字符串，如下图
新建打包脚本
Cloud-Jenkins-9001.bat
@echo OFF
:START
:TUNNEL
cd /d D:\SpringCloudSeata\SpringCloudStudy\Jenkins-02
mvn --settings D:\MAVEN\apache-maven-3.6.1\conf\riest-study-settings.xml clean install package -Dmaven.test.skip=true
PAUSE
goto TUNNEL
Cloud-Jenkins-9002.bat
@echo OFF
:START
:TUNNEL
cd /d D:\SpringCloudSeata\SpringCloudStudy\Jenkins
mvn --settings D:\MAVEN\apache-maven-3.6.1\conf\riest-study-settings.xml clean install package -Dmaven.test.skip=true
PAUSE
goto TUNNEL
如图:
将打包脚本聚合在一起
@echo OFF
call E:\210107TEST\PackageBat\SinglePackage\Cloud-Jenkins-9001.bat
call E:\210107TEST\PackageBat\SinglePackage\Cloud-Jenkins-9002.bat
windows复制脚本
@echo off
del E:\210107TEST\Testjar\*.*
copy D:\mavenjar\com\riest\Jenkins\1.0.0\Jenkins-1.0.0.jar E:\210107TEST\Testjar
copy D:\mavenjar\com\riest\Jenkins-02\1.0.0\Jenkins-02-1.0.0.jar E:\210107TEST\Testjar
TIMEOUT /T 5
远程主主机的备份shell（判断是否需要备份,存在备份即可）
#!/bin/bash
if [ ! -d "/root/download/Testjar/" ];then
echo "文件夹不存在"
else
echo "**文件夹存在,开始备份**"
#获得当前的时间
ctime=$(date +%Y%m%d%H%M%S)
#创建文件夹
mkdir -p /root/download/backup/${ctime}
#将需要备份的数据copy
cp -r /root/download/Testjar/* /root/download/backup/${ctime}
# 压缩备份 这里有两个问题
#1、给压缩文件起名字的时候不能用前面的ctime,暂不清楚原因
#2、压缩文件会将/usr/local/backup全部备份,后续解决
tar -zcPvf /root/download/backup/backup$(date +%Y%m%d%H%M%S).tar.gz /root/download/backup/${ctime}
#删除copy文件
rm -rf /root/download/backup/${ctime}
fi
上传jar包到远程主机
@echo off
pass:服务器密码
ip:服务器ip
E:\210107TEST\Testjar:文件路径
/root/download:远程主机命令
pscp -r -P 22 -pw pass E:\210107TEST\Testjar root@服务器ip:/root/download
启动上传的jar(三个命令组成:start、stop、restart)
#!/bin/sh
# jar 路径以及名称，根据实际情况来
export Jenkins_01=/root/download/Testjar/Jenkins-1.0.0.jar
export Jenkins_02=/root/download/Testjar/Jenkins-02-1.0.0.jar
# 启动参数,根据实际情况配置
export Jenkins_01_xms=-Xms256m-Xmx256m-Xmn128m-Duser.timezone=GMT+8
export Jenkins_02_xms=-Xms256m-Xmx256m-Xmn128m-Duser.timezone=GMT+8
# 启动端口
export Jenkins_01_port=9092
export Jenkins_02_port=9091
case "$1" in
start)
## 启动权限01
echo "--------Jenkins9092 开始启动--------------"
nohup java -jar $Jenkins_01 $Jenkins_01_xms &
Jenkins_01_pid=`lsof -i:$Jenkins_01_port|grep "LISTEN"|awk ‘{print $2}‘`
until [ -n "$Jenkins_01_pid" ]
do
Jenkins_01_pid=`lsof -i:$Jenkins_01_port|grep "LISTEN"|awk ‘{print $2}‘`
done
echo "Jenkins_01 pid is $Jenkins_01_pid"
sleep 10
echo "--------Jenkins9092 启动成功--------------"
## 启动工作流01
echo "--------Jenkins9091 开始启动--------------"
nohup java -jar $Jenkins_02 $Jenkins_02_xms &
Jenkins_02_pid=`lsof -i:$Jenkins_02_port|grep "LISTEN"|awk ‘{print $2}‘`
until [ -n "$Jenkins_02_pid" ]
do
Jenkins_02_pid=`lsof -i:$Jenkins_02_port|grep "LISTEN"|awk ‘{print $2}‘`
done
echo "Jenkins_02 pid is $Jenkins_02_pid"
sleep 10
echo "--------Jenkins9091 启动成功--------------"
echo "===startAll success==="
;;
stop)
P_ID=`ps -ef | grep -w $Jenkins_01 | grep -v "grep" | awk ‘{print $2}‘`
if [ "$P_ID" == "" ]; then
echo "===Jenkins-01 process not exists or stop success"
else
kill -9 $P_ID
echo "Jenkins-01 killed success"
fi
P_ID=`ps -ef | grep -w $Jenkins_02 | grep -v "grep" | awk ‘{print $2}‘`
if [ "$P_ID" == "" ]; then
echo "===Jenkins-02 process not exists or stop success"
else
kill -9 $P_ID
echo "Jenkins-02 killed success"
fi
echo "===stop success==="
;;
restart)
$0 stop
sleep 10
$0 start
echo "===restart success==="
;;
esac
exit 0
一键启动脚本(windows)
@echo OFF
chcp 65001
echo "启动，开始打包"
call E:\210107TEST\PackageBat\PolyPackage\Maven-Cloud-Package.bat
echo "启动，开始复制"
call E:\210107TEST\CopyBat\CopyJar.bat
echo "启动，远程调用linux服务器上的sh备份脚本..."
call E:\210107TEST\BackUpLiunx\backup.bat
echo "启动，开始上传"
call E:\210107TEST\UploadLiunx\UploadLiunx.bat
echo "启动，远程调用linux服务器上的sh启动脚本..."
call E:\210107TEST\StratJar\startjar.bat
@cmd.exe
exist
8、遗留问题
首先给所有shell脚本授权
chmod +x xxx.sh
9、bug解决
**-bash: ./xxx.sh: /bin/sh^M: bad interpreter: No such file or directory **
原因：Windows环境下dos格式文件传输到unix系统时,会在每行的结尾多一个^M，所以在执行的时候出现了这种现象
解决办法,执行
sed -i -e ‘s/\r$//‘ xxx.sh
failed to run command `java‘: No such file or directory
解决办法：
首先排除服务器是否有java 环境,当然这一般不可能
第一步:
vim /etc/profile
将自己java配置的环境变量copy出来,如下
第二步执行:
vim ~/.bashrc
第三步将复制的java环境变量copy出来复制到最后,如下
第三步:
source ~/.bashrc