需要部署nginx的https环境，之前是yum安装的openssl，版本比较低，如下：
[root@nginx ~]# yum install -y pcre pcre-devel openssl openssl-devel gcc [root@nginx ~]# openssl version -a OpenSSL 1.0.1e-fips 11 Feb 2013 built on: Wed Mar 22 21:43:28 UTC 2017 platform: linux-x86_64 options: bn(64,64) md2(int) rc4(16x,int) des(idx,cisc,16,int) idea(int) blowfish(idx) compiler: gcc -fPIC -DOPENSSL_PIC -DZLIB -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -DKRB5_MIT -m64 -DL_ENDIAN -DTERMIO -Wall -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -Wa,--noexecstack -DPURIFY -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DMD5_ASM -DAES_ASM -DVPAES_ASM -DBSAES_ASM -DWHIRLPOOL_ASM -DGHASH_ASM OPENSSLDIR: "/etc/pki/tls" engines: rdrand dynamic
默认yum安装的openssl版本是1.0.1，现在需要将版本升级到1.1.0。升级的操作记录如下：
[root@nginx ~]# wget https://www.openssl.org/source/openssl-1.1.0g.tar.gz [root@nginx ~]# tar -zvxf openssl-1.1.0g.tar.gz [root@nginx ~]# cd openssl-1.1.0g [root@nginx openssl-1.1.0g]# ./config shared zlib [root@nginx openssl-1.1.0g]# make [root@nginx openssl-1.1.0g]# make install [root@nginx openssl-1.1.0g]# mv /usr/bin/openssl /usr/bin/openssl.bak [root@nginx openssl-1.1.0g]# mv /usr/include/openssl /usr/include/openssl.bak [root@nginx openssl-1.1.0g]# find / -name openssl /etc/pki/ca-trust/extracted/openssl /data/software/nginx-1.12.2/auto/lib/openssl /data/software/openssl-1.1.0g/apps/openssl /data/software/openssl-1.1.0g/include/openssl /usr/lib64/openssl /usr/local/share/doc/openssl /usr/local/include/openssl /usr/local/bin/openssl /usr/include/openssl /usr/bin/openssl [root@nginx openssl-1.1.0g]# ln -s /usr/local/bin/openssl /usr/bin/openssl [root@nginx openssl-1.1.0g]# ln -s /usr/local/include/openssl /usr/include/openssl [root@external-lb01 ~]# find / -name "libssl*" /data/software/openssl-1.1.0g/libssl.pc /data/software/openssl-1.1.0g/libssl.so /data/software/openssl-1.1.0g/libssl.a /data/software/openssl-1.1.0g/libssl.so.1.1 /data/software/openssl-1.1.0g/util/libssl.num /usr/lib64/libssl3.so /usr/lib64/pkgconfig/libssl.pc /usr/lib64/libssl.so.1.0.1e /usr/lib64/libssl.so /usr/lib64/libssl.so.10 /usr/local/lib64/libssl.a /usr/local/lib64/pkgconfig/libssl.pc /usr/local/lib64/libssl.so /usr/local/lib64/libssl.so.1.1 [root@nginx openssl-1.1.0g]# echo "/usr/local/lib64/" >> /etc/ld.so.conf [root@nginx openssl-1.1.0g]# ldconfig [root@nginx openssl-1.1.0g]# openssl version -a OpenSSL 1.1.0g 2 Nov 2017 built on: reproducible build, date unspecified platform: linux-x86_64 compiler: gcc -DZLIB -DDSO_DLFCN -DHAVE_DLFCN_H -DNDEBUG -DOPENSSL_THREADS -DOPENSSL_NO_STATIC_ENGINE -DOPENSSL_PIC -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DRC4_ASM -DMD5_ASM -DAES_ASM -DVPAES_ASM -DBSAES_ASM -DGHASH_ASM -DECP_NISTZ256_ASM -DPADLOCK_ASM -DPOLY1305_ASM -DOPENSSLDIR="\"/usr/local/ssl\"" -DENGINESDIR="\"/usr/local/lib64/engines-1.1\"" -Wa,--noexecstack OPENSSLDIR: "/usr/local/ssl" ENGINESDIR: "/usr/local/lib64/engines-1.1"
########## openssl升级后编译nginx出现的问题 ###########
如上将本机的openssl升级后，由于之前编译的nginx里没有stream模块，现在需要手动平滑添加stream模块，操作如下：
检查下，发现nginx没有安装stream模块 [root@external-lb01 ~]# /data/nginx/sbin/nginx -V nginx version: nginx/1.12.2 built by gcc 4.4.7 20120313 (Red Hat 4.4.7-18) (GCC) built with OpenSSL 1.1.0g 2 Nov 2017 TLS SNI support enabled configure arguments: --prefix=/data/nginx --user=www --group=www --with-http_ssl_module --with-http_flv_module --with-http_stub_status_module --with-http_gzip_static_module --with-pcre 操作之前，一定要备份一下之前的nginx安装目录，防止操作失败进行回滚！ [root@external-lb01 ~]# cp -r /data/nginx /mnt/nginx.bak 之前的编译命令是： [root@external-lb01 vhosts]# cd /data/software/nginx-1.12.2 [root@external-lb01 nginx-1.12.2]# ./configure --prefix=/data/nginx --user=www --group=www --with-http_ssl_module --with-http_flv_module --with-http_stub_status_module --with-http_gzip_static_module --with-pcre 现在需要手动添加stream，编译命令如下： [root@external-lb01 vhosts]# cd /data/software/nginx-1.12.2 [root@external-lb01 nginx-1.12.2]# ./configure --prefix=/data/nginx --user=www --group=www --with-http_ssl_module --with-http_flv_module --with-http_stub_status_module --with-http_gzip_static_module --with-pcre --with-stream 报错如下： ...... ./configure: error: SSL modules require the OpenSSL library. You can either do not enable the modules, or install the OpenSSL library into the system, or build the OpenSSL library statically from the source with nginx by using --with-openssl=<path> option. 原因分析：是由于openssl升级所致！ [root@external-lb01 nginx-1.12.2]# openssl version -a OpenSSL 1.1.0g 2 Nov 2017 built on: reproducible build, date unspecified platform: dist compiler: cc -DNDEBUG -DOPENSSL_NO_DYNAMIC_ENGINE -DOPENSSLDIR="\"/usr/local/ssl\"" -DENGINESDIR="\"/usr/local/lib/engines-1.1\"" OPENSSLDIR: "/usr/local/ssl" ENGINESDIR: "/usr/local/lib/engines-1.1 所以编译命令需要改为： [root@external-lb01 nginx-1.12.2]# ./configure --prefix=/data/nginx --user=www --group=www --with-http_ssl_module --with-http_flv_module --with-http_stub_status_module --with-http_gzip_static_module --with-pcre --with-stream --with-openssl=/usr/local/ssl 然后进行make，千万注意！！！！一定不要make install！！！否则会自动覆盖掉之前的配置！！！ [root@external-lb01 nginx-1.12.2]# make 又报错如下： ....... make[1]: *** [/usr/local/ssl/.openssl/include/openssl/ssl.h] Error 127 make[1]: Leaving directory `/usr/local/src/nginx-1.9.9' make: *** [build] Error 2 解决办法： [root@external-lb01 nginx-1.12.2]# cd auto/lib/openssl [root@external-lb01 openssl]# cp conf /mnt/ [root@external-lb01 openssl]# vim conf 将 CORE_INCS="$CORE_INCS $OPENSSL/.openssl/include" CORE_DEPS="$CORE_DEPS $OPENSSL/.openssl/include/openssl/ssl.h" CORE_LIBS="$CORE_LIBS $OPENSSL/.openssl/lib/libssl.a" CORE_LIBS="$CORE_LIBS $OPENSSL/.openssl/lib/libcrypto.a" CORE_LIBS="$CORE_LIBS $NGX_LIBDL" 修改为 CORE_INCS="$CORE_INCS $OPENSSL/include" CORE_DEPS="$CORE_DEPS $OPENSSL/include/openssl/ssl.h" CORE_LIBS="$CORE_LIBS $OPENSSL/lib/libssl.a" CORE_LIBS="$CORE_LIBS $OPENSSL/lib/libcrypto.a" CORE_LIBS="$CORE_LIBS $NGX_LIBDL" 接着继续make安装 [root@external-lb01 nginx-1.12.2]# make 又报错说找不到下面两个文件 /usr/local/ssl/lib/libssl.a /usr/local/ssl/lib/libcrypto.a 解决办法： [root@external-lb01 nginx-1.12.2]# mkdir /usr/local/ssl/lib [root@external-lb01 nginx-1.12.2]# ln -s /usr/local/lib64/libssl.a /usr/local/ssl/lib/libssl.a [root@external-lb01 nginx-1.12.2]# ln -s /usr/local/lib64/libcrypto.a /usr/local/ssl/lib/libcrypto.a 然后make就可以了 [root@external-lb01 nginx-1.12.2]# make 最后进行平滑操作 [root@external-lb01 nginx-1.12.2]# cp -f /data/software/nginx-1.12.2/objs/nginx /data/nginx/sbin/nginx [root@external-lb01 nginx-1.12.2]# pkill -9 nginx [root@external-lb01 nginx-1.12.2]# /data/nginx/sbin/nginx 检查下，发现nginx已经安装了stream模块了 [root@external-lb01 nginx-1.12.2]# /data/nginx/sbin/nginx -V nginx version: nginx/1.12.2 built by gcc 4.4.7 20120313 (Red Hat 4.4.7-18) (GCC) built with OpenSSL 1.1.0g 2 Nov 2017 TLS SNI support enabled configure arguments: --prefix=/data/nginx --user=www --group=www --with-http_ssl_module --with-http_flv_module --with-http_stub_status_module --with-http_gzip_static_module --with-pcre --with-stream --with-openssl=/usr/local/ssl
#########################################################
如上升级openssl版本后, 导致某些服务编译安装失败的坑, 如果短时间解决不来, 最好回滚到之前的默认版本:
openssl由默认的OpenSSL 1.0.1e升级到OpenSSL 1.1.1e后, 编译安装keepalived, 出现下面报错: ......... /usr/local/src/keepalived-1.3.5/keepalived/check/check_ssl.c:70: undefined reference to `OPENSSL_init_ssl' ......... 由于openssl升级后, 可能会导致一个应用编译安装失败, 遇到的有nginx, keepalived等, 不得已的办法就是将openssl回滚到之前默认的版本状态, 操作方法如下: 查看openssl, 然后删除升级后的openssl [root@localhost ~]# find / -name openssl [root@localhost ~]# rm -rf /usr/local/src/openssl-1.1.1 [root@localhost ~]# rm -rf /usr/local/bin/openssl [root@localhost ~]# rm -rf /usr/local/share/doc/openssl [root@localhost ~]# rm -rf /usr/local/include/openssl 然后查看下openssl版本 [root@localhost ~]# which openssl /usr/bin/openssl [root@localhost ~]# openssl version -a 报错说/usr/local/bin/openssl 找不到这个文件 然后重启机器 [root@localhost ~]# init 6 重启机器后, 查看openssl版本, 如果正常查出是默认版本, 则回滚正常 [root@localhost ~]# openssl version -a 如果还是报错"/usr/local/bin/openssl 找不到这个文件", 则需要卸载掉openssl, 重新安装! 特别注意: 卸载openssl之前, 要确保安装了rz, sz命令(yum install -y lrzsz), 方便后续从别的机器上传文件 [root@localhost ~]# rpm -qa|grep openssl [root@localhost ~]# rpm -e openssl-devel-1.0.1e-57.el6.x86_64 --nodeps [root@localhost ~]# rpm -e openssl-1.0.1e-57.el6.x86_64 --nodeps openssl卸载后, 使用yum安装会报错 [root@localhost ~]# yum install -y openssl openssl-devel 报错: libssl.so.10: cannot open shared object file: No such file or directory libcrypto.so.10: cannot open shared object file: No such file or directory 然后从别的正常机器(默认openssl版本的机器)上拷贝上面两个文件(先sz到本地, 然后rz上传到本机) 即从别的机器下载libssl.so.1.0.1e 和 libcrypto.so.1.0.1e 文件到本机的/usr/lib64下, 授权777, 并做ln软链接 [root@localhost ~]# cd /usr/lib64/ [root@localhost lib64]# ll libssl.so.10 lrwxrwxrwx 1 root root 16 Dec 20 17:16 libssl.so.10 -> libssl.so.1.0.1e [root@localhost lib64]# ll libssl.so.1.0.1e -rwxr-xr-x 1 root root 443416 Mar 23 2017 libssl.so.1.0.1e [root@localhost lib64]# ll libcrypto.so.10 lrwxrwxrwx 1 root root 19 Dec 20 17:16 libcrypto.so.10 -> libcrypto.so.1.0.1e [root@localhost lib64]# ll libcrypto.so.1.0.1e -rwxr-xr-x 1 root root 1971488 Mar 23 2017 libcrypto.so.1.0.1e [root@localhost lib64]# cat /etc/ld.so.conf include ld.so.conf.d/*.conf /usr/lib64/ [root@localhost lib64]# ldconfig 然后重启服务器 [root@localhost lib64]# init 6 [root@localhost lib64]# openssl version -a OpenSSL 1.0.1e-fips 11 Feb 2013 built on: Wed Mar 22 21:43:28 UTC 2017 platform: linux-x86_64 options: bn(64,64) md2(int) rc4(16x,int) des(idx,cisc,16,int) idea(int) blowfish(idx) compiler: gcc -fPIC -DOPENSSL_PIC -DZLIB -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -DKRB5_MIT -m64 -DL_ENDIAN -DTERMIO -Wall -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -Wa,--noexecstack -DPURIFY -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DMD5_ASM -DAES_ASM -DVPAES_ASM -DBSAES_ASM -DWHIRLPOOL_ASM -DGHASH_ASM OPENSSLDIR: "/etc/pki/tls" engines: rdrand dynamic
################################################################
curl不支持https的处理方法（"Protocol https not supported or disabled in libcurl"）
在使用curl访问kubernetes-apiservers时报错： [root@bkevin ~]# curl -s --cacert ca.pem --cert admin.pem --key admin-key.pem https://192.168.81.172:6443/metrics Protocol https not supported or disabled in libcurl 发现是因为当前系统的curl命令不支持https协议，验证如下： [root@bkevin ~]# curl -V curl 7.64.1 (x86_64-pc-linux-gnu) libcurl/7.64.1 OpenSSL/1.0.2k zlib/1.2.8 Release-Date: 2019-03-27 Protocols: dict file ftp ftps gopher http imap imaps pop3 telnet tftp Features: AsynchDNS HTTPS-proxy IPv6 Largefile libz NTLM NTLM_WB SSL TLS-SRP UnixSockets 通过上面可以看到当前curl并不支持https协议。若用curl命令访问https时就会报错： Protocol https not supported or disabled in libcurl 下面是针对以上问题的处理办法： 若需要让curl支持https协议，需要安装openssl并在curl中使之生效！！！！ [root@bkevin ~]# openssl version -a OpenSSL 1.0.1e-fips 11 Feb 2013 built on: Wed Mar 22 21:43:28 UTC 2017 platform: linux-x86_64 options: bn(64,64) md2(int) rc4(16x,int) des(idx,cisc,16,int) idea(int) blowfish(idx) compiler: gcc -fPIC -DOPENSSL_PIC -DZLIB -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -DKRB5_MIT -m64 -DL_ENDIAN -DTERMIO -Wall -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -Wa,--noexecstack -DPURIFY -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DMD5_ASM -DAES_ASM -DVPAES_ASM -DBSAES_ASM -DWHIRLPOOL_ASM -DGHASH_ASM OPENSSLDIR: "/etc/pki/tls" engines: rdrand dynamic 下载并安装openssl包（发现当前系统openssl是默认安装的，不用管，直接如下编译安装，然后编译安装curl）： # wget https://www.openssl.org/source/openssl-1.0.2k.tar.gz # wget https://www.openssl.org/source/openssl-fips-2.0.14.tar.gz 安装openssl-fips： # tar xvf openssl-fips-2.0.14.tar.gz # cd openssl-fips-2.0.14&&./config&&make&&make install 安装openssl： # tar xvf openssl-1.0.2k.tar.gz # ./config shared --prefix=/usr/local/ssl&& make && make install 更新ld # echo "/usr/local/ssl/lib" >> /etc/ld.so.conf # ldconfig -v 配置openssl库（如果提示已经有了该路径，就mv或unlink去掉之前的软连接关系） # cp /usr/local/ssl/lib/libssl.so.1.0.0 /usr/lib64 # cp/usr/local/ssl/lib/libcrypto.so.1.0.0 /usr/lib64 # chmod 555 /usr/lib64/libssl.so.1.0.0 # chmod 555/usr/lib64/libcrypto.so.1.0.0 # mv /usr/lib64/libcrypto.so.10 /usr/lib64/libcrypto.so.10_bak #这个文件最好先不要动，可能会导致后续ssh启动失败！故次文件最好先不动！ # mv /usr/lib64/libssl.so.10 /usr/lib64/libssl.so.10_bak # mv /usr/bin/openssl /usr/bin/openssl_bak # mv /usr/include/openssl /usr/include/openssl_bak # ln -s /usr/lib64/libcrypto.so.1.0.0 /usr/lib64/libcrypto.so.10 #这个文件最好先不要动，可能会导致后续ssh启动失败！故这个文件先不动！ # ln -s /usr/lib64/libssl.so.1.0.0 /usr/lib64/libssl.so.10 # ln -s /usr/local/ssl/bin/openssl /usr/bin/openssl # ln -s/usr/local/ssl/include/openssl /usr/include/openssl 查看openssl版本 (发现openssl版本已经更新了) [root@bkevin ~]# openssl version -a OpenSSL 1.0.2k 26 Jan 2017 built on: reproducible build, date unspecified platform: linux-x86_64 options: bn(64,64) rc4(16x,int) des(idx,cisc,16,int) idea(int) blowfish(idx) compiler: gcc -I. -I.. -I../include -fPIC -DOPENSSL_PIC -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -Wa,--noexecstack -m64 -DL_ENDIAN -O3 -Wall -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DRC4_ASM -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DMD5_ASM -DAES_ASM -DVPAES_ASM -DBSAES_ASM -DWHIRLPOOL_ASM -DGHASH_ASM -DECP_NISTZ256_ASM OPENSSLDIR: "/usr/local/ssl/ssl" 重新编译curl（找到之前编译安装的curl路径，或者直接重新下载curl二进制包进行编译安装）。注意编译安装curl时一定要添加"--with-ssl"，带上openssl参数进行编译安装！ # cd /usr/local/src/curl-7.64.1 # ./configure --with-ssl=/usr/local/ssl # make # make install 查看curl是否已经支持https协议： [root@bkevin ~]# curl -V curl 7.64.1 (x86_64-pc-linux-gnu) libcurl/7.64.1 OpenSSL/1.0.2k zlib/1.2.8 Release-Date: 2019-03-27 Protocols: dict file ftp ftps gopher http https imap imaps pop3 pop3s rtsp smb smbs smtp smtps telnet tftp Features: AsynchDNS HTTPS-proxy IPv6 Largefile libz NTLM NTLM_WB SSL TLS-SRP UnixSockets 由上面可知，curl现在已经可以支持https协议了，故可以使用curl访问https了 [root@bkevin ~]# curl -s --cacert ca.pem --cert admin.pem --key admin-key.pem https://192.168.81.172:6443/metrics |head -5 # HELP APIServiceOpenAPIAggregationControllerQueue1_adds Total number of adds handled by workqueue: APIServiceOpenAPIAggregationControllerQueue1 # TYPE APIServiceOpenAPIAggregationControllerQueue1_adds counter APIServiceOpenAPIAggregationControllerQueue1_adds 18601 # HELP APIServiceOpenAPIAggregationControllerQueue1_depth Current depth of workqueue: APIServiceOpenAPIAggregationControllerQueue1 # TYPE APIServiceOpenAPIAggregationControllerQueue1_depth gauge ######################################################################################################################## ######################################################################################################################## 需要特别注意： 上面在更新openssl版本后，最好不要退出当前终端！！！ 因为如上OpenSSL版本更新后，可能导致ssh远程连接失败问题！！！ [root@bkevin ~]# ssh -p22 root@192.168.36.12 ssh_exchange_identification: read: Connection reset by peer [root@bkevin ~]# ssh -v -p22 root@192.168.36.12 OpenSSH_7.4p1, OpenSSL 1.0.2k-fips 26 Jan 2017 debug1: Reading configuration data /etc/ssh/ssh_config debug1: /etc/ssh/ssh_config line 58: Applying options for * debug1: Connecting to 192.168.36.12 [172.20.36.12] port 22. debug1: Connection established. debug1: permanently_set_uid: 0/0 debug1: identity file /root/.ssh/id_rsa type 1 debug1: key_load_public: No such file or directory debug1: identity file /root/.ssh/id_rsa-cert type -1 debug1: key_load_public: No such file or directory debug1: identity file /root/.ssh/id_dsa type -1 debug1: key_load_public: No such file or directory debug1: identity file /root/.ssh/id_dsa-cert type -1 debug1: key_load_public: No such file or directory debug1: identity file /root/.ssh/id_ecdsa type -1 debug1: key_load_public: No such file or directory debug1: identity file /root/.ssh/id_ecdsa-cert type -1 debug1: key_load_public: No such file or directory debug1: identity file /root/.ssh/id_ed25519 type -1 debug1: key_load_public: No such file or directory debug1: identity file /root/.ssh/id_ed25519-cert type -1 debug1: Enabling compatibility mode for protocol 2.0 debug1: Local version string SSH-2.0-OpenSSH_7.4 ssh_exchange_identification: read: Connection reset by peer 解决办法：在ssh在服务端更改配置文件修改： [root@bkevin ~]# vi /etc/hosts.allow ######################### sshd: ALL ##允许所有ip主机均能连接本机 [root@bkevin ~]# /etc/init.d/sshd restart Stopping sshd: [FAILED] Starting sshd: /usr/sbin/sshd: /usr/lib64/libcrypto.so.10: no version information available (required by /usr/sbin/sshd) /usr/sbin/sshd: /usr/lib64/libcrypto.so.10: no version information available (required by /usr/sbin/sshd) /usr/sbin/sshd: /usr/lib64/libcrypto.so.10: no version information available (required by /usr/sbin/sshd) OpenSSL version mismatch. Built against 1000105f, you have 100020bf 解决办法： [root@bkevin ~]# unlink /usr/lib64/libcrypto.so.10 [root@bkevin ~]# cp /usr/lib64/libcrypto.so.10_bak /usr/lib64/libcrypto.so.10 [root@bkevin ~]# /etc/init.d/sshd restart Stopping sshd: [FAILED] Starting sshd: [ OK ] [root@bz4paas02zk1001 ~]# /etc/init.d/sshd restart Stopping sshd: [ OK ] Starting sshd: [ OK ] 再次尝试远程ssh连接，就正常了！ [root@bkevin ~]# ssh -p22 root@192.168.36.12 Authorized only. All activity will be monitored and reported Last login: Mon Mar 2 09:06:17 2020 from 172.20.20.65 [root@02zk1001 ~]#