linux集群之LVS入门和企业级实战（续二）

2022-03-27 20:47:36

三、LVS 持续性连接

LVS的持久连接：

持久连接即是不考虑LVS的转发方法，确保所有来自同一个用户的连接转发到同一个RealServer上。

lvs持久连接适用于大部分调度算法。当某一种请求需要定向到一个real server 时，就要用到持久连接
一般应用到：ssl（http.https等）、ftp。
-p //表示此连接为持久连接
N //表示维持此持久连接的时间。默认6分钟。当超过这个时间后，如果网页还没有关掉，仍处于激活状态，重新复位时间为2分钟。

持久连接的类型：

1.PCC(persistent client connector,持久用户连接)同一个用户所有的请求在超时范围之内都被定位到同一个RealServer上，这个时候在指定端口的时候使用的是0端口，就是所有的请求都转发出去。

[root@localhost ~]# ipvsadm -A -t 10.40.0.51:0 -s rr -p

[root@localhost ~]# ipvsadm -a -t 10.40.0.51:0 -r 192.168.1.11 -g

[root@localhost ~]# ipvsadm -a -t 10.40.0.51:0 -r 192.168.1.12 -g

[root@localhost ~]# ipvsadm -ln

IP Virtual Server version 1.2.1 (size=4096)

Prot LocalAddress:Port Scheduler Flags

-> RemoteAddress:Port Forward Weight ActiveConn InActConn

TCP 10.40.0.51:0 rr persistent 360

-> 192.168.1.11:0 Route 1 0 0

-> 192.168.1.12:0 Route 1 0 0

2.PPC(persistent port connector)用户的所有请求在超时范围内按照端口定位到不同的RS上。，只对一个服务进行持久链接。

[root@localhost ~]# ipvsadm -A -t 10.40.0.51:22 -s rr -p

[root@localhost ~]# ipvsadm -a -t 10.40.0.51:22 -r 192.168.1.11:22 -g

[root@localhost ~]# ipvsadm -a -t 10.40.0.51:22 -r 192.168.1.12:22 -g

[root@localhost ~]# ipvsadm -A -t 10.40.0.51:80 -s rr -p

[root@localhost ~]# ipvsadm -a -t 10.40.0.51:80 -r 192.168.1.12:80 -g

[root@localhost ~]# ipvsadm -a -t 10.40.0.51:80 -r 192.168.1.11:80 -g

[root@localhost ~]# ipvsadm -ln

IP Virtual Server version 1.2.1 (size=4096)

Prot LocalAddress:Port Scheduler Flags

-> RemoteAddress:Port Forward Weight ActiveConn InActConn

TCP 10.40.0.51:22 rr persistent 360

-> 192.168.1.11:22 Route 1 0 0

-> 192.168.1.12:22 Route 1 0 0

TCP 10.40.0.51:80 rr persistent 360

-> 192.168.1.11:80 Route 1 0 0

-> 192.168.1.12:80 Route 1 0 0

3.防火墙标记：把相关联的端口在防火墙上打上同样的标记，用户在访问两个相关联的服务的时候，就会定位到同一个RealServer上。
4.FTP connection：由于ftp使用的是两个端口号，所以需要单独列出来。

参考文献：

http://www.360doc.com/content/17/0314/14/29704676_636776330.shtml

http://blog.itpub.net/124805/viewspace-1047686/

https证书及CA创建过程

假设ca放在192.168.1.15上，httpd服务器是192.168.1.11、192.168.1.12

1、创建CA,并生成自签证书PEM（windwos系统的格式通常是DER）

[root@ 192.168.1.15 CA]# (umask 077; openssl genrsa 2048 >private/cakey.pem)

Generating RSA private key, 2048 bit long modulus

........+++

........................................+++

e is 65537 (0x10001)

[root@ 192.168.1.15 CA]# openssl req -new -x509 -in private/cakey.pem -out cacert1.pem -days 3650

Generating a 2048 bit RSA private key

........................................+++

....................................................+++

writing new private key to 'privkey.pem'

Enter PEM pass phrase:

140021996894024:error:28069065:lib(40):UI_set_result:result too small:ui_lib.c:869:You must type in 4 to 1024 characters

140021996894024:error:0906406D:PEM routines:PEM_def_callback:problems getting password:pem_lib.c:111:

140021996894024:error:0907E06F:PEM routines:DO_PK8PKEY:read key:pem_pk8.c:130:

[root@localhost CA]# openssl req -new -x509 -key private/cakey.pem -out cacert1.pem -days 3650

You are about to be asked to enter information that will be incorporated

into your certificate request.

What you are about to enter is what is called a Distinguished Name or a DN.

There are quite a few fields but you can leave some blank

For some fields there will be a default value,

If you enter '.', the field will be left blank.

-----

Country Name (2 letter code) [XX]:CN

State or Province Name (full name) []:HENAN

Locality Name (eg, city) [Default City]:ZZ

Organization Name (eg, company) [Default Company Ltd]:MAGEEDU

Organizational Unit Name (eg, section) []:TECH

Common Name (eg, your name or your server's hostname) []:ca.magedu.com

Email Address []:ca@magedu.com

[root@ 192.168.1.15 CA]# touch index.txt

[root@ 192.168.1.15 CA]# echo “01”>serial

2、创建服务器的证书签发请求CSR

[root@192.168.1.12 ~]# cd /etc/httpd/conf/&& mkdir ssl&& cd ssl &&(umask 077;openssl genrsa 1024 >httpd.key)

Generating RSA private key, 1024 bit long modulus

................++++++

......................................++++++

e is 65537 (0x10001)

[root@192.168.1.12 /etc/httpd/conf/ssl]# openssl req -new -key httpd.key -out httpd.csr

You are about to be asked to enter information that will be incorporated

into your certificate request.

What you are about to enter is what is called a Distinguished Name or a DN.

There are quite a few fields but you can leave some blank

For some fields there will be a default value,

If you enter '.', the field will be left blank.

-----

Country Name (2 letter code) [XX]:CN

State or Province Name (full name) []:HENAN

Locality Name (eg, city) [Default City]:ZZ

Organization Name (eg, company) [Default Company Ltd]:MAGEEDU

Organizational Unit Name (eg, section) []:TECH

Common Name (eg, your name or your server's hostname) []:test.magedu.com

Email Address []:

Please enter the following 'extra' attributes

to be sent with your certificate request

A challenge password []:

An optional company name []:

[root@192.168.1.12/etc/httpd/conf/ssl]$ls

httpd.csr httpd.key

3、CA颁发给服务器证书

[root@ 192.168.1.15 CA]# scp -r 192.168.1.12:/etc/httpd/conf/ssl/httpd.csr /root/

root@192.168.1.12's password:

httpd.csr 100% 647 0.6KB/s 00:00

[root@ 192.168.1.15 CA]# openssl ca -in /root/httpd.csr -out /root/httpd.crt -days 265

Using configuration from /etc/pki/tls/openssl.cnf

unable to load number from /etc/pki/CA/serial

error while loading serial number

140408525461320:error:0D066096:asn1 encoding routines:a2i_ASN1_INTEGER:short line:f_int.c:215:

[root@ 192.168.1.15 CA]# ls

cacert1.pem certs httpd.csr newcerts privkey.pem

cacert.pem crl index.txt private serial

[root@ 192.168.1.15 CA]# echo "01">serial

[root@192.168.1.15 CA]# cat serial

[root@localhost CA]# openssl ca -in /root/httpd.csr -out /root/httpd.crt -days 265

Using configuration from /etc/pki/tls/openssl.cnf

Check that the request matches the signature

Signature ok

Certificate Details:

Serial Number: 1 (0x1)

Validity

Not Before: Mar 14 12:42:02 2017 GMT

Not After : Dec 4 12:42:02 2017 GMT

Subject:

countryName = CN

stateOrProvinceName = HENAN

organizationName = MAGEEDU

organizationalUnitName = TECH

commonName = test.magedu.com

X509v3 extensions:

X509v3 Basic Constraints:

CA:FALSE

Netscape Comment:

OpenSSL Generated Certificate

X509v3 Subject Key Identifier:

A8:9F:C2:4C:1D:64:A1:A3:3E:07:C5:2D:81:73:7E:4B:55:F8:CD:A0

X509v3 Authority Key Identifier:

keyid:A3:7A:FD:8A:E0:F0:FF:E0:5F:F1:BE:F8:11:BA:7A:BD:53:8E:58:38

Certificate is to be certified until Dec 4 12:42:02 2017 GMT (265 days)

Sign the certificate? [y/n]:y

1 out of 1 certificate requests certified, commit? [y/n]y

Write out database with 1 new entries

Data Base Updated

4、http的服务器安装配置证书、安装mod_ssl模块。

[root@192.168.1.12 ~]# scp 192.168.1.15:/root/httpd.crt /etc/httpd/conf/ssl/

root@192.168.1.15s password:

httpd.crt 100% 3066 3.0KB/s 00:00

[root@192.168.1.12 /etc/httpd/conf/ssl]# yum install mod_ssl

5、修改/etc/httpd/conf.d/ssl.conf配置文件。

# Server Certificate:

# Point SSLCertificateFile at a PEM encoded certificate. If

# the certificate is encrypted, then you will be prompted for a

# pass phrase. Note that a kill -HUP will prompt again. A new

# certificate can be generated using the genkey(1) command.

#SSLCertificateFile /etc/pki/tls/certs/localhost.crt

SSLCertificateFile /etc/httpd/conf/ssl/httpd.crt

# Server Private Key:

# If the key is not combined with the certificate, use this

# directive to point at the key file. Keep in mind that if

# you've both a RSA and a DSA private key you can configure

# both in parallel (to also allow the use of DSA ciphers, etc.)

#SSLCertificateKeyFile /etc/pki/tls/private/localhost.key

SSLCertificateKeyFile /etc/httpd/conf/ssl/httpd.key

# General setup for the virtual host, inherited from global configuration

DocumentRoot "/var/www/html"

#ServerName www.example.com:443

6、对于多台实现负载均衡功能的httpd服务器，可以直接将ssl.conf和证书复制到其他httpd服务器上即可。

[root@192.168.1.12 /etc/httpd/conf/ssl]# scp -rp /etc/httpd/conf/ssl 192.168.1.11:/etc/httpd/conf/

The authenticity of host '192.168.1.11 (192.168.1.11)' can't be established.

RSA key fingerprint is ed:f5:8b:3a:70:4e:d6:7d:16:59:aa:52:14:34:5d:1f.

Are you sure you want to continue connecting (yes/no)? yes

Warning: Permanently added '192.168.1.11' (RSA) to the list of known hosts.

root@192.168.1.11's password:

httpd.crt 100% 3066 3.0KB/s 00:00

httpd.csr 100% 647 0.6KB/s 00:00

httpd.key 100% 891 0.9KB/s 00:00

[root@192.168.1.12 /etc/httpd/conf/ssl]# scp -rp /etc/httpd/conf.d/ssl.conf 192.168.1.11:/etc/httpd/conf.d/

root@192.168.1.11's password:

ssl.conf 100% 9568 9.3KB/s 00:00

[root@192.168.1.12 /etc/httpd/conf/ssl]#

[root@192.168.1.12 /etc/httpd/conf/ssl]# netstat -tunlp |grep [80,443]

tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 1344/sshd

tcp 0 0 :::80 :::* LISTEN 3300/httpd

tcp 0 0 :::22 :::* LISTEN 1344/sshd

tcp 0 0 :::443 :::* LISTEN 3300/httpd

参考文献：

证书颁发及文件名后缀介绍

常见ipvs的持久链接，方式为防火墙标记：

[root@localhost ~]# iptables -t mangle -A PREROUTING -d 10.40.0.51 -p tcp --dport 80 -j MARK --set-mark 9

[root@localhost ~]# iptables -t mangle -A PREROUTING -d 10.40.0.51 -p tcp --dport 443 -j MARK --set-mark 9

[root@localhost ~]# iptables -L -t mangle

Chain PREROUTING (policy ACCEPT)