基于redis 3.x搭建集群环境

由于我团队开发的在线坐席系统,即将面对线上每周3000W的下行投放客户,产品的咨询量可能会很大,基于前期,200W的投放时,前10分钟,大概800问题量,平均一个客户大概8个问题,也就是说每分钟10个客户,折算3000W的话,就是每分钟150客户。系统容量应该问题不是太大,考虑一下极端情况(叠加效应),或者留下富余5倍,也就不到800吧,我们的生产环境,需要扩容消费者服务器,redis作为调度,需要扩容,由现在的一主从升级到一个集群(3对主从的cluster)。

这里大概记录下我们开发环境的redis集群的配置和安装过程,这些属于运维的工作,但是也是系统架构的范畴,这些若没有实战的经验,不要和我说什么架构设计。。。我就是搞这个的,我要对我的系统负责,但是又不会出现系统的过度设计。

有三台机器,准备搭建redis集群环境,redis的版本是3.2.8,源码安装,安装过程略去。

10.90.7.2
10.90.7.10
10.90.2.102

每个上面部署两个redis的实例,端口配置信息如下:

10.90.7.2 7000/7010
10.90.7.10 7001/7011
10.90.2.102 7002/7012

三台的redis的配置采用近乎一样的配置,除了因为端口的不同造成的一点点差异。这里拿10.90.7.2的配置7000为例:

在/opt/redis-3.2.8下面创建目录tkcluster。将/opt/redis-3.2.8下面的redis.conf文件copy到tkcluster目录下,重命名为7000.conf,修改的内容如下,其他的采用默认值。

bind 10.90.7.2
protected-mode no
port
daemonize yes
pidfile /var/run/redis_7000.pid
appendonly yes
appendfilename "appendonly7000.aof"
cluster-enabled yes
cluster-config-file nodes-.conf
cluster-node-timeout
notify-keyspace-events Ex

配置完后,copy一下7000.conf到7010.conf,将上述相应的7000改成7010的参数即可完成7010.conf的配置。为了方便启动redis-server。写一个简单shell脚本:

#!/bin/bash

/opt/redis-3.2./src/redis-server .conf
echo "status from 7000: $?"
sleep
/opt/redis-3.2./src/redis-server .conf
echo "status from 7010: $?"

运行一下后,成功。得到文件列表:

[root@localhost tkcluster]# ll
总计
-rw-r--r-- root root - : .conf
-rw-r--r-- root root - : .conf
-rw-r--r-- root root - : appendonly7000.aof
-rw-r--r-- root root - : appendonly7010.aof
-rw-r--r-- root root - : dump.rdb
-rw-r--r-- root root - : nodes-.conf
-rw-r--r-- root root - : nodes-.conf
-rwxr-xr-x root root - : stcluster.sh

检查下,集群创建的指令是否能工作:

[root@bogon src]# ./redis-trib.rb --help
/usr/bin/env: ruby: 没有那个文件或目录

系统里面没有ruby这个环境,需要安装一下。

这个过程有点让人头疼,开始,在10.90.7.10的机器上进行安装,通过yum,下面是我遇到的问题

[root@bogon src]# yum install -y ruby

源找到了,很快安装成功。继续尝试:

[root@bogon src]# ./redis-trib.rb --help
/usr/share/rubygems/rubygems/core_ext/kernel_require.rb::in `require': cannot load such file -- redis (LoadError)
from /usr/share/rubygems/rubygems/core_ext/kernel_require.rb::in `require'
from ./redis-trib.rb::in `<main>'

这个错误,呵呵,熟悉ruby的话,很容易看出来,是ruby的环境需要访问redis,但是系统里面没有ruby的redis客户端程序,需要安装一下ruby的redis客户端插件。

[root@bogon src]# gem install redis
Fetching: redis-4.0..gem (%)
ERROR: Error installing redis:
redis requires Ruby version >= 2.2..

这个错误,是因为ruby版本太低,需要升级,采用下面的方式可以完成:

[root@bogon src]# curl -L get.rvm.io | bash -s stable
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
:: --:--:-- ::
:: :: --:--:--
Downloading https://github.com/rvm/rvm/archive/1.29.3.tar.gz
Downloading https://github.com/rvm/rvm/releases/download/1.29.3/1.29.3.tar.gz.asc
curl: () The requested URL returned error: Could not download 'https://github.com/rvm/rvm/releases/download/1.29.3/1.29.3.tar.gz.asc'.
curl returned status ''. Creating group 'rvm' Installing RVM to /usr/local/rvm/
Installation of RVM in /usr/local/rvm/ is almost complete: * First you need to add all users that will be using rvm to 'rvm' group,
and logout - login again, anyone using rvm will be operating with `umask u=rwx,g=rwx,o=rx`. * To start using RVM you need to run `source /etc/profile.d/rvm.sh`
in all your open shell windows, in rare cases you need to reopen all shell windows.

执行完成后,按照上面的提示,首先推出ssh,然后再登陆ssh。然后执行source /etc/profile.d/rvm.sh指令。接下来执行一下rvm list known指令,看看当前查看rvm库中已知的ruby版本

[root@bogon redis-3.2.]# rvm list known
# MRI Rubies
[ruby-]1.8.[-p420]
[ruby-]1.8.[-head] # security released on head
[ruby-]1.9.[-p431]
[ruby-]1.9.[-p330]
[ruby-]1.9.[-p551]
[ruby-]2.0.[-p648]
[ruby-]2.1[.]
[ruby-]2.2[.]
[ruby-]2.3[.]
[ruby-]2.4[.]
ruby-head # for forks use: rvm install ruby-head-<name> --url https://github.com/github/ruby.git --branch 2.2 # JRuby
jruby-1.6[.]
jruby-1.7[.]
jruby[-9.1.13.0]
jruby-head # Rubinius
rbx-[.4.3]
rbx-2.3[.]
rbx-2.4[.]
rbx-[.5.8]
rbx-[.]
rbx-head # Opal
opal # Minimalistic ruby implementation - ISO :
mruby-1.0.
mruby-1.1.
mruby-1.2.
mruby-[.3.0]
mruby[-head] # Ruby Enterprise Edition
ree-1.8.
ree[-1.8.][-2012.02] # Topaz
topaz # MagLev
maglev[-head]
maglev-1.0. # Mac OS X Snow Leopard Or Newer
macruby-0.10
macruby-0.11
macruby[-0.12]
macruby-nightly
macruby-head # IronRuby
ironruby[-1.1.]
ironruby-head
[root@bogon redis-3.2.]#

查看后,看到前面有ruby的版本比较多,高于2.0.0的有4个,选一个即可。这里就选择2.3.4吧。

[root@bogon redis-3.2.]# rvm install 2.3.
Searching for binary rubies, this might take some time.
Found remote file https://rvm_io.global.ssl.fastly.net/binaries/centos/7/x86_64/ruby-2.3.4.tar.bz2
Checking requirements for centos.
Installing requirements for centos.
Installing required packages: bison, bzip2, libyaml-devel......
Requirements installation successful.
ruby-2.3. - #configure
ruby-2.3. - #download
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
25.2M 25.2M 149k :: :: --:--:-- 165k
No checksum for downloaded archive, recording checksum in user configuration.
ruby-2.3. - #validate archive
ruby-2.3. - #extract
ruby-2.3. - #validate binary
ruby-2.3. - #setup
ruby-2.3. - #gemset created /usr/local/rvm/gems/ruby-2.3.@global
ruby-2.3. - #importing gemset /usr/local/rvm/gemsets/global.gems..............................
ruby-2.3. - #generating global wrappers........
Error running 'run_gem_wrappers_regenerate',
please read /usr/local/rvm/log/1511253510_ruby-2.3./gemset.wrappers.global.log
ruby-2.3. - #gemset created /usr/local/rvm/gems/ruby-2.3.
ruby-2.3. - #importing gemsetfile /usr/local/rvm/gemsets/default.gems evaluated to empty gem list
ruby-2.3. - #generating default wrappers.........
Error running 'run_gem_wrappers_regenerate',
please read /usr/local/rvm/log/1511253510_ruby-2.3./gemset.wrappers.default.log

在这个地方,我的10.90.7.10的机器总是报错。。。

执行启用新的ruby版本:

[root@bogon redis-3.2.]# rvm use 2.3.
Using /usr/local/rvm/gems/ruby-2.3.
Error loading RubyGems plugin "/usr/local/rvm/gems/ruby-2.3.4@global/gems/executable-hooks-1.3.2/lib/rubygems_plugin.rb": /lib64/libcrypto.so.: version `OPENSSL_1.0.2' not found (required by /usr/local/rvm/rubies/ruby-2.3.4/lib/ruby/2.3.0/x86_64-linux/openssl.so) - /usr/local/rvm/rubies/ruby-2.3.4/lib/ruby/2.3.0/x86_64-linux/openssl.so (LoadError)
Error loading RubyGems plugin "/usr/local/rvm/gems/ruby-2.3.4@global/gems/gem-wrappers-1.3.2/lib/rubygems_plugin.rb": /lib64/libcrypto.so.: version `OPENSSL_1.0.2' not found (required by /usr/local/rvm/rubies/ruby-2.3.4/lib/ruby/2.3.0/x86_64-linux/openssl.so) - /usr/local/rvm/rubies/ruby-2.3.4/lib/ruby/2.3.0/x86_64-linux/openssl.so (LoadError)

这个错误是找openssl 1.0.2的依赖,需要安装。我以源码来安装这个吧。先从执行:

[root@bogon openssl-1.0.2l]# wget  wget https://www.openssl.org/source/openssl-1.0.2.tar.gz

下载完毕后,解压安装:

[root@bogon openssl-1.0.]#  ./config -fPIC --prefix=/usr/local/openssl/ enable-shared
Operating system: x86_64-whatever-linux2
Configuring for linux-x86_64
Configuring for linux-x86_64
no-ec_nistp_64_gcc_128 [default] OPENSSL_NO_EC_NISTP_64_GCC_128 (skip dir)
no-gmp [default] OPENSSL_NO_GMP (skip dir)
no-jpake [experimental] OPENSSL_NO_JPAKE (skip dir)
no-krb5 [krb5-flavor not specified] OPENSSL_NO_KRB5
no-libunbound [experimental] OPENSSL_NO_LIBUNBOUND (skip dir)
no-md2 [default] OPENSSL_NO_MD2 (skip dir)
no-rc5 [default] OPENSSL_NO_RC5 (skip dir)
no-rfc3779 [default] OPENSSL_NO_RFC3779 (skip dir)
no-sctp [default] OPENSSL_NO_SCTP (skip dir)
no-ssl-trace [default] OPENSSL_NO_SSL_TRACE (skip dir)
no-store [experimental] OPENSSL_NO_STORE (skip dir)
no-unit-test [default] OPENSSL_NO_UNIT_TEST (skip dir)
no-zlib [default]
no-zlib-dynamic [default]
IsMK1MF=
CC =gcc
CFLAG =-fPIC -DOPENSSL_PIC -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -fPIC -Wa,--noexecstack -m64 -DL_ENDIAN -DTERMIO -O3 -Wall -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DMD5_ASM -DAES_ASM -DVPAES_ASM -DBSAES_ASM -DWHIRLPOOL_ASM -DGHASH_ASM -DECP_NISTZ256_ASM
EX_LIBS =-ldl
CPUID_OBJ =x86_64cpuid.o
BN_ASM =x86_64-gcc.o x86_64-mont.o x86_64-mont5.o x86_64-gf2m.o rsaz_exp.o rsaz-x86_64.o rsaz-avx2.o
EC_ASM =ecp_nistz256.o ecp_nistz256-x86_64.o
DES_ENC =des_enc.o fcrypt_b.o
AES_ENC =aes-x86_64.o vpaes-x86_64.o bsaes-x86_64.o aesni-x86_64.o aesni-sha1-x86_64.o aesni-sha256-x86_64.o aesni-mb-x86_64.o
BF_ENC =bf_enc.o
CAST_ENC =c_enc.o
RC4_ENC =rc4-x86_64.o rc4-md5-x86_64.o
RC5_ENC =rc5_enc.o
MD5_OBJ_ASM =md5-x86_64.o
SHA1_OBJ_ASM =sha1-x86_64.o sha256-x86_64.o sha512-x86_64.o sha1-mb-x86_64.o sha256-mb-x86_64.o
RMD160_OBJ_ASM=
CMLL_ENC =cmll-x86_64.o cmll_misc.o
MODES_OBJ =ghash-x86_64.o aesni-gcm-x86_64.o
ENGINES_OBJ =
PROCESSOR =
RANLIB =/usr/bin/ranlib
ARFLAGS =
PERL =/usr/bin/perl
SIXTY_FOUR_BIT_LONG mode
DES_UNROLL used
DES_INT used
RC4_CHUNK is unsigned long
e_os2.h => include/openssl/e_os2.h
making links in crypto...
make[]: 进入目录“/opt/redis/openssl-1.0./crypto”
crypto.h => ../include/openssl/crypto.h
opensslv.h => ../include/openssl/opensslv.h
opensslconf.h => ../include/openssl/opensslconf.h
ebcdic.h => ../include/openssl/ebcdic.h
symhacks.h => ../include/openssl/symhacks.h
ossl_typ.h => ../include/openssl/ossl_typ.h
constant_time_test.c => ../test/constant_time_test.c
making links in crypto/objects...
make[]: 进入目录“/opt/redis/openssl-1.0./crypto/objects”
objects.h => ../../include/openssl/objects.h
obj_mac.h => ../../include/openssl/obj_mac.h
make[]: 离开目录“/opt/redis/openssl-1.0./crypto/objects”
making links in crypto/md4...
make[]: 进入目录“/opt/redis/openssl-1.0./crypto/md4”
md4.h => ../../include/openssl/md4.h
md4test.c => ../../test/md4test.c
md4.c => ../../apps/md4.c
make[]: 离开目录“/opt/redis/openssl-1.0./crypto/md4”
making links in crypto/md5...
make[]: 进入目录“/opt/redis/openssl-1.0./crypto/md5”
md5.h => ../../include/openssl/md5.h
md5test.c => ../../test/md5test.c
make[]: 离开目录“/opt/redis/openssl-1.0./crypto/md5”
making links in crypto/sha...
make[]: 进入目录“/opt/redis/openssl-1.0./crypto/sha”
sha.h => ../../include/openssl/sha.h
shatest.c => ../../test/shatest.c
sha1test.c => ../../test/sha1test.c
sha256t.c => ../../test/sha256t.c
sha512t.c => ../../test/sha512t.c
make[]: 离开目录“/opt/redis/openssl-1.0./crypto/sha”
making links in crypto/mdc2...
make[]: 进入目录“/opt/redis/openssl-1.0./crypto/mdc2”
mdc2.h => ../../include/openssl/mdc2.h
mdc2test.c => ../../test/mdc2test.c
make[]: 离开目录“/opt/redis/openssl-1.0./crypto/mdc2”
making links in crypto/hmac...
make[]: 进入目录“/opt/redis/openssl-1.0./crypto/hmac”
hmac.h => ../../include/openssl/hmac.h
hmactest.c => ../../test/hmactest.c
make[]: 离开目录“/opt/redis/openssl-1.0./crypto/hmac”
making links in crypto/ripemd...
make[]: 进入目录“/opt/redis/openssl-1.0./crypto/ripemd”
ripemd.h => ../../include/openssl/ripemd.h
rmdtest.c => ../../test/rmdtest.c
make[]: 离开目录“/opt/redis/openssl-1.0./crypto/ripemd”
making links in crypto/whrlpool...
make[]: 进入目录“/opt/redis/openssl-1.0./crypto/whrlpool”
whrlpool.h => ../../include/openssl/whrlpool.h
wp_test.c => ../../test/wp_test.c
make[]: 离开目录“/opt/redis/openssl-1.0./crypto/whrlpool”
making links in crypto/des...
make[]: 进入目录“/opt/redis/openssl-1.0./crypto/des”
des.h => ../../include/openssl/des.h
des_old.h => ../../include/openssl/des_old.h
destest.c => ../../test/destest.c
make[]: 离开目录“/opt/redis/openssl-1.0./crypto/des”
making links in crypto/aes...
make[]: 进入目录“/opt/redis/openssl-1.0./crypto/aes”
aes.h => ../../include/openssl/aes.h
make[]: 离开目录“/opt/redis/openssl-1.0./crypto/aes”
making links in crypto/rc2...
make[]: 进入目录“/opt/redis/openssl-1.0./crypto/rc2”
rc2.h => ../../include/openssl/rc2.h
rc2test.c => ../../test/rc2test.c
make[]: 离开目录“/opt/redis/openssl-1.0./crypto/rc2”
making links in crypto/rc4...
make[]: 进入目录“/opt/redis/openssl-1.0./crypto/rc4”
rc4.h => ../../include/openssl/rc4.h
rc4test.c => ../../test/rc4test.c
make[]: 离开目录“/opt/redis/openssl-1.0./crypto/rc4”
making links in crypto/idea...
make[]: 进入目录“/opt/redis/openssl-1.0./crypto/idea”
idea.h => ../../include/openssl/idea.h
ideatest.c => ../../test/ideatest.c
make[]: 离开目录“/opt/redis/openssl-1.0./crypto/idea”
making links in crypto/bf...
make[]: 进入目录“/opt/redis/openssl-1.0./crypto/bf”
blowfish.h => ../../include/openssl/blowfish.h
bftest.c => ../../test/bftest.c
make[]: 离开目录“/opt/redis/openssl-1.0./crypto/bf”
making links in crypto/cast...
make[]: 进入目录“/opt/redis/openssl-1.0./crypto/cast”
cast.h => ../../include/openssl/cast.h
casttest.c => ../../test/casttest.c
make[]: 离开目录“/opt/redis/openssl-1.0./crypto/cast”
making links in crypto/camellia...
make[]: 进入目录“/opt/redis/openssl-1.0./crypto/camellia”
camellia.h => ../../include/openssl/camellia.h
make[]: 离开目录“/opt/redis/openssl-1.0./crypto/camellia”
making links in crypto/seed...
make[]: 进入目录“/opt/redis/openssl-1.0./crypto/seed”
seed.h => ../../include/openssl/seed.h
make[]: 离开目录“/opt/redis/openssl-1.0./crypto/seed”
making links in crypto/modes...
make[]: 进入目录“/opt/redis/openssl-1.0./crypto/modes”
modes.h => ../../include/openssl/modes.h
make[]: 离开目录“/opt/redis/openssl-1.0./crypto/modes”
making links in crypto/bn...
make[]: 进入目录“/opt/redis/openssl-1.0./crypto/bn”
bn.h => ../../include/openssl/bn.h
bntest.c => ../../test/bntest.c
exptest.c => ../../test/exptest.c
make[]: 离开目录“/opt/redis/openssl-1.0./crypto/bn”
making links in crypto/ec...
make[]: 进入目录“/opt/redis/openssl-1.0./crypto/ec”
ec.h => ../../include/openssl/ec.h
ectest.c => ../../test/ectest.c
make[]: 离开目录“/opt/redis/openssl-1.0./crypto/ec”
making links in crypto/rsa...
make[]: 进入目录“/opt/redis/openssl-1.0./crypto/rsa”
rsa.h => ../../include/openssl/rsa.h
rsa_test.c => ../../test/rsa_test.c
make[]: 离开目录“/opt/redis/openssl-1.0./crypto/rsa”
making links in crypto/dsa...
make[]: 进入目录“/opt/redis/openssl-1.0./crypto/dsa”
dsa.h => ../../include/openssl/dsa.h
dsatest.c => ../../test/dsatest.c
make[]: 离开目录“/opt/redis/openssl-1.0./crypto/dsa”
making links in crypto/ecdsa...
make[]: 进入目录“/opt/redis/openssl-1.0./crypto/ecdsa”
ecdsa.h => ../../include/openssl/ecdsa.h
ecdsatest.c => ../../test/ecdsatest.c
make[]: 离开目录“/opt/redis/openssl-1.0./crypto/ecdsa”
making links in crypto/dh...
make[]: 进入目录“/opt/redis/openssl-1.0./crypto/dh”
dh.h => ../../include/openssl/dh.h
dhtest.c => ../../test/dhtest.c
make[]: 离开目录“/opt/redis/openssl-1.0./crypto/dh”
making links in crypto/ecdh...
make[]: 进入目录“/opt/redis/openssl-1.0./crypto/ecdh”
ecdh.h => ../../include/openssl/ecdh.h
ecdhtest.c => ../../test/ecdhtest.c
make[]: 离开目录“/opt/redis/openssl-1.0./crypto/ecdh”
making links in crypto/dso...
make[]: 进入目录“/opt/redis/openssl-1.0./crypto/dso”
dso.h => ../../include/openssl/dso.h
make[]: 离开目录“/opt/redis/openssl-1.0./crypto/dso”
making links in crypto/engine...
make[]: 进入目录“/opt/redis/openssl-1.0./crypto/engine”
engine.h => ../../include/openssl/engine.h
enginetest.c => ../../test/enginetest.c
make[]: 离开目录“/opt/redis/openssl-1.0./crypto/engine”
making links in crypto/buffer...
make[]: 进入目录“/opt/redis/openssl-1.0./crypto/buffer”
buffer.h => ../../include/openssl/buffer.h
make[]: 离开目录“/opt/redis/openssl-1.0./crypto/buffer”
making links in crypto/bio...
make[]: 进入目录“/opt/redis/openssl-1.0./crypto/bio”
bio.h => ../../include/openssl/bio.h
make[]: 离开目录“/opt/redis/openssl-1.0./crypto/bio”
making links in crypto/stack...
make[]: 进入目录“/opt/redis/openssl-1.0./crypto/stack”
stack.h => ../../include/openssl/stack.h
safestack.h => ../../include/openssl/safestack.h
make[]: 离开目录“/opt/redis/openssl-1.0./crypto/stack”
making links in crypto/lhash...
make[]: 进入目录“/opt/redis/openssl-1.0./crypto/lhash”
lhash.h => ../../include/openssl/lhash.h
make[]: 离开目录“/opt/redis/openssl-1.0./crypto/lhash”
making links in crypto/rand...
make[]: 进入目录“/opt/redis/openssl-1.0./crypto/rand”
rand.h => ../../include/openssl/rand.h
randtest.c => ../../test/randtest.c
make[]: 离开目录“/opt/redis/openssl-1.0./crypto/rand”
making links in crypto/err...
make[]: 进入目录“/opt/redis/openssl-1.0./crypto/err”
err.h => ../../include/openssl/err.h
make[]: 离开目录“/opt/redis/openssl-1.0./crypto/err”
making links in crypto/evp...
make[]: 进入目录“/opt/redis/openssl-1.0./crypto/evp”
evp.h => ../../include/openssl/evp.h
evp_test.c => ../../test/evp_test.c
evptests.txt -> ../../test/evptests.txt
make[]: 离开目录“/opt/redis/openssl-1.0./crypto/evp”
making links in crypto/asn1...
make[]: 进入目录“/opt/redis/openssl-1.0./crypto/asn1”
asn1.h => ../../include/openssl/asn1.h
asn1_mac.h => ../../include/openssl/asn1_mac.h
asn1t.h => ../../include/openssl/asn1t.h
make[]: 离开目录“/opt/redis/openssl-1.0./crypto/asn1”
making links in crypto/pem...
make[]: 进入目录“/opt/redis/openssl-1.0./crypto/pem”
pem.h => ../../include/openssl/pem.h
pem2.h => ../../include/openssl/pem2.h
make[]: 离开目录“/opt/redis/openssl-1.0./crypto/pem”
making links in crypto/x509...
make[]: 进入目录“/opt/redis/openssl-1.0./crypto/x509”
x509.h => ../../include/openssl/x509.h
x509_vfy.h => ../../include/openssl/x509_vfy.h
make[]: 离开目录“/opt/redis/openssl-1.0./crypto/x509”
making links in crypto/x509v3...
make[]: 进入目录“/opt/redis/openssl-1.0./crypto/x509v3”
x509v3.h => ../../include/openssl/x509v3.h
v3nametest.c => ../../test/v3nametest.c
make[]: 离开目录“/opt/redis/openssl-1.0./crypto/x509v3”
making links in crypto/conf...
make[]: 进入目录“/opt/redis/openssl-1.0./crypto/conf”
conf.h => ../../include/openssl/conf.h
conf_api.h => ../../include/openssl/conf_api.h
make[]: 离开目录“/opt/redis/openssl-1.0./crypto/conf”
making links in crypto/txt_db...
make[]: 进入目录“/opt/redis/openssl-1.0./crypto/txt_db”
txt_db.h => ../../include/openssl/txt_db.h
make[]: 离开目录“/opt/redis/openssl-1.0./crypto/txt_db”
making links in crypto/pkcs7...
make[]: 进入目录“/opt/redis/openssl-1.0./crypto/pkcs7”
pkcs7.h => ../../include/openssl/pkcs7.h
make[]: 离开目录“/opt/redis/openssl-1.0./crypto/pkcs7”
making links in crypto/pkcs12...
make[]: 进入目录“/opt/redis/openssl-1.0./crypto/pkcs12”
pkcs12.h => ../../include/openssl/pkcs12.h
make[]: 离开目录“/opt/redis/openssl-1.0./crypto/pkcs12”
making links in crypto/comp...
make[]: 进入目录“/opt/redis/openssl-1.0./crypto/comp”
comp.h => ../../include/openssl/comp.h
make[]: 离开目录“/opt/redis/openssl-1.0./crypto/comp”
making links in crypto/ocsp...
make[]: 进入目录“/opt/redis/openssl-1.0./crypto/ocsp”
ocsp.h => ../../include/openssl/ocsp.h
make[]: 离开目录“/opt/redis/openssl-1.0./crypto/ocsp”
making links in crypto/ui...
make[]: 进入目录“/opt/redis/openssl-1.0./crypto/ui”
ui.h => ../../include/openssl/ui.h
ui_compat.h => ../../include/openssl/ui_compat.h
make[]: 离开目录“/opt/redis/openssl-1.0./crypto/ui”
making links in crypto/krb5...
make[]: 进入目录“/opt/redis/openssl-1.0./crypto/krb5”
krb5_asn.h => ../../include/openssl/krb5_asn.h
make[]: 离开目录“/opt/redis/openssl-1.0./crypto/krb5”
making links in crypto/cms...
make[]: 进入目录“/opt/redis/openssl-1.0./crypto/cms”
cms.h => ../../include/openssl/cms.h
make[]: 离开目录“/opt/redis/openssl-1.0./crypto/cms”
making links in crypto/pqueue...
make[]: 进入目录“/opt/redis/openssl-1.0./crypto/pqueue”
pqueue.h => ../../include/openssl/pqueue.h
make[]: 离开目录“/opt/redis/openssl-1.0./crypto/pqueue”
making links in crypto/ts...
make[]: 进入目录“/opt/redis/openssl-1.0./crypto/ts”
ts.h => ../../include/openssl/ts.h
make[]: 离开目录“/opt/redis/openssl-1.0./crypto/ts”
making links in crypto/srp...
make[]: 进入目录“/opt/redis/openssl-1.0./crypto/srp”
srp.h => ../../include/openssl/srp.h
srptest.c => ../../test/srptest.c
make[]: 离开目录“/opt/redis/openssl-1.0./crypto/srp”
making links in crypto/cmac...
make[]: 进入目录“/opt/redis/openssl-1.0./crypto/cmac”
cmac.h => ../../include/openssl/cmac.h
make[]: 离开目录“/opt/redis/openssl-1.0./crypto/cmac”
make[]: 离开目录“/opt/redis/openssl-1.0./crypto”
making links in ssl...
make[]: 进入目录“/opt/redis/openssl-1.0./ssl”
ssl.h => ../include/openssl/ssl.h
ssl2.h => ../include/openssl/ssl2.h
ssl3.h => ../include/openssl/ssl3.h
ssl23.h => ../include/openssl/ssl23.h
tls1.h => ../include/openssl/tls1.h
dtls1.h => ../include/openssl/dtls1.h
kssl.h => ../include/openssl/kssl.h
srtp.h => ../include/openssl/srtp.h
ssltest.c => ../test/ssltest.c
heartbeat_test.c => ../test/heartbeat_test.c
make[]: 离开目录“/opt/redis/openssl-1.0./ssl”
making links in engines...
make[]: 进入目录“/opt/redis/openssl-1.0./engines”
making links in engines/ccgost...
make[]: 进入目录“/opt/redis/openssl-1.0./engines/ccgost”
make[]: 对“links”无需做任何事。
make[]: 离开目录“/opt/redis/openssl-1.0./engines/ccgost”
make[]: 离开目录“/opt/redis/openssl-1.0./engines”
making links in apps...
make[]: 进入目录“/opt/redis/openssl-1.0./apps”
make[]: 对“links”无需做任何事。
make[]: 离开目录“/opt/redis/openssl-1.0./apps”
making links in test...
make[]: 进入目录“/opt/redis/openssl-1.0./test”
make[]: 对“links”无需做任何事。
make[]: 离开目录“/opt/redis/openssl-1.0./test”
making links in tools...
make[]: 进入目录“/opt/redis/openssl-1.0./tools”
make[]: 对“links”无需做任何事。
make[]: 离开目录“/opt/redis/openssl-1.0./tools”
generating dummy tests (if needed)...
make[]: 进入目录“/opt/redis/openssl-1.0./test”
make[]: 对“generate”无需做任何事。
make[]: 离开目录“/opt/redis/openssl-1.0./test” Configured for linux-x86_64.
[root@bogon openssl-1.0.]

然后进行make操作,生成可执行文件。 完成后,看看当前的版本。openssl安装后,执行redis-trib还是报错。。。

到此为止,因为这个过程我反复尝试各种修正openssl的操作,都报错,应该是我的这个机器环境什么地方出问题了,不折腾这个了,既然redis配置的是集群的环境,那么,我在另外的一台机器上处理,应该也没有什么问题。

接下来的,配置操作是在10.90.7.2的服务器上进行的,一切换一种方式进行操作,这次主要是基于源码的安装。

源码安装ruby 2.4.2。从 http://www.ruby-lang.org/en/downloads/下载需要的版本。 解压后,进入ruby目录,分别进行./configure,./make, ./make install

查看版本信息,得到下面的内容,说明安装成功。

[root@localhost ~]# ruby -v
ruby 2.4.2p198 (-- revision ) [x86_64-linux]

继续执行redis-trib.rb指令。

[root@localhost src]# ./redis-trib.rb --help
/usr/local/lib/ruby/2.4./rubygems/core_ext/kernel_require.rb::in `require': cannot load such file -- redis (LoadError)
from /usr/local/lib/ruby/2.4./rubygems/core_ext/kernel_require.rb::in `require'
from ./redis-trib.rb::in `<main>'

这个错误和上面的基于yum安装的一个错误,缺少ruby版本的redis。依然下载源码安装。https://rubygems.org/downloads/redis-4.0.1.gem下载完毕后,用gem进行安装。

[root@localhost opt]# gem install redis-4.0..gem
Successfully installed redis-4.0.
Parsing documentation for redis-4.0.
Installing ri documentation for redis-4.0.
Done installing documentation for redis after seconds
gem installed
[root@localhost opt]#

再次执行redis-trib指令,检查redis-trib是否可以操作集群。

[root@localhost src]# ./redis-trib.rb
Usage: redis-trib <command> <options> <arguments ...> create host1:port1 ... hostN:portN
--replicas <arg>
check host:port
info host:port
fix host:port
--timeout <arg>
reshard host:port
--from <arg>
--to <arg>
--slots <arg>
--yes
--timeout <arg>
--pipeline <arg>
rebalance host:port
--weight <arg>
--auto-weights
--use-empty-masters
--timeout <arg>
--simulate
--pipeline <arg>
--threshold <arg>
add-node new_host:new_port existing_host:existing_port
--slave
--master-id <arg>
del-node host:port node_id
set-timeout host:port milliseconds
call host:port command arg arg .. arg
import host:port
--from <arg>
--copy
--replace
help (show this help) For check, fix, reshard, del-node, set-timeout you can specify the host and port of any working node in the cluster.
[root@localhost src]#

这次,说明redis的起集群的操作成功,下一步准备集群创建。

查看目前启动的几台redis机器信息(info)

[root@localhost src]# ./redis-trib.rb info 10.90.7.2:
[ERR] Sorry, can't connect to node 10.90.7.2:7001
[root@localhost src]# ./redis-trib.rb info 10.90.7.2:
10.90.7.2: (...) -> keys | slots | slaves.
[OK] keys in masters.
0.00 keys per slot on average.
[root@localhost src]#
[root@localhost src]# ./redis-trib.rb info 10.90.7.2:
10.90.7.2: (79ebc53a...) -> keys | slots | slaves.
[OK] keys in masters.
0.00 keys per slot on average.
[root@localhost src]#
[root@localhost src]# ./redis-trib.rb info 10.90.7.10:
10.90.7.10: (7fd72f31...) -> keys | slots | slaves.
[OK] keys in masters.
0.00 keys per slot on average.
[root@localhost src]# ./redis-trib.rb info 10.90.7.10:
10.90.7.10: (d5a0fa35...) -> keys | slots | slaves.
[OK] keys in masters.
0.00 keys per slot on average.
[root@localhost src]# ./redis-trib.rb info 10.90.2.102:
10.90.2.102: (8d539663...) -> keys | slots | slaves.
[OK] keys in masters.
0.00 keys per slot on average.
[root@localhost src]# ./redis-trib.rb info 10.90.2.102:
10.90.2.102: (175061fa...) -> keys | slots | slaves.
[OK] keys in masters.
0.00 keys per slot on average.
[root@localhost src]#

创建redis集群,通过redis-trib命令实现:

[root@localhost src]# ./redis-trib.rb create --replicas  10.90.7.2: 10.90.7.2: 10.90.7.10: 10.90.7.10: 10.90.2.102: 10.90.2.102:
>>> Creating cluster
>>> Performing hash slots allocation on nodes...
Using masters:
10.90.7.2:
10.90.7.10:
10.90.2.102:
Adding replica 10.90.7.10: to 10.90.7.2:
Adding replica 10.90.7.2: to 10.90.7.10:
Adding replica 10.90.2.102: to 10.90.2.102:
M: 549417379f072c2f9551c1e662fbf0ef3ecc73eb 10.90.7.2:
slots:- ( slots) master
S: 79ebc53a5193dac898ec01fe2c5b0023990a379b 10.90.7.2:
replicates 7fd72f31bf4faaae4f30cec6ac8ae1ab9cdaa597
M: 7fd72f31bf4faaae4f30cec6ac8ae1ab9cdaa597 10.90.7.10:
slots:- ( slots) master
S: d5a0fa35859c40e8588bf8648e036ab152bcece7 10.90.7.10:
replicates 549417379f072c2f9551c1e662fbf0ef3ecc73eb
M: 8d53966305162f6f0e53714f22e554a38f74d67b 10.90.2.102:
slots:- ( slots) master
S: 175061fa6addecc8eaaf9b999372a9d5a8964d44 10.90.2.102:
replicates 8d53966305162f6f0e53714f22e554a38f74d67b
Can I set the above configuration? (type 'yes' to accept): yes
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join.....
>>> Performing Cluster Check (using node 10.90.7.2:)
M: 549417379f072c2f9551c1e662fbf0ef3ecc73eb 10.90.7.2:
slots:- ( slots) master
additional replica(s)
S: 79ebc53a5193dac898ec01fe2c5b0023990a379b 10.90.7.2:
slots: ( slots) slave
replicates 7fd72f31bf4faaae4f30cec6ac8ae1ab9cdaa597
S: d5a0fa35859c40e8588bf8648e036ab152bcece7 10.90.7.10:
slots: ( slots) slave
replicates 549417379f072c2f9551c1e662fbf0ef3ecc73eb
M: 7fd72f31bf4faaae4f30cec6ac8ae1ab9cdaa597 10.90.7.10:
slots:- ( slots) master
additional replica(s)
M: 8d53966305162f6f0e53714f22e554a38f74d67b 10.90.2.102:
slots:- ( slots) master
additional replica(s)
S: 175061fa6addecc8eaaf9b999372a9d5a8964d44 10.90.2.102:
slots: ( slots) slave
replicates 8d53966305162f6f0e53714f22e554a38f74d67b
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All slots covered.

指令查看redis集群信息:

[root@localhost src]# ./redis-cli -c -h 10.90.7.2 -p
10.90.7.2:> info
# Server
redis_version:3.2.
redis_git_sha1:
redis_git_dirty:
redis_build_id:6319bd4d85613682
redis_mode:cluster
os:Linux 2.6.-.el5 x86_64
arch_bits:
multiplexing_api:epoll
gcc_version:4.1.
process_id:
run_id:428293cc0d9842e0ba2f3917322a0deb136196ee
tcp_port:
uptime_in_seconds:
uptime_in_days:
hz:
lru_clock:
executable:/opt/redis-3.2./src/redis-server
config_file:/opt/redis-3.2./tkcluster/.conf # Clients
connected_clients:
client_longest_output_list:
client_biggest_input_buf:
blocked_clients: # Memory
used_memory:
used_memory_human:2.31M
used_memory_rss:
used_memory_rss_human:2.88M
used_memory_peak:
used_memory_peak_human:2.31M
total_system_memory:
total_system_memory_human:.70G
used_memory_lua:
used_memory_lua_human:.00K
maxmemory:
maxmemory_human:0B
maxmemory_policy:noeviction
mem_fragmentation_ratio:1.25
mem_allocator:jemalloc-4.0. # Persistence
loading:
rdb_changes_since_last_save:
rdb_bgsave_in_progress:
rdb_last_save_time:
rdb_last_bgsave_status:ok
rdb_last_bgsave_time_sec:
rdb_current_bgsave_time_sec:-
aof_enabled:
aof_rewrite_in_progress:
aof_rewrite_scheduled:
aof_last_rewrite_time_sec:-
aof_current_rewrite_time_sec:-
aof_last_bgrewrite_status:ok
aof_last_write_status:ok
aof_current_size:
aof_base_size:
aof_pending_rewrite:
aof_buffer_length:
aof_rewrite_buffer_length:
aof_pending_bio_fsync:
aof_delayed_fsync: # Stats
total_connections_received:
total_commands_processed:
instantaneous_ops_per_sec:
total_net_input_bytes:
total_net_output_bytes:
instantaneous_input_kbps:0.04
instantaneous_output_kbps:0.00
rejected_connections:
sync_full:
sync_partial_ok:
sync_partial_err:
expired_keys:
evicted_keys:
keyspace_hits:
keyspace_misses:
pubsub_channels:
pubsub_patterns:
latest_fork_usec:
migrate_cached_sockets: # Replication
role:master
connected_slaves:
slave0:ip=10.90.7.10,port=,state=online,offset=,lag=
master_repl_offset:
repl_backlog_active:
repl_backlog_size:
repl_backlog_first_byte_offset:
repl_backlog_histlen: # CPU
used_cpu_sys:0.09
used_cpu_user:0.11
used_cpu_sys_children:0.01
used_cpu_user_children:0.00 # Cluster
cluster_enabled: # Keyspace
10.90.7.2:>

在10.90.7.2上设置一个数据:

10.90.7.2:> set hello
OK
10.90.7.2:> get hello
""

查看一下集群的信息点:

10.90.7.2:> cluster info
cluster_state:ok
cluster_slots_assigned:
cluster_slots_ok:
cluster_slots_pfail:
cluster_slots_fail:
cluster_known_nodes:
cluster_size:
cluster_current_epoch:
cluster_my_epoch:
cluster_stats_messages_sent:
cluster_stats_messages_received:

从另外一个主机上,测试看看:

[root@bogon src]# ./redis-cli -c -h 10.90.7.10 -p 7001
10.90.7.10:> info
# Server
redis_version:3.2.
redis_git_sha1:
redis_git_dirty:
redis_build_id:b50c19b6d7e8c0c8
redis_mode:cluster
os:Linux 3.10.-.el7.x86_64 x86_64
arch_bits:
multiplexing_api:epoll
gcc_version:4.8.
process_id:
run_id:5b03ae96c0af7156d9ee0fae9ab96f9c56c7f6b7
tcp_port:
uptime_in_seconds:
uptime_in_days:
hz:
lru_clock:
executable:/opt/redis/redis-3.2./src/redis-server
config_file:/opt/redis/redis-3.2./tkcluster/.conf # Clients
connected_clients:
client_longest_output_list:
client_biggest_input_buf:
blocked_clients: # Memory
used_memory:
used_memory_human:2.31M
used_memory_rss:
used_memory_rss_human:9.57M
used_memory_peak:
used_memory_peak_human:2.31M
total_system_memory:
total_system_memory_human:.15G
used_memory_lua:
used_memory_lua_human:.00K
maxmemory:
maxmemory_human:0B
maxmemory_policy:noeviction
mem_fragmentation_ratio:4.15
mem_allocator:jemalloc-4.0. # Persistence
loading:
rdb_changes_since_last_save:
rdb_bgsave_in_progress:
rdb_last_save_time:
rdb_last_bgsave_status:ok
rdb_last_bgsave_time_sec:
rdb_current_bgsave_time_sec:-
aof_enabled:
aof_rewrite_in_progress:
aof_rewrite_scheduled:
aof_last_rewrite_time_sec:-
aof_current_rewrite_time_sec:-
aof_last_bgrewrite_status:ok
aof_last_write_status:ok
aof_current_size:
aof_base_size:
aof_pending_rewrite:
aof_buffer_length:
aof_rewrite_buffer_length:
aof_pending_bio_fsync:
aof_delayed_fsync: # Stats
total_connections_received:
total_commands_processed:
instantaneous_ops_per_sec:
total_net_input_bytes:
total_net_output_bytes:
instantaneous_input_kbps:0.02
instantaneous_output_kbps:0.00
rejected_connections:
sync_full:
sync_partial_ok:
sync_partial_err:
expired_keys:
evicted_keys:
keyspace_hits:
keyspace_misses:
pubsub_channels:
pubsub_patterns:
latest_fork_usec:
migrate_cached_sockets: # Replication
role:master
connected_slaves:
slave0:ip=10.90.7.2,port=,state=online,offset=,lag=
master_repl_offset:
repl_backlog_active:
repl_backlog_size:
repl_backlog_first_byte_offset:
repl_backlog_histlen: # CPU
used_cpu_sys:35.23
used_cpu_user:19.00
used_cpu_sys_children:0.00
used_cpu_user_children:0.00 # Cluster
cluster_enabled: # Keyspace
10.90.7.10:> info
# Server
redis_version:3.2.
redis_git_sha1:
redis_git_dirty:
redis_build_id:b50c19b6d7e8c0c8
redis_mode:cluster
os:Linux 3.10.-.el7.x86_64 x86_64
arch_bits:
multiplexing_api:epoll
gcc_version:4.8.
process_id:
run_id:5b03ae96c0af7156d9ee0fae9ab96f9c56c7f6b7
tcp_port:
uptime_in_seconds:
uptime_in_days:
hz:
lru_clock:
executable:/opt/redis/redis-3.2.8/src/redis-server
config_file:/opt/redis/redis-3.2.8/tkcluster/7001
.conf # Clients
connected_clients:
client_longest_output_list:
client_biggest_input_buf:
blocked_clients: # Memory
used_memory:
used_memory_human:2.31M
used_memory_rss:
used_memory_rss_human:9.57M
used_memory_peak:
used_memory_peak_human:2.31M
total_system_memory:
total_system_memory_human:.15G
used_memory_lua:
used_memory_lua_human:.00K
maxmemory:
maxmemory_human:0B
maxmemory_policy:noeviction
mem_fragmentation_ratio:4.15
mem_allocator:jemalloc-4.0. # Persistence
loading:
rdb_changes_since_last_save:
rdb_bgsave_in_progress:
rdb_last_save_time:
rdb_last_bgsave_status:ok
rdb_last_bgsave_time_sec:
rdb_current_bgsave_time_sec:-
aof_enabled:
aof_rewrite_in_progress:
aof_rewrite_scheduled:
aof_last_rewrite_time_sec:-
aof_current_rewrite_time_sec:-
aof_last_bgrewrite_status:ok
aof_last_write_status:ok
aof_current_size:
aof_base_size:
aof_pending_rewrite:
aof_buffer_length:
aof_rewrite_buffer_length:
aof_pending_bio_fsync:
aof_delayed_fsync: # Stats
total_connections_received:
total_commands_processed:
instantaneous_ops_per_sec:
total_net_input_bytes:
total_net_output_bytes:
instantaneous_input_kbps:0.04
instantaneous_output_kbps:0.00
rejected_connections:
sync_full:
sync_partial_ok:
sync_partial_err:
expired_keys:
evicted_keys:
keyspace_hits:
keyspace_misses:
pubsub_channels:
pubsub_patterns:
latest_fork_usec:
migrate_cached_sockets: # Replication
role:master
connected_slaves:
slave0:ip=10.90.7.2,port=,state=online,offset=,lag=
master_repl_offset:
repl_backlog_active:
repl_backlog_size:
repl_backlog_first_byte_offset:
repl_backlog_histlen: # CPU
used_cpu_sys:35.23
used_cpu_user:19.00
used_cpu_sys_children:0.00
used_cpu_user_children:0.00 # Cluster
cluster_enabled: # Keyspace
10.90.7.10:7001> get hello
-> Redirected to slot [866] located at 10.90.7.2:7000
"1"
10.90.7.2:7000>

选择database的操作:

10.90.7.2:> select
OK
10.90.7.2:> select
(error) ERR SELECT is not allowed in cluster mode
10.90.7.2:>

特别说明,redis集群下,不支持database的非0的选择,其实是不支持select指令了,但是对于默认的database 0又默认支持。有点设计的不美妙

另外,备注:集群环境下,不支持slaveof的配置,即配置文件里面,不能启动slaveof的配置。否则如我开始没有注意到这个,将conf文件中的slaveof指定了。就出现下面的错误。

[root@localhost tkcluster]# ./stcluster.sh
status from : *** FATAL CONFIG FILE ERROR ***
Reading the configuration file, at line
>>> 'slaveof 10.90.7.2 7010'
slaveof directive not allowed in cluster mode
status from :
[root@localhost tkcluster]#

OK,到此,redis的集群环境搭建到此结束,成功完成!

上一篇:[补档][Poi2010]Monotonicity 2


下一篇:LCA最近公共祖先(Tarjan离线算法)