Redis Sentinel的功能：
Monitoring. Sentinel constantly checks if your master and slave instances are working as expected.
Automatic failover. If a master is not working as expected, Sentinel can start a failover process where a slave is promoted to master, the other additional slaves are reconfigured to use the new master, and the applications using the Redis server informed about the new address to use when connecting.
Configuration provider. Sentinel acts as a source of authority for clients service discovery: clients connect to Sentinels in order to ask for the address of the current Redis master responsible for a given service. If a failover occurs, Sentinels will report the new address.
Notification. Sentinel can notify the system administrator, another computer programs, via an API, that something is wrong with one of the monitored Redis instances.
Sentinel的架构：
大于等于3的奇数个Sentinel进程分布在不同的主机上。
动态配置Sentinel
sentinel monitor <master-group-name> <ip> <port> <quorum>
sentinel monitor redis6379 192.168.116.133 6379 2
sentinel set redis6379 auth-pass abcdef
sentinel set redis6379 down-after-milliseconds 30000
sentinel set redis6379 parallel-syncs 1
sentinel set redis6379 failover-timeout 180000
31456:X 20 Apr 21:42:30.376 # +monitor master redis6379 192.168.116.133 6379 quorum 2
31456:X 20 Apr 21:42:30.542 # +set master redis6379 192.168.116.133 6379 auth-pass abcdef
31456:X 20 Apr 21:42:30.572 # +set master redis6379 192.168.116.133 6379 down-after-milliseconds 30000
31456:X 20 Apr 21:42:30.599 # +set master redis6379 192.168.116.133 6379 parallel-syncs 1
31456:X 20 Apr 21:42:31.325 * +sentinel sentinel 716af730eacde6031e31758fbd19726f939d07ca 192.168.116.133 26379 @ redis6379 192.168.116.133 6379
31456:X 20 Apr 21:42:31.468 # +set master redis6379 192.168.116.133 6379 failover-timeout 180000
31456:X 20 Apr 21:42:31.950 * +sentinel sentinel 593f1d32376602e8bb576d504ff44eedc1f88764 192.168.116.134 26380 @ redis6379 192.168.116.133 6379
31456:X 20 Apr 21:42:45.539 * +slave slave 192.168.116.134:6379 192.168.116.134 6379 @ redis6379 192.168.116.133 6379
通过Sentinel获取master的信息：
sentinel master redis6379
sentinel slaves redis6379
sentinel sentinels redis6379
sentinel get-master-addr-by-name redis6379
在redis6379上执行DEBUG sleep 40，引发故障转移：
第一个Sentinel的日志：
52808:X 20 Apr 21:47:26.657 # +sdown master redis6379 192.168.116.133 6379
52808:X 20 Apr 21:47:26.725 # +odown master redis6379 192.168.116.133 6379 #quorum 2/2
52808:X 20 Apr 21:47:26.725 # +new-epoch 11
52808:X 20 Apr 21:47:26.725 # +try-failover master redis6379 192.168.116.133 6379
52808:X 20 Apr 21:47:26.726 # +vote-for-leader 593f1d32376602e8bb576d504ff44eedc1f88764 11
52808:X 20 Apr 21:47:26.729 # a4f178284d257df80743ff237444c07b8a0919eb voted for 593f1d32376602e8bb576d504ff44eedc1f88764 11
52808:X 20 Apr 21:47:26.731 # 716af730eacde6031e31758fbd19726f939d07ca voted for 593f1d32376602e8bb576d504ff44eedc1f88764 11
52808:X 20 Apr 21:47:26.793 # +elected-leader master redis6379 192.168.116.133 6379
52808:X 20 Apr 21:47:26.793 # +failover-state-select-slave master redis6379 192.168.116.133 6379
52808:X 20 Apr 21:47:26.861 # +selected-slave slave 192.168.116.134:6379 192.168.116.134 6379 @ redis6379 192.168.116.133 6379
52808:X 20 Apr 21:47:26.862 * +failover-state-send-slaveof-noone slave 192.168.116.134:6379 192.168.116.134 6379 @ redis6379 192.168.116.133 6379
52808:X 20 Apr 21:47:26.953 * +failover-state-wait-promotion slave 192.168.116.134:6379 192.168.116.134 6379 @ redis6379 192.168.116.133 6379
52808:X 20 Apr 21:47:27.778 # +promoted-slave slave 192.168.116.134:6379 192.168.116.134 6379 @ redis6379 192.168.116.133 6379
52808:X 20 Apr 21:47:27.778 # +failover-state-reconf-slaves master redis6379 192.168.116.133 6379
52808:X 20 Apr 21:47:27.842 # +failover-end master redis6379 192.168.116.133 6379
52808:X 20 Apr 21:47:27.842 # +switch-master redis6379 192.168.116.133 6379 192.168.116.134 6379
52808:X 20 Apr 21:47:27.843 * +slave slave 192.168.116.133:6379 192.168.116.133 6379 @ redis6379 192.168.116.134 6379
52808:X 20 Apr 21:47:45.756 * +convert-to-slave slave 192.168.116.133:6379 192.168.116.133 6379 @ redis6379 192.168.116.134 6379
第二个
31452:X 20 Apr 21:47:25.999 # +sdown master redis6379 192.168.116.133 6379
31452:X 20 Apr 21:47:26.734 # +new-epoch 11
31452:X 20 Apr 21:47:26.735 # +vote-for-leader 593f1d32376602e8bb576d504ff44eedc1f88764 11
31452:X 20 Apr 21:47:27.117 # +odown master redis6379 192.168.116.133 6379 #quorum 2/2
31452:X 20 Apr 21:47:27.117 # Next failover delay: I will not start a failover before Sat Apr 20 21:53:27 2019
31452:X 20 Apr 21:47:27.850 # +config-update-from sentinel 593f1d32376602e8bb576d504ff44eedc1f88764 192.168.116.134 26380 @ redis6379 192.168.116.133 6379
31452:X 20 Apr 21:47:27.850 # +switch-master redis6379 192.168.116.133 6379 192.168.116.134 6379
31452:X 20 Apr 21:47:27.850 * +slave slave 192.168.116.133:6379 192.168.116.133 6379 @ redis6379 192.168.116.134 6379
第三个
31456:X 20 Apr 21:47:26.732 # +new-epoch 11
31456:X 20 Apr 21:47:26.733 # +vote-for-leader 593f1d32376602e8bb576d504ff44eedc1f88764 11
31456:X 20 Apr 21:47:27.848 # +config-update-from sentinel 593f1d32376602e8bb576d504ff44eedc1f88764 192.168.116.134 26380 @ redis6379 192.168.116.133 6379
31456:X 20 Apr 21:47:27.848 # +switch-master redis6379 192.168.116.133 6379 192.168.116.134 6379
31456:X 20 Apr 21:47:27.849 * +slave slave 192.168.116.133:6379 192.168.116.133 6379 @ redis6379 192.168.116.134 6379
在Sentinel上执行Sentinel failover redis6379，强制故障转移：
发起的sentinel的日志：
52808:X 20 Apr 22:45:09.364 # Executing user requested FAILOVER of 'redis6379'
52808:X 20 Apr 22:45:09.364 # +new-epoch 12
52808:X 20 Apr 22:45:09.364 # +try-failover master redis6379 192.168.116.134 6379
52808:X 20 Apr 22:45:09.374 # +vote-for-leader 593f1d32376602e8bb576d504ff44eedc1f88764 12
52808:X 20 Apr 22:45:09.374 # +elected-leader master redis6379 192.168.116.134 6379
52808:X 20 Apr 22:45:09.375 # +failover-state-select-slave master redis6379 192.168.116.134 6379
52808:X 20 Apr 22:45:09.477 # +selected-slave slave 192.168.116.133:6379 192.168.116.133 6379 @ redis6379 192.168.116.134 6379
52808:X 20 Apr 22:45:09.477 * +failover-state-send-slaveof-noone slave 192.168.116.133:6379 192.168.116.133 6379 @ redis6379 192.168.116.134 6379
52808:X 20 Apr 22:45:09.533 * +failover-state-wait-promotion slave 192.168.116.133:6379 192.168.116.133 6379 @ redis6379 192.168.116.134 6379
52808:X 20 Apr 22:45:10.062 # +promoted-slave slave 192.168.116.133:6379 192.168.116.133 6379 @ redis6379 192.168.116.134 6379
52808:X 20 Apr 22:45:10.062 # +failover-state-reconf-slaves master redis6379 192.168.116.134 6379
52808:X 20 Apr 22:45:10.127 # +failover-end master redis6379 192.168.116.134 6379
52808:X 20 Apr 22:45:10.127 # +switch-master redis6379 192.168.116.134 6379 192.168.116.133 6379
52808:X 20 Apr 22:45:10.130 * +slave slave 192.168.116.134:6379 192.168.116.134 6379 @ redis6379 192.168.116.133 6379
第二个
31456:X 20 Apr 22:45:09.494 # +new-epoch 12
31456:X 20 Apr 22:45:10.144 # +config-update-from sentinel 593f1d32376602e8bb576d504ff44eedc1f88764 192.168.116.134 26380 @ redis6379 192.168.116.134 6379
31456:X 20 Apr 22:45:10.144 # +switch-master redis6379 192.168.116.134 6379 192.168.116.133 6379
31456:X 20 Apr 22:45:10.147 * +slave slave 192.168.116.134:6379 192.168.116.134 6379 @ redis6379 192.168.116.133 6379
31456:X 20 Apr 22:45:20.237 * +convert-to-slave slave 192.168.116.134:6379 192.168.116.134 6379 @ redis6379 192.168.116.133 6379
第三个
31452:X 20 Apr 22:45:09.495 # +new-epoch 12
31452:X 20 Apr 22:45:10.147 # +config-update-from sentinel 593f1d32376602e8bb576d504ff44eedc1f88764 192.168.116.134 26380 @ redis6379 192.168.116.134 6379
31452:X 20 Apr 22:45:10.148 # +switch-master redis6379 192.168.116.134 6379 192.168.116.133 6379
31452:X 20 Apr 22:45:10.151 * +slave slave 192.168.116.134:6379 192.168.116.134 6379 @ redis6379 192.168.116.133 6379
调低redis6379的一个slave的slave-priority值，在Sentinel上执行Sentinel failover redis6379，强制故障转移：
发起的sentinel的日志：
52808:X 20 Apr 22:58:54.960 # Executing user requested FAILOVER of 'redis6379'
52808:X 20 Apr 22:58:54.961 # +new-epoch 13
52808:X 20 Apr 22:58:54.961 # +try-failover master redis6379 192.168.116.133 6379
52808:X 20 Apr 22:58:54.992 # +vote-for-leader 593f1d32376602e8bb576d504ff44eedc1f88764 13
52808:X 20 Apr 22:58:54.992 # +elected-leader master redis6379 192.168.116.133 6379
52808:X 20 Apr 22:58:54.992 # +failover-state-select-slave master redis6379 192.168.116.133 6379
52808:X 20 Apr 22:58:55.055 # +selected-slave slave 192.168.116.134:6382 192.168.116.134 6382 @ redis6379 192.168.116.133 6379
52808:X 20 Apr 22:58:55.055 * +failover-state-send-slaveof-noone slave 192.168.116.134:6382 192.168.116.134 6382 @ redis6379 192.168.116.133 6379
52808:X 20 Apr 22:58:55.114 * +failover-state-wait-promotion slave 192.168.116.134:6382 192.168.116.134 6382 @ redis6379 192.168.116.133 6379
52808:X 20 Apr 22:58:56.118 # +promoted-slave slave 192.168.116.134:6382 192.168.116.134 6382 @ redis6379 192.168.116.133 6379
52808:X 20 Apr 22:58:56.118 # +failover-state-reconf-slaves master redis6379 192.168.116.133 6379
52808:X 20 Apr 22:58:56.193 * +slave-reconf-sent slave 192.168.116.134:6379 192.168.116.134 6379 @ redis6379 192.168.116.133 6379
52808:X 20 Apr 22:58:56.676 * +slave-reconf-inprog slave 192.168.116.134:6379 192.168.116.134 6379 @ redis6379 192.168.116.133 6379
52808:X 20 Apr 22:58:56.677 * +slave-reconf-done slave 192.168.116.134:6379 192.168.116.134 6379 @ redis6379 192.168.116.133 6379
52808:X 20 Apr 22:58:56.742 # +failover-end master redis6379 192.168.116.133 6379
52808:X 20 Apr 22:58:56.743 # +switch-master redis6379 192.168.116.133 6379 192.168.116.134 6382
52808:X 20 Apr 22:58:56.743 * +slave slave 192.168.116.134:6379 192.168.116.134 6379 @ redis6379 192.168.116.134 6382
52808:X 20 Apr 22:58:56.743 * +slave slave 192.168.116.133:6379 192.168.116.133 6379 @ redis6379 192.168.116.134 6382
第二个
31456:X 20 Apr 22:58:55.066 # +new-epoch 13
31456:X 20 Apr 22:58:56.204 # +config-update-from sentinel 593f1d32376602e8bb576d504ff44eedc1f88764 192.168.116.134 26380 @ redis6379 192.168.116.133 6379
31456:X 20 Apr 22:58:56.204 # +switch-master redis6379 192.168.116.133 6379 192.168.116.134 6382
31456:X 20 Apr 22:58:56.205 * +slave slave 192.168.116.134:6379 192.168.116.134 6379 @ redis6379 192.168.116.134 6382
31456:X 20 Apr 22:58:56.205 * +slave slave 192.168.116.133:6379 192.168.116.133 6379 @ redis6379 192.168.116.134 6382
31456:X 20 Apr 22:59:06.247 * +convert-to-slave slave 192.168.116.133:6379 192.168.116.133 6379 @ redis6379 192.168.116.134 6382
第三个
31452:X 20 Apr 22:58:55.085 # +new-epoch 13
31452:X 20 Apr 22:58:56.204 # +config-update-from sentinel 593f1d32376602e8bb576d504ff44eedc1f88764 192.168.116.134 26380 @ redis6379 192.168.116.133 6379
31452:X 20 Apr 22:58:56.204 # +switch-master redis6379 192.168.116.133 6379 192.168.116.134 6382
31452:X 20 Apr 22:58:56.205 * +slave slave 192.168.116.134:6379 192.168.116.134 6379 @ redis6379 192.168.116.134 6382
31452:X 20 Apr 22:58:56.205 * +slave slave 192.168.116.133:6379 192.168.116.133 6379 @ redis6379 192.168.116.134 6382
将config命令rename-command，在Sentinel上执行Sentinel failover redis6379，强制故障转移（失败）：
发起的sentinel的日志：
52808:X 21 Apr 12:03:33.980 # Executing user requested FAILOVER of 'redis6379'
52808:X 21 Apr 12:03:33.980 # +new-epoch 19
52808:X 21 Apr 12:03:33.980 # +try-failover master redis6379 192.168.116.133 6379
52808:X 21 Apr 12:03:34.028 # +vote-for-leader 593f1d32376602e8bb576d504ff44eedc1f88764 19
52808:X 21 Apr 12:03:34.028 # +elected-leader master redis6379 192.168.116.133 6379
52808:X 21 Apr 12:03:34.028 # +failover-state-select-slave master redis6379 192.168.116.133 6379
52808:X 21 Apr 12:03:34.096 # +selected-slave slave 192.168.116.134:6382 192.168.116.134 6382 @ redis6379 192.168.116.133 6379
52808:X 21 Apr 12:03:34.096 * +failover-state-send-slaveof-noone slave 192.168.116.134:6382 192.168.116.134 6382 @ redis6379 192.168.116.133 6379
52808:X 21 Apr 12:03:34.156 * +failover-state-wait-promotion slave 192.168.116.134:6382 192.168.116.134 6382 @ redis6379 192.168.116.133 6379
52808:X 21 Apr 12:06:34.202 # -failover-abort-slave-timeout master redis6379 192.168.116.133 6379
第二个
31456:X 21 Apr 12:03:35.313 # +new-epoch 19
第三个
31452:X 21 Apr 12:03:35.181 # +new-epoch 19
Sentinel命令 - SENTINEL reset <pattern>
SENTINEL reset <pattern> This command will reset all the masters with matching name. The pattern argument is a glob-style pattern. The reset process clears any previous state in a master (including a failover in progress), and removes every slave and sentinel already discovered and associated with the master.
该命令解决了下面两个问题：如何删除Sentinel节点，和删除无用的slave。
删除Sentinel节点：
sentinel reset *（Sentinel节点间间隔30秒）
顺带说下，添加Sentinel节点：
All you need to do is to start the new Sentinel configured to monitor the currently active master.
删除无用的slave（Sentinel不会主动删除slave）：
sentinel reset redis6379