原理：
|
1 2 3 4 5 6 7 |
Redis集群采用一致性哈希槽的方式将集群中每个主节点都分配一定的哈希槽，对写入的数据进行哈希后分配到某个主节点进行存储。 集群使用公式（CRC16 key）& 16384计算键key数据那个槽 16384个slot均匀分布在各个节点上 集群中每个主节点将承担一部分槽点的维护，而槽点中存储着数据，每个主节点都有至少一个从节点用于高可用。 |
节点通信方式：
|
1 2 3 4 5 6 7 |
开启一个端口 设置的端口号+10000，用于集群之间节点通信交换信息 每个节点默认每秒10次选择随机5个节点发送ping消息，将自身信息和知道的集群信息传递，收到ping消息后返回pong消息做回复，最后通过这种随机的消息交换，最终每个节点将获得所有信息 当某个主节点挂掉，所有节点将会发现主节点挂掉了，作为主节点的从节点，就会接替主节点的工作，然后告诉所有其它节点，他成为了主。这样其它存活节点，就将它们维护的信息表更新从节点将接任做主，如果都挂掉集群将报错。当从一个节点操作，根据一致性哈希计算后将存储在其中一个主节点中，从节点将同步主的数据 redis cluster是去中心化的，集群中的每个节点都是平等的关系，每个节点都保存各自的数据和整个集群的状态。每个节点都和其他所有节点连接，而且这些连接保持活跃。 |
Redis分布式集群（部署）：
环境
| 主机 | 内网 | 外网 | 部署服务 | 端口 |
|---|---|---|---|---|
| redis01 | 172.16.1.81 | 10.0.0.81 | redis集群 | 6381/6384 |
| redis02 | 172.16.1.82 | 10.0.0.82 | redis集群 | 6382/6385 |
| redis03 | 172.16.1.83 | 10.0.0.83 | redis集群 | 6383/6386 |
1.停止掉redis和哨兵
2.创建工作目录
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 |
[root@redis01 ~]# mkdir /usr/local/redis/cluster [root@redis02 ~]# mkdir /usr/local/redis/cluster [root@redis03 ~]# mkdir /usr/local/redis/cluster [root@redis01 ~]# mkdir /var/lib/redis [root@redis02 ~]# mkdir /var/lib/redis [root@redis03 ~]# mkdir /var/lib/redis [root@redis01 ~]# mkdir -p /var/log/redis [root@redis02 ~]# mkdir -p /var/log/redis [root@redis03 ~]# mkdir -p /var/log/redis [root@redis01 ~]# touch /var/log/redis/redis-6381.log [root@redis02 ~]# touch /var/log/redis/redis-6382.log [root@redis03 ~]# touch /var/log/redis/redis-6383.log |
3.创建工作目录并编写配置文件
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 |
[root@redis01 ~]# vim /usr/local/redis/cluster/redis6381.conf # Redis-cluster-01 daemonize yes port 6381 dir /var/lib/redis ## 启动集群模式 cluster-enabled yes ## 设置集群配置文件名（配置文件是由redis自己创建的，不需要手动创建） cluster-config-file redis-cluster-6381.conf ## 集群内部沟通超时时间 cluster-node-timeout 5000 ## 是否集群内所有的节点都能够正常提供服务才算集群正常 cluster-require-full-coverage no bind 0.0.0.0 protected-mode no save "" appendonly no requirepass 1 masterauth 1 logfile "/var/log/redis/redis-6381.log" ######################################################################## [root@redis01 ~]# vim /usr/local/redis/cluster/redis6382.conf # Redis-cluster-01 daemonize yes port 6382 dir /var/lib/redis ## 启动集群模式 cluster-enabled yes ## 设置集群配置文件名（配置文件是由redis自己创建的，不需要手动创建） cluster-config-file redis-cluster-6382.conf ## 集群内部沟通超时时间 cluster-node-timeout 5000 ## 是否集群内所有的节点都能够正常提供服务才算集群正常 cluster-require-full-coverage no bind 0.0.0.0 protected-mode no save "" appendonly no requirepass 1 masterauth 1 logfile "/var/log/redis/redis-6382.log" ######################################################################## [root@redis01 ~]# vim /usr/local/redis/cluster/redis6383.conf # Redis-cluster-03 daemonize yes port 6383 dir /var/lib/redis cluster-enabled yes cluster-config-file redis-cluster-6383.conf cluster-node-timeout 5000 # 是否集群内所有的节点都能够正常提供服务才算集群正常 cluster-require-full-coverage no bind 0.0.0.0 protected-mode no save "" appendonly no requirepass 1 masterauth 1 logfile "/var/log/redis/redis-6383.log" ######################################################################## [root@redis01 ~]# vim /usr/local/redis/cluster/redis6384.conf # Redis-cluster-04 daemonize yes port 6384 dir /var/lib/redis cluster-enabled yes cluster-config-file redis-cluster-6384.conf cluster-node-timeout 5000 # 是否集群内所有的节点都能够正常提供服务才算集群正常 cluster-require-full-coverage no bind 0.0.0.0 protected-mode no save "" appendonly no requirepass 1 masterauth 1 logfile "/var/log/redis/redis-6384.log" ######################################################################## [root@redis01 ~]# vim /usr/local/redis/cluster/redis6385.conf # Redis-cluster-05 daemonize yes port 6385 dir /var/lib/redis cluster-enabled yes cluster-config-file redis-cluster-6385.conf cluster-node-timeout 5000 # 是否集群内所有的节点都能够正常提供服务才算集群正常 cluster-require-full-coverage no bind 0.0.0.0 protected-mode no save "" appendonly no requirepass 1 masterauth 1 logfile "/var/log/redis/redis-6385.log" ######################################################################## [root@redis01 ~]# vim /usr/local/redis/cluster/redis6386.conf # Redis-cluster-06 daemonize yes port 6386 dir /var/lib/redis cluster-enabled yes cluster-config-file redis-cluster-6386.conf cluster-node-timeout 5000 # 是否集群内所有的节点都能够正常提供服务才算集群正常 cluster-require-full-coverage no bind 0.0.0.0 protected-mode no save "" appendonly no requirepass 1 masterauth 1 logfile "/var/log/redis/redis-6386.log" |
4.启动集群
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 |
[root@redis01 ~]# /usr/local/redis/bin/redis-server /usr/local/redis/cluster/redis6381.conf [root@redis01 ~]# /usr/local/redis/bin/redis-server /usr/local/redis/cluster/redis6384.conf [root@redis02 ~]# /usr/local/redis/bin/redis-server /usr/local/redis/cluster/redis6382.conf [root@redis02 ~]# /usr/local/redis/bin/redis-server /usr/local/redis/cluster/redis6385.conf [root@redis03 ~]# /usr/local/redis/bin/redis-server /usr/local/redis/cluster/redis6383.conf [root@redis03 ~]# /usr/local/redis/bin/redis-server /usr/local/redis/cluster/redis6386.conf #部署在一台可以用以下方法启动 for i in 81 82 83 84 85 86; do /usr/local/redis/bin/redis-server /usr/local/redis/cluster/redis63$i.conf; done |
5.查看是否启动成功
|
1 2 |
[root@redis01 /usr/local/redis/cluster]# ps -ef | grep redis |
6.集群握手
|
1 2 3 4 5 6 7 8 9 10 11 12 |
#集群握手的目的是为了绑定集群,可以都向一个握手 [root@redis01 ~]# redis-cli -a 1 -p 6381 cluster meet 172.16.1.81 6384 [root@redis01 ~]# redis-cli -a 1 -p 6381 cluster meet 172.16.1.82 6382 [root@redis01 ~]# redis-cli -a 1 -p 6381 cluster meet 172.16.1.82 6385 [root@redis01 ~]# redis-cli -a 1 -p 6381 cluster meet 172.16.1.83 6383 [root@redis01 ~]# redis-cli -a 1 -p 6381 cluster meet 172.16.1.83 6386 #部署在一台可以用以下方法集群握手 for i in 6382 6383 6384 6385 6386; do redis-cli -a 1 -p 6381 cluster meet 172.16.1.81 $i done |
7.查看握手结果
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 |
[root@redis01 ~]# redis-cli -a 1 -p 6381 cluster nodes Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe. d721544df39cdd192ce31eaf062ead33ce4759ba 172.16.1.83:6383@16383 master - 0 1608355980186 4 connected d003d10db9d88acd4cc486d8a12a6a733f7b7ec3 172.16.1.81:6384@16384 master - 0 1608355980000 3 connected eafcb901a751f345dcb63beb3117ab3ec136dae8 172.16.1.82:6385@16385 master - 0 1608355981000 0 connected 7a15703fe4fc2119ab545e83178254e41af612fd 172.16.1.81:6381@16381 myself,master - 0 1608355980000 1 connected 3c5ae9a66a1505a75873e0eb128fb70481f11105 172.16.1.82:6382@16382 master - 0 1608355981192 2 connected 8da363b93f4aedb3039eccdec8483b442e4d98e5 172.16.1.83:6386@16386 master - 0 1608355980000 5 connected #查看集群状态 [root@redis01 ~]# redis-cli -a 1 -p 6381 cluster info Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe. cluster_state:fail #不可用 cluster_slots_assigned:0 #卡槽0个 cluster_slots_ok:0 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 #集群节点已变成6个 cluster_size:0 cluster_current_epoch:5 cluster_my_epoch:1 cluster_stats_messages_ping_sent:759 cluster_stats_messages_pong_sent:764 cluster_stats_messages_meet_sent:5 cluster_stats_messages_sent:1528 cluster_stats_messages_ping_received:764 cluster_stats_messages_pong_received:764 cluster_stats_messages_received:1528 |
8.分配数据槽
1）编写脚本
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 |
#在redis01编写脚本 [root@redis01 ~]# vim /usr/local/redis/cluster/addslotes.sh #!/bin/bash start=$1 end=$2 port=$3 IP="172.16.1.81" password=$4 for i in `seq ${start} ${end}` do /usr/local/redis/bin/redis-cli -h ${IP} -a $4 -p $3 cluster addslots $i done ########################################################################### #在redis02编写脚本 [root@redis02 ~]# vim /usr/local/redis/cluster/addslotes.sh #!/bin/bash start=$1 end=$2 port=$3 IP="172.16.1.82" password=$4 for i in `seq ${start} ${end}` do /usr/local/redis/bin/redis-cli -h ${IP} -a $4 -p $3 cluster addslots $i done ########################################################################### #在redis03编写脚本 [root@redis03 ~]# vim /usr/local/redis/cluster/addslotes.sh #!/bin/bash start=$1 end=$2 port=$3 IP="172.16.1.83" password=$4 for i in `seq ${start} ${end}` do /usr/local/redis/bin/redis-cli -h ${IP} -a $4 -p $3 cluster addslots $i done ########################################################################## #部署在一台可以用以下脚本 #!/bin/bash start=$1 end=$2 port=$3 IP="172.16.1.81" password=$4 for i in `seq ${start} ${end}` do /usr/local/redis/bin/redis-cli -h ${IP} -a $4 -p $3 cluster addslots $i done |
2）执行脚本
|
1 2 3 4 5 6 7 8 9 10 11 |
#redis01执行脚本 [root@redis01 ~]# chmod +x /usr/local/redis/cluster/addslotes.sh [root@redis01 ~]# sh /usr/local/redis/cluster/addslotes.sh 1 5461 6381 1 #redis02执行脚本 [root@redis02 ~]# chmod +x /usr/local/redis/cluster/addslotes.sh [root@redis02 ~]# sh /usr/local/redis/cluster/addslotes.sh 5462 10922 6382 1 #redis03执行脚本 [root@redis03 ~]# chmod +x /usr/local/redis/cluster/addslotes.sh [root@redis03 ~]# sh /usr/local/redis/cluster/addslotes.sh 10923 16383 6383 1 |
3）查看分配卡槽的结果
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 |
[root@redis03 ~]# redis-cli -a 1 -h 172.16.1.83 -p 6383 Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe. 172.16.1.83:6383> CLUSTER INFO cluster_state:ok cluster_slots_assigned:16383 cluster_slots_ok:16383 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:5 cluster_my_epoch:4 cluster_stats_messages_ping_sent:8127 cluster_stats_messages_pong_sent:8086 cluster_stats_messages_sent:16213 cluster_stats_messages_ping_received:8085 cluster_stats_messages_pong_received:8127 cluster_stats_messages_meet_received:1 cluster_stats_messages_received:16213 172.16.1.83:6383> [root@redis01 /usr/local/redis/cluster]# redis-cli -a 1 -p 6381 cluster nodes bad625528f48ad23e088081065a868037a039f00 172.16.1.81:6382@16382 master - 0 1608290357201 1 connected 5462-10922 # 数据槽区间 b4a792c45576350ecec01e41b4e75fe56e6477fa 172.16.1.81:6383@16383 master - 0 1608290356000 2 connected 10923-16383 # 数据槽区间 f84e056919622b9455405da125059de31d2fc1ec 172.16.1.81:6381@16381 myself,master - 0 1608290355000 0 connected 1-5461 # 数据槽区间 |
9.绑定主从关系
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 |
格式： redis-cli -a [密码] -p [端口] cluster replicate [主节点ID] 查看主节点ID redis-cli -a [密码] -p [端口] cluster nodes #把6384绑定到[主节点ID] [root@redis01 ~]# redis-cli -h 172.16.1.81 -p 6384 -a 1 cluster replicate 7a15703fe4fc2119ab545e83178254e41af612fd OK #把6385绑定到[主节点ID] [root@redis01 ~]# redis-cli -h 172.16.1.82 -p 6385 -a 1 cluster replicate 3c5ae9a66a1505a75873e0eb128fb70481f11105 OK #把6386绑定到[主节点ID] [root@redis03 ~]# redis-cli -h 172.16.1.83 -p 6386 -a 1 cluster replicate d721544df39cdd192ce31eaf062ead33ce4759ba OK |
10.再次查看
|
1 2 3 4 5 6 7 8 9 |
[root@redis03 ~]# redis-cli -h 172.16.1.81 -p 6381 -a 1 CLUSTER NODES Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe. d721544df39cdd192ce31eaf062ead33ce4759ba 172.16.1.83:6383@16383 master - 0 1608362698141 4 connected 10923-16383 d003d10db9d88acd4cc486d8a12a6a733f7b7ec3 172.16.1.81:6384@16384 slave 7a15703fe4fc2119ab545e83178254e41af612fd 0 1608362698000 1 connected eafcb901a751f345dcb63beb3117ab3ec136dae8 172.16.1.82:6385@16385 slave 3c5ae9a66a1505a75873e0eb128fb70481f11105 0 1608362699148 2 connected 7a15703fe4fc2119ab545e83178254e41af612fd 172.16.1.81:6381@16381 myself,master - 0 1608362698000 1 connected 1-5461 3c5ae9a66a1505a75873e0eb128fb70481f11105 172.16.1.82:6382@16382 master - 0 1608362698645 2 connected 5462-10922 8da363b93f4aedb3039eccdec8483b442e4d98e5 172.16.1.83:6386@16386 slave d721544df39cdd192ce31eaf062ead33ce4759ba 0 1608362698000 4 connected |
11.测试集群
|
1 2 3 4 5 6 |
[root@redis01 /usr/local/redis/cluster]# redis-cli -c -a 1 -p 6381 127.0.0.1:6381> set a b -> Redirected to slot [15495] located at 172.16.1.81:6383 OK -c #以集群模式打开客户端 |