Redis sentinel

[ile]

New here.

How to configure Redis sentinel, which requires a separate config file for each node, and even knowledge of IP addresses (as I believe)?

If someone has a better solution for high availability Redis, that's also welcomed.

[scart88]

Hey @ile,

I've been working on a solution for High Availability Redis using 1 master, 2 replicas and 3 sentinels.

It's not a perfect solution, so if anyone is interested in improving it, that's welcomed.

In this setup, I'm using 3 Proxmox nodes connected in a Private VLAN/28 and my VMs can talk/ping to each other using private IPv4 addresses, however, you are free to use any cloud VM that has a VPC.

# Use accessory services (secrets come from .env).
accessories:
  redis_master:
    image: redis:7.0
    host: 10.4.0.30
    port: "10.4.0.30:6379:6379"
    add-host: 10.4.0.30
    cmd: 'redis-server /data/redis.conf'
    files:
      - config/redis/redis.conf:/redis_master/redis.conf
    directories:
      - redis_master:/data
  redis_replica_1:
    image: redis:7.0
    host: 10.4.0.31
    port: "10.4.0.31:6379:6379"
    add-host: 10.4.0.31
    cmd: 'redis-server /data/redis.conf' # 'redis-server --appendonly yes --replicaof 10.4.0.30 6379'
    files:
      - config/redis/replica_1.conf:/redis_replica_1/redis.conf
    directories:
      - redis_replica_1:/data
  redis_replica_2:
    image: redis:7.0
    host: 10.4.0.32
    port: "10.4.0.32:6379:6379"
    add-host: 10.4.0.32
    cmd: 'redis-server /data/redis.conf' #'redis-server --appendonly yes --replicaof 10.4.0.30 6379'
    files:
      - config/redis/replica_2.conf:/redis_replica_2/redis.conf
    directories:
      - redis_replica_2:/data

  redis_sentinel_1:
    image: redis:7.0
    host: 10.4.0.33
    port: "10.4.0.33:26379:26379"
    add-host: 10.4.0.33
    cmd: 'redis-sentinel /data/sentinel.conf'
    files:
      - config/redis/sentinel_1/sentinel.conf:/redis_sentinel_1/sentinel.conf
    directories:
      - redis_sentinel_1:/data

  redis_sentinel_2:
    image: redis:7.0
    host: 10.4.0.34
    port: "10.4.0.34:26379:26379"
    add-host: 10.4.0.34
    cmd: 'redis-sentinel /data/sentinel.conf'
    files:
      - config/redis/sentinel_2/sentinel.conf:/redis_sentinel_2/sentinel.conf
    directories:
      - redis_sentinel_2:/data

  redis_sentinel_3:
    image: redis:7.0
    host: 10.4.0.35
    port: "10.4.0.35:26379:26379"
    add-host: 10.4.0.35
    cmd: 'redis-sentinel /data/sentinel.conf'
    files:
      - config/redis/sentinel_3/sentinel.conf:/redis_sentinel_3/sentinel.conf
    directories:
      - redis_sentinel_3:/data

This requires you to create a few config files.

Create the following files and folders in your MRSK config folder:

Redis Master

1. config/redis/redis.conf

bind 0.0.0.0 ::
tcp-backlog 511
timeout 0
tcp-keepalive 300
daemonize no
supervised no
pidfile /var/run/redis.pid
appendonly yes

Redis Replicas

1. config/redis/replica_1.conf

bind 0.0.0.0 ::
tcp-backlog 511
timeout 0
tcp-keepalive 300
daemonize no
supervised no
pidfile /var/run/redis.pid
appendonly yes
replicaof 10.4.0.30 6379
replica-announce-ip 10.4.0.31

2. config/redis/replica_2.conf

bind 0.0.0.0 ::
tcp-backlog 511
timeout 0
tcp-keepalive 300
daemonize no
supervised no
pidfile /var/run/redis.pid
appendonly yes
replicaof 10.4.0.30 6379
replica-announce-ip 10.4.0.32

Sentinel

1. config/redis/sentinel_1/sentinel.conf

sentinel monitor mymaster 10.4.0.30 6379 2
sentinel announce-ip 10.4.0.33
sentinel announce-port 26379

protected-mode no
port 26379
daemonize no
pidfile /var/run/redis-sentinel.pid
logfile ""
dir /tmp

acllog-max-len 128

# sentinel monitor mymaster 127.0.0.1 6379 2
# sentinel down-after-milliseconds mymaster 30000
# sentinel parallel-syncs mymaster 1
# sentinel failover-timeout mymaster 180000
sentinel deny-scripts-reconfig yes
SENTINEL resolve-hostnames no
SENTINEL announce-hostnames no
# SENTINEL master-reboot-down-after-period mymaster 0

2. config/redis/sentinel_2/sentinel.conf

sentinel monitor mymaster 10.4.0.30 6379 2
sentinel announce-ip 10.4.0.34
sentinel announce-port 26379

....

3. config/redis/sentinel_3/sentinel.conf

sentinel monitor mymaster 10.4.0.30 6379 2
sentinel announce-ip 10.4.0.35
sentinel announce-port 26379

....

irb(main):002:1* SENTINELS = [
                                           { host: "10.4.0.33", port: 26379 },
                                           { host: "10.4.0.34", port: 26379 }, 
                                           { host: "10.4.0.35", port: 26379}]
                                           
=> [{:host=>"10.4.0.33", :port=>26379}, {:host=>"10.4.0.34", :port=>26379}, {:host=>"10.4.0.35", :port=>26379}]
irb(main):004:0> redis = Redis.new(name: "mymaster", sentinels: SENTINELS, role: :master)
=> #<Redis client v5.0.6 for redis://10.4.0.31:6379/0>
irb(main):005:0> redis.ping
=> "PONG"

If anyone have better knowledge and can improve this config please feel free to post it here.

Thank you!

[ile]

Thank you for the whole solution, I will go with this.