Redis Cluster High Availability and Failover Guide

Introduction: why Redis high availability matters now
Official documentation and primary sources
Sentinel vs Cluster mode comparison
- Architecture comparison table
- When to choose which
Configuring and operating Sentinel
Configuring and operating Cluster mode
Failure simulation and recovery script
Operational pitfalls and failure cases
Performance tuning checklist
- Network and OS level
- Redis level
Production operations checklist
Monitoring and alerting setup
Replication in depth: PSYNC and partial synchronization
Persistence strategy: RDB vs AOF
Conclusion
References

Introduction: why Redis high availability matters now

Redis has established itself as core infrastructure for modern applications — an in-memory data store used for caching, session management, real-time leaderboards, message queues and more. As of 2026, with microservice architectures now commonplace, the dependency on Redis has grown further, and it is common for a single Redis instance failure to translate directly into downtime for the whole system.

A high-availability Redis setup is essential in the following situations in particular:

Session store: if Redis fails, every user is logged out at once
Distributed locks: if Redis goes down, concurrency control fails and data consistency is damaged
Real-time cache: a surge of cache misses overloads the database (cache stampede)
Event streaming: Redis Streams based pipelines stop

This article compares Redis's two high-availability strategies, Sentinel and Cluster mode, in depth, and covers how to configure them in a real production environment, the failure detection and automatic failover mechanisms, and practical troubleshooting cases.

Official documentation and primary sources

The Redis official documentation and primary sources referenced in this article are as follows:

Redis Sentinel Documentation - https://redis.io/docs/management/sentinel/
Redis Cluster Specification - https://redis.io/docs/reference/cluster-spec/
Redis Cluster Tutorial - https://redis.io/docs/management/scaling/
Redis Replication - https://redis.io/docs/management/replication/
Redis Persistence (RDB/AOF) - https://redis.io/docs/management/persistence/
redis-py Cluster Client - https://redis-py.readthedocs.io/en/stable/clustering.html

Sentinel vs Cluster mode comparison

Architecture comparison table

Item	Sentinel	Cluster
Purpose	High availability (HA)	High availability + horizontal scaling
Minimum nodes	Sentinel 3 + Master 1 + Replica 1	Master 3 + Replica 3 (6 in total)
Data distribution	Not possible (single master)	Automatic hash-slot distribution (16384 slots)
Write scaling	Not possible	Multi-master supported
Read scaling	READONLY on replicas	READONLY on replicas
Failure detection	Sentinel quorum vote	Cluster bus gossip protocol
Failover	Executed by the Sentinel leader	Replica promotes itself (majority vote)
Client complexity	Must be Sentinel-aware	Must handle MOVED/ASK redirection
Multi-key commands	Unrestricted	Only within the same hash slot
Maximum data size	Limited by single-node memory	Scales in proportion to the node count
Operational complexity	Moderate	High
Where it fits	HA for a single dataset	Large data volume + high throughput

When to choose which

# Decision flow
#
# Q1: does the entire dataset fit in the memory of a single node (say 64GB)?
#   YES -> go to Q2
#   NO  -> Cluster mode is required
#
# Q2: is write throughput satisfied by a single master?
#   YES -> Sentinel mode recommended
#   NO  -> Cluster mode needed
#
# Q3: are multi-key transactions (MULTI/EXEC) frequent?
#   YES -> prefer Sentinel (Cluster requires hash-tag design)
#   NO  -> either mode works
#
# Q4: how large is the ops team, and how much infrastructure experience does it have?
#   small     -> Sentinel (simpler to operate)
#   dedicated -> Cluster (complex but powerful)

Configuring and operating Sentinel

The Sentinel configuration file

Sentinel must be deployed as at least 3 independent processes. Each Sentinel monitors the master and, when a failure occurs, performs failover through quorum-based agreement.

# sentinel.conf - basic configuration
port 26379
daemonize yes
logfile "/var/log/redis/sentinel.log"
dir "/var/lib/redis/sentinel"

# Master monitoring settings
# sentinel monitor <master-name> <ip> <port> <quorum>
sentinel monitor mymaster 10.0.1.10 6379 2

# Time after which the master counts as unresponsive (milliseconds)
sentinel down-after-milliseconds mymaster 5000

# Failover timeout (milliseconds)
# If failover does not complete within this time, it is retried
sentinel failover-timeout mymaster 60000

# Number of replicas that sync to the new master at the same time
# Lower is safer (a replica cannot serve reads while syncing)
sentinel parallel-syncs mymaster 1

# When authentication is required
sentinel auth-pass mymaster StrongP@ssw0rd!

# Sentinel's own authentication (Redis 6.2+)
requirepass SentinelP@ss!

# Notification script (run when a failure occurs)
sentinel notification-script mymaster /opt/redis/notify.sh

# Script run after failover completes
sentinel client-reconfig-script mymaster /opt/redis/reconfig.sh

Master Redis configuration

# redis.conf - master settings
bind 0.0.0.0
port 6379
requirepass StrongP@ssw0rd!
masterauth StrongP@ssw0rd!

# Replication settings
replica-serve-stale-data yes
replica-read-only yes
repl-diskless-sync yes
repl-diskless-sync-delay 5

# Persistence settings
save 900 1
save 300 10
save 60 10000
appendonly yes
appendfsync everysec

# Memory settings
maxmemory 4gb
maxmemory-policy allkeys-lru

# Network tuning
tcp-backlog 511
tcp-keepalive 300
timeout 0

How failover works

Sentinel's failure detection and failover proceed in the following stages:

# === Stage 1: SDOWN (Subjective Down) ===
# An individual Sentinel detects no master response for down-after-milliseconds
# At this point it is only that one Sentinel's subjective judgement

# Check the Sentinel log
# +sdown master mymaster 10.0.1.10 6379

# === Stage 2: ODOWN (Objective Down) ===
# When at least `quorum` Sentinels agree on SDOWN, it becomes ODOWN
# This is the objective failure decision

# +odown master mymaster 10.0.1.10 6379 #quorum 2/2

# === Stage 3: Sentinel leader election ===
# A Raft-based algorithm elects the leader that will perform the failover
# A majority of votes is required
# 3 Sentinels -> at least 2 votes needed

# === Stage 4: replica selection ===
# The leader Sentinel picks the best replica
# Selection criteria (in priority order):
#   1. the node with the lowest replica-priority value (0 is excluded)
#   2. the node with the largest replication offset (most up-to-date data)
#   3. the node whose run ID sorts first lexicographically

# === Stage 5: failover execution ===
# Send REPLICAOF NO ONE to the selected replica
# Point the remaining replicas at the new master
# Sentinel configuration files are updated automatically

# Monitor failover status
redis-cli -p 26379 SENTINEL masters
redis-cli -p 26379 SENTINEL get-master-addr-by-name mymaster
redis-cli -p 26379 SENTINEL replicas mymaster
redis-cli -p 26379 SENTINEL sentinels mymaster

Connecting to Sentinel from Python

import redis
from redis.sentinel import Sentinel

# Connect with the list of Sentinel instances
sentinel = Sentinel(
    [
        ("10.0.1.20", 26379),
        ("10.0.1.21", 26379),
        ("10.0.1.22", 26379),
    ],
    socket_timeout=0.5,
    sentinel_kwargs={"password": "SentinelP@ss!"},
)

# Look up the master address
master_host, master_port = sentinel.discover_master("mymaster")
print(f"Current master: {master_host}:{master_port}")

# Master connection (for writes)
master = sentinel.master_for(
    "mymaster",
    socket_timeout=0.5,
    password="StrongP@ssw0rd!",
    db=0,
)

# Replica connection (for reads)
replica = sentinel.slave_for(
    "mymaster",
    socket_timeout=0.5,
    password="StrongP@ssw0rd!",
    db=0,
)

# Writes go to the master
master.set("session:user:1001", "active")

# Reads go to a replica (spreading the read load)
session = replica.get("session:user:1001")
print(f"Session status: {session}")

# On failover, reconnects automatically to the new master
# redis-py's SentinelConnectionPool handles this on its own
try:
    master.set("key", "value")
except redis.exceptions.ConnectionError:
    # Sentinel reports the new master -> automatic reconnect
    print("Failover detected, reconnecting...")
    master.set("key", "value")  # the retry connects to the new master

Configuring and operating Cluster mode

Cluster node configuration

# redis-cluster.conf - settings common to every node
port 7000
cluster-enabled yes
cluster-config-file nodes-7000.conf
cluster-node-timeout 5000

# Cluster bus port (default: port + 10000)
# cluster-port 17000

# Authentication
requirepass ClusterP@ss!
masterauth ClusterP@ss!

# Replication settings
replica-serve-stale-data yes
replica-read-only yes
repl-diskless-sync yes

# Persistence
appendonly yes
appendfsync everysec
save 900 1
save 300 10

# Memory
maxmemory 8gb
maxmemory-policy allkeys-lru

# Network
bind 0.0.0.0
tcp-backlog 511
tcp-keepalive 300

# Whether to take the whole cluster down when any part of the data is unavailable
# yes: reject everything if even one slot is uncovered
# no: keep serving keys in the covered slots (recommended)
cluster-require-full-coverage no

# Automatically move a spare replica from another master to a master that has none
cluster-allow-replica-migration yes

Cluster creation and management commands

# Create a cluster from 6 nodes (3 masters + 3 replicas)
redis-cli -a ClusterP@ss! --cluster create \
  10.0.1.10:7000 10.0.1.11:7001 10.0.1.12:7002 \
  10.0.1.10:7003 10.0.1.11:7004 10.0.1.12:7005 \
  --cluster-replicas 1

# Check cluster information
redis-cli -a ClusterP@ss! -c -h 10.0.1.10 -p 7000 CLUSTER INFO
# cluster_state:ok
# cluster_slots_assigned:16384
# cluster_slots_ok:16384
# cluster_size:3
# cluster_known_nodes:6

# List the nodes
redis-cli -a ClusterP@ss! -c -h 10.0.1.10 -p 7000 CLUSTER NODES
# <node-id> 10.0.1.10:7000@17000 myself,master - 0 0 1 connected 0-5460
# <node-id> 10.0.1.11:7001@17001 master - 0 1234567890 2 connected 5461-10922
# <node-id> 10.0.1.12:7002@17002 master - 0 1234567890 3 connected 10923-16383
# ...

# Check slot distribution
redis-cli -a ClusterP@ss! -c -h 10.0.1.10 -p 7000 CLUSTER SLOTS

# Check cluster health
redis-cli -a ClusterP@ss! --cluster check 10.0.1.10:7000

# === Adding a node ===
# Add a new master node
redis-cli -a ClusterP@ss! --cluster add-node \
  10.0.1.13:7006 10.0.1.10:7000

# Add a new replica node (as a replica of a specific master)
redis-cli -a ClusterP@ss! --cluster add-node \
  10.0.1.13:7007 10.0.1.10:7000 \
  --cluster-slave --cluster-master-id <master-node-id>

# === Resharding ===
# Move slots from an existing master to a new master
redis-cli -a ClusterP@ss! --cluster reshard 10.0.1.10:7000 \
  --cluster-from <source-node-id> \
  --cluster-to <target-node-id> \
  --cluster-slots 4096 \
  --cluster-yes

# Automatic rebalancing (even slot distribution)
redis-cli -a ClusterP@ss! --cluster rebalance 10.0.1.10:7000 \
  --cluster-threshold 2 \
  --cluster-use-empty-masters

# === Removing a node ===
# Move the slots to other nodes first, then remove it
redis-cli -a ClusterP@ss! --cluster del-node \
  10.0.1.10:7000 <node-id-to-remove>

Hash slots and hash tags

Redis Cluster uses the CRC16 hash function to map a key to one of 16384 slots. To use multi-key commands (MGET, MSET, pipelines), the keys involved must live in the same slot.

import redis
from redis.cluster import RedisCluster

# Cluster client connection
rc = RedisCluster(
    startup_nodes=[
        {"host": "10.0.1.10", "port": 7000},
        {"host": "10.0.1.11", "port": 7001},
        {"host": "10.0.1.12", "port": 7002},
    ],
    password="ClusterP@ss!",
    decode_responses=True,
    # MOVED/ASK redirection is handled automatically
    skip_full_coverage_check=True,
)

# Basic usage
rc.set("user:1001:name", "Kim")
rc.set("user:1001:email", "kim@example.com")

# Use a hash tag to place keys in the same slot
# The string inside the braces determines the slot
rc.set("{user:1001}:name", "Kim")
rc.set("{user:1001}:email", "kim@example.com")
rc.set("{user:1001}:session", "abc123")

# Now a single MGET can fetch them (same slot)
values = rc.mget(
    "{user:1001}:name",
    "{user:1001}:email",
    "{user:1001}:session",
)
print(values)  # ['Kim', 'kim@example.com', 'abc123']

# Pipelines also work within the same slot
pipe = rc.pipeline()
pipe.hset("{order:5001}:info", "status", "pending")
pipe.hset("{order:5001}:info", "amount", "15000")
pipe.expire("{order:5001}:info", 3600)
pipe.execute()

# Check the slot
slot = rc.cluster_keyslot("{user:1001}:name")
print(f"Slot: {slot}")

# Query cluster information
info = rc.cluster_info()
print(f"Cluster state: {info['cluster_state']}")
print(f"Known nodes: {info['cluster_known_nodes']}")

The Cluster failover mechanism

Redis Cluster performs failover through the cluster nodes themselves, without Sentinel.

# === Cluster failure detection flow ===

# 1. Gossip protocol
#    Every node uses the cluster bus (port+10000) to
#    PING a random node every second and check for a PONG

# 2. PFAIL (Probable Fail)
#    When no PONG arrives within cluster-node-timeout,
#    the node is marked PFAIL (subjective judgement)

# 3. FAIL (confirmed failure)
#    When a majority of masters agree on PFAIL, it becomes FAIL
#    A FAIL message is broadcast across the cluster

# 4. Replica promotion
#    The failed master's replica starts an election
#    The other masters vote (a majority is required)
#    The winning replica is promoted to the new master

# === Manual failover ===
# Run on the replica node (for planned maintenance)
redis-cli -a ClusterP@ss! -h 10.0.1.10 -p 7003 CLUSTER FAILOVER

# TAKEOVER: force promotion without the other masters' consent (emergencies only)
redis-cli -a ClusterP@ss! -h 10.0.1.10 -p 7003 CLUSTER FAILOVER TAKEOVER

# Check cluster status after failover
redis-cli -a ClusterP@ss! --cluster check 10.0.1.10:7000

Failure simulation and recovery script

Before deploying to production, always simulate the failure scenarios and validate the recovery procedure.

#!/bin/bash
# failover-simulation.sh - Redis failure simulation and verification script

REDIS_CLI="redis-cli -a ClusterP@ss!"
MASTER_HOST="10.0.1.10"
MASTER_PORT=7000

echo "=== Step 1: check the current cluster state ==="
$REDIS_CLI -c -h $MASTER_HOST -p $MASTER_PORT CLUSTER INFO | head -5
$REDIS_CLI -c -h $MASTER_HOST -p $MASTER_PORT CLUSTER NODES

echo ""
echo "=== Step 2: insert test data ==="
for i in $(seq 1 100); do
  $REDIS_CLI -c -h $MASTER_HOST -p $MASTER_PORT \
    SET "test:failover:$i" "value_$i" EX 300 > /dev/null 2>&1
done
echo "inserted 100 test keys"

echo ""
echo "=== Step 3: kill the master process (failure simulation) ==="
# Warning: never run this in production
$REDIS_CLI -c -h $MASTER_HOST -p $MASTER_PORT DEBUG SLEEP 30 &
# Or terminate the actual process
# ssh $MASTER_HOST "redis-cli -p $MASTER_PORT -a ClusterP@ss! SHUTDOWN NOSAVE"

echo "starting master node failure simulation"
echo "waiting for cluster-node-timeout (5 seconds)..."
sleep 10

echo ""
echo "=== Step 4: check cluster state after failover ==="
# Check the cluster state through another node
$REDIS_CLI -c -h 10.0.1.11 -p 7001 CLUSTER INFO | head -5
$REDIS_CLI -c -h 10.0.1.11 -p 7001 CLUSTER NODES

echo ""
echo "=== Step 5: verify data integrity ==="
FOUND=0
MISSING=0
for i in $(seq 1 100); do
  RESULT=$($REDIS_CLI -c -h 10.0.1.11 -p 7001 \
    GET "test:failover:$i" 2>/dev/null)
  if [ -n "$RESULT" ]; then
    FOUND=$((FOUND + 1))
  else
    MISSING=$((MISSING + 1))
  fi
done
echo "verification result: keys found=$FOUND, keys lost=$MISSING"

echo ""
echo "=== Step 6: recover the failed node ==="
# Rejoin the failed node as a replica
echo "restarting the failed node makes it rejoin automatically as a replica."
echo "manual check: redis-cli --cluster check 10.0.1.11:7001"

Operational pitfalls and failure cases

Case 1: split brain

Split brain is the situation where a network partition leaves two or more nodes acting as master at the same time. It leads to data inconsistency and data loss.

How it happens: 2 of the 3 Sentinels lose network connectivity to the master and elect a new master, while the original master is still accepting client writes

Preventive settings:

# redis.conf - split brain prevention
# Allow writes only while at least N replicas are connected
min-replicas-to-write 1

# Maximum replica lag (seconds)
# If no ACK arrives for longer than this, the connection is treated as lost
min-replicas-max-lag 10

# What the settings above mean:
# If the master is cut off from the network,
# it has 0 connected replicas and therefore refuses writes
# -> preventing data written to the old master from conflicting with the new master

A real failure case: at company A, a temporary partition occurred during a network equipment swap in production. Because min-replicas-to-write had not been set, both masters accepted writes, and after failover completed the data on the original master (roughly 30 seconds' worth) was lost. The setting above was applied afterwards to prevent a recurrence.

Case 2: running out of memory (OOM)

# The problem: maxmemory was left unset and Redis consumed all system memory
# The Linux OOM killer terminated the Redis process

# Commands to check
redis-cli INFO memory
# used_memory_human:12.45G
# used_memory_peak_human:14.23G
# maxmemory_human:0B           <- no limit! dangerous!
# mem_fragmentation_ratio:1.35

# Preventive settings
# redis.conf
maxmemory 8gb
maxmemory-policy allkeys-lru

# Memory usage monitoring script
# crontab -e
# */5 * * * * /opt/redis/check_memory.sh

# check_memory.sh example
# USED=$(redis-cli INFO memory | grep used_memory_bytes | cut -d: -f2 | tr -d '\r')
# MAX=8589934592  # 8GB
# RATIO=$(echo "scale=2; $USED * 100 / $MAX" | bc)
# if (( $(echo "$RATIO > 80" | bc -l) )); then
#   echo "WARNING: Redis memory usage at ${RATIO}%" | \
#     mail -s "Redis Memory Alert" ops@company.com
# fi

Case 3: timeouts during Cluster resharding

When a large number of keys move during resharding, client requests are served through ASK redirection, and a temporary latency increase can occur at that point.

# Check cluster state during resharding
redis-cli -a ClusterP@ss! -c -h 10.0.1.10 -p 7000 CLUSTER NODES
# Slots shown as IMPORTING/MIGRATING mean resharding is in progress

# Throttling the resharding rate (recommended in large environments)
redis-cli -a ClusterP@ss! --cluster reshard 10.0.1.10:7000 \
  --cluster-from <source-id> \
  --cluster-to <target-id> \
  --cluster-slots 500 \
  --cluster-timeout 10000 \
  --cluster-pipeline 100 \
  --cluster-yes

# Recovering from an abnormally interrupted resharding
redis-cli -a ClusterP@ss! --cluster fix 10.0.1.10:7000

Case 4: permission problems on the Sentinel configuration file

Sentinel rewrites its configuration file automatically during failover. If the file is not writable, the failover itself still succeeds, but on a Sentinel restart it refers to the old master information and connects to the wrong node.

# Check sentinel.conf permissions
ls -la /etc/redis/sentinel.conf
# -rw-r--r-- 1 redis redis 1234 Mar 14 10:00 sentinel.conf

# Grant write permission to the Redis user
chown redis:redis /etc/redis/sentinel.conf
chmod 640 /etc/redis/sentinel.conf

# Confirm that Sentinel is modifying the configuration file
# Check whether the master IP in sentinel.conf changed after failover
grep "sentinel monitor" /etc/redis/sentinel.conf

Performance tuning checklist

Network and OS level

# /etc/sysctl.conf - kernel parameter tuning

# Increase the TCP backlog size
net.core.somaxconn = 65535

# Allow memory overcommit (needed when Redis forks)
vm.overcommit_memory = 1

# Disable THP (Transparent Huge Pages)
# With THP, Redis memory usage can spike and latency can increase
# echo never > /sys/kernel/mm/transparent_hugepage/enabled

# Raise the file descriptor limit
# /etc/security/limits.conf
# redis soft nofile 65536
# redis hard nofile 65536

Redis level

# 1. Slow log settings (record commands taking 10ms or more)
slowlog-log-slower-than 10000
slowlog-max-len 128

# Check the slow log
redis-cli SLOWLOG GET 10
redis-cli SLOWLOG LEN
redis-cli SLOWLOG RESET

# 2. Enable latency monitoring
latency-monitor-threshold 100

# Check latency events
redis-cli LATENCY LATEST
redis-cli LATENCY HISTORY event-name

# 3. Client output buffer limits
# Prevents master memory from exploding when a replica is slow
client-output-buffer-limit replica 256mb 64mb 60

# 4. Connection pool settings (client side)
# maxclients defaults to 10000
maxclients 10000

Production operations checklist

Always go through the items below before deploying:

Configuration

Have you chosen appropriately between Sentinel and Cluster mode?
Do you meet the minimum node count (Sentinel 3+, Cluster 6+)?
Is authentication (requirepass, masterauth) configured on every node?
Are maxmemory and a suitable eviction policy set?
Is a persistence strategy (RDB/AOF) configured?
Are you preventing split brain with min-replicas-to-write?

Monitoring

Are you monitoring memory usage on each node?
Are you tracking replication lag?
Do you review the slow log regularly?
Are alerts configured for Sentinel/Cluster events?
Is cluster state (cluster_state) part of your health check?

Failure preparedness

Have you run a failover simulation and measured the recovery time?
Are the backup and restore procedures documented?
Have you tested the network partition scenario?
Are the configuration file write permissions correct on every node?
Does the client library support automatic reconnection on failover?

OS level

Is vm.overcommit_memory = 1 set?
Are Transparent Huge Pages disabled?
Is the file descriptor limit high enough?
Is net.core.somaxconn larger than Redis tcp-backlog?
Have you tuned vm.swappiness to keep swap usage to a minimum?

Monitoring and alerting setup

#!/usr/bin/env python3
"""Redis cluster health check and alerting script"""

import redis
from redis.cluster import RedisCluster
import smtplib
from email.mime.text import MIMEText
import json
import time


def check_cluster_health(hosts, password):
    """Check the overall state of the cluster."""
    alerts = []

    try:
        rc = RedisCluster(
            startup_nodes=[{"host": h, "port": p} for h, p in hosts],
            password=password,
            decode_responses=True,
            socket_timeout=3,
        )

        # 1. Check cluster state
        info = rc.cluster_info()
        if info.get("cluster_state") != "ok":
            alerts.append(
                f"CRITICAL: cluster_state = {info.get('cluster_state')}"
            )

        # 2. Check slot coverage
        slots_ok = int(info.get("cluster_slots_ok", 0))
        if slots_ok < 16384:
            alerts.append(
                f"CRITICAL: insufficient slot coverage - {slots_ok}/16384"
            )

        # 3. Check memory usage on each node
        for node in rc.cluster_nodes():
            node_info = rc.info(target_nodes=node)
            used = node_info.get("used_memory", 0)
            maxmem = node_info.get("maxmemory", 0)
            if maxmem > 0:
                usage_pct = (used / maxmem) * 100
                if usage_pct > 85:
                    alerts.append(
                        f"WARNING: node memory at {usage_pct:.1f}%"
                    )

        # 4. Check replication lag
        for node in rc.cluster_nodes():
            repl_info = rc.info(section="replication", target_nodes=node)
            lag = repl_info.get("master_repl_offset", 0)
            if lag > 1000000:  # lag of 1MB or more
                alerts.append(
                    f"WARNING: replication lag detected - offset lag: {lag}"
                )

    except redis.exceptions.ConnectionError as e:
        alerts.append(f"CRITICAL: cluster connection failed - {str(e)}")
    except Exception as e:
        alerts.append(f"ERROR: health check failed - {str(e)}")

    return alerts


def send_alert(alerts, smtp_config):
    """Send the alert email"""
    if not alerts:
        return

    body = "Redis cluster alert:\n\n"
    body += "\n".join(f"  - {a}" for a in alerts)
    body += f"\n\nchecked at: {time.strftime('%Y-%m-%d %H:%M:%S')}"

    msg = MIMEText(body)
    msg["Subject"] = f"[Redis Alert] {len(alerts)} issue(s) detected"
    msg["From"] = smtp_config["from"]
    msg["To"] = smtp_config["to"]

    with smtplib.SMTP(smtp_config["host"], smtp_config["port"]) as server:
        server.send_message(msg)


if __name__ == "__main__":
    CLUSTER_HOSTS = [
        ("10.0.1.10", 7000),
        ("10.0.1.11", 7001),
        ("10.0.1.12", 7002),
    ]

    alerts = check_cluster_health(CLUSTER_HOSTS, "ClusterP@ss!")
    if alerts:
        print("Issues detected:")
        for a in alerts:
            print(f"  {a}")
        # send_alert(alerts, SMTP_CONFIG)
    else:
        print("Cluster health: OK")

Replication in depth: PSYNC and partial synchronization

Redis replication splits into full synchronization (full sync) and partial synchronization (partial sync, PSYNC). When a replica briefly loses its connection and reconnects, sending only the delta via PSYNC avoids the cost of resending the entire dataset.

# Replication backlog settings (master)
# The memory buffer that makes partial synchronization possible
repl-backlog-size 128mb

# How long the backlog is kept (after every replica disconnects)
repl-backlog-ttl 3600

# Diskless replication (an advantage when the network is faster than the disk)
repl-diskless-sync yes
repl-diskless-sync-delay 5
repl-diskless-sync-period 0

# Check replication status
redis-cli INFO replication
# role:master
# connected_slaves:2
# slave0:ip=10.0.1.11,port=6379,state=online,offset=1234567,lag=0
# slave1:ip=10.0.1.12,port=6379,state=online,offset=1234567,lag=1
# master_replid:abc123...
# master_repl_offset:1234567
# repl_backlog_active:1
# repl_backlog_size:134217728

When PSYNC fails and a full sync happens, the master starts creating an RDB snapshot, and that process consumes a lot of CPU and memory. Setting repl-backlog-size large enough is the key. Setting it to at least the write rate per second (MB/s) multiplied by the expected disconnection time (seconds) is recommended.

Persistence strategy: RDB vs AOF

Item	RDB (snapshot)	AOF (Append Only File)
How it works	Periodic full snapshot	Records every write command
Data safety	Can lose data since the last snapshot	Minimal loss, per the fsync policy
File size	Small (compressed)	Large (a command log)
Recovery speed	Fast	Slow (commands are replayed)
Performance impact	Temporary stall on fork	I/O on every fsync
Recommended pairing	Use together with AOF	Use together with RDB

# Recommended: use RDB and AOF together
save 900 1
save 300 10
save 60 10000

appendonly yes
appendfsync everysec

# AOF rewrite trigger conditions
auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 64mb

# Mixed mode (Redis 4.0+, recommended)
# On AOF rewrite, an RDB-format preamble followed by the AOF log
aof-use-rdb-preamble yes

Conclusion

A highly available Redis setup demands a deeper understanding than simply switching Sentinel or Cluster on. The difference between quorum and majority, how hash slots are distributed, split brain prevention strategy, sizing the replication backlog, choosing a persistence strategy — every one of these is connected to the others.

Before deploying to production, always run a failure simulation, build out the monitoring, and go through every item on the checklist above. The min-replicas-to-write and maxmemory settings in particular are the ones that head off the most frequent causes of failure, so make sure you apply them.

References

Redis Sentinel Documentation - Sentinel architecture and configuration guide
Redis Cluster Specification - the Cluster protocol specification
Redis Cluster Tutorial - Cluster setup and operations tutorial
Redis Replication - the replication mechanism and PSYNC
Redis Persistence - RDB and AOF persistence strategies
redis-py Documentation - the official Python Redis client documentation
Redis Administration - operations and administration guide
Redis Latency Problems Troubleshooting - latency problem diagnosis guide