does django-redis support retry on TimeOut Error while reading data?

[prasad5141]

I am using django-redis package version (4.12.1),

I am getting a TimeOut Exception(Intermittent issue) while reading data from the redis cache, I would like to Retry for some N number of time to fetch the data

I tried using these settings but retry is not happening "REDIS_CLIENT_KWARGS": {"retry_on_timeout": True}

Please let me know if any other settings are available to retry while reading data from redis using django-redis package.

[barkhabol]

@prasad5141 Were you able to execute the retry logic for n number of times by any chance? Thanks in advance.

[esseti]

Is there any news on how to use the retry?

[WisdomPill]

sorry for the late reply, retry is not supported

[prasad5141]

Hi @WisdomPill ,

Thanks for your reply, Shall I implement Retry?

Plan of Implementation:

• User will specify the number of times to retry while reading data from Redis. (it will be in settings.py)
• Code should retry a user-specified number of times, if still not able to get the data after the specified count will throw an error max retry exceeded

Please suggest to me if you have any other plan for retry, and I will try to implement it.

[WisdomPill]

okay, but by default the number of retries should be zero to not change the default behaviour

[esseti]

It could be a set of settings within the redis configuration.

one starting point could be to "copy" what redis-py does: https://redis-py.readthedocs.io/en/stable/retry.html

so the setting could look like:

CACHES = {
    "default": {
        "BACKEND": "django_redis.cache.RedisCache",
        "LOCATION": "redis://127.0.0.1:6379/1",
        "OPTIONS": {
            "RETRY": 3
            "RETRY_ON_ERROR":[...]
        }
    }
}

Still. not sure what could be supported. A retry for starting will be lovely.

[esseti]

any update? @prasad5141

[amingilani]

@esseti Mind if I take this over? I have some void time coming up and would like to contribute to this package :)

[amingilani]

I'd love some clarifications:

Redis client supports 3 parameters to configure the retry behaviour:

retry: Retry instance with a Backoff strategy and the max number of retries

retry_on_error: list of Exceptions to retry on

retry_on_timeout: if True, retry on TimeoutError only

Does this signature sound fine? I can create a Retry class along with a few Backoff classes?

[esseti]

I just saw this: where you able to implement it ?

[amingilani]

@esseti I haven't implemented it, I'm still waiting on clarification for how the interface should look like. Let me know when you can please.

[esseti]

Not sure what i can/cannot confirm, but just mimic the parameters of the client should work. that is:

• Retry: int: default 1
• RetryBackoff: one of the https://redis-py.readthedocs.io/en/stable/backoff.html#backoff-label or the default: NoBackoff()
• retry_on_error: bool
• retry_on_timeout: bool

[WisdomPill]

it is something that has to be setup at the connection factory level, following the documentation of redis-py it seems to me that it can be done by adding the right parameters here.

So I changed my mind, maybe it is supported, but I didn't try it yet, I think it can be already done using some configuration like the following

from redis.backoff import ExponentialBackoff
from redis.retry import Retry

retry = Retry(ExponentialBackoff(), 3)
retry_on_error=[BusyLoadingError, ConnectionError, TimeoutError]
CACHES = {
    "default": {
        "BACKEND": "django_redis.cache.RedisCache",
        "LOCATION": "redis://127.0.0.1:6379/1",
        "OPTIONS": {
            "REDIS_CLIENT_KWARGS": {
               "retry": retry,
               "retry_on_error": retry_on_error
            }
        }
    }
}

or for timeouts only

from redis.backoff import ExponentialBackoff
from redis.retry import Retry

retry = Retry(ExponentialBackoff(), 3)
CACHES = {
    "default": {
        "BACKEND": "django_redis.cache.RedisCache",
        "LOCATION": "redis://127.0.0.1:6379/1",
        "OPTIONS": {
            "REDIS_CLIENT_KWARGS": {
               "retry": retry,
               "retry_on_timeout": True
            }
        }
    }
}

to make it easier and cleaner we could override REDIS_CLIENT_KWARGS based on OPTIONS like @esseti suggested.

to start @amingilani I would try to write some tests mimicking the redis-py ones

[akshetpandey]

This already works, but the documentation is wrong. You have to use CONNECTION_POOL_KWARGS instead of REDIS_CLIENT_KWARGS.