I believe there is something odd in the utils/retry.py decorator which should be put to attention. Not sure if it is intentional or not but at least I spent some time on it to discover the reason behind
the issue is that the delay time is not reset after a successful call. With this behaviour you could basically create an indefinite sleep period if you don’t use the max_delay property, already after a few calls if you have a reasonable sized backoff time.
as the RetryDelay class is not locally scoped the state will preserved basically within your worker class. This is not a major issue however, if you have a few failures your sleep time will increase very rapidly for future failed requests.
I believe we should reset the delay to the initial value after a successful call. I don’t think the max delay is fair in this one as it will then basically replace the whole idea behind incremental increase of the sleeping time.
I can make a PR but I would first better understand why this has been designed in this way. At least it should be documented I think.