Rxify: Retry with Exponential Backoff in RxJava

Published in

Over Engineering

3 min readMar 6, 2019

At Over, we faced an issue wherein the app was running OutOfMemory when trying to export a project with high-resolution images. This made us think of various approaches for retrying our export mechanism.

Whenever we think of retrying, there are quite a few options available to us. Create a custom solution with recursion perhaps or maybe an iterative approach. One could also come up with a solution with co-routines. It all depends upon the implementation of the crash-site. In our case, exportAt(scale) is the crash-site.

Fortunately (or unfortunately — whichever way you look at it 😜) our exportAt(scale) function is reactive. Even with Rx, for retrying we have multiple options to choose from depending upon the problem statement. Let us have a look at the problem statement then.

Problem:

Given functionexportAt(scale): Single<Result>, retry the function whenever Exception occurs. We also need to change the input scale to the function upon each retry. The assumption here is if we were not able to export our project at scale 1.0 then we can retry by reducing the scale either linearly ( 1.0 , 0.9 , 0.8 , 0.7 and so on) or exponentially ( 1.0 , 0.5 , 0.25 , 0.0625 and so on) depending upon the retry mechanism we end up choosing.

What are our options here? With Rx we have an operator called retry() which has a few overloaded options. Our problem demands that we change our original Single exportAt(scale) each time we want to retry() to take a different scale. None of the overloaded variations of the retry() operator would let us change our scale as a function of the number of times we are retrying (At least from what I could make out from the documentation 🤔). So, we need to think of another way.

Solution:

The solution I finally came up with makes use of various helpful operators (spells). Let us look at a few of the operators first before jumping to the final solution.

Range : Observable.range(0, 5)

It emits values 0, 1, 2, 3, 4 and 5.

ConcatMap :

ConcatMap operator is like flatMap() with two differences. First, concatMap() preserves order and second, it doesn’t subscribe to the next value until the first one completes.

TakeUntil : (`Take`um `Until`lum ;)

takeUntil(stopPredicateFunction) accepts values until the stopPredicateFunction resolves to true.

Here’s the full code-snippet with exponential backoff :

Observable.range emits values 0 to 5, where 5 is the max. number of times we wish to retry.
map() converts our input values into scale = 1/2^(input) i.e.1.0, 0.5, 0.25 …. (Reducing exponentially). You can provide it any function, depending upon the retry mechanism you choose. Linear for example could look like : scale = 1.0f — 0.1 * input producing values 1.0 , 0.9, 0.8, 0.7 and so on.
concatMap() subscribes to the next scale value only when one value completes. So, we will not subscribe to exportAt(0.9) until we are done processing exportAt(1.0).
Then, map() and onErrorReturn() wrap our exportAt(scale) output inside a Result wrapper which is a sealed class as follows :

sealed class Result {
    data class Success(val double: Double): Result()
    data class Failed(val error: Throwable): Result()
}

5. With takeUntil(Result.Success): We keep on retrying till our exportAt(scale) successfully exports the Project and thus emits a Result.Success.

6. Finally, with lastOrError() we take the last value which will be Result.Success in case we were successful in 5 attempts. Or we get our exportAt(scale) exception wrapped inside Result.Error in case we couldn’t succeed even in 5 attempts.

7. We can then show User appropriate feedback depending upon whether we were successful in exporting the project or not.

And with this, we are done!

This wraps up our approach to implementing exponential backoff with Rx. I am sure that there are alternative solutions to this problem, let me know if you find a different or better way in the comments below.

Hope it helps. Find me on twitter @ragdroid.