Skip to content

Testing Retry Logic Without Real Sleeps

Retry logic is easy to get subtly wrong — off-by-one attempt counts, backoff that does not grow, errors retried that should not be — and easy to leave untested because the tests are slow. A retry with 1, 2 and 4 second backoffs takes seven seconds to exercise for real; a handful of such tests turns a fast suite into a slow one. Measured with pytest: the real-time test took 7.01 s. Injecting a fake sleep that records delays, overriding tenacity's sleep with retry_with(sleep=...), and running on a virtual-time loop with the looptime plugin all ran the same scenario in milliseconds — three such tests together took 0.09 s — and asserted the exact delays, [1.0, 2.0, 4.0]. One obvious approach failed: monkeypatching asyncio.sleep in the module had no effect, because the retry function had bound asyncio.sleep as a default argument when it was defined; the test still took 7.01 s and saw no recorded delays. This guide makes retry logic testable and tests what matters.

Prerequisites

1. Make the sleep injectable

The simplest seam is a sleep parameter that defaults to asyncio.sleep. Tests pass a fake that records the delays and returns immediately:

async def call_with_retry(fn, attempts: int = 5, base: float = 1.0, sleep=asyncio.sleep):
    for i in range(attempts):
        try:
            return await fn()
        except Transient:
            if i == attempts - 1:
                raise
            await sleep(base * 2 ** i)


async def test_backoff_schedule():
    fn, calls = failing_times(3)
    delays: list[float] = []

    async def fake_sleep(seconds: float) -> None:
        delays.append(seconds)

    assert await call_with_retry(fn, sleep=fake_sleep) == "ok"
    assert delays == [1.0, 2.0, 4.0]
    assert calls["n"] == 4

The test asserts both the number of attempts and the exact schedule, and runs in microseconds. Recording delays is better than merely skipping them: a bug that stops the backoff from growing, or that sleeps after the final attempt, shows up as a wrong list. Jittered backoff needs a seeded random.Random passed in the same way, or assertions on ranges instead of exact values.

Verify: changing the backoff formula in the code under test makes the test fail.

Run time of one retry test (3 failures, 1-2-4 s backoff) 5 horizontal bars comparing real asyncio.sleep with the others. Run time of one retry test (3 failures, 1-2-4 s backoff) real asyncio.sleep 7.01 s monkeypatched module asyncio.sleep (no effect) 7.01 s, failed injected fake sleep ~ms tenacity retry_with(sleep=fake) ~ms looptime virtual clock ~ms pytest 9.1, pytest-asyncio; the three fast variants ran together in 0.09 s. Fake the clock, but fake it where the code actually reads it.

2. Know why monkeypatching asyncio.sleep can fail

Patching asyncio.sleep in the module under test looks equivalent, and was not:

async def test_monkeypatch(monkeypatch):
    delays = []

    async def fake_sleep(d, *a, **k):
        delays.append(d)

    monkeypatch.setattr("retrylib.asyncio.sleep", fake_sleep)   # patches the module attribute...
    await call_with_retry(fn)                                    # ...but the default arg kept the original
    assert delays == [1.0, 2.0, 4.0]                             # FAILED: [] == [...], test took 7.01 s

sleep=asyncio.sleep in the signature is evaluated once, when the function is defined, so the function keeps a reference to the original coroutine function no matter what the module attribute later points to. Patching works only where code looks the name up at call time (await asyncio.sleep(...) in the body). Prefer an explicit parameter or a clock object over patching module globals: it is visible in the signature and cannot silently miss.

Verify: each retry test asserts on recorded delays, so a patch that does not take effect fails the test instead of just making it slow.

3. Override tenacity's sleep for library-based retries

tenacity-decorated functions expose retry_with, which returns a copy of the function with different settings — including the sleep function:

@retry(stop=stop_after_attempt(5), wait=wait_exponential(multiplier=1, min=1, max=60),
       retry=retry_if_exception_type(Transient), reraise=True)
async def fetch(fn):
    return await fn()


async def test_tenacity_schedule():
    delays: list[float] = []

    async def fake_sleep(seconds: float) -> None:
        delays.append(seconds)

    patched = fetch.retry_with(sleep=fake_sleep)
    assert await patched(failing_times(3)[0]) == "ok"
    assert delays == [1.0, 2.0, 4.0]

Tested: the copy ran instantly and recorded [1.0, 2.0, 4.0]. Because retry_with leaves the production function untouched, tests can also override stop or wait to exercise edge cases — a single attempt, zero waits — without global state. Test the configuration you ship, though: the schedule assertion belongs on the real wait settings, with only the sleep faked.

Verify: the tenacity tests run in milliseconds and fail if the decorator's wait changes.

How should this retry test control time? A decision on Who owns the sleep call with 4 outcomes. How should this retry test control time? Who owns the sleep call? your own helper inject sleep / clock record delays tenacity decorator fn.retry_with(sleep=fake) same config code you cannot change virtual loop time (looptime) sleeps fast-forward monkeypatch module global avoid default args bind early Fake time at the seam the code actually uses.

4. Use a virtual-time event loop for code you cannot change

When retry logic is buried in a library, or uses asyncio.timeout alongside sleeps, faking one function is not enough. A virtual-time loop makes every timer fire instantly while keeping their order and the loop's clock consistent:

# pip install looptime  (a pytest plugin)
@pytest.mark.looptime
async def test_retry_with_virtual_time():
    loop = asyncio.get_running_loop()
    start, wall = loop.time(), time.perf_counter()
    assert await call_with_retry(failing_times(3)[0]) == "ok"
    assert loop.time() - start == pytest.approx(7.0)        # virtual seconds
    assert time.perf_counter() - wall < 0.5                  # real seconds

Tested with looptime 0.7: the loop's clock advanced exactly 7.0 virtual seconds while the test finished in milliseconds. This also covers timeouts: an asyncio.timeout(10) around the retries fires at virtual second 10, so deadline behaviour can be tested as easily as backoff. It works only for code that waits on the event loop's clock; time.sleep or time.time() comparisons in the code under test are not affected.

Verify: a test that combines retries with an overall timeout sees the timeout at the expected virtual time.

5. Test the decisions, not just the schedule

Fast tests make it cheap to cover the cases that actually break in production:

@pytest.mark.parametrize("exc, attempts", [
    (Transient(), 5),                  # retried until the limit
    (ValueError("bad input"), 1),      # not retried
    (asyncio.CancelledError(), 1),     # never retried
])
async def test_retry_decisions(exc, attempts):
    calls = {"n": 0}

    async def fn():
        calls["n"] += 1
        raise exc

    with pytest.raises(type(exc)):
        await call_with_retry(fn, sleep=no_sleep)
    assert calls["n"] == attempts


async def test_gives_up_with_full_schedule():
    delays = []
    with pytest.raises(Transient):
        await call_with_retry(failing_times(99)[0], sleep=recorder(delays))
    assert delays == [1.0, 2.0, 4.0, 8.0]        # no sleep after the last attempt

Tested: giving up after five attempts recorded four delays — no pointless sleep after the final failure. Cover which exceptions are retried, which are not, that cancellation is never retried, the exact give-up point and that the original exception is what callers see. Each takes microseconds, so there is no reason to leave any of them out.

Verify: the parametrized decision tests fail if a non-retryable error is added to the retry list.

Retry behaviours worth a fast test A grid of 6 rows by 3 columns. Retry behaviours worth a fast test behaviour assertion example result backoff schedule recorded delays [1.0, 2.0, 4.0] give-up point attempt count 5 attempts no sleep after last attempt delays length 4 delays for 5 attempts non-retryable error attempt count 1 cancellation attempt count 1, re-raised exception seen by caller pytest.raises(type) original type Each runs in microseconds once time is faked.

Verification

Retry logic is tested well when:

  • No retry test sleeps for real; time is injected, overridden or virtual.
  • Tests record and assert delays, so ineffective fakes fail loudly.
  • Decisions are covered: retried, not retried, cancelled, give-up point.
  • The shipped configuration is what the tests exercise.

Diagnostic Hook: run the suite with --durations=10. Any retry or timeout test near the top is sleeping for real — often because a patch did not take effect, as with the default-argument binding measured here.

Pitfalls & edge cases

  • Real sleeps in tests. Measured: 7.01 s for one scenario.
  • Monkeypatching a name bound as a default argument. Tested: no effect.
  • Asserting only the final result. A broken schedule still returns "ok".
  • Faking time.sleep for async code. Async retries wait on the loop.

Frequently Asked Questions

How do I test retry backoff without waiting?

Inject the sleep function into the retry code and pass a fake that records delays, or run the test on a virtual-time loop. In testing, the same 7-second scenario ran in milliseconds and asserted delays of [1.0, 2.0, 4.0].

Why doesn't monkeypatching asyncio.sleep speed up my test?

The code probably captured asyncio.sleep earlier, for example as a default argument evaluated at definition time; the patch then never reaches it. In testing, the patched test still took 7.01 s.

How do I test tenacity retries quickly?

Call fn.retry_with(sleep=fake_sleep) to get a copy of the decorated function that uses your fake sleep while keeping the same stop and wait configuration.

What is looptime?

A pytest plugin that runs tests on an event loop with virtual time, so asyncio sleeps and timeouts complete instantly while loop.time() advances as if they had elapsed.