Running Periodic Tasks Without Drift

A metrics exporter is supposed to flush every ten seconds. After a day, dashboards show gaps and double points, because the flush now happens at :03, :14, :25 — each cycle ten seconds plus however long the flush took. A token refresher meant to run every five minutes slowly slides past the token's expiry. A health checker scheduled per instance fires on all two hundred instances in the same millisecond, every minute. All three use the natural-looking loop while True: await work(); await asyncio.sleep(interval), which schedules the next run relative to when the previous one finished, so every run's duration and every bit of loop lag is added to the schedule forever. This guide replaces it with a scheduler that targets absolute tick times on the loop clock, decides deliberately what happens when a tick is missed or a run overlaps the next one, spreads a fleet with jitter, survives exceptions, and stops cleanly.

Prerequisites

• Python 3.11+, standard library only.
• Task lifecycle and cancellation from Task Scheduling & Lifecycle and cancelling a task and waiting for it to finish.
• Loop lag awareness from measuring event loop lag in production: a busy loop delays every timer.

1. Measure the drift of sleep-after-work

Run both schedules side by side with a 100 ms interval and 30 ms of work per tick, recording when each tick actually starts.

import asyncio
import time


async def work() -> None:
    await asyncio.sleep(0.03)                      # 30 ms of real work per tick


async def naive(interval: float, ticks: int, starts: list[float]) -> None:
    t0 = time.monotonic()
    for _ in range(ticks):
        starts.append(time.monotonic() - t0)
        await work()
        await asyncio.sleep(interval)              # relative to when the work finished


async def fixed_rate(interval: float, ticks: int, starts: list[float]) -> None:
    loop = asyncio.get_running_loop()
    t0 = time.monotonic()
    next_at = loop.time()
    for _ in range(ticks):
        starts.append(time.monotonic() - t0)
        await work()
        next_at += interval                        # relative to the schedule
        await asyncio.sleep(max(0.0, next_at - loop.time()))


async def main() -> None:
    for schedule in (naive, fixed_rate):
        starts: list[float] = []
        await schedule(0.1, 11, starts)
        print(f"{schedule.__name__:>10}: tick 10 at {starts[10]:.3f}s (ideal 1.000s)")


asyncio.run(main())

The naive loop started its eleventh tick at 1.303 seconds — ten ticks times 30 ms of work — and keeps sliding by 30 ms per tick for as long as it runs. The fixed-rate loop started it at 1.000 seconds. Over a day at a ten-second interval with one second of work, the naive schedule loses nearly 8,000 ticks' worth of alignment; the fixed-rate one loses none.

Verify: the naive tick-10 time exceeds the ideal by roughly ticks × work duration, and the fixed-rate time stays within a millisecond or two of it.

2. Schedule against absolute ticks on the loop clock

Compute each tick as start + n × interval rather than "now plus interval". Use loop.time(), the monotonic clock the loop's own timers use, so system clock changes cannot move the schedule and so tests can control time as in controlling time in asyncio tests.

import asyncio
from collections.abc import Awaitable, Callable


async def every(interval: float, job: Callable[[], Awaitable[None]], *,
                align: bool = False) -> None:
    """Run `job` at start + n * interval, forever, on the loop's monotonic clock."""
    loop = asyncio.get_running_loop()
    now = loop.time()
    first = (now // interval + 1) * interval if align else now   # align to multiples of interval
    tick = 0
    while True:
        target = first + tick * interval
        delay = target - loop.time()
        if delay > 0:
            await asyncio.sleep(delay)
        await job()
        tick += 1

align=True snaps ticks to multiples of the interval on the monotonic clock, which keeps several periodic jobs with related intervals (10 s and 60 s, say) firing together rather than at arbitrary offsets. It does not align to wall-clock minutes; for "run at :00 every minute" semantics use a cron-style scheduler, as in scheduling cron jobs inside an asyncio service.

Verify: log loop.time() % interval at each run with align=True; it stays close to zero rather than creeping upwards.

3. Decide what happens when a tick is missed

If a run takes longer than the interval, or the loop was blocked, the next target is already in the past. Three policies are defensible, and the naive every() above silently picks the worst one for most jobs — it fires every missed tick back to back.

import asyncio
from collections.abc import Awaitable, Callable
from enum import Enum


class Missed(Enum):
    SKIP = "skip"            # jump to the next future tick (metrics, polling)
    CATCH_UP = "catch_up"    # run once per missed tick (accounting, sampling)
    RUN_ONCE = "run_once"    # run once now, then realign (refreshing a token)


async def every_with_policy(interval: float, job: Callable[[], Awaitable[None]],
                            policy: Missed = Missed.SKIP,
                            on_missed: Callable[[int], None] = lambda n: None) -> None:
    loop = asyncio.get_running_loop()
    start = loop.time()
    tick = 0
    while True:
        target = start + tick * interval
        lateness = loop.time() - target
        if lateness >= interval and policy is not Missed.CATCH_UP:
            behind = int(lateness // interval)                  # ticks that are already past
            tick += behind
            target = start + tick * interval
            if policy is Missed.SKIP:
                on_missed(behind + 1)                           # export it: this is a signal
                tick += 1                                       # wait for the next future tick
                continue
            on_missed(behind)                                   # RUN_ONCE: the late run covers one
        delay = target - loop.time()
        if delay > 0:
            await asyncio.sleep(delay)
        await job()
        tick += 1


async def main() -> None:
    runs: list[float] = []
    missed: list[int] = []
    loop = asyncio.get_running_loop()
    t0 = loop.time()

    async def slow_job() -> None:
        runs.append(round(loop.time() - t0, 2))
        await asyncio.sleep(0.25 if len(runs) == 2 else 0.01)   # one run overruns 2.5 ticks

    task = asyncio.create_task(every_with_policy(0.1, slow_job, Missed.SKIP, missed.append))
    await asyncio.sleep(0.65)
    task.cancel()
    await asyncio.gather(task, return_exceptions=True)
    print("runs at:", runs, "missed:", missed)


asyncio.run(main())

With SKIP, the overrunning second run is followed by the next future tick, and the missed ticks are reported rather than replayed — the right choice for exporters and pollers, where two back-to-back runs add load without adding information. CATCH_UP suits jobs where every interval must be accounted for, provided the job is cheap. RUN_ONCE suits refreshers: do the late work once, then return to the schedule.

Verify: with SKIP, the recorded run times jump from the overrun to the next multiple of 0.1 and missed contains the number of skipped ticks; switching to CATCH_UP produces back-to-back runs instead.

4. Keep overlapping runs and failures from breaking the schedule

Two more decisions belong in the scheduler, not in every job. A run that raises must not end the periodic task forever, and a run that is still going when the next tick arrives needs a rule: wait (serialise), skip, or run concurrently. Serialising is the default above; adding a timeout per run bounds how long one bad run can delay the schedule.

import asyncio
import logging
from collections.abc import Awaitable, Callable

log = logging.getLogger("periodic")


def guarded(name: str, job: Callable[[], Awaitable[None]], timeout: float) -> Callable[[], Awaitable[None]]:
    async def run() -> None:
        try:
            async with asyncio.timeout(timeout):
                await job()
        except TimeoutError:
            log.warning("%s: run exceeded %.1fs and was cancelled", name, timeout)
        except Exception:
            log.exception("%s: run failed; schedule continues", name)
    return run


async def main() -> None:
    attempts = 0

    async def flaky() -> None:
        nonlocal attempts
        attempts += 1
        if attempts == 2:
            raise RuntimeError("upstream 500")

    task = asyncio.create_task(every_with_policy(0.05, guarded("flush", flaky, timeout=0.04)),
                               name="periodic.flush")
    await asyncio.sleep(0.22)
    task.cancel()
    await asyncio.gather(task, return_exceptions=True)
    print("runs attempted:", attempts)                            # 5: the failure did not stop it


logging.basicConfig(level=logging.ERROR)
asyncio.run(main())

Catching Exception — not BaseException — lets cancellation still stop the scheduler while ordinary failures are logged and the next tick proceeds. The per-run timeout, set below the interval, guarantees that a hung dependency cannot freeze the schedule; its cancellation must be handled correctly by the job, as described in preventing CancelledError leaks in cleanup.

Verify: about five runs are attempted in 220 ms with a 50 ms interval, one error is logged, and cancelling the task ends the loop immediately.

5. Spread a fleet with jitter and stop cleanly

When every instance starts its periodic job at deploy time with the same interval, they all hit the shared dependency together, forever. A random initial offset, fixed per instance, spreads the load while keeping each instance's own schedule regular. And the periodic task needs an owner that cancels it and waits during shutdown.

import asyncio
import random


async def every_jittered(interval: float, job, *, rng: random.Random | None = None) -> None:
    rng = rng or random.Random()
    await asyncio.sleep(rng.uniform(0, interval))        # one-time offset, then a fixed rate
    await every_with_policy(interval, job, Missed.SKIP)


class PeriodicService:
    def __init__(self) -> None:
        self._tasks: set[asyncio.Task] = set()

    def start(self, name: str, interval: float, job) -> None:
        task = asyncio.create_task(every_jittered(interval, guarded(name, job, interval * 0.8)),
                                   name=f"periodic.{name}")
        self._tasks.add(task)
        task.add_done_callback(self._tasks.discard)

    async def stop(self, grace: float = 5.0) -> None:
        for task in self._tasks:
            task.cancel()
        if self._tasks:
            await asyncio.wait(self._tasks, timeout=grace)


async def main() -> None:
    service = PeriodicService()
    service.start("heartbeat", 0.05, lambda: asyncio.sleep(0))
    await asyncio.sleep(0.2)
    await service.stop()
    print("periodic tasks left:", len(service._tasks))            # 0


asyncio.run(main())

Jitter is applied once, as an offset, rather than to every interval: per-tick jitter reintroduces drift and makes the interval irregular for the job itself. The service object holds strong references, names each task for task dumps and metrics, and gives shutdown a bounded wait.

Verify: across several instances, first-run times are spread over one interval while each instance's later runs stay on its own fixed ticks; stop() leaves no periodic tasks running.

Verification

Periodic work is scheduled correctly when:

• No cumulative drift: the Nth run starts within milliseconds of start + N × interval, regardless of run duration.
• Missed ticks follow a chosen policy: skip, catch up or run once, with missed-tick counts exported.
• One bad run is contained: exceptions are logged and timeouts cancel runs, while the schedule continues.
• Fleets are spread: instances start with a per-instance offset and do not fire in unison.
• Shutdown is clean: periodic tasks are owned, cancelled and awaited with a bounded grace period.

Pitfalls & edge cases

• Using time.time() for schedules. Wall-clock adjustments by NTP or an operator move the next tick forwards or backwards. Schedule on loop.time() and use wall-clock time only for cron-style "at 03:00" jobs.
• Blocking work inside the job. A synchronous call in the job delays the loop and every other timer. Offload it with asyncio.to_thread().
• Catching BaseException in the guard. That swallows cancellation, so the periodic task can never be stopped and shutdown hangs.
• Per-tick random sleeps as "jitter". Randomising every interval makes the schedule drift and the job's cadence unpredictable. Offset once.
• Periodic tasks in every worker process. With several server workers, each runs its own copy. Run singleton jobs in one designated process, or coordinate with a lock, as covered for background jobs and task queues.

Frequently Asked Questions

Why does my asyncio periodic task drift over time?

Because awaiting asyncio.sleep(interval) after the work schedules the next run relative to when the work finished, so the duration of every run and any event loop lag are added to the schedule permanently. Compute each run's target as start plus tick number times the interval and sleep only until that target instead.

How do I run a coroutine every N seconds in asyncio?

Loop forever, computing the next target time from the loop's monotonic clock as start + n × interval, sleeping for the difference to now, running the job, and incrementing n. Wrap the job so exceptions are logged and a per-run timeout applies, and decide explicitly what to do when a tick is already in the past.

What should happen when a periodic job takes longer than its interval?

Choose a policy. Skipping missed ticks suits metrics and polling, running once per missed tick suits accounting-style jobs, and running once late before realigning suits refreshers. Export the number of missed ticks, because consistent overruns mean the interval or the job needs to change.

How do I stop many instances from running their periodic jobs at the same time?

Add a random initial delay between zero and one interval when each instance starts, then run on a fixed schedule from that offset. This spreads load across the interval while keeping every instance's own runs regular. Avoid adding random delays to each interval, which causes drift.

Related

• Task Scheduling & Lifecycle — up to the topic overview for timers, tasks and cancellation.
• Using loop.call_later and timer handles — the lower-level timers these schedules are built on.
• Asyncio Fundamentals & Event Loop Architecture — the section overview for the loop and its timer heap.