Skip to content

Choosing asyncio Lock vs Semaphore vs Event

The most common asyncio coordination bug is not a subtle race — it is picking the wrong primitive for the symptom. Engineers reach for a Lock when they need a concurrency cap (and accidentally serialize everything down to one-at-a-time), or grab an Event when they need a work queue (and silently drop items, because an Event carries no payload and merges signals). This guide is a decision procedure: start from the symptom you actually have, and it routes you to the right primitive — including the answer "you need none."

The reason this confusion is so persistent is that the four primitives overlap in shape — all four are objects you await on, all four park your coroutine until something changes — while differing entirely in intent. The decision below keys off intent, phrased as the concrete symptom you observe: "two callers corrupt shared state," "I open too many sockets at once," "everyone has to wait for warm-up," "a consumer must block until the data is ready." Map the symptom, not the vocabulary; the wrong mental label ("I need to lock this") is exactly what leads to the wrong primitive.

Prerequisites

Decision tree: symptom to asyncio primitive A top-down decision tree starting from whether the shared region awaits, then branching to Lock, Semaphore, Event, or Condition based on the coordination symptom. Symptom to primitive Do you await inside the shared region? no No primitive needed yes: what do you need? Lock one at a time mutual exclusion Semaphore N at a time cap concurrency Event signal readiness wake all waiters Condition wait predicate over shared state If the real need is "hand items between producers and consumers" → use asyncio.Queue If state is shared with OS threads → none of these; use threading primitives

1. Do I even need a primitive?

Ask first: does the shared region await? Under cooperative scheduling a coroutine cannot be preempted mid-statement; control only leaves at an await. So a read-modify-write that contains no await is already atomic relative to other coroutines, and wrapping it in a Lock adds overhead and contention for nothing.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
import asyncio

counter = 0

async def increment_no_await() -> None:
    global counter
    counter += 1          # no await between read and write — already atomic

async def increment_needs_lock(lock: asyncio.Lock, store: dict) -> None:
    async with lock:
        current = store["v"]
        await asyncio.sleep(0)   # <-- yields here; a peer could interleave
        store["v"] = current + 1 # without the lock this loses updates

What to verify: scan the shared region for await. None? Delete the lock. If there is an await between a read and a dependent write, you need exclusion — continue to step 2.

2. I need mutual exclusion → Lock

Symptom: exactly one coroutine may be inside the region at a time because it read-modify-writes shared state across an await (lazy init, a cache refresh, a non-reentrant client call).

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
import asyncio

class Once:
    def __init__(self) -> None:
        self._lock = asyncio.Lock()
        self._value: str | None = None

    async def get(self) -> str:
        if self._value is not None:        # fast path, no await
            return self._value
        async with self._lock:
            if self._value is None:        # re-check under the lock
                await asyncio.sleep(0.1)   # the expensive, await-ing init
                self._value = "ready"
            return self._value

What to verify: run two concurrent callers and assert the expensive init ran once. Add a counter inside the locked block; it must increment exactly once. The double-check is what makes queued callers reuse the result instead of redoing the work.

3. I need to cap N-at-once → Semaphore

Symptom: many coroutines should run in parallel, but no more than N simultaneously — bounded fan-out to an API, a connection pool, a rate-limited service. This is not exclusion; you want concurrency, just capped.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
import asyncio

async def call(sem: asyncio.Semaphore, i: int) -> int:
    async with sem:                        # up to N proceed together
        await asyncio.sleep(0.1)           # stand-in for a network call
        return i

async def main() -> None:
    sem = asyncio.Semaphore(10)            # never more than 10 in flight
    async with asyncio.TaskGroup() as tg:
        tasks = [tg.create_task(call(sem, i)) for i in range(100)]
    assert sorted(t.result() for t in tasks) == list(range(100))

asyncio.run(main())

If you used a Lock here you would get serial execution — 100 sequential calls instead of 10-wide parallelism. The semaphore is the concurrency knob.

What to verify: instrument a live in-flight counter; its maximum must equal the semaphore count, not 1 and not the task count. Size N to the scarcest downstream limit, as covered under the parent reference.

A useful tell that you have confused this case with mutual exclusion: if your "lock" is being acquired by code that is fundamentally independent — separate URLs, separate rows, separate files — then exclusion is the wrong model. Independent work does not need to be serialized; it needs to be capped. Reserve the Lock for the case where the coroutines genuinely contend over one shared thing, and use the Semaphore whenever the only constraint is "not too many at once."

4. I need to signal an event/readiness → Event

Symptom: one coroutine reaches a milestone (config loaded, warm-up done, shutdown requested) and one or many others should proceed once it does. No data is handed over — only a fact ("it happened").

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
import asyncio

async def waiter(name: str, ready: asyncio.Event) -> str:
    await ready.wait()                     # parks until set(); sticky after
    return f"{name} go"

async def main() -> None:
    ready = asyncio.Event()
    tasks = [asyncio.create_task(waiter(f"w{i}", ready)) for i in range(3)]
    await asyncio.sleep(0.2)               # do the startup work
    ready.set()                            # release all waiters at once
    print(await asyncio.gather(*tasks))

asyncio.run(main())

Do not use an Event to pass work items — it has no payload and collapses multiple set() calls into one state. If you need to move items, that is a queue, not an event.

What to verify: confirm all waiters wake from a single set(), and that a coroutine which calls wait() after set() returns immediately (the flag is sticky).

The stickiness is the property people most often get wrong in both directions. It is exactly right for one-shot milestones — "the cache is warm" stays true, so a latecomer should not block. It is exactly wrong for recurring pulses, where you would have to clear() between signals and then race against waiters that have not yet looped back to wait(). If your signal fires more than once, you have outgrown Event; an asyncio.Queue (one sentinel per pulse) or a Condition over an explicit counter expresses repeated signaling without the lost-wakeup race.

5. I need to wait for a predicate over shared state → Condition

Symptom: a coroutine must block until shared state satisfies a condition richer than "an item exists" — "queue depth ≥ K," "all shards reported," "the system left maintenance mode." A producer mutates that state and notifies waiters.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
import asyncio

class Gate:
    def __init__(self, need: int) -> None:
        self._cond = asyncio.Condition()
        self._count = 0
        self._need = need

    async def arrive(self) -> None:
        async with self._cond:
            self._count += 1
            self._cond.notify_all()        # state changed; wake waiters

    async def wait_for_quorum(self) -> None:
        async with self._cond:
            while self._count < self._need:   # while, never if
                await self._cond.wait()

What to verify: the waiter must re-test the predicate in a while loop and stay blocked until quorum, then proceed exactly once. If your predicate is simply "is there an item," stop — use an asyncio.Queue instead; it is the same machinery with safer ergonomics.

Verification

Across the five paths, the observable proof of a correct choice is specific to the symptom:

  • Lock (correct serialization): an instrumented in-region counter never exceeds 1, and the guarded one-shot work executes exactly once under concurrent callers.
  • Semaphore (bounded concurrency): a live in-flight gauge peaks at exactly N regardless of how many tasks are submitted; throughput scales with N, not with task count.
  • Event (broadcast wakeup): a single set() releases every current waiter, and late waiters return immediately because the flag is sticky.
  • Condition (predicate wakeup): waiters stay parked until the predicate holds, survive spurious wakeups via the while re-check, and proceed once.

If none of these gauges move as expected, you likely picked the wrong primitive for the symptom — re-run the decision tree from step 1.

Pitfalls & edge cases

  • None of these are thread-safe. They coordinate coroutines on one loop only. If a worker thread touches the same state, no asyncio primitive helps — use threading primitives, a thread-safe queue.Queue, or share state between tasks and threads correctly.
  • Forgetting async with. A bare await sem.acquire() with no matching release() on an exception path leaks a permit and slowly throttles you to zero. Prefer async with; if you must acquire manually, release in finally.
  • Use BoundedSemaphore to catch over-release. A plain Semaphore lets a stray release() raise the count above its start value, silently inflating your cap. BoundedSemaphore raises ValueError at the offending call.
  • Event is sticky once set. It does not auto-reset. If you need a repeatable signal, call event.clear() deliberately — and beware the race where a waiter misses the brief set/clear window. For repeated signaling, a queue is usually the right tool.
  • Condition needs its lock held. You must be inside async with cond: to call wait(), notify(), or notify_all(), and you must re-check the predicate with while, not if, or a spurious or stale wakeup proceeds on a false predicate.

Frequently Asked Questions

When can I skip an asyncio.Lock entirely?

When the shared region contains no await. Cooperative scheduling means a coroutine cannot be preempted mid-statement, so a read-modify-write with no await is already atomic relative to other coroutines and a lock only adds contention.

Why does using a Lock to cap concurrency make everything slow?

A Lock admits exactly one coroutine at a time, so guarding a fan-out with it serializes the work to one-at-a-time. To run N in parallel with a ceiling, use asyncio.Semaphore(N) instead; the semaphore count is your concurrency knob.

Can I use an asyncio.Event as a work queue?

No. An Event carries no payload and merges multiple set() calls into a single sticky state, so items are lost. To hand work items between producers and consumers, use asyncio.Queue, which provides payloads and backpressure.