Skip to content

Configuring the asyncio Logger

asyncio reports its problems through the standard logging module — under the logger named asyncio — and through the loop's exception handler. What it reports, and at what cost, depends on configuration that most services leave at defaults. Measured on Python 3.14: with default settings, the asyncio logger reported a task whose exception was never retrieved, but said nothing about a coroutine that blocked the loop for 150 ms; in debug mode, it also logged "Executing took ..." for the blocker — and creating and awaiting a task cost 62.6 µs instead of 3.24 µs, about 19× more. Logging itself is a blocking call: with a handler whose write took 1 ms, 2,000 requests that each logged one line took 2,196 ms and stalled the loop for up to 1,985 ms; moving the handler behind a QueueHandler and a QueueListener thread brought that to 35 ms with 17 ms of lag, against 23 ms and 9 ms with logging off. This guide configures both the asyncio logger and your own logging for an async service.

Prerequisites

1. Route the asyncio logger explicitly

asyncio logs under logging.getLogger("asyncio"). Its messages include unhandled task exceptions, unclosed resources, slow callbacks in debug mode and, at DEBUG level, implementation details such as the selector in use. Give it its own level and let it propagate to your structured handlers:

import logging

logging.basicConfig(level=logging.INFO, format="%(asctime)s %(name)s %(levelname)s %(message)s")
logging.getLogger("asyncio").setLevel(logging.WARNING)          # problems, not chatter

Measured with the asyncio logger at DEBUG: in normal mode it emitted "Using selector: EpollSelector" at DEBUG and "Task exception was never retrieved" at ERROR, with the task's representation and traceback. WARNING keeps the second and drops the first. Do not silence the logger entirely: its ERROR messages are frequently the only trace of a background task that failed, as covered in debugging unawaited coroutines in large codebases.

Verify: an exception in an unawaited task produces an ERROR record from the asyncio logger in your log pipeline.

2. Install an exception handler for structured reports

Unhandled task exceptions, callback errors and protocol failures go to the loop's exception handler, whose default formats them as a log message. A custom handler can turn the context dictionary into structured fields:

log = logging.getLogger("asyncio")

def exception_handler(loop, context):
    exc = context.get("exception")
    task = context.get("future") or context.get("task")
    log.error(
        "asyncio: %s",
        context.get("message"),
        exc_info=exc,
        extra={"task_name": task.get_name() if hasattr(task, "get_name") else None},
    )

async def main():
    asyncio.get_running_loop().set_exception_handler(exception_handler)
    ...

The context includes message, and depending on the case exception, future or task, handle, protocol and transport. Set the handler at the start of main(), so it is installed before any task can fail, and never let it raise. The full treatment — which keys appear when, chaining to the default handler, and alerting — is in installing a custom exception handler on the event loop; here the point is only that its output should land in the same structured pipeline as the asyncio logger.

Verify: a failing unawaited task produces one structured record with the task name and traceback.

What the asyncio logger reported A grid of 4 rows by 4 columns. What the asyncio logger reported message level normal mode debug mode Using selector: EpollSelector DEBUG yes yes Task exception was never retrieved ERROR yes yes, with source traceback Executing <Task ...> took 0.150 seconds WARNING no yes create + await a task - 3.24 us 62.6 us Python 3.14; slow_callback_duration set to 0.1 s.

3. Keep log handlers off the event loop

logger.info() runs every handler synchronously in the caller's thread. On the event loop, a handler that writes to a slow destination — a full pipe, a network log collector, a busy disk — blocks every coroutine:

class SlowHandler(logging.Handler):
    def emit(self, record):
        time.sleep(0.001)                      # a 1 ms write

Measured with 2,000 concurrent requests logging one line each: 2,196 ms in total and a maximum loop lag of 1,985 ms, against 23 ms and 9 ms with logging disabled. Put a queue between the loop and the real handlers, so the loop only enqueues records and a background thread does the writing:

import logging.handlers, queue

log_queue: queue.SimpleQueue = queue.SimpleQueue()
real_handlers = [logging.StreamHandler(), logging.FileHandler("app.log")]

root = logging.getLogger()
root.handlers = [logging.handlers.QueueHandler(log_queue)]
listener = logging.handlers.QueueListener(log_queue, *real_handlers, respect_handler_level=True)
listener.start()
# at shutdown: listener.stop()  -- flushes what is queued

Measured with the same slow handler behind the queue: 35 ms and 17 ms of lag. The queue is unbounded by default, so a sink that is slower than the log rate on average grows memory; size it, or reduce log volume, as described in flushing telemetry and logs before exit.

Verify: loop lag under load is the same with logging enabled as with it disabled, give or take a few milliseconds.

2,000 requests each logging one line, handler write of 1 ms 3 horizontal bars comparing logging off with the others. 2,000 requests each logging one line, handler write of 1 ms logging off 8.6 ms QueueHandler + listener thread 17.3 ms slow handler on the loop 1,985 ms Total time: 23 ms, 35 ms and 2,196 ms respectively. Logging is I/O; keep it off the loop like any other.

4. Use debug mode in development, not production

Debug mode — asyncio.run(main(), debug=True), PYTHONASYNCIODEBUG=1 or python -X dev — makes asyncio check more and report more: slow callbacks over loop.slow_callback_duration, the source traceback of where a failing task was created, calls from the wrong thread, unawaited coroutines with their origin. It pays for that on every operation:

async def main():
    loop = asyncio.get_running_loop()
    loop.slow_callback_duration = 0.05          # report steps longer than 50 ms
    ...

asyncio.run(main(), debug=os.environ.get("ASYNC_DEBUG") == "1")

Measured: in debug mode, a coroutine that blocked for 150 ms produced a WARNING, "Executing > took 0.150 seconds"; without debug mode it produced nothing. Creating and awaiting a task cost 62.6 µs in debug mode against 3.24 µs without — mostly capturing a stack for every task. Run tests and local development in debug mode; in production, detect blocking with a cheap lag probe instead, as in measuring event loop lag in production.

Verify: CI runs the test suite with debug mode on, and production does not.

5. Put task context into every log line

In concurrent code, log lines from different requests interleave. Add the request and task identity to every record with a filter, so lines can be grouped again:

request_id = contextvars.ContextVar("request_id", default="-")

class TaskContextFilter(logging.Filter):
    def filter(self, record):
        record.request_id = request_id.get()
        try:
            task = asyncio.current_task()
        except RuntimeError:                    # not in an event loop thread
            task = None
        record.task = task.get_name() if task else "-"
        return True

handler.addFilter(TaskContextFilter())
handler.setFormatter(logging.Formatter("%(asctime)s %(request_id)s %(task)s %(levelname)s %(message)s"))

With a QueueHandler, attach the filter to the QueueHandler itself — filters run when the record is created, on the loop thread where the context and current task are known, not later in the listener thread where they are not. The request ID comes from the context variable set by middleware, as in propagating request IDs with contextvars.

Verify: interleaved log lines from concurrent requests can be separated by request ID and task name.

What does this logging concern need? A decision on What do you need from logging with 4 outcomes. What does this logging concern need? What do you need from logging? asyncio's own problems 'asyncio' logger at WARNING never retrieved: ERROR structured task errors loop.set_exception_handler name + traceback slow log sinks QueueHandler + QueueListener lag 1,985 ms to 17 ms blocking callbacks debug mode in dev, lag probe in prod 19x task cost Attach context filters on the loop thread, before the queue.

Verification

Logging in an async service is configured well when:

  • The asyncio logger is routed at WARNING, and its ERROR records reach your log pipeline.
  • A custom exception handler emits structured records for unhandled task errors.
  • Handlers run behind a queue, and loop lag does not depend on log volume.
  • Debug mode runs in tests, a lag probe in production, and every line carries request and task identity.

Diagnostic Hook: compare loop lag with logging at its normal level and at WARNING during a load test. If the difference is more than a few milliseconds, a handler is running on the loop — often one added later by a library or an APM agent, outside the queue you configured.

Pitfalls & edge cases

  • Expecting slow-callback warnings without debug mode. Measured: none appeared.
  • Debug mode in production. Measured: 19× the cost per task.
  • Handlers on the loop. Measured: 1,985 ms of loop lag from 1 ms writes.
  • Filters on the listener side of a queue. Context and current task are gone there.

Frequently Asked Questions

What does the asyncio logger log?

Under the name "asyncio": unhandled task exceptions ("Task exception was never retrieved"), unclosed resources, the selector in use at DEBUG and, in debug mode only, callbacks slower than slow_callback_duration.

Does Python logging block the asyncio event loop?

Yes, handlers run synchronously: with a 1 ms handler, 2,000 logged requests stalled the loop for up to 1,985 ms. A QueueHandler with a QueueListener thread cut that to 17 ms.

Should I enable asyncio debug mode in production?

No: it raised the cost of creating and awaiting a task from 3.24 µs to 62.6 µs. Use it in tests and development; use a loop-lag probe in production.

How do I add request IDs to async log lines?

Use a logging.Filter that reads a ContextVar and asyncio.current_task(), attached where records are created, on the loop thread.