Skip to content

Writing Files Atomically from Async Code

A file written in place is broken for as long as the write takes: open with "wb" truncates it to zero, and the new contents arrive over the following milliseconds. A crash, kill or power loss in that window leaves a truncated or mixed file — the configuration that will not parse, the cache that crashes the next start, the checkpoint that resumes from garbage. Tested on ext4 by killing a process that repeatedly rewrote a 20 MiB file: writing in place left a torn or truncated file in 19 of 30 kills; writing to a temporary file, fsyncing it and os.replace-ing it over the original left 0 of 30 damaged — every kill left either the complete old version or the complete new one. The fsync makes the operation slow and variable: an 8 MiB atomic write took 10 ms on average, and doing it inline on the event loop stalled the loop for up to 101 ms; through asyncio.to_thread, the worst stall was 1 ms. This guide implements the pattern for async code.

Prerequisites

1. Write to a temporary file, then rename

os.replace swaps a directory entry in one step: other processes see the old file or the new one, never a mixture. Write the new contents to a temporary file in the same directory and replace the original with it:

import os
import tempfile


def write_atomic(path: str, data: bytes) -> None:
    directory = os.path.dirname(os.path.abspath(path))
    fd, tmp = tempfile.mkstemp(dir=directory, prefix=".tmp-", suffix=os.path.basename(path))
    try:
        with os.fdopen(fd, "wb") as f:
            f.write(data)
            f.flush()
            os.fsync(f.fileno())                 # contents on disk before the rename
        os.replace(tmp, path)                    # atomic swap of the directory entry
    except BaseException:
        os.unlink(tmp)                           # never leave temp files behind
        raise
    dir_fd = os.open(directory, os.O_RDONLY)
    try:
        os.fsync(dir_fd)                         # the rename itself survives a power loss
    finally:
        os.close(dir_fd)

The temporary file must be on the same filesystem as the target — a rename across filesystems is a copy, not an atomic swap — which is why it goes in the same directory rather than /tmp. mkstemp gives a unique name, so concurrent writers do not trample each other's temporary files. Measured with a writer killed at random 30 times, the target was always complete.

Verify: kill a process repeatedly while it rewrites a file in a loop; every surviving version of the file is complete and valid.

Damaged files after killing a writer mid-write, 30 trials 2 horizontal bars comparing write in place (open 'wb') with the others. Damaged files after killing a writer mid-write, 30 trials write in place (open 'wb') 19 of 30 damaged temp file + fsync + os.replace 0 of 30 damaged ext4, 20 MiB file rewritten in a loop by a child process killed with SIGKILL after 0.2-0.6 s. In-place writes are broken for the whole duration of the write.

2. Keep the fsync off the event loop

Each step — write, fsync, rename, directory fsync — is a blocking system call, and fsync waits for the disk. Run the whole function in a thread:

async def save(path: str, data: bytes) -> None:
    await asyncio.to_thread(write_atomic, path, data)

Measured on ext4 with 8 MiB writes: about 10 ms per write either way, but called inline the event loop stalled for up to 101 ms on one of the writes — every other task, request and timer in the process waited. Through asyncio.to_thread the worst stall was 1 ms. fsync latency varies with what else the disk is doing, so an inline call that is usually quick will occasionally be very slow. Running the whole function in one thread call, rather than offloading each step separately, also keeps the steps in order without extra coordination.

Verify: with asyncio debug mode on and slow_callback_duration at 50 ms, saving files produces no slow-callback warnings.

Worst event loop stall during 8 MiB atomic writes 2 horizontal bars comparing inline on the event loop with the others. Worst event loop stall during 8 MiB atomic writes inline on the event loop 101 ms asyncio.to_thread 1 ms Python 3.14 on ext4; both averaged 10 ms per write; a 1 ms heartbeat task measured its own lateness. fsync time is unpredictable; keep it off the thread that serves everything else.

3. Serialize writers to the same file

Atomic replacement guarantees readers see a complete file. It does not decide which complete file wins when two tasks save at once — the last rename wins, which may be the older data if it started first and finished last:

import collections

_locks: dict[str, asyncio.Lock] = collections.defaultdict(asyncio.Lock)


async def save(path: str, data: bytes) -> None:
    async with _locks[os.path.abspath(path)]:          # one writer per file at a time, in order
        await asyncio.to_thread(write_atomic, path, data)

The per-path lock serializes writes within the process, in the order they acquired it. Across processes, use a file lock (fcntl.flock on a separate lock file, taken in the thread) or design so only one process writes each file. For read-modify-write updates — load, change, save — hold the lock across all three steps, or two concurrent updates will each lose the other's change.

Verify: a test that issues 100 concurrent saves with increasing version numbers ends with the highest version on disk.

4. Apply it to the formats you write

Most files written by services are small structured documents. Wrap the encoding in the same helper, and write text with an explicit encoding:

import json


async def save_json(path: str, obj) -> None:
    data = json.dumps(obj, indent=2, sort_keys=True).encode("utf-8")
    await save(path, data)


async def save_checkpoint(path: str, position: int) -> None:
    await save_json(path, {"next": position, "saved_at": time.time()})

Encode before entering the thread so serialization errors surface before anything touches the disk. Checkpoints, as in checkpointing progress in long-running async jobs, are the most important use: a torn checkpoint turns a resumable job into one that must start over, or worse, resumes from a wrong position. For large outputs — exports, downloads — stream to the temporary file in chunks and rename at the end, as in downloading many URLs concurrently with progress.

Verify: every file your service writes goes through the atomic helper; grep for open(..., "w") on persistent paths finds nothing else.

5. Clean up and know the limits

Crashes between creating and renaming the temporary file leave .tmp-* files behind. Remove stale ones at startup, and know where the guarantees stop:

import glob
import time


def remove_stale_temps(directory: str, older_than: float = 3600) -> int:
    removed = 0
    for tmp in glob.glob(os.path.join(directory, ".tmp-*")):
        if time.time() - os.path.getmtime(tmp) > older_than:
            os.unlink(tmp)
            removed += 1
    return removed

The age threshold avoids deleting the temporary file of a write in progress in another process. Network filesystems (NFS, SMB) and some FUSE filesystems do not all provide atomic rename or reliable fsync; object stores have no rename at all — there, write the new object under a new key and switch a pointer, or rely on the store's atomic single-object PUT. On container filesystems, write persistent files to a mounted volume, not the container layer.

Verify: after an induced crash and restart, stale temporary files are removed and the target file is valid.

The atomic write sequence A flow of 5 stages. The atomic write sequence mkstemp in same dir unique name write + flush data in page cache fsync(file) data on disk os.replace atomic swap fsync(dir) rename on disk Run all five steps in one thread call; readers only ever see a whole file.

Verification

Files are written atomically when:

  • Every persistent write goes through temp file, fsync and os.replace in the same directory.
  • The sequence runs in a thread, never on the event loop.
  • Writers to the same file are serialized, including read-modify-write cycles.
  • Stale temporary files are cleaned up at startup.

Diagnostic Hook: log parse failures of your own files at startup, and count stale temporary files found. Parse failures on files you wrote mean a non-atomic code path; stale temporary files mean crashes during writes — harmless with this pattern, but a sign worth investigating.

Pitfalls & edge cases

  • Writing in place. Measured: 19 of 30 kills damaged the file.
  • Temporary files in /tmp. A rename across filesystems is not atomic.
  • fsync on the event loop. Measured stalls up to 101 ms.
  • Unserialized concurrent saves. Each file is complete, but an older version can win.

Frequently Asked Questions

How do I write a file atomically in Python?

Write to a temporary file in the same directory, flush and os.fsync it, then os.replace it over the target and fsync the directory. Readers then see either the old or the new file, never a partial one.

Is os.replace atomic?

On POSIX filesystems, renaming within one filesystem replaces the directory entry atomically. In testing, 30 random kills during writes never left a damaged target with this method.

Should fsync run in asyncio.to_thread?

Yes. fsync waits for the disk and its latency varies; inline it stalled the event loop for up to 101 ms in testing, against 1 ms through to_thread.

How do I stop concurrent async saves from overwriting newer data?

Serialize writes per file with an asyncio.Lock, and hold the lock across the whole read-modify-write cycle when updating existing contents.