io_uring for people who already know epoll

If you have written a network service in C or Rust on Linux in the last twenty years, you know epoll. You know the dance: epoll_create1, epoll_ctl, epoll_wait, set non-blocking, read until EAGAIN, write until EAGAIN, mind the edge-triggered semantics, mind EPOLLRDHUP, mind EPOLLET + EPOLLONESHOT and the dozen footnotes attached to each.

io_uring is the Linux answer to “what if we stopped doing one syscall per I/O and did batched, async, completion-based I/O instead.” It landed in 5.1 (2019) and has been growing every kernel release since. By 2026 the API surface is large and the ecosystem has finally caught up.

This post is not an introduction to async programming. It assumes you know what epoll is, why you’d reach for it, and where you’ve been bitten. It is a walk through the io_uring model from that starting point: what changes, what stays the same, and where the new sharp edges live.

The mental model

Forget readiness for a moment. epoll is a readiness API: it tells you a file descriptor is ready for I/O, then you issue the syscall that actually does the I/O.

io_uring is a completion API: you tell the kernel what I/O you want done, and the kernel tells you when it’s done. There is no separate “do the actual read” step; the kernel performs the read into your buffer.

This is the same model as Windows IOCP and (older) Solaris event ports. Linux skipped completion-based I/O for two decades while the rest of the world used readiness.

In practice it shows up as two ring buffers shared between user-space and the kernel:

• A submission queue (SQ). Userspace writes Submission Queue Entries (SQEs). Each SQE is a 64-byte struct describing one operation: “read 4096 bytes from fd 3 into buffer X.”
• A completion queue (CQ). The kernel writes Completion Queue Entries (CQEs). Each is 16 bytes: “operation Y returned 4096.”

You correlate submissions with completions through a user_data field that you control. Typically a pointer to your own request struct.

  user-space                          kernel
  ----------                          ------
  fill SQE  ──►  submission queue ──► picks up
                                       performs I/O
  reaps CQE ◄──  completion queue ◄── posts result

One syscall pushes submissions and pulls completions: io_uring_enter. With IORING_SETUP_SQPOLL, none at all; a kernel thread polls the SQ on your behalf.

A minimal example

The full setup is verbose, so most people reach for liburing. Here is a “read a file” the io_uring way:

#include <liburing.h>
#include <fcntl.h>
#include <stdio.h>
#include <string.h>

int main(void) {
    struct io_uring ring;
    if (io_uring_queue_init(8, &ring, 0) < 0) { perror("queue_init"); return 1; }

    int fd = open("./hello.txt", O_RDONLY);
    char buf[256] = {0};
    struct iovec iov = { .iov_base = buf, .iov_len = sizeof(buf) };

    struct io_uring_sqe *sqe = io_uring_get_sqe(&ring);
    io_uring_prep_readv(sqe, fd, &iov, 1, /*offset=*/0);
    sqe->user_data = 0xCAFEBABE;        // correlation token

    io_uring_submit(&ring);

    struct io_uring_cqe *cqe;
    io_uring_wait_cqe(&ring, &cqe);

    if (cqe->res < 0) {
        fprintf(stderr, "read failed: %s\n", strerror(-cqe->res));
    } else {
        printf("read %d bytes: %.*s", cqe->res, cqe->res, buf);
    }
    io_uring_cqe_seen(&ring, cqe);

    io_uring_queue_exit(&ring);
    return 0;
}

Compile with gcc -o ur ur.c -luring. Two things worth noticing:

1. The read happens asynchronously, but we immediately wait for completion. This is the simplest mode. In a real service you submit many ops and only block when there’s nothing else to do.
2. The error is in cqe->res as a negative errno, not in the thread-local errno. There is no errno for completions; the kernel cannot reach into your TLS from a worker thread.

What replaces what

Coming from epoll, the mental translation is roughly:

Most epoll operations have a direct io_uring equivalent. The interesting entries are at the bottom of the list, and that’s where the wins are.

Where io_uring actually wins

Three places, ordered roughly by the size of the win:

1. Syscall amortization

Every read or write in epoll-land is at least one syscall: two context switches and a TLB flush at minimum. With io_uring you can submit a hundred reads with one io_uring_enter, or with SQPOLL, zero. Under load, per-syscall cost dominates the per-request budget, and removing it is the single biggest performance win.

Numbers from an edge proxy I worked on last year (Linux 6.6, EPYC 7402P, 25 GbE NIC, single ring per worker, four workers, wrk2 driving fixed RPS, p50 latency reported; reproduce at your own risk because tail behaviour does not match):

The closer the workload sits to “lots of small ops on hot connections,” the more io_uring wins. SQPOLL adds a kernel thread per ring, which is great for one busy ring and bad if you have a thousand mostly-idle ones.

2. Operations that epoll can’t async

A handful of syscalls do not integrate with epoll because they have no readiness signal:

• open() — inode lookup can block on disk.
• stat(), fstatat() — same.
• mkdir, unlink, rename — metadata ops, all blocking.
• fsync, fdatasync — explicitly blocking.

In epoll-land you punt these to a thread pool or accept that they block. In io_uring they’re regular SQEs:

io_uring_prep_openat(sqe, AT_FDCWD, path, O_RDONLY, 0);
io_uring_prep_statx(sqe, AT_FDCWD, path, 0, STATX_BASIC_STATS, &stx);
io_uring_prep_fsync(sqe, fd, IORING_FSYNC_DATASYNC);
io_uring_prep_unlinkat(sqe, AT_FDCWD, path, 0);

Disk-heavy workloads see real gains here: HTTP servers serving lots of small static files, mail servers, databases doing a lot of small fsyncs.

3. Zero-copy paths

io_uring_prep_send_zc() (Linux 6.0+) does a true zero-copy send. Buffer ownership stays with the kernel until you get the completion, at which point the buffer is yours again.

For TLS termination this matters a lot: the user→kernel copy on the unencrypted side is often the dominant cost in a splice+sendfile pipeline. With send_zc plus kTLS you keep the data in kernel-pinned pages from read through to the NIC.

Where io_uring doesn’t help

Worth being explicit about:

• CPU-bound work. If your service spends 80% of its time parsing JSON and 5% in syscalls, you save 5%.
• Small connection counts. Below a few thousand concurrent connections, epoll is fine and arguably easier to reason about.
• Workloads already using splice / sendfile. You’re already mostly zero-copy.
• Mixed-blocking workloads where blocking ops are uncommon. The thread-pool model is genuinely fine.

There is also a non-trivial engineering cost: io_uring’s API surface is large, has changed across kernel versions, and the completion-based model is a different shape from what your existing code expects. Rewriting an epoll-based service to use io_uring properly is weeks, not days.

The sharp edges

Now the parts that are not in the keynote.

Buffer lifetimes

In epoll-land, read(fd, buf, n) either returns immediately or doesn’t. The buffer is yours either way.

In io_uring-land, you submit a buffer, return to your event loop, do other things, and the buffer is still owned by the kernel until the CQE arrives. If you free it, reuse it, or hand it to a different SQE, you have a heap-corruption bug at best and a memory-disclosure bug at worst.

In Rust this means you cannot use &mut [u8] for io_uring buffers; the borrow can’t outlive the function call but the operation can. The mature Rust io_uring crates (tokio-uring, rio, glommio) all use owned buffer types like Box<[u8]> or BytesMut for exactly this reason.

Cancellation is hard

Cancelling an in-flight op requires submitting an io_uring_prep_cancel SQE that points at the original op’s user_data. The cancellation may succeed (the original op returns -ECANCELED), or it may race with the completion (the op finishes and the cancel returns -ENOENT).

A correct cancel-on-timeout typically looks like:

// Original op.
struct io_uring_sqe *sqe = io_uring_get_sqe(&ring);
io_uring_prep_recv(sqe, fd, buf, n, 0);
sqe->user_data = (uintptr_t)req;
sqe->flags |= IOSQE_IO_LINK;

// Linked timeout. Auto-cancels the previous op if it does not finish in 5s.
struct __kernel_timespec ts = { .tv_sec = 5 };
struct io_uring_sqe *to = io_uring_get_sqe(&ring);
io_uring_prep_link_timeout(to, &ts, 0);
to->user_data = (uintptr_t)req_timeout_marker;

io_uring_submit(&ring);

IOSQE_IO_LINK is the feature for this. It chains SQEs so the next one runs only when the previous one finishes (or, with link-timeout semantics, when the timeout fires).

Versioning and feature probing

io_uring features land in kernels continuously. IORING_OP_FOO may not exist on the kernel you’re running on. The portable way to handle this is to call io_uring_get_probe() once at startup and ask whether each op is supported:

struct io_uring_probe *p = io_uring_get_probe();
if (!io_uring_opcode_supported(p, IORING_OP_SEND_ZC)) {
    // fall back to non-zero-copy send
}
io_uring_free_probe(p);

Skipping this is how you ship a binary that works in your dev VM (kernel 6.6) and returns -EINVAL for half its SQEs on the prod 5.10 LTS.

Security

io_uring has had a bumpy security history . Some hyperscalers (Google’s ChromeOS, parts of Android) disable it by default. The blast radius of a kernel bug inside io_uring is wider than in epoll because many more code paths are reachable from a single user-space SQE.

If you run io_uring in production: subscribe to kernel security announcements, run a current LTS kernel, and consider seccomp-ing the io_uring_* syscalls in processes that don’t need them.

A starting setup

The set of flags that has served me well for busy network services:

struct io_uring_params params = {0};
params.flags = IORING_SETUP_SINGLE_ISSUER   // 6.0+: one thread submits
             | IORING_SETUP_COOP_TASKRUN    // 5.19+: don't wake task on completion
             | IORING_SETUP_DEFER_TASKRUN;  // 6.1+: batch completions until enter()

if (io_uring_queue_init_params(4096, &ring, &params) < 0) {
    // ...
}

// Register a fixed buffer pool. Avoids per-op buffer registration overhead.
io_uring_register_buffers(&ring, iovecs, n_iovecs);

// Register file descriptors. Same reason.
io_uring_register_files(&ring, fds, n_fds);

These three flags plus two registrations give you most of the io_uring throughput story. The remaining 10–20% comes from buffer rings (IORING_REGISTER_PBUF_RING) and multishot accept / recv, both worth their own posts.

Should you switch?

Three questions to answer first:

1. Is your service syscall-bound? Profile it. If read/write/recv/send/accept sit at the top of your flamegraph, io_uring will help. If not, look at your application before you look at the kernel interface.
2. Are you on Linux ≥ 6.1 in production? On older LTS releases (Debian 11 ships 5.10, Ubuntu 22.04 ships 5.15, RHEL 9 ships 5.14) you get an older io_uring without SINGLE_ISSUER, without DEFER_TASKRUN, and with a weaker send_zc. The decision is harder; sometimes the answer is “wait for the next LTS.”
3. Do you have engineering bandwidth for the rewrite? It is rarely a small change. Buffer ownership, lifetimes, and error paths all move at once.

If the answer to all three is yes, io_uring is one of the highest-ROI rewrites you can do. If any is no, the right answer is probably “later.”

If you take exactly one thing away: do not ship the rewrite to chase a benchmark you have not run on your actual workload. The wins are real but they are not uniform, and “we got 30% on wrk2” rarely survives contact with the request shape you serve in production.

Further reading:

• Jens Axboe’s original io_uring paper — concise, written by the author.
• Linux Kernel Source, io_uring/, especially io_uring/io_uring.c. Readable and well-commented.
• Programming with io_uring by Jens Axboe, the canonical longer reference.
• tokio-uring — Rust integration, with a useful design document.