epoll is Linux’s readiness-notification API: the application registers file descriptors, then asks the kernel which of those descriptors are ready for an operation such as read or write.

Prerequisites

Read IO Selectors first. eventfd explains how another thread can wake an epoll loop. Threads helps with the thread-per-connection baseline, and OS Page Cache helps with the disk-I/O caveats.

The diagram separates readiness from completion. epoll tells the application “this descriptor should not block now”; it does not perform the read or write for the application.

The Problem

A blocking file descriptor call waits inside the kernel until the operation can make progress. A thread can block in read waiting for bytes from a pipe, accept waiting for a new TCP connection, or write waiting for buffer space.

One thread per descriptor is easy to understand because each thread follows one blocking operation at a time. The cost is that concurrency is paid for with more kernel-scheduled threads. A selector loop pays with explicit state instead: the loop waits for readiness events, then advances whichever descriptor can make progress.

The Mechanism

An epoll instance is an in-kernel object with two conceptual lists:

StructureMeaning
Interest listFile descriptors the process asked the kernel to watch.
Ready listWatched descriptors that the kernel currently considers ready for one or more requested operations.

The usual lifecycle is:

  1. epoll_create1 creates an epoll instance and returns an epoll file descriptor.
  2. epoll_ctl adds, modifies, or removes watched descriptors.
  3. epoll_wait blocks until the ready list has events, then copies a batch of events back to user space.
  4. The application calls read, write, accept, or another operation on each ready descriptor.

epoll_wait returns readiness, not bytes. After a socket becomes readable, the application still has to call read. After a socket becomes writable, the application still has to call write.

Level-Triggered And Edge-Triggered

epoll has two common delivery modes:

ModeMeaningPractical rule
Level-triggeredKeep reporting the descriptor while the readiness condition remains true.Simpler; repeated notifications are expected until the condition is cleared.
Edge-triggeredReport when the descriptor changes into a ready state.Use non-blocking descriptors and drain reads/writes until EAGAIN.

EAGAIN means “try again later”: for a non-blocking descriptor, the operation would have blocked, so the event loop should stop working on that descriptor and return to epoll_wait.

Edge-triggered mode is easy to misuse. If a socket receives 2 KiB, the event loop reads only 1 KiB, and no new data arrives, there may be no second edge to wake the loop. The unread 1 KiB remains in the socket buffer, but the application can sleep because it failed to drain the descriptor.

What epoll Can Watch Well

epoll works naturally with descriptors whose readiness changes over time, such as sockets, pipes, timers, and event file descriptors. Linux epoll_ctl rejects regular disk files with EPERM, so a program cannot register an ordinary file and wait for “file read completed” events.

Disk I/O can still block. Opening or reading a file can wait on metadata, page-cache misses, storage queue slots, or filesystem work. epoll is not the right API for being notified when a regular file read has completed.

The boundary looks like this:

Operationepoll helps?Why
Accept a new TCP connectionYesThe listening socket becomes readable when a connection can be accepted.
Read bytes from a socket or pipeYesThe descriptor becomes readable when bytes are queued.
Write bytes to a socket or pipeYesThe descriptor becomes writable when buffer space exists.
Wait for a timerYestimerfd exposes timer expiration as descriptor readiness.
Receive a thread notificationYeseventfd exposes application notifications as descriptor readiness.
Read a regular disk fileNoRegular files are not supported by epoll_ctl; file reads are not reported as completion events.

Programs that must avoid blocking on disk add another mechanism: a disk-I/O thread pool, asynchronous file APIs, sendfile where it fits, or io_uring.

State Lives In The Application

The blocking function style stores progress on the call stack. An event-driven program stores progress in application-owned objects because work resumes over multiple readiness events.

For a descriptor that receives bytes and later writes bytes, that object usually needs:

FieldWhy it exists
File descriptorThe descriptor to read from or write to.
Input buffer and offsetInput may arrive over multiple reads.
Output buffer, length, and offsetwrite may accept only part of the pending output.
State enumThe event loop needs to know whether the descriptor is reading, writing, waiting for external work, or closing.

Partial writes happen routinely. A successful write(fd, buf, 16384) may return 4096, meaning only 4 KiB reached the socket buffer. The event loop records the remaining 12 KiB and waits for another writable event.

The HTTP file-server state machine belongs in Multi-Threaded epoll HTTP File Server Architecture. The epoll note only covers the readiness mechanism.

When To Use

Use epoll when the workload is mostly descriptor readiness: many network sockets, pipes, timers, or eventfd-style coordination objects. It is a good fit for a single-threaded or per-core event loop where the application owns the state machine.

Avoid treating epoll as a universal asynchronous I/O interface. It multiplexes readiness for suitable file descriptors; it does not turn arbitrary blocking work into completion events.

See also

Sources

  • Linux man-pages: epoll(7) and epoll_ctl(2).