What is the difference between ThreadPool worker threads and I/O completion threads, and does it matter on Linux?

1. Windows: I/O completion ports

An IOCP is a kernel queue of completion packets. Handles (sockets, files opened with overlapped I/O) are associated with the port. When an overlapped operation finishes, the kernel posts a packet; a thread blocked in GetQueuedCompletionStatus dequeues it.

ReadAsync -> ReadFile(OVERLAPPED) returns ERROR_IO_PENDING
             ... device works ... (no thread)
kernel    -> posts completion packet to IOCP
poller    -> GetQueuedCompletionStatusEx dequeues batch
          -> completes the Task / IValueTaskSource
          -> continuation queued to worker pool

In .NET Framework, a pool of "I/O threads" ran the callbacks directly. Since .NET 7, the managed portable thread pool handles IOCP with a few poller threads that dequeue completions in batches and dispatch the work to workers.

2. Linux: epoll and SocketAsyncEngine

epoll is a readiness model, not a completion model: it tells you a socket can be read or written without blocking.

ReceiveAsync -> try recv() non-blocking
                 -> data available: complete synchronously (no wait at all)
                 -> EAGAIN: register interest, return pending
epoll thread -> epoll_wait() reports socket readable
             -> performs recv(), completes the operation
             -> continuation queued to worker pool

• The runtime creates a small number of SocketAsyncEngine instances (each with an epoll thread), scaled with processor count.
• Many operations complete synchronously when data is already available, avoiding any thread hop.
• An environment switch (DOTNET_SYSTEM_NET_SOCKETS_INLINE_COMPLETIONS) can run continuations directly on epoll threads; it can reduce latency for some workloads but risks stalling the event loop, so it's an expert-only setting.

3. File I/O on Linux

Regular files are always "ready" from epoll's perspective, so readiness doesn't help. .NET's FileStream/RandomAccess async APIs on Unix run the blocking pread/pwrite calls on pool threads. That's correct and usually fast (page cache), but:

• Each concurrent slow read (cold disk, network file system) occupies a pool thread.
• Thousands of concurrent file reads can create pool pressure.
• Linux has io_uring for true async file I/O, but .NET's file stack doesn't use it by default; check the release notes of your runtime version before relying on it.

4. "There is no thread": the scaling math

5. Where continuations run

So CPU-heavy code after an await competes for worker threads, never for I/O pollers.

6. Version notes

7. Try it yourself

1. Run the code example and confirm 200 pending socket reads use only a handful of pool threads.
2. Change the client to synchronous stream.Read inside Task.Run and watch thread count and queue length climb.
3. On Linux, start 500 concurrent File.ReadAllBytesAsync calls on large files on a slow volume and watch thread count; repeat on Windows.
4. Print GetAvailableThreads on Linux and Windows; observe that the I/O number doesn't track real activity.
5. Capture dotnet-trace with the Microsoft-Windows-DotNETRuntime:ThreadPool keyword and see worker vs I/O events.

using System.Net.Sockets;
using System.Threading;

ThreadPool.GetAvailableThreads(out int workers, out int io);
Console.WriteLine($"Available workers {workers}, completion port {io}");

using var listener = new TcpListener(System.Net.IPAddress.Loopback, 0);
listener.Start();
int port = ((System.Net.IPEndPoint)listener.LocalEndpoint).Port;

var clients = new List<Task>();
for (int i = 0; i < 200; i++)
{
    clients.Add(Task.Run(async () =>
    {
        using var client = new TcpClient();
        await client.ConnectAsync("127.0.0.1", port);
        var buffer = new byte[1];
        // Pending read: no thread is blocked while waiting.
        await client.GetStream().ReadAsync(buffer).AsTask()
            .WaitAsync(TimeSpan.FromSeconds(2))
            .ContinueWith(_ => { });
    }));
}

await Task.Delay(500);
Console.WriteLine($"200 pending reads, pool threads = {ThreadPool.ThreadCount}");
await Task.WhenAll(clients);