How does the operating system scheduler decide which thread runs next, and what do preemption, time slices, priorities and container CPU limits mean for a .NET service?

1. CFS quota throttling, drawn

limit = 1 CPU  ->  quota 100 ms CPU per 100 ms period

period 1:  [####################.....................]   used 100 ms in the first 25 ms
            8 threads busy x 12.5 ms = 100 ms  ->  THROTTLED for 75 ms
period 2:  [###.........................................]   normal
period 3:  [####################.....................]   burst again -> throttled

Average CPU looks like ~40%, but requests arriving during a throttled window wait up to 75 ms.

More threads make throttling worse: they burn the shared quota in parallel.

2. Reading throttling data

# Inside the container (cgroup v2)
cat /sys/fs/cgroup/cpu.stat
# nr_periods 12000
# nr_throttled 3600        <- 30% of periods throttled
# throttled_usec 210000000

# Prometheus / Azure Monitor metric
# container_cpu_cfs_throttled_periods_total / container_cpu_cfs_periods_total

3. How .NET reacts to CPU limits

Override with DOTNET_PROCESSOR_COUNT when you have a reason (for example a fractional limit where rounding up isn't what you want).

4. Sizing guidance

• Set CPU requests from measured steady-state usage so the scheduler places pods sensibly.
• Set limits high enough to absorb bursts (start-up, JIT, GC, traffic spikes), or omit CPU limits where your platform policy allows and rely on requests for fairness.
• Watch throttling, not just average CPU.
• Remember that start-up (JIT compilation, ReadyToRun loading, cache warm-up) is CPU-heavy; tight limits make cold starts and readiness slow.

5. Starvation, priority inversion and why locks matter

• Starvation: a low-priority thread never runs because higher-priority ones are always runnable.
• Priority inversion: a high-priority thread waits for a lock held by a low-priority thread that can't run. OSes mitigate it with boosts, but the cleanest fix is short critical sections and no priority games.

6. Affinity and NUMA

• On large VMs with several NUMA nodes, keeping threads and their memory on the same node avoids remote memory access. The GC has NUMA awareness; GCHeapAffinitizeMask and related settings exist for specialised tuning.
• Most services shouldn't set affinity manually.

7. Try it yourself

1. Run a CPU-bound loop on 8 threads in a container with docker run --cpus=1 and print Environment.ProcessorCount; read /sys/fs/cgroup/cpu.stat before and after.
2. Measure the latency of a tiny request every 10 ms while the CPU-bound work runs, and observe the spikes caused by throttling.
3. Repeat with --cpus=4 and compare.

namespace Scheduling;

using System.Diagnostics;

public static class Program
{
    public static async Task Main()
    {
        Console.WriteLine($"ProcessorCount (respects container CPU limit): {Environment.ProcessorCount}");
        ThreadPool.GetMinThreads(out int minWorkers, out int minIo);
        Console.WriteLine($"Thread pool min workers: {minWorkers}, min I/O: {minIo}");

        using var cts = new CancellationTokenSource(TimeSpan.FromSeconds(10));

        // Background CPU burners
        var burners = Enumerable.Range(0, 8)
            .Select(_ => Task.Run(() => { while (!cts.IsCancellationRequested) { } }))
            .ToArray();

        // Measure how late a 10 ms tick runs: large values indicate throttling or run-queue waits
        var worst = TimeSpan.Zero;
        var sw = Stopwatch.StartNew();
        while (!cts.IsCancellationRequested)
        {
            var before = sw.Elapsed;
            await Task.Delay(10);
            var lateness = sw.Elapsed - before - TimeSpan.FromMilliseconds(10);
            if (lateness > worst) worst = lateness;
        }
        await Task.WhenAll(burners);
        Console.WriteLine($"Worst scheduling delay: {worst.TotalMilliseconds:F1} ms");

        if (File.Exists("/sys/fs/cgroup/cpu.stat"))
        {
            Console.WriteLine(await File.ReadAllTextAsync("/sys/fs/cgroup/cpu.stat"));
        }
    }
}

docker run --rm --cpus=1 my-scheduling-demo   # compare with --cpus=4