A tour of CPU architecture for software engineers: cores and hardware threads, the register file, the ALU and other execution units, the control unit, the cache hierarchy and the clock, and why each matters for .NET performance.
A CPU has one or more cores, each able to run instructions independently; with SMT (Hyper-Threading) one core presents two logical processors that share its execution units. Inside a core, registers hold the values being worked on, the ALU does integer arithmetic and logic (with separate units for floating point and SIMD), and the control unit fetches and decodes instructions and directs data between registers, execution units and memory. Caches (L1, L2 per core, L3 shared) keep recently used data close, because RAM is about 100 times slower than a register. A clock drives it all, but real speed depends on how many instructions complete per cycle and how often you wait for memory.
| Part | What it does | Why a .NET developer cares |
|---|---|---|
| Core | Independent instruction pipeline with its own registers and execution units | Parallelism: Parallel.For, the thread pool and Server GC scale with cores |
| Hardware thread (SMT) | Second instruction stream sharing one core's units | Environment.ProcessorCount counts these; two SMT threads don't give 2× throughput |
| Registers | A few dozen named slots inside the core (x64: 16 general-purpose, plus vector registers; Arm64: 31 general-purpose, plus 32 vector) | The JIT's register allocator decides which locals live here; spills to the stack cost time |
| ALU | Integer add, subtract, compare, bitwise logic, shifts | Simple integer work is cheap: about one cycle |
| FPU and vector units | Floating point and SIMD (SSE/AVX on x64, NEON on Arm64) | Vector128<T>, Vector256<T> and vectorised BCL methods use them |
| Load/store units and AGUs | Compute addresses and move data to and from memory | Memory access, not arithmetic, is usually the bottleneck |
| Control unit | Fetches, decodes and schedules instructions; handles branches | Branch mispredictions and pipeline stalls come from here |
| Caches | L1 (per core, ~32–64 KB data + instruction), L2 (per core, ~1–2 MB), L3 (shared, tens of MB) | Data layout and access patterns decide performance |
| Clock | Ticks a few billion times per second (3–5 GHz) | One cycle is about 0.2–0.3 ns; a RAM access is hundreds of cycles |
| Operation | Rough cost |
|---|---|
| Register access, integer add | ~1 cycle |
| L1 cache hit | ~4 cycles (~1 ns) |
| L2 hit | ~12 cycles |
| L3 hit | ~40 cycles |
| Main memory | ~200–300 cycles (~60–100 ns) |
| Branch mispredict | ~15–20 cycles |
A team moved an API from 4 vCPU to 8 vCPU containers expecting double throughput and got about 30%. The 8 vCPUs were 4 physical cores with SMT, and the hot path was memory-bound (cache misses on a large dictionary). Reducing the working set (smaller keys, a FrozenDictionary for read-only data) gave more than the extra vCPUs did.
Checks a working mental model of the hardware that code runs on.
Green flags: names the parts and what each does, relates them to .NET (JIT registers, SIMD, ProcessorCount, GC), knows rough latency numbers, understands SMT.
Red flags: vague "the CPU does the calculations"; no idea what registers or caches are for; believes GHz is the main performance factor.
+--------------------------- core ----------------------------+
| instruction -> decode -> scheduler -> execution units |
L1i ----> | fetch (IP) (control unit) ALU ALU FPU |
| SIMD load/store|
| register file: RAX..R15 / X0..X30, vector regs |
L1d <---> | |
+-------------------------------------------------------------+
|
L2 (per core)
|
L3 (shared by cores) <---> RAM (DDR)
| Kind | x64 | Arm64 |
|---|---|---|
| General-purpose | 16 × 64-bit (RAX, RBX, RCX, RDX, RSI, RDI, RBP, RSP, R8–R15) | 31 × 64-bit (X0–X30) plus SP |
| Instruction pointer | RIP | PC |
| Flags | RFLAGS | NZCV condition flags |
| Vector | 16 XMM/YMM (128/256-bit); 32 ZMM (512-bit) with AVX-512 | 32 × 128-bit (V0–V31) |
The calling convention says which registers carry arguments (for example RCX, RDX, R8, R9 on Windows x64; X0–X7 on Arm64). The JIT follows the platform convention, which is why small methods with few parameters are cheap to call, and why inlining (removing the call) helps even more.
Environment.ProcessorCount returns logical processors, or the CPU limit when running in a container with a quota (for example --cpus=2 gives 2).Parallel defaults are derived from it. Over-reporting CPUs (for example in a container without limits on a 64-core host) leads to too many GC heaps and threads.DOTNET_PROCESSOR_COUNT overrides it when needed.Performance ≈ instructions × cycles per instruction ÷ clock rate. A 3 GHz core that averages 4 instructions per cycle beats a 4 GHz core waiting on memory half the time. That's why data layout, allocations and branch behaviour matter more than raw GHz.
Environment.ProcessorCount on your machine and inside docker run --cpus=2.lscpu (Linux) or look at Task Manager → Performance → CPU (Windows) and find cores, logical processors and cache sizes.DOTNET_JitDisasm on a small method with a few locals and identify which registers the JIT used for arguments and the return value.namespace CpuParts;
using System.Runtime.InteropServices;
using System.Runtime.Intrinsics;
public static class Program
{
public static void Main()
{
Console.WriteLine($"Architecture: {RuntimeInformation.ProcessArchitecture}");
Console.WriteLine($"Logical processors: {Environment.ProcessorCount}");
Console.WriteLine($"64-bit process: {Environment.Is64BitProcess}");
Console.WriteLine($"Vector128 hardware: {Vector128.IsHardwareAccelerated}");
Console.WriteLine($"Vector256 hardware: {Vector256.IsHardwareAccelerated}");
Console.WriteLine($"Vector512 hardware: {Vector512.IsHardwareAccelerated}");
// Integer division by a constant: the JIT turns this into multiply + shift.
Console.WriteLine(DivideBy10(12345));
}
private static int DivideBy10(int value) => value / 10;
}
# Show the generated machine code and the registers used
DOTNET_JitDisasm=DivideBy10 dotnet run -c Release
# Linux: cores, threads per core and cache sizes
lscpu | grep -E 'Model name|Thread|Core|cache'
Mistake: Treating logical processors as full cores.
Why it is wrong: two SMT threads share one core's execution units and caches, so CPU-bound .NET code (for example a Parallel.For over a large array) rarely scales 2x when vCPUs double.
Instead: check physical cores versus logical processors and measure scaling with BenchmarkDotNet or a load test before buying more vCPUs.
Mistake: Thinking the ALU is the bottleneck.
Why it is wrong: an integer add takes about a cycle; a cache miss to RAM takes hundreds. Most slow .NET code is waiting on data: pointer-chasing through List<object> of reference types, large dictionaries, LINQ chains that allocate.
Instead: profile cache misses and allocations first. Prefer arrays of structs, Span<T> and smaller working sets for hot paths.
Mistake: Equating clock speed with performance. Why it is wrong: instructions per cycle, cache sizes and branch prediction vary hugely between CPU generations and workloads. Instead: benchmark the real workload on the real VM size.
Mistake: Ignoring container CPU limits.
Why it is wrong: if limits are missing, the process sees every host core and creates too many GC heaps and thread-pool threads, wasting memory and adding contention.
Instead: set CPU limits, check Environment.ProcessorCount inside the container, and use DOTNET_PROCESSOR_COUNT or DOTNET_GCHeapCount when needed.
Mistake: Assuming every machine has the same instruction set.
Why it is wrong: AVX-512 or particular Arm64 features may be missing on the production VM, so code tuned on a developer laptop behaves differently.
Instead: branch on Vector512.IsHardwareAccelerated or Avx2.IsSupported, and always keep a scalar fallback (or use the cross-platform Vector128/256 APIs).
Seniors use CPU architecture to reason about performance, not to recite parts. The working model is: arithmetic is nearly free, predictable branches are cheap, memory is expensive unless it is in cache, and cores only help if the work is parallel and does not fight over shared data.
| Part | Typical cost | .NET lever |
|---|---|---|
| Registers / ALU | ~1 cycle | Inlining, fewer locals in hot loops, let the JIT optimise (Release, tiered PGO) |
| SIMD units | 4-16 values per instruction | Vector128/256/512<T>, vectorised BCL (Span.IndexOf, Sum) |
| L1/L2/L3 | 4 / 12 / 40 cycles | Contiguous arrays, structs, smaller objects, FrozenDictionary for read-only data |
| RAM | 200-300 cycles | Fewer allocations, avoid pointer-chasing, reduce working set |
| Branch predictor | 15-20 cycles per mispredict | Predictable branches, branch-free code in tight loops |
| Cores / SMT | Parallelism, shared caches | Size thread pool and GC from real core counts, avoid false sharing |
This model explains why allocation-heavy code is slow, why a smaller data structure beats a cleverer algorithm at small sizes, and why doubling vCPUs often disappoints. It also drives cloud sizing: compute-optimised versus memory-optimised VM families, and Arm64 instances for price-performance. Security note: shared cores and caches are also the root of side-channel attacks (Spectre-class), which is why cloud providers isolate tenants by core and why some mitigations cost measurable performance.
The CPU repeats one logical loop billions of times a second. Fetch: read the instruction at the address in the instruction pointer (RIP on x64, PC on Arm64) from the L1 instruction cache and move the pointer on. Decode: work out the operation and operands (x64 instructions become micro-operations). Execute: run it on the ALU or another unit, then access memory and write back the result. Jumps, calls and returns load a new value into the instruction pointer, which is how if, loops and method calls work.
The .NET JIT compiles IL into optimised machine code (tiered: quick Tier 0 first, then optimised Tier 1 with dynamic PGO for hot methods) and writes it into executable memory. The CPU then runs it exactly like native code. Real cores pipeline these stages and execute many instructions out of order in parallel, while still retiring results in program order so the code behaves as if it ran one instruction at a time.
Pipelining splits instruction processing into stages and overlaps them, like an assembly line: while one instruction executes, the next is decoding and the one after is being fetched. Modern cores are also superscalar (they issue several instructions per cycle to multiple execution units) and out-of-order (they run any instruction whose inputs are ready, using register renaming and a reorder buffer, then retire results in program order). The pipeline stalls when an instruction waits for a previous result (a data dependency), when a branch is mispredicted and work must be thrown away, or when data isn't in cache. In C#, a loop where each iteration depends on the last, like a single running sum, can't use that parallelism; multiple accumulators or SIMD can.
x64 comes from a CISC lineage with variable-length instructions and strong memory ordering. Arm64 is a RISC load/store design with fixed 4-byte instructions, more registers, and a weaker memory model. For .NET, IL and pure managed code run unchanged because the JIT targets whichever CPU it's on. What needs checking: native dependencies must have arm64 builds, container images must be multi-arch, RIDs such as linux-arm64 must be published, hand-written SIMD must have Arm64 or cross-platform paths, and code with data races that happened to work on x64 can break on Arm64. Then benchmark, because Arm64 VMs such as Azure's Cobalt-based sizes often give better price-performance for web workloads.
Last updated 23 Oct 2026.