A .NET service that processes large batches spends a lot of time in garbage collection, and memory profiling shows most allocations are short-lived arrays. How do you make array-heavy code fast? Explain ArrayPool, the Large Object Heap, cache locality, copying APIs, spans and SIMD.

1. Why short-lived arrays hurt

The .NET GC is generational: new objects go into Gen0, which is collected frequently and cheaply if most objects are dead. Short-lived small arrays should be cheap — but:

• A high allocation rate (hundreds of MB/s) triggers Gen0 GCs many times per second; each pauses threads (briefly) and touches caches.
• Objects alive at a Gen0 GC (e.g. buffers in flight across an await) get promoted to Gen1/Gen2, making later collections more expensive ("mid-life crisis").
• Large objects (≥ 85,000 bytes) skip Gen0 and go straight to the LOH, which is logically part of Gen2. Allocating many large temporary arrays forces Gen2 collections, which scan the whole heap, and the LOH isn't compacted by default, so it fragments.
• Every new array must be zeroed by the runtime, which costs memory bandwidth for large buffers.

2. How ArrayPool<T>.Shared works

The shared pool (TlsOverPerCoreLockedStacksArrayPool<T>):

• Buckets arrays by power-of-two sizes (16, 32, 64, ... up to ~1 GB in .NET 8+; earlier versions capped at 1 MB for pooling).
• Keeps a small thread-local cache per bucket for the fastest path, then per-core stacks with light locking.
• Trims pooled arrays under memory pressure (registered with the GC's memory pressure callbacks).
• Rent(n) rounds up to the bucket size; requests larger than the largest bucket allocate a fresh array that isn't pooled on return.

Pooled arrays are not cleared on Rent — they contain old data. Never assume zeros, and pass clearArray: true on Return for secrets.

3. Ownership bugs with pooling

Pooling reintroduces manual memory-management bugs:

• Use after return: another caller rents the same array and you overwrite each other's data.
• Double return: the array ends up in the pool twice and is handed to two callers.
• Leak: forgetting to return just loses the pooling benefit (the GC still collects it) — the least harmful bug.

Mitigations: try/finally, clear ownership (IMemoryOwner<T> + using), keep rent/return in the same method where possible, and code review.

4. stackalloc rules

const int MaxStack = 256;
byte[]? rented = null;
Span<byte> buf = length <= MaxStack
    ? stackalloc byte[MaxStack]
    : (rented = ArrayPool<byte>.Shared.Rent(length));
try { /* use buf[..length] */ }
finally { if (rented is not null) ArrayPool<byte>.Shared.Return(rented); }

Stack space is ~1 MB per thread by default (less on some platforms); large or input-dependent stackalloc risks StackOverflowException, which cannot be caught. Use a constant size and fall back to the pool. [SkipLocalsInit] avoids zeroing stack buffers in hot methods.

5. The Pinned Object Heap and uninitialised arrays

• GC.AllocateArray<T>(n, pinned: true) (.NET 5+) allocates on the Pinned Object Heap, for long-lived buffers passed to native I/O, avoiding fragmentation from fixed/GCHandle pinning in the normal heap.
• GC.AllocateUninitializedArray<T>(n) skips zeroing for large buffers you will fully overwrite immediately (only for unmanaged T).

6. Cache hierarchy numbers (typical server CPU)

Sequential access lets hardware prefetchers load lines before you need them; pointer-chasing through arrays of object references defeats them. That is why struct[] and spans are often 2–10× faster than equivalent class[] traversals on large datasets.

7. SIMD in .NET

• Vector<T>: hardware-sized vectors (16 bytes on SSE, 32 on AVX2, 64 with AVX-512 when enabled).
• Vector128/256/512<T>: fixed sizes with cross-platform operations; Vector512.IsHardwareAccelerated checks.
• TensorPrimitives: ready-made vectorised operations (Sum, Dot, Add, Multiply, Max, CosineSimilarity...) over spans — the recommended starting point.
• The BCL vectorises IndexOf, SequenceEqual, Fill, CopyTo, Base64, UTF-8 transcoding and more automatically.

Vectorisation needs contiguous data (arrays/spans of primitives or blittable structs) — another reason to fix data layout first.

8. Measuring improvement

Use BenchmarkDotNet with [MemoryDiagnoser] for micro-level changes (bytes allocated per operation should drop to ~0 on hot paths), then re-run dotnet-counters under realistic load to confirm GC time and allocation rate improved end to end. Watch for regressions in memory retention (pools hold memory) using dotnet-gcdump.

Try it yourself

1. Write a loop that allocates a 100 KB array per iteration and watch loh-size and gen-2-gc-count in dotnet-counters; switch to ArrayPool and compare.
2. Benchmark summing a float[] with a loop, Vector<float>, and TensorPrimitives.Sum.
3. Benchmark iterating 1 million class points vs struct points.
4. Deliberately use a pooled array after returning it in a multi-threaded test and observe corruption — then fix the ownership.

using System.Buffers;
using System.Numerics.Tensors;
using System.Security.Cryptography;

namespace Arrays;

public readonly record struct Sample(float Value, int SensorId);   // struct: contiguous in arrays

public static class ArrayPerformance
{
    // Hash a stream with a pooled buffer: no per-call large allocation
    public static async Task<byte[]> HashAsync(Stream input, CancellationToken ct)
    {
        byte[] buffer = ArrayPool<byte>.Shared.Rent(128 * 1024);   // may be larger than requested
        try
        {
            using var hash = IncrementalHash.CreateHash(HashAlgorithmName.SHA256);
            int read;
            while ((read = await input.ReadAsync(buffer.AsMemory(0, 128 * 1024), ct)) > 0)
            {
                hash.AppendData(buffer, 0, read);                     // use only the bytes read
            }
            return hash.GetHashAndReset();
        }
        finally
        {
            ArrayPool<byte>.Shared.Return(buffer, clearArray: true); // documents may hold personal data
        }
    }

    // Vectorised maths over contiguous data
    public static float Average(ReadOnlySpan<float> values)
        => values.IsEmpty ? 0 : TensorPrimitives.Sum(values) / values.Length;

    // Large buffer that will be fully overwritten: skip zeroing
    public static byte[] CreateScratch(int size) => GC.AllocateUninitializedArray<byte>(size);

    public static void Main()
    {
        float[] readings = new float[1_000];
        readings.AsSpan().Fill(21.5f);                                  // vectorised fill
        Console.WriteLine(Average(readings));                           // 21.5

        Sample[] samples = [new(1.5f, 1), new(2.5f, 2)];
        float total = 0;
        foreach (ref readonly var s in samples.AsSpan()) total += s.Value;  // no copies, no checks
        Console.WriteLine(total);

        using var ms = new MemoryStream("hello"u8.ToArray());
        Console.WriteLine(Convert.ToHexString(HashAsync(ms, default).GetAwaiter().GetResult()));
    }
}