A problem only happens in production and you cannot attach a debugger. How do you capture and analyse a memory dump of a .NET process?

1. What is inside a dump

 process memory snapshot (.dmp / core file)
 +-- threads: registers + native and managed stacks
 +-- managed heap: gen 0/1/2, Large Object Heap, Pinned Object Heap
 |     every object, its fields and its references
 +-- loaded modules: DLL images and versions
 +-- runtime data structures: method tables, sync blocks,
                              thread pool state, GC handles, finaliser queue

 analysis = dump + DAC (the runtime's data access component for THAT
            runtime build) + PDBs
          --> readable stacks, objects, roots and exceptions

The DAC (mscordaccore) knows how to read the runtime's internal structures for one specific runtime build. That is why the runtime version matters, and why dotnet-dump and Visual Studio download the matching DAC automatically when they can.

2. Choosing the right diagnostic

dotnet-gcdump is a lighter option for leaks: it records only types, counts, sizes and references, not object contents, so it is much smaller and holds far less sensitive data. Use a full dump when you need field values.

3. Dump types

4. Capturing, platform by platform

The crash dump variables:

dotnet-dump and dotnet-monitor talk to the runtime through its diagnostics IPC channel. If the app sets DOTNET_EnableDiagnostics=0, they can't connect.

5. What collection costs

 dotnet-dump collect
   --> runtime suspends the process
   --> dump writer copies memory to disk     (seconds for a large heap)
   --> process resumes

• A full dump of a 4 GB process writes roughly 4 GB and can pause the process long enough to fail health checks.
• On a multi-instance service, take the dump from one instance, ideally after removing it from the load balancer.
• Check there is enough disk space where the dump is written.

6. The analysis toolkit

Inside dotnet-dump analyze <file>:

Or open the dump in Visual Studio (File > Open, then Debug with Managed Only). The Threads, Call Stack, Parallel Stacks and Tasks windows work as if the process were paused, and the managed memory view sorts types by size and shows paths to root.

7. Leak analysis, step by step

 dump A (800 MB)                      dump B (2.5 GB, six hours later)
 dumpheap -stat                       dumpheap -stat
   Count    TotalSize  Type             Count      TotalSize  Type
   120,000  490 MB     Byte[]           610,000    2.4 GB     Byte[]
   120,000  9 MB       Entry<...>       610,000    45 MB      Entry<...>
                              |
                              v  the type that grew most
 dumpheap -type System.Byte[] -short   --> pick a few addresses
 gcroot 00007f3a2c01d2a8
   static field LeakyCache.Entries
     -> ConcurrentDictionary -> Node -> Byte[]
                              |
                              v
 the holder: an unbounded static dictionary keyed by raw query strings

Common holders:

8. Hang analysis, step by step

1. Take two dumps a few seconds apart.
2. Run clrstack -all in both. Threads with identical stacks in both are stuck; threads whose stacks changed are working.
3. For stuck threads in Monitor.Enter, run syncblk to find the owner. For threads in Lock.Enter (.NET 9+ System.Threading.Lock), read the stacks, because syncblk won't list it.
4. If no thread is blocked but requests never finish, run dumpasync to find the pending async chain.
5. Run threadpool to check for starvation (a long queue, all workers busy).

9. Making dumps readable

• Keep PDBs with every deployment, or in a symbol server, so stacks show file and line.
• Record the exact runtime version deployed (dotnet --list-runtimes in the image, or RuntimeInformation.FrameworkDescription in a log at start-up).
• dotnet-dump works on any OS for dumps from any OS. A Linux core dump can be analysed on Windows and the reverse.
• dotnet-symbol -o ./symbols <dump> downloads matching symbols for the modules in the dump.

10. Security and compliance

A full dump contains everything in memory: access tokens, connection strings, decrypted payloads and customer records. Under the Privacy Act 1988 and the Australian Privacy Principles that is personal information you are responsible for.

• Decide in advance who may collect dumps, and log every collection.
• Store dumps encrypted in a controlled location, for example a private storage account in Australia East with short retention.
• Never attach dumps to tickets, chat or email.
• Prefer dotnet-gcdump or Triage dumps when they are enough.
• Expose dotnet-monitor only with authentication and never publicly.
• Delete dumps when the investigation ends.

11. .NET 11 awareness

The dump format and the diagnostics tools are stable and versioned with the runtime. Update dotnet-dump and dotnet-monitor alongside the runtime, and check the .NET 11 diagnostics release notes for new analysis commands.

12. Try it yourself

1. Run the code example's leaky console app and note its PID.
2. Run dotnet-dump collect -p <pid> --type Heap -o a.dmp, wait 30 seconds, and collect b.dmp.
3. In each, run dumpheap -stat and compare the System.Byte[] counts.
4. Use dumpheap -type System.Byte[] -short, then gcroot on one address to find the static field.
5. Run clrstack -all and threadpool to see what threads are doing.
6. Open b.dmp in Visual Studio, Debug with Managed Only, and compare the experience.
7. Collect dotnet-gcdump collect -p <pid> and compare its size with the heap dump.

A small app that leaks through a static cache, to practise on:

using System.Collections.Concurrent;

Console.WriteLine($"PID {Environment.ProcessId}: run dotnet-dump collect -p {Environment.ProcessId} --type Heap");

var requestId = 0;
while (true)
{
    var key = $"report?customer={Random.Shared.Next()}&page={requestId++}";
    LeakyCache.GetOrAdd(key);
    if (requestId % 10_000 == 0)
    {
        Console.WriteLine($"{requestId} requests, {GC.GetTotalMemory(false) / 1_048_576} MB");
    }
    await Task.Delay(1);
}

public static class LeakyCache
{
    // Unbounded, keyed by raw query strings: grows forever.
    private static readonly ConcurrentDictionary<string, byte[]> Entries = new();

    public static byte[] GetOrAdd(string key) => Entries.GetOrAdd(key, _ => new byte[4_096]);
}

Capture and analyse:

dotnet tool install --global dotnet-dump
dotnet-dump collect -p 12345 --type Heap -o ./dumps/api-1.dmp
dotnet-dump analyze ./dumps/api-1.dmp
> dumpheap -stat
> dumpheap -type System.Collections.Concurrent.ConcurrentDictionary -short
> gcroot 00007f3a2c01d2a8
> clrstack -all
> pe

Automatic crash dumps (container or App Service environment variables):

DOTNET_DbgEnableMiniDump=1
DOTNET_DbgMiniDumpType=4
DOTNET_DbgMiniDumpName=/home/dumps/%e.%p.dmp

The fixed cache with a size limit and expiry (Microsoft.Extensions.Caching.Memory):

using Microsoft.Extensions.Caching.Memory;

var cache = new MemoryCache(new MemoryCacheOptions { SizeLimit = 10_000 });

byte[] GetReport(string normalisedKey) =>
    cache.GetOrCreate(normalisedKey, entry =>
    {
        entry.Size = 1;
        entry.SlidingExpiration = TimeSpan.FromMinutes(10);
        return new byte[4_096];
    })!;

Console.WriteLine(GetReport("report:customer:42:page:1").Length);