How does Parallel.ForEachAsync work, and how does it compare with Task.WhenAll and SemaphoreSlim for concurrent I/O?

1. How it works internally (simplified)

Parallel.ForEachAsync(source, options, body):
    create linked CTS (options.CancellationToken)
    start N worker tasks (N = MaxDegreeOfParallelism)
    each worker:
        loop:
            lock source enumerator; MoveNext / MoveNextAsync; take item
            await body(item, linkedToken)
            on exception: record it, cancel linked CTS, stop
    await all workers; throw aggregated exceptions if any

The workers share one enumerator, which is why items are pulled lazily and why IAsyncEnumerable<T> sources work naturally.

2. The Parallel.ForEach + async lambda bug

Parallel.ForEach(ids, async id => await SyncAsync(id));   // compiles: lambda is Action<int> -> async void
Console.WriteLine("Done");                                  // prints immediately; work still running; exceptions crash

Parallel.ForEach waits for the synchronous part of each delegate only. Use Parallel.ForEachAsync.

3. Continue-on-error pattern

var failures = new ConcurrentQueue<(int Id, Exception Error)>();

await Parallel.ForEachAsync(ids, new ParallelOptions { MaxDegreeOfParallelism = 16, CancellationToken = ct },
    async (id, token) =>
    {
        try { await SyncAsync(id, token); }
        catch (Exception ex) when (ex is not OperationCanceledException) { failures.Enqueue((id, ex)); }
    });

if (!failures.IsEmpty) await WriteFailureReportAsync(failures, ct);

4. Ordered results without locks

var results = new Customer[ids.Length];
await Parallel.ForEachAsync(Enumerable.Range(0, ids.Length), opts, async (i, token) =>
    results[i] = await LoadAsync(ids[i], token));   // each index written by exactly one worker

5. Related APIs

6. Observability

Log or meter: items processed, failures, per-item latency, and in-flight count. A degree set too high shows up as rising dependency latency and 429/503s; too low shows up as idle dependency capacity and long batch times.

7. Try it yourself

1. Run the code example: 200 items × 100 ms at degree 20 ≈ 1 s; check the failure count.
2. Remove the try/catch and see the loop stop on the first failure.
3. Replace with Parallel.ForEach(ids, async id => ...) and observe "Done" printing immediately.
4. Replace with unbounded Task.WhenAll against a local test server with a connection limit and watch errors.
5. Feed an IAsyncEnumerable<int> source that yields slowly and confirm items are processed as they arrive.
6. Vary the degree (4, 16, 64) against a test API and plot throughput vs latency.

using System.Collections.Concurrent;
using System.Diagnostics;

int[] ids = Enumerable.Range(1, 200).ToArray();
var results = new string[ids.Length];
var failures = new ConcurrentBag<(int Id, string Error)>();

var sw = Stopwatch.StartNew();
await Parallel.ForEachAsync(
    ids.Select((id, index) => (id, index)),
    new ParallelOptions { MaxDegreeOfParallelism = 20 },
    async (item, token) =>
    {
        try
        {
            results[item.index] = await FetchAsync(item.id, token); // keeps input order
        }
        catch (HttpRequestException ex)
        {
            failures.Add((item.id, ex.Message)); // continue on per-item failure
        }
    });

Console.WriteLine($"Done in {sw.ElapsedMilliseconds} ms, failures: {failures.Count}");
Console.WriteLine(results[0]);

// .NET 8+: integer ranges
await Parallel.ForAsync(0, 5, async (i, token) => await Task.Delay(10, token));

static async Task<string> FetchAsync(int id, CancellationToken ct)
{
    await Task.Delay(100, ct); // stands in for an HTTP call
    if (id % 50 == 0) throw new HttpRequestException($"Partner rejected {id}");
    return $"customer-{id}";
}