How do the concurrent collections in .NET work? Explain ConcurrentDictionary (including the GetOrAdd and AddOrUpdate caveats), ConcurrentQueue, ConcurrentStack, ConcurrentBag and BlockingCollection, why compound operations are still not atomic, and when you would use Channel<T> instead.

How I would answer this in an interview

The concurrent collections in System.Collections.Concurrent make individual operations thread-safe using fine-grained locks or lock-free techniques. The senior points are: knowing which caveats each one has (especially ConcurrentDictionary.GetOrAdd and AddOrUpdate running factories more than once), that sequences of operations are still not atomic, and that for producer/consumer pipelines Channel<T> is usually the better modern choice.

1. ConcurrentDictionary<TKey,TValue>

• Reads (TryGetValue, indexer get) are lock-free.
• Writes take a lock on one stripe of buckets (a fixed array of locks, by default related to processor count), so different keys can be written concurrently.
• Atomic operations: TryAdd, TryRemove, TryUpdate(key, new, comparisonValue), GetOrAdd, AddOrUpdate, TryRemove(KeyValuePair) (remove only if value matches).

The GetOrAdd/AddOrUpdate caveat: the value factory runs outside the lock. If two threads call GetOrAdd(key, factory) for a missing key at the same time, both factories can run; only one result is stored and both callers get that stored value, but the losing factory's work (and side effects) happened anyway. For AddOrUpdate, the update factory can run multiple times under contention (it retries with TryUpdate).

Consequences: expensive or side-effecting factories (database calls, creating HttpClients, opening connections) may execute more than once. Fix with Lazy<T>:

private readonly ConcurrentDictionary<string, Lazy<Task<Rates>>> _cache = new();

public Task<Rates> GetRatesAsync(string currency) =>
    _cache.GetOrAdd(currency, static c => new Lazy<Task<Rates>>(() => LoadRatesAsync(c))).Value;

The Lazy objects are cheap and may be created twice, but only the stored one's Value is ever evaluated, and LazyThreadSafetyMode.ExecutionAndPublication (default) ensures it runs once. Caching the Task also means concurrent callers share a single in-flight load (no cache stampede) — but remember to evict faulted tasks.

Other notes: Count, IsEmpty, Keys, Values, ToArray() take all locks (expensive); enumeration is lock-free and not a snapshot — it may or may not see concurrent changes but never throws.

2. ConcurrentQueue<T> and ConcurrentStack<T>

• Lock-free FIFO and LIFO using Interlocked operations. ConcurrentQueue<T> uses linked array segments; ConcurrentStack<T> is a linked list of nodes (allocates per push).
• API: Enqueue/TryDequeue/TryPeek; Push/TryPop/PushRange/TryPopRange.
• They don't block when empty — a consumer must poll or sleep, which wastes CPU or adds latency. That's the gap BlockingCollection<T> and Channel<T> fill.

3. ConcurrentBag<T>

• An unordered collection optimised for the case where the same thread adds and removes items (e.g. object pools, work-stealing). Each thread has a local list; TryTake takes from the local list first and steals from other threads only when empty.
• Misused as a general thread-safe list, it performs poorly (stealing is expensive) and provides no ordering. For "collect results from parallel work", prefer ConcurrentQueue<T>, or have each task return its result and use Task.WhenAll.

4. BlockingCollection<T>

• A wrapper (default over ConcurrentQueue<T>) adding blocking Take, bounded capacity (Add blocks when full), and CompleteAdding + GetConsumingEnumerable() for producer/consumer.
• It blocks threads — fine in a dedicated thread, harmful in ASP.NET Core or async code because blocked thread-pool threads cause thread-pool starvation.

5. Why compound operations are still not atomic

Each call is thread-safe, but sequences of calls are not:

if (!_dict.ContainsKey(key)) _dict[key] = Create(key);    // check-then-act race
var count = _dict[key]; _dict[key] = count + 1;          // read-modify-write race: lost updates
if (_queue.Count > 0) _queue.TryDequeue(out var item);   // Count is stale immediately

Use the atomic APIs (GetOrAdd, AddOrUpdate, TryUpdate in a loop, TryDequeue), Interlocked on values, or a lock around the whole sequence. And mutable values stored in a concurrent dictionary (e.g. a List<T>) are not protected at all.

6. When to use Channel<T> instead

System.Threading.Channels is the modern async producer/consumer primitive:

var channel = Channel.CreateBounded<Order>(new BoundedChannelOptions(1_000)
{
    FullMode = BoundedChannelFullMode.Wait,   // backpressure
    SingleReader = true
});

// producer
await channel.Writer.WriteAsync(order, ct);

// consumer (e.g. in a BackgroundService)
await foreach (var o in channel.Reader.ReadAllAsync(ct)) await ProcessAsync(o, ct);

Advantages over BlockingCollection/ConcurrentQueue: async waiting (no blocked threads), built-in backpressure with bounded capacity and full modes (Wait, DropOldest, DropNewest, DropWrite), completion and error propagation, and optimisations when SingleReader/SingleWriter are set. Use channels for in-process pipelines and background work queues; use concurrent collections for shared state (caches, registries, counters).

7. A production example

A service cached per-tenant HttpClient-based API clients with _clients.GetOrAdd(tenantId, id => CreateClient(id)), where CreateClient fetched credentials from Key Vault. Under a burst after a deployment, dozens of requests for the same tenant raced, Key Vault was hit many times per tenant and started throttling (429), and some clients were created and discarded without disposal. Wrapping values in Lazy<Task<TenantClient>> made creation happen exactly once per tenant and shared the in-flight task. Later we switched to IHttpClientFactory named clients and cached only credentials, with a HybridCache for stampede protection.