Explain HTTP from a backend engineer's point of view: methods and idempotency, status codes, important headers, and the practical differences between HTTP/1.1, HTTP/2 and HTTP/3.

1. Anatomy of a request and response

GET /orders/42 HTTP/1.1
Host: api.example.com
Accept: application/json
Authorization: Bearer eyJ...
If-None-Match: "v7"
traceparent: 00-4bf92f3577b34da6a3ce929d0e0e4736-00f067aa0ba902b7-01

HTTP/1.1 304 Not Modified
ETag: "v7"
Cache-Control: private, max-age=60

2. Caching and conditional requests

• Cache-Control: public, max-age=300 lets CDNs and browsers cache; private limits to the client; no-store forbids caching.
• ETag + If-None-Match lets a client revalidate cheaply (304, no body).
• ETag + If-Match on PUT/PATCH gives optimistic concurrency: 412 Precondition Failed if someone else changed it.
• Vary: Accept-Encoding, Authorization tells caches which request headers change the response.

3. HTTP/1.1 limits

connection 1: [req A ───── resp A][req B ── resp B]
connection 2: [req C ─── resp C]
... browsers open ~6 per host; slow responses block the rest

Pipelining exists but is effectively unused because responses must return in order.

4. HTTP/2

one TCP connection
├─ stream 1: HEADERS GET /a  ... DATA
├─ stream 3: HEADERS GET /b  ... DATA     frames interleaved
└─ stream 5: HEADERS POST /c ... DATA

• Multiplexing removes the need for many connections; HPACK shrinks repeated headers (cookies, auth).
• In .NET, HttpClient uses HTTP/2 when the server negotiates it via ALPN over TLS and you set Version = 2.0 (default request version is 1.1). SocketsHttpHandler.EnableMultipleHttp2Connections opens more connections when stream limits are reached.
• Long-lived HTTP/2 connections can unbalance L4 load balancers (all streams go to one backend).

5. HTTP/3

Same semantics, carried on QUIC. Faster connection setup, better on lossy mobile networks, connection migration. Requires UDP 443 open; negotiated via Alt-Svc.

6. Semantics are the contract

Proxies, caches, SDKs and retry policies all rely on method and status semantics. Returning 200 with {"error": ...} breaks retries, monitoring and caching. Use correct codes and ProblemDetails (RFC 9457) bodies.

7. Try it yourself

1. curl -v https://example.com and read every header.
2. curl -sI --http2 https://example.com vs --http1.1.
3. Add app.UseOutputCache() or ETag support to an endpoint and test If-None-Match with curl.
4. Simulate a retry of POST with and without an idempotency key and count created rows.

// Idempotent POST using an Idempotency-Key header.
// The unique key in IdempotencyRecords is the real guard: if two retries race,
// the second insert fails and we return the response the first one stored.
app.MapPost("/orders", async (
    [FromHeader(Name = "Idempotency-Key")] string? key,
    CreateOrder cmd, OrdersDb db, CancellationToken ct) =>
{
    if (string.IsNullOrWhiteSpace(key))
        return Results.Problem("Idempotency-Key header is required", statusCode: 400);

    var existing = await db.IdempotencyRecords.AsNoTracking()
        .SingleOrDefaultAsync(r => r.Key == key, ct);
    if (existing is not null)
        return Results.Json(existing.ResponseBody, statusCode: existing.StatusCode);

    var order = Order.Create(cmd);
    db.Orders.Add(order);
    db.IdempotencyRecords.Add(new IdempotencyRecord(key, 201, order.ToDto()));

    try
    {
        // Order and key are saved in one transaction: both or neither.
        await db.SaveChangesAsync(ct);
    }
    catch (DbUpdateException ex) when (ex.IsUniqueViolation())
    {
        // A concurrent request with the same key won the race.
        db.ChangeTracker.Clear();
        var winner = await db.IdempotencyRecords.AsNoTracking()
            .SingleAsync(r => r.Key == key, ct);
        return Results.Json(winner.ResponseBody, statusCode: winner.StatusCode);
    }

    return Results.Created($"/orders/{order.Id}", order.ToDto());
});

// IsUniqueViolation is a small extension: for SQL Server it checks
// SqlException.Number 2627 or 2601; for PostgreSQL, SqlState "23505".

// Overload protection: the built-in rate limiter returns 429 with Retry-After
builder.Services.AddRateLimiter(o =>
{
    o.RejectionStatusCode = StatusCodes.Status429TooManyRequests;
    o.AddFixedWindowLimiter("reports", w => { w.PermitLimit = 10; w.Window = TimeSpan.FromSeconds(30); });
    o.OnRejected = (ctx, _) =>
    {
        if (ctx.Lease.TryGetMetadata(MetadataName.RetryAfter, out var retryAfter))
            ctx.HttpContext.Response.Headers.RetryAfter = ((int)retryAfter.TotalSeconds).ToString();
        return ValueTask.CompletedTask;
    };
});

app.UseRateLimiter();
app.MapGet("/reports", GetReports).RequireRateLimiting("reports");

// Client: prefer HTTP/2 to an internal API, fall back to HTTP/1.1 if not negotiated
using var http = new HttpClient(new SocketsHttpHandler
{
    EnableMultipleHttp2Connections = true,
    PooledConnectionLifetime = TimeSpan.FromMinutes(5) // pick up DNS and scale-out changes
})
{
    BaseAddress = new Uri("https://orders.internal/"),
    DefaultRequestVersion = HttpVersion.Version20,
    DefaultVersionPolicy = HttpVersionPolicy.RequestVersionOrLower
};