What is a Stream in .NET? Explain how FileStream, MemoryStream and wrapper streams like GZipStream fit together, and the common mistakes when reading from a stream.

1. Read semantics

int n = stream.Read(buffer, 0, 4096);

• n > 0: you got between 1 and 4096 bytes. Not necessarily 4096.
• n == 0: end of stream.
• Network and pipe streams often return partial reads.

Ask for 4096 ──► got 1460 (one TCP segment) ──► ask again ──► got 2636

.NET 7+ added ReadExactly(buffer) (throws EndOfStreamException if the stream ends early) and ReadAtLeast(buffer, minimum).

2. Position and seeking

After writing to a MemoryStream, Position is at the end. Reading returns nothing until you reset it:

write "hello"   [h e l l o]
                           ^ Position = 5
read            → 0 bytes
Position = 0    [h e l l o]
                 ^
read            → "hello"

3. Disposal chains

Disposing an outer stream disposes the inner one by default. StreamReader, StreamWriter, GZipStream and others have a leaveOpen parameter when you need the inner stream afterwards (for example, writing a gzip body into a response stream you do not own).

For compression and encryption, disposing (or Flush/FlushFinalBlock) writes the final block. If you read the inner MemoryStream before disposing the GZipStream, the data is incomplete.

4. Async I/O

• FileStream opened with useAsync: true or FileOptions.Asynchronous uses real async I/O (IOCP on Windows). On Linux, file I/O is executed on the thread pool, because regular files do not support epoll readiness.
• .NET 6 rewrote FileStream for much better performance and correct Position tracking with async calls.
• Prefer CopyToAsync, ReadAsync(Memory<byte>) and WriteAsync(ReadOnlyMemory<byte>) overloads, which avoid extra allocations.

5. Beyond Stream: Pipelines

System.IO.Pipelines (PipeReader/PipeWriter) handles buffer management and partial messages for you. Kestrel uses it internally. It is a good fit for high-throughput protocol parsing; for ordinary file and HTTP work, Stream is enough.

6. Try it yourself

1. Write "hello" to a MemoryStream and call new StreamReader(ms).ReadToEnd() before and after ms.Position = 0.
2. Compress into a MemoryStream with GZipStream and print ms.Length before and after disposing the GZipStream.
3. Read from a NetworkStream (or HttpClient response stream) into a 64 KiB buffer in a loop and log each n to see partial reads.

// Stream a gzipped CSV from HTTP to the database without loading it into memory
using var response = await http.GetAsync(url, HttpCompletionOption.ResponseHeadersRead, ct);
response.EnsureSuccessStatusCode();

await using var body = await response.Content.ReadAsStreamAsync(ct);
await using var gzip = new GZipStream(body, CompressionMode.Decompress);
using var reader = new StreamReader(gzip, Encoding.UTF8);

var batch = new List<Order>(capacity: 5_000);
while (await reader.ReadLineAsync(ct) is { } line)
{
    batch.Add(Order.Parse(line));
    if (batch.Count == batch.Capacity)
    {
        await repository.BulkInsertAsync(batch, ct);
        batch.Clear();
    }
}
if (batch.Count > 0) await repository.BulkInsertAsync(batch, ct);

// Reading a fixed-size header correctly
var header = new byte[16];
await stream.ReadExactlyAsync(header, ct);           // .NET 7+: loops for you

// Manual equivalent on older code
int read = 0;
while (read < header.Length)
{
    int n = await stream.ReadAsync(header.AsMemory(read), ct);
    if (n == 0) throw new EndOfStreamException();
    read += n;
}

// Compress to memory: dispose the GZipStream before using the bytes
var output = new MemoryStream();
await using (var gzip = new GZipStream(output, CompressionLevel.Optimal, leaveOpen: true))
{
    await JsonSerializer.SerializeAsync(gzip, report, cancellationToken: ct);
} // final block written here
output.Position = 0;                                   // rewind before reading
return output;