Files, directories, metadata, inodes and links, plus efficient directory enumeration in .NET with EnumerateFiles and EnumerationOptions.
A file is a named sequence of bytes plus metadata (size, timestamps, owner, permissions). On Linux the data and metadata live in an inode; the name is just an entry in a directory. A directory is a special file that maps names to inodes (Linux) or file records (NTFS MFT), forming a tree.
In .NET, use Directory.EnumerateFiles / DirectoryInfo.EnumerateFiles (lazy, streams results) rather than GetFiles (builds the whole array first), and pass EnumerationOptions to recurse, skip inaccessible folders and control case matching.
| Part | What it holds |
|---|---|
| Name | Stored in the directory entry, not in the file itself |
| Data | Blocks on disk (or extents) |
| Metadata | Size, created/modified/accessed times, owner, permissions, attributes (hidden, read-only) |
| Identity | Inode number (Linux/macOS), file ID in the MFT (NTFS) |
The OS does not care what is inside. .json or .png is only a naming convention; the content format is up to applications.
A directory is a table of name → inode entries. That explains several behaviours:
| API | Behaviour | Use when |
|---|---|---|
Directory.GetFiles |
Reads everything into a string[] before returning |
Small folders, you need a count or random access |
Directory.EnumerateFiles |
Lazy IEnumerable<string>, yields as it reads |
Large folders, early exit, streaming into a pipeline |
DirectoryInfo.EnumerateFiles |
Yields FileInfo with metadata already loaded |
You need size or dates too (avoids a second system call per file) |
FileSystemEnumerable<T> |
Low-level, custom filter and projection on the raw entry | Hot paths with millions of entries |
A nightly archive job listed an upload folder with 2 million files using Directory.GetFiles(path, "*", SearchOption.AllDirectories). It took 4 minutes before the first file was processed, allocated hundreds of MB of strings, and failed outright when it hit one folder without read permission. Switching to new DirectoryInfo(path).EnumerateFiles("*", new EnumerationOptions { RecurseSubdirectories = true, IgnoreInaccessible = true }) started processing straight away, kept memory flat, and logged inaccessible folders instead of crashing.
Checks basic understanding of what the file system stores and whether the candidate knows the streaming enumeration APIs.
Green flags
EnumerateFiles is lazy and GetFiles is eager.EnumerationOptions and inaccessible folders.Red flags
Directory /var/app/ Inode table Data blocks
┌───────────────────────┐ ┌───────────────────────┐ ┌─────────┐
│ "config.json" → 1201 │─────►│ 1201: size 812, uid 1000, │───────►│ {"a":1} │
│ "logs" → 1305 │ │ mode 0644, mtime, │ └─────────┘
│ "current.json"→ 1201 │──────┘ links = 2 │
└───────────────────────┘ │ 1305: directory │
(hard link: two names, └───────────────────────┘
one inode)
ls -i shows inode numbers; stat file shows metadata. A file system can run out of inodes before it runs out of space (millions of tiny files), which shows up as "No space left on device" while df -h shows free space. Check with df -i.
NTFS stores every file as a record in the Master File Table (MFT). Small files can live entirely inside their MFT record. Directories are B-tree indexes of names, which is why lookups in big folders stay fast. NTFS also supports alternate data streams, hard links, symbolic links and junctions.
File.CreateSymbolicLink("current.json", "config.v2.json");
var info = new FileInfo("current.json");
Console.WriteLine(info.LinkTarget); // config.v2.json
var target = info.ResolveLinkTarget(returnFinalTarget: true);
Creating symbolic links on Windows needs Developer Mode or admin rights.
GetFiles: [read all entries] ──────────────────► return array ─► process
EnumerateFiles: [read batch] ► process ► [read batch] ► process ► ...
Under the hood .NET calls getdents64 (Linux) or NtQueryDirectoryFile (Windows) in batches. DirectoryInfo.EnumerateFiles fills FileInfo from the same batch, so you avoid a stat per file.
| Option | Default | Why it matters |
|---|---|---|
RecurseSubdirectories |
false | Walk the tree |
IgnoreInaccessible |
true (with EnumerationOptions) | Skip folders you cannot read instead of throwing |
MatchCasing |
PlatformDefault | Make *.JPG vs *.jpg consistent across OSes |
MatchType |
Simple | Win32 keeps legacy DOS wildcard quirks |
AttributesToSkip |
Hidden | System | Include or exclude hidden files |
MaxRecursionDepth |
int.MaxValue | Limit deep trees |
Note the legacy SearchOption overloads use old behaviour (no ignore inaccessible, Win32 matching), so prefer the EnumerationOptions overload.
Directory.GetFiles(dir).First() vs Directory.EnumerateFiles(dir).First().df -i on Linux before and after to see inode usage grow.ln a.txt b.txt), edit b.txt, and read a.txt.// Stream a large tree, skip folders we cannot read, get size without extra stat calls
var options = new EnumerationOptions
{
RecurseSubdirectories = true,
IgnoreInaccessible = true,
MatchCasing = MatchCasing.CaseInsensitive,
AttributesToSkip = FileAttributes.System
};
long totalBytes = 0;
foreach (var file in new DirectoryInfo("/data/uploads").EnumerateFiles("*.jpg", options))
{
totalBytes += file.Length;
if (file.LastWriteTimeUtc < DateTime.UtcNow.AddDays(-30))
{
await ArchiveAsync(file); // processing starts on the first file found
}
}
// High-performance custom enumeration: filter on the raw entry, no FileInfo allocations
using System.IO.Enumeration;
var bigFiles = new FileSystemEnumerable<string>(
"/data/uploads",
(ref FileSystemEntry entry) => entry.ToFullPath(),
new EnumerationOptions { RecurseSubdirectories = true })
{
ShouldIncludePredicate = (ref FileSystemEntry entry) =>
!entry.IsDirectory && entry.Length > 100 * 1024 * 1024
};
foreach (var path in bigFiles)
Console.WriteLine(path);
Mistake: Using Directory.GetFiles with SearchOption.AllDirectories on large trees.
Why it is wrong: It materialises every path before returning, uses a lot of memory, and throws UnauthorizedAccessException on the first unreadable folder.
Instead: Use EnumerateFiles with EnumerationOptions { RecurseSubdirectories = true, IgnoreInaccessible = true }.
Mistake: Thinking the file name is stored inside the file. Why it is wrong: The name lives in the directory entry, so hard links, instant renames and deleted-but-open files make no sense under that model. Instead: Explain the directory entry to inode (or MFT record) split.
Mistake: Calling new FileInfo(path).Length for every path from Directory.EnumerateFiles.
Why it is wrong: It adds a stat system call per file, which is very slow on network shares.
Instead: Use DirectoryInfo.EnumerateFiles, which returns FileInfo objects already filled from the directory read.
Mistake: Assuming every file system behaves like the one on your laptop.
Why it is wrong: Case sensitivity, path length, locking and metadata speed differ between ext4, NTFS, APFS and SMB or NFS mounts.
Instead: Set MatchCasing explicitly, test on the target OS, and avoid metadata-heavy loops over network storage.
Mistake: Assuming free disk space means you can create files.
Why it is wrong: Millions of tiny files can exhaust inodes while df -h still shows free space.
Instead: Check df -i and shard large flat folders into subfolders by date or hash prefix.
The directory entry versus inode split explains a lot of real behaviour: instant renames, hard links, deleted-but-open files keeping space, and inode exhaustion. For enumeration, the rule is to stream rather than materialise, and to design folder layout before worrying about the API.
| File system | Directory indexing | Notable traits for apps |
|---|---|---|
| ext4 | htree (hashed B-tree) | Case-sensitive, fixed inode count at format time |
| XFS | B+ trees | Dynamic inodes, strong with large files and parallel I/O |
| NTFS | B+ tree indexes in the MFT | Case-insensitive by default, mandatory share locks, alternate data streams |
| ReFS | B+ trees, copy-on-write | Integrity streams, block cloning; used for Dev Drive on Windows |
| APFS | B-trees, copy-on-write | Snapshots, clones; usually case-insensitive on macOS |
| SMB or NFS share | Server-side | Every metadata call is a network round trip |
| API | Memory | Time to first result | When to use it |
|---|---|---|---|
GetFiles |
Whole array | After full scan | Small folders |
EnumerateFiles |
Flat | Immediate | Large trees, early exit |
DirectoryInfo.EnumerateFiles |
Flat | Immediate | Need size or dates too |
FileSystemEnumerable<T> |
Minimal | Immediate | Millions of entries, hot paths |
\ (and accepts /), Linux and macOS use only /. Never hard-code \; use Path.Combine/Path.Join or Path.DirectorySeparatorChar.C:\data\a.txt or /data/a.txt are absolute; data/a.txt is resolved against the current working directory, which differs between dotnet run, a service and a container. Base paths on AppContext.BaseDirectory or configuration.Path.Combine drops earlier parts if a later part is rooted (Combine("/app", "/etc/passwd") gives /etc/passwd). Path.Join just concatenates with one separator.Config.json vs config.json works on your laptop and fails in the Linux container.rwx bits for owner, group and others (for example 0644, 0755). On a directory, x means "can enter" and w means "can create, delete or rename entries". Root bypasses most checks.The container error usually means the app runs as a non-root user (the official .NET 10 images define user app, UID 1654, via $APP_UID) while the mounted folder is owned by root with 0755. Fix it by giving that UID or a shared group write access (chown, Kubernetes fsGroup, or volume mount options), not by running the container as root or using chmod 777.
A file descriptor (Linux, a small integer) or handle (Windows, an opaque value) is the process's reference to a kernel object: an open file, socket, pipe, event, mutex, process or thread. The kernel owns the object, and the process holds an index into its own table.
They are limited (e.g. ulimit -n often 1,024 soft by default on Linux shells, much higher in many containers and under systemd), so leaks eventually break the app.
In .NET, SafeHandle wraps the raw value: it releases it in a critical finaliser if you forget, guarantees release even under thread aborts or async exceptions, and prevents handle recycling bugs by reference counting during P/Invoke calls. You still call Dispose (using) to release deterministically.
Last updated 23 Oct 2026.