Chapter 14: Files, Streams, and Serialization

In this chapter you learn how to work with the file system using the System.IO namespace, how to create and consume Stream types, how to read and write binary and text data, how to serialize objects to JSON, how to use asynchronous I/O with buffering, and how to ensure resources are cleaned up with using. The focus is on C# syntax and patterns; .NET types and runtime behavior are woven in where they directly affect your code.

Working with paths and directories

The Path, Directory, DirectoryInfo, File, and FileInfo types provide the building blocks for working with files and folders. You can combine segments, normalize separators, inspect the current directory, and enumerate content without touching file data. These APIs are cross-platform; separators are handled for you on Windows, macOS, and Linux.

Common Path operations

Use Path.Combine to join segments; use Path.GetExtension, GetFileName, and GetDirectoryName to dissect a path. Path.Join avoids adding redundant separators; Path.GetTempPath returns a suitable temporary location.

using System;
using System.IO;

string baseDir = Environment.CurrentDirectory;
string sub = "data";
string file = "report.txt";
string full = Path.Combine(baseDir, sub, file);
Console.WriteLine(full);
Console.WriteLine(Path.GetExtension(full));  // .txt
Console.WriteLine(Path.GetFileName(full));   // report.txt
Console.WriteLine(Path.GetDirectoryName(full));

Creating and enumerating directories

Directory.CreateDirectory makes a folder if it does not exist. Enumerate with Directory.EnumerateFiles and Directory.EnumerateDirectories; the Enumerate methods stream results which is better for large trees than the Get methods that materialize arrays.

string root = Path.Combine(baseDir, "logs");
Directory.CreateDirectory(root);

foreach (string path in Directory.EnumerateFiles(root, "*.log", SearchOption.AllDirectories))
{
  Console.WriteLine(path);
}

File and directory metadata

Query and modify attributes with File.GetAttributes, File.SetAttributes, and the info types. Timestamps are available as creation, last access, and last write times; use UTC where possible.

var info = new FileInfo(full);
if (info.Exists)
{
  Console.WriteLine($"{info.Length} bytes; modified {info.LastWriteTimeUtc:u}");
}

Streams and readers

A Stream represents a sequence of bytes that can be read and or written. Concrete types include FileStream for files, MemoryStream for in-memory buffers, NetworkStream for sockets, and GZipStream for compression. Text helpers such as StreamReader and StreamWriter layer character encoding on top of byte streams.

Using FileStream effectively

FileStream exposes buffered I/O with synchronous and asynchronous members. Opening with FileMode, FileAccess, and FileShare lets you control intent precisely. For modern code prefer the FileStream constructor that accepts FileStreamOptions when you need fine control.

using var fs = new FileStream(
  path: full,
  mode: FileMode.OpenOrCreate,
  access: FileAccess.ReadWrite,
  share: FileShare.Read);

// Read or write bytes with fs.Read, fs.Write, or their async counterparts

StreamReader and StreamWriter for text

Wrap a stream with StreamReader for decoding and with StreamWriter for encoding. Specify Encoding.UTF8 to be explicit and portable.

using var writer = new StreamWriter(full, append: true, encoding: System.Text.Encoding.UTF8);
writer.WriteLine("Hello");
writer.Write("World");

using var reader = new StreamReader(full, System.Text.Encoding.UTF8, detectEncodingFromByteOrderMarks: true);
string all = reader.ReadToEnd();
Console.WriteLine(all);

Binary and text I/O

Choose text I/O when you need human readable content that respects encodings and line endings. Choose binary I/O for compact data or when you must preserve exact bytes. BinaryReader and BinaryWriter provide typed methods for primitives; endianness matters for cross-system exchange.

Writing and reading text

The File convenience methods are terse and allocate complete buffers; use streams if you need streaming or partial reads.

string[] lines = { "alpha", "beta", "gamma" };
File.WriteAllLines(full, lines, System.Text.Encoding.UTF8);

foreach (string line in File.ReadLines(full, System.Text.Encoding.UTF8))
{
  Console.WriteLine(line);
}

Writing and reading binary

With BinaryWriter and BinaryReader you control the exact layout. Document the layout or include a simple header with a magic value and version number.

using var bfs = new FileStream(full + ".bin", FileMode.Create, FileAccess.Write, FileShare.None);
using var bw = new BinaryWriter(bfs);
bw.Write(0xCAFEBABE);  // magic
bw.Write(1);           // version
bw.Write(3.14159);     // double
bw.Write(true);        // bool
bw.Write("hello");     // length-prefixed UTF-8 by default

bfs.Flush();

using var rfs = new FileStream(full + ".bin", FileMode.Open, FileAccess.Read, FileShare.Read);
using var br = new BinaryReader(rfs);
int magic = br.ReadInt32();
int version = br.ReadInt32();
double pi = br.ReadDouble();
bool ok = br.ReadBoolean();
string text = br.ReadString();
Console.WriteLine($"{magic:x}; v{version}; {pi}; {ok}; {text}");

JSON serialization

System.Text.Json provides fast JSON serialization and deserialization. Use JsonSerializer.Serialize and JsonSerializer.Deserialize for simple cases; configure JsonSerializerOptions for naming policies, comments, numbers, and other behavior. Attributes on your types control property names, null handling, and conversions.

Basic JsonSerializer usage

Define plain objects with public getters and setters. Anonymous types can be serialized to JSON but not deserialized back. Records provide concise immutable models.

using System.Text.Json;
using System.Text.Json.Serialization;

public record Book(
  string Title,
  string Author,
  int Year);

var model = new Book("This is C# & .NET", "Robin", 2025);

var options = new JsonSerializerOptions
{
  WriteIndented = true,
  PropertyNamingPolicy = JsonNamingPolicy.CamelCase
};

string json = JsonSerializer.Serialize(model, options);
Console.WriteLine(json);

// {"title":"This is C# & .NET","author":"Robin","year":2025}

Deserializing and dealing with missing data

When reading unknown JSON shape, target JsonDocument or JsonElement for DOM-style access. For streaming large payloads use Utf8JsonReader and Utf8JsonWriter.

string incoming = "{\"title\":\"This is C#\",\"meta\":{\"tags\":[\"csharp\",\"dotnet\"],\"isbn\":\"...\"}}";
using JsonDocument doc = JsonDocument.Parse(incoming);
JsonElement root = doc.RootElement;
Console.WriteLine(root.GetProperty("title").GetString());
if (root.TryGetProperty("meta", out var meta) && meta.TryGetProperty("isbn", out var isbn))
{
  Console.WriteLine(isbn.GetString());
}

Async I/O and buffering

Asynchronous I/O prevents blocking threads while the operating system performs disk or network work. Use await with ReadAsync, WriteAsync, and higher-level helpers such as File.ReadAllTextAsync. Buffer sizes affect throughput and memory; choose values that suit your device and workload.

Asynchronous file operations

These helpers read or write whole files. For streaming, call async members on streams. Configure FileStreamOptions to enable or disable buffering as needed.

using System.Buffers;
using System.IO;
using System.Text;
using System.Threading.Tasks;

string path = Path.Combine(Environment.CurrentDirectory, "async.txt");
await File.WriteAllTextAsync(path, "Hello async I/O", Encoding.UTF8);

await using var fs = new FileStream(path, new FileStreamOptions
{
  Mode = FileMode.Open,
  Access = FileAccess.Read,
  Share = FileShare.Read,
  BufferSize = 64 * 1024,
  Options = FileOptions.Asynchronous | FileOptions.SequentialScan
});

byte[] buffer = ArrayPool<byte>.Shared.Rent(8192);
try
{
  int read;
  while ((read = await fs.ReadAsync(buffer.AsMemory(0, buffer.Length))) > 0)
  {
    // process bytes...
  }
}
finally
{
  ArrayPool<byte>.Shared.Return(buffer);
}

Working with MemoryStream and pipelines

MemoryStream is convenient for small to medium buffers. For high-throughput scenarios consider System.IO.Pipelines which offers producer and consumer friendly primitives. The examples in this book keep to Stream for simplicity.

Resource cleanup with using

Types that hold unmanaged resources implement IDisposable or IAsyncDisposable. Use the using statement or the using declaration to ensure timely cleanup even if exceptions occur. Async disposal releases resources that need asynchronous finalization such as flushing async pipelines.

using statement and declaration

The classic using statement scopes the resource to a block. The declaration form disposes when the enclosing scope ends which can reduce nesting. Choose the form that is clearer for the reader of your code.

// Statement form
using (var stream = File.OpenRead(full))
{
  // use stream
}

// Declaration form
using var stream2 = File.OpenRead(full);
// use stream2
// disposed at end of scope

Async disposal with await using

When a type implements IAsyncDisposable, use await using. File streams support async disposal; network streams and writers often do as well.

await using var asyncStream = new FileStream(full, FileMode.Open, FileAccess.Read, FileShare.Read, 4096, FileOptions.Asynchronous);
// use asyncStream with await
// disposed asynchronously at end of scope