Chapter 13: Testing, Benchmarking, and Docs

Quality in Rust is built into the toolchain. You can write tests next to your code, run them with a single command, measure performance with reliable harnesses, and publish documentation generated from your comments. This chapter shows how unit and integration tests work, how to add property tests with proptest, how to benchmark with criterion, how to write doc tests, and how to wire everything into a simple continuous integration setup.

Unit and integration tests

Rust has two primary testing styles. Unit tests live close to the code they validate. Integration tests exercise your public API through the crate boundary. Both styles use the same test runner that cargo invokes. You can mix and match; start with focused unit tests, then add integration tests for end to end behaviour.

Module based unit tests with #[cfg(test)]

A common pattern places unit tests in a private tests module within the same file. The module is compiled only during tests; it is excluded from normal builds. This keeps helper functions and fixtures close to the code under test without leaking them into your final binary or library.

// src/math.rs
pub fn add(a: i32, b: i32) -> i32 {
  a + b
}

#[cfg(test)]
mod tests {
  use super::*;

  #[test]
  fn adds_positive_numbers() {
    assert_eq!(add(2, 3), 5);
  }

  #[test]
  fn adds_negative_numbers() {
    assert_eq!(add(-2, -3), -5);
  }

  #[test]
  #[should_panic(expected = "overflow")]
  fn panics_on_overflow_in_debug() {
    let _ = add(i32::MAX, 1);
  }
}

Run unit tests for the current crate with cargo test. Use cargo test name_fragment to run a subset. The runner filters by test function names and module paths.

Integration tests in the tests/ directory

Integration tests go in a top level tests/ folder and are compiled as separate crates. Each .rs file becomes its own test crate that depends on your library crate. This ensures you exercise only the public interface, not private internals.

// src/lib.rs
pub fn slugify(s: &str) -> String {
  s.to_lowercase().replace(' ', "-")
}

// tests/slug_test.rs
use mycrate::slugify;

#[test]
fn turns_spaces_into_hyphens() {
  assert_eq!(slugify("Hello World"), "hello-world");
}

Use cargo test --test slug_test to run a single integration file. Add a tests/common.rs module or a submodule tree like tests/common/mod.rs when you need shared helpers across multiple integration tests.

Assertions, matching, and custom messages

Rust’s standard library provides helpful assertion macros. Use equality assertions for exact results; use pattern matching when you only care about shape or variants; add custom messages to clarify failures.

#[test]
fn examples_of_asserts() {
  let value = Some(10);
  assert!(value.is_some(), "expected Some(…)");
  assert_eq!(2 * 3, 6, "basic arithmetic must hold");
  match value {
    Some(v) if v > 5 => {}
    other => panic!("unexpected value: {:?}", other),
  }
}

Fixtures and test organisation

Keep tests readable by extracting setup code into small helpers. Prefer values and builders over mutable global state. For filesystem fixtures, create a temporary directory and clean up using tempfile. Table driven tests are straightforward; iterate over cases and assert each outcome.

#[derive(Debug)]
struct Case {
  input: &'static str,
  want: &'static str,
}

#[test]
fn table_driven_slugify() {
  let cases = [
    Case { input: "Hello World", want: "hello-world" },
    Case { input: "À bientôt",   want: "à-bientôt" },
  ];

  for c in cases {
    assert_eq!(slugify(c.input), c.want, "input: {}", c.input);
  }
}

Property testing with proptest

Property tests state invariants that must always hold, then generate many inputs automatically. This complements example based tests. Instead of a single example for each behaviour, you declare properties like idempotence, symmetry, or round trip encoding, and the framework searches for counterexamples.

Declaring properties and strategies

Install proptest in Cargo.toml. A strategy describes how to generate values. You can compose strategies to build complex inputs. The macro proptest! expands into multiple tests that cover many random cases with shrinking.

# Cargo.toml
[dev-dependencies]
proptest = "1"

# tests/prop_slug.rs
use mycrate::slugify;
use proptest::prelude::*;

proptest! {
  #[test]
  fn slug_is_lowercase(ref s in "\\PC*") {
    let slug = slugify(s);
    prop_assert!(slug.chars().all(|c| !c.is_uppercase()));
  }

  #[test]
  fn idempotent(input in "\\PC*") {
    let once = slugify(&input);
    let twice = slugify(&once);
    prop_assert_eq!(once, twice);
  }
}

When a failure occurs, proptest shrinks the input to a minimal counterexample. This makes debugging practical; you get a small case that reproduces the issue.

Composing strategies for structured data

Strategies are first class values. You can map and filter them to model constraints. The following example builds a simple URL like scheme://host/path and checks a round trip parse property.

use proptest::prelude::*;

#[derive(Debug, Clone, PartialEq, Eq)]
struct SimpleUrl {
  scheme: String,
  host: String,
  path: String,
}

fn parse_simple(s: &str) -> Option<SimpleUrl> {
  let parts: Vec<_> = s.splitn(3, '/').collect();
  if parts.len() < 3 { return None; }
  let scheme_host = parts[0];
  let path = format!("/{}", parts[2]);
  let mut it = scheme_host.splitn(2, "://");
  let scheme = it.next()?.to_string();
  let host = it.next()?.to_string();
  Some(SimpleUrl { scheme, host, path })
}

fn render_simple(u: &SimpleUrl) -> String {
  format!("{}://{}{}", u.scheme, u.host, u.path)
}

proptest! {
  #[test]
  fn parse_render_roundtrip(
    scheme in "(http|https)",
    host in "[a-z]{1,8}\\.example\\.com",
    path in "/[a-z0-9/]{0,16}"
  ) {
    let url = SimpleUrl { scheme: scheme.to_string(), host, path };
    let s = render_simple(&url);
    prop_assert_eq!(parse_simple(&s), Some(url));
  }
}

Benchmarks and criterion

Microbenchmarks help you track performance over time. The criterion crate provides statistics, warm up, resampling, and regression detection. It is more reliable than the unstable built in harness. Benchmarks live under benches/. Each file defines groups of related measurements.

Setting up criterion with benches/

Add criterion as a development dependency. Create a bench file in benches/, then run cargo bench. The harness records results to target/criterion and can compare against a baseline.

# Cargo.toml
[dev-dependencies]
criterion = { version = "0.5", features = ["html_reports"] }

# benches/slug_bench.rs
use criterion::{black_box, criterion_group, criterion_main, Criterion};
use mycrate::slugify;

fn bench_slugify(c: &mut Criterion) {
  let input = "The quick brown fox jumps over the lazy dog";
  c.bench_function("slugify baseline", |b| {
    b.iter(|| slugify(black_box(input)))
  });
}

criterion_group!(benches, bench_slugify);
criterion_main!(benches);

Open the HTML report to visualise distributions; it appears under target/criterion/…/report/index.html. For stable results, pin CPU frequency scaling and close background applications that create noise.

Comparing algorithms and tracking regressions

Benchmarks are most useful when you compare competing approaches. Group related functions, then examine which inputs change the ordering. Criterion can track changes between runs and flag regressions with its statistical tests.

use criterion::{Criterion, BenchmarkId, criterion_group, criterion_main};

fn algo_a(s: &str) -> usize { s.bytes().filter(|b| b.is_ascii()).count() }
fn algo_b(s: &str) -> usize { s.chars().filter(|c| c.is_ascii()).count() }

fn compare_algos(c: &mut Criterion) {
  let mut group = c.benchmark_group("ascii_count");
  for size in [16usize, 256, 4096] {
    let input = "a".repeat(size);
    group.bench_with_input(BenchmarkId::new("bytes", size), &input, |b, s| {
      b.iter(|| algo_a(s))
    });
    group.bench_with_input(BenchmarkId::new("chars", size), &input, |b, s| {
      b.iter(|| algo_b(s))
    });
  }
  group.finish();
}

criterion_group!(benches, compare_algos);
criterion_main!(benches);

Doc tests and examples

Rust can compile and run examples embedded in your documentation. This prevents code drift. A doc comment marked with triple slashes becomes part of your public documentation and its code blocks are treated as tests by default. You can also publish standalone examples under examples/; these build like small binaries that depend on your crate.

Writing rustdoc comments with executable blocks

Use fenced code blocks with rust language tags. Prefer concise, self contained snippets. If your example requires extern crate or use lines, include them so the snippet compiles in isolation.

// src/lib.rs

/// Convert a string into a slug.
/// 
/// # Examples
/// 
/// ```rust
/// use mycrate::slugify;
/// let s = slugify("Hello World");
/// assert_eq!(s, "hello-world");
/// ```
pub fn slugify(s: &str) -> String {
  s.to_lowercase().replace(' ', "-")
}

Run all doc tests with cargo test --doc. Use attributes like ignore or no_run for examples that cannot execute, such as those that require network access. Keep these rare; tested code gives the strongest guarantee.

The examples/ directory for runnable demos

Place small programs under examples/ to demonstrate usage. They appear in documentation and can be executed directly. This is helpful for tutorials and quick start guides that readers can compile and run without scaffolding.

// examples/cli.rs
use mycrate::slugify;

fn main() {
  let arg = std::env::args().nth(1).unwrap_or_else(|| "Hello World".into());
  println!("{}", slugify(&arg));
}

Run the example with cargo run --example cli -- "Some Input". Keep examples focused; a single concept per file helps readers learn quickly.

Continuous integration basics

Continuous integration runs your checks on every push. Even a lightweight pipeline improves confidence. Start with stable Rust; add nightly only when needed. Cache builds to keep runs fast. Fail the build on any test or lint error so that problems never merge undetected.

A minimal GitHub Actions workflow

Create a workflow file that installs the toolchain, caches dependencies, runs tests, and saves criterion reports as artifacts. This example uses stable Rust; adjust names and paths to match your crate. The parts with … indicate optional steps.

# .github/workflows/ci.yml
name: CI

on:
  push:
    branches: [ main ]
  pull_request:

jobs:
  build-test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: dtolnay/rust-toolchain@stable
      - uses: Swatinem/rust-cache@v2
      - name: Build
        run: cargo build --verbose
      - name: Lint
        run: cargo clippy --all-targets --all-features -- -D warnings
      - name: Format check
        run: cargo fmt -- --check
      - name: Unit and integration tests
        run: cargo test --all-features
      - name: Doc tests
        run: cargo test --doc
      - name: Benchmarks & reports
        run: cargo bench || true  # allow benches to run without failing the job
      - name: Upload criterion reports
        if: always()
        uses: actions/upload-artifact@v4
        with:
          name: criterion-reports
          path: target/criterion/**/report/index.html

Keep CI fast by avoiding unnecessary feature matrices. Add a second job for nightly or additional platforms later. For crates with network calls, provide feature flags that swap real clients for fakes during tests.

Catching undefined behaviour with sanitizers

You can enable sanitizers in CI to catch memory and thread issues early. Use a nightly toolchain for the thread sanitizer; address and leak sanitizers work on stable for many targets. Add dedicated jobs because these flags slow builds.

# Example CI step
- name: Address sanitizer
  run: RUSTFLAGS="-Zsanitizer=address" RUSTDOCFLAGS="-Zsanitizer=address" \
       cargo +nightly test -Zbuild-std --target x86_64-unknown-linux-gnu

Summary of test types

The following table contrasts common test and example locations. Use it as a quick reference when deciding where to place new checks.

Checklist before publishing

Before releasing a version, run through a short checklist. Confirm formatting, linting, tests, doc tests, and examples. Review benchmark trends. Ensure your CI workflow reflects these steps so that every pull request gets the same scrutiny.

# Local quality sweep
cargo fmt -- --check
cargo clippy --all-targets --all-features -- -D warnings
cargo test --all-features
cargo test --doc
cargo bench  # review reports under target/criterion/…