Chapter 17: Observability and Operations

Once an application reaches production, observability becomes the compass that guides every fix and improvement. Clear logs, useful metrics, distributed traces, and predictable configuration help you understand what the system is doing even when you are far from the terminal. This chapter brings these pieces together so your Node services remain transparent, measurable, and steady under load.

Logging strategy and structure

Logs are the first signal you check during an incident. They should remain structured, easy to search, and context rich. A few predictable fields make dashboards meaningful and give operators a reliable trail to follow.

Shaping structured logs

Emit logs as structured objects rather than free text. Include fields such as method, path, status, duration, and a correlation ID. This improves filtering and chart building in tools such as Loki, Elastic, or cloud providers.

// logger.ts
export function logEvent(data: Record<string, any>) {
  process.stdout.write(JSON.stringify({ time: Date.now(), ...data }) + "\n");
}

Integrating logging with frameworks

Fastify bundles structured logging with pino. Express and Koa require external helpers. Wrap your logger in middleware so every request receives a consistent lifecycle entry.

// express-logger.ts
import { logEvent } from "./logger";

export function requestLogger(req, res, next) {
  const start = Date.now();
  res.on("finish", () => {
    logEvent({
      method: req.method,
      path: req.url,
      status: res.statusCode,
      duration: Date.now() - start
    });
  });
  next();
}

Log levels and rotation

Use levels such as info, warn, error, and debug. Keep production logs tight by reducing noise at the debug level. Rotate logs by size or time so disks stay healthy and old files compress cleanly.

Metrics, tracing, and profiling

Metrics provide continuous summaries of system health, traces show how a request moves through the system, and profilers reveal where CPU or memory slows the service. Together these tools create a full picture of runtime behaviour.

Metrics with Prometheus format

Export counters, gauges, and histograms so Prometheus and similar collectors can scrape them. Metrics help you notice slow climbs in error counts or latency before they become incidents.

// metrics.ts
let requests = 0;

export async function metrics(req, res) {
  if (req.url === "/metrics") {
    res.setHeader("Content-Type", "text/plain; version=0.0.4");
    return res.end(`requests_total ${requests}`);
  }
  requests++;
}

Distributed tracing

Distributed traces connect spans across services. Each span records timing and context so you can follow a request from edge to database. OpenTelemetry provides the shared vocabulary and exporters for popular backends.

// otel-setup.ts
import { NodeSDK } from "@opentelemetry/sdk-node";
import { getNodeAutoInstrumentations } from "@opentelemetry/auto-instrumentations-node";

const sdk = new NodeSDK({
  instrumentations: [getNodeAutoInstrumentations()]
});

sdk.start();

Profiling CPU and memory

Profilers identify hot paths and leaks. Use Node’s built in --inspect and Chrome DevTools or sampling profilers through clinic when performance dips. Symptoms often appear under concurrency so reproduce load while profiling.

// run with: node --inspect app.js
// then open chrome://inspect

Health checks and graceful shutdown

Reliable services advertise their state and shut down gently when asked. Health checks help orchestrators place traffic safely, and graceful shutdown prevents dropped requests during deploys or restarts.

Liveness and readiness endpoints

Expose two endpoints. Liveness confirms the process is running. Readiness confirms the process is ready to accept traffic. Kubernetes and other orchestrators use these signals to route or withhold requests.

// health.ts
export function liveness(_req, res) {
  res.end("ok");
}

export function readiness(_req, res) {
  const healthy = true; // check db or cache here
  res.statusCode = healthy ? 200 : 503;
  res.end(healthy ? "ready" : "not-ready");
}

Graceful shutdown

On signals such as SIGTERM, stop accepting new connections and finish in flight requests before exiting. This avoids dropped responses during deploys.

// graceful.ts
export function attachGraceful(server) {
  const shutdown = () => {
    server.close(() => process.exit(0));
  };
  process.on("SIGTERM", shutdown);
  process.on("SIGINT", shutdown);
}

Dependency checks

Check key dependencies such as the database, cache, or message queue. Readiness should fail if any critical component becomes unreachable for more than a short span.

Managing configuration across environments

Configuration varies between development, staging, and production. The aim is to keep application code static while configuration flows in from the environment. This keeps deployments reproducible and secure.

Environment variables and safe parsing

Read values from the environment and validate them on startup so errors appear early. Use a small schema to enforce presence and type safety.

// config.ts
import { z } from "zod";

const Schema = z.object({
  PORT: z.string().transform(Number),
  DATABASE_URL: z.string().url(),
  REDIS_URL: z.string().url()
});

export const config = Schema.parse(process.env);

Secret management

Avoid storing sensitive values inside repositories. Use secret managers, encrypted files, or cloud vaults. Retrieve secrets at startup and keep them in memory for the lifetime of the process.

Configuration layering

Layer defaults with environment overrides. For example, include a base file with harmless defaults, then layer environment specific files, then apply environment variables. This pattern keeps development smooth without leaking production values.

Observability closes the loop between development and operations. By combining logs, metrics, traces, health checks, and reliable configuration, you build services that remain transparent and trustworthy even as they grow.