Chapter 21: Concurrency and Parallelism

Modern C++ offers portable building blocks for running work in parallel, communicating safely between threads, and reasoning about memory effects. This chapter orients you around the core primitives and patterns, then shows techniques for composing higher level task flows. The aim is correctness first, performance second, and clarity always.

Threads and thread management

A thread is an independent path of execution that runs alongside others. In C++, std::thread launches a function concurrently, while std::jthread (C++20) adds cooperative cancellation with std::stop_token and automatically joins in its destructor. Prefer std::jthread for simpler lifetime management; use std::thread when you need precise control and will remember to join or detach.

Creating and joining threads

Construct a thread with a callable and arguments. Always finish with join (wait for completion) or detach (let it run independently), since an unjoined std::thread at destruction is a program error. With std::jthread you get RAII joining automatically, which reduces footguns.

#include <thread>
#include <iostream>

void work(int id) {
  std::cout << "hello from " << id << "\n";
}

int main() {
  std::thread t(work, 1);
  // do other things...
  t.join();  // required for std::thread
}

Cooperative cancellation

Cooperative cancellation uses tokens. A thread checks stop_token periodically and exits promptly when requested. This avoids forcibly terminating threads and keeps invariants intact.

#include <thread>
#include <stop_token>
#include <chrono>

void ticking(std::stop_token st) {
  while (!st.stop_requested()) {
    // do a unit of work
    std::this_thread::sleep_for(std::chrono::milliseconds(10));
  }
}

int main() {
  std::jthread jt(ticking);
  // later...
  jt.request_stop();  // cooperative
} 

Thread affinity, counts, and structure

Use std::thread::hardware_concurrency() as a heuristic for the number of worker threads. It reports a hint, not a promise. Structure your design so that thread counts are configurable, since the best number depends on I/O versus CPU mix and the runtime environment.

Mutexes, locks, and condition variables

Shared data accessed by multiple threads needs synchronization. A mutex provides mutual exclusion. Locks manage mutex ownership using RAII. Condition variables allow threads to sleep until a predicate becomes true, which avoids busy waiting.

Protecting shared state with a mutex

Wrap reads and writes to shared data with a lock. Use std::scoped_lock for simple critical sections or std::unique_lock when you need to unlock and relock around waits.

#include <mutex>
#include <vector>

std::mutex m;
std::vector<int> data;

void push(int x) {
  std::scoped_lock lk(m);
  data.push_back(x);
}

Waiting with a condition variable

A condition variable pairs with a mutex and a predicate. Always check the predicate in a loop, since spurious wakeups can occur. Use wait or wait_for with a lambda that inspects the current state.

#include <mutex>
#include <condition_variable>
#include <queue>

std::mutex m;
std::condition_variable cv;
std::queue<int> q;
bool done = false;

void producer() {
  for (int i = 0; i < 10; ++i) {
    {
      std::lock_guard lg(m);
      q.push(i);
    }
    cv.notify_one();
  }
  {
    std::lock_guard lg(m);
    done = true;
  }
  cv.notify_all();
}

void consumer() {
  for (;;) {
    std::unique_lock lk(m);
    cv.wait(lk, [] { return !q.empty() || done; });
    if (!q.empty()) {
      int v = q.front(); q.pop();
      lk.unlock();
      // process v
    } else if (done) {
      break;
    }
  }
}

Deadlocks, ordering, and std::scoped_lock

Deadlock arises when threads acquire multiple locks in different orders. Either enforce a global lock ordering or acquire multiple mutexes at once using std::scoped_lock a{m1, m2} which uses a deadlock avoidance protocol. Release locks before expensive work to shorten contention windows.

Futures, promises, and std::async

Futures carry results from asynchronous operations. A std::promise sets the value, and a paired std::future retrieves it. std::async creates a future-producing task without manual thread management. Futures propagate exceptions to the waiting side, which simplifies error handling.

One-shot result channels

Use a promise and future when a single producer needs to deliver a single value (or exception) to a consumer. The consumer calls get() to block until the result is ready, or uses timed waits with wait_for.

#include <future>
#include <string>

std::string compute() { return "result"; }

int main() {
  std::promise<std::string> p;
  std::future<std::string> f = p.get_future();
  std::thread t([pr = std::move(p)]() mutable {
    try { pr.set_value(compute()); }
    catch (...) { pr.set_exception(std::current_exception()); }
  });
  auto s = f.get();  // throws if the worker threw
  t.join();
}

std::async launch policies

std::async may run the task on a new thread or defer execution until you call get(). The policy is a hint unless you pass std::launch::async (guarantees concurrent execution) or std::launch::deferred (runs in the waiting thread). Treat std::async as a convenience for fire and wait workflows.

#include <future>
#include <numeric>
#include <vector>

long sum(const std::vector<int>& v) { 
  return std::accumulate(v.begin(), v.end(), 0L); 
}

int main() {
  std::vector<int> v(1'000'000, 1);
  auto fut = std::async(std::launch::async, sum, std::cref(v));
  long s = fut.get();
}

Packaging work

std::packaged_task wraps a callable so that invoking it sets a future. This is handy for thread pools and work queues, since you can move the task into a worker and later wait on the paired future for the result.

Atomics and memory order

Atomics allow threads to read and write shared variables without data races. Each operation has a memory ordering that constrains how operations appear across cores. When in doubt, use the default memory_order_seq_cst, which is simple and correct. Weaker orders can improve performance when used with care.

Basic atomic operations

Integral atomics support load, store, read-modify-write operations like fetch_add. The default ordering is sequentially consistent. Start there, measure, then consider relaxed or acquire-release only if you fully understand the visibility rules.

#include <atomic>

std::atomic<int> counter{0};

void inc() { counter.fetch_add(1); }
int get() { return counter.load(); }

Acquire, release, and fences

Acquire on loads pairs with release on stores to establish happens-before edges. This ensures that writes before the release become visible to reads after the acquire. Fences are lower level barriers; prefer operation-specific orderings on atomics rather than manual fences unless you have a proven need.

Lock-free queries and progress

is_lock_free() tells you whether an atomic is implemented without locks on the current platform. Lock free does not mean wait free; there may still be livelock when many threads contend. Choose algorithms with clear progress guarantees where it matters.

Task based parallelism and executors

Task based design focuses on units of work and dependencies rather than manual thread lifetimes. In C++ today you can build task graphs with a thread pool, package callables as tasks, and connect them via futures or queues. Standard algorithms with the std::execution policies run many loops in parallel without custom threading.

Parallel algorithms

Pass an execution policy to eligible algorithms to enable parallel or vectorized execution. Use par for multithreaded execution and par_unseq to allow parallelism and unsequenced vectorization when safe. The body must be free of data races and side effects that depend on order.

#include <algorithm>
#include <execution>
#include <vector>

int main() {
  std::vector<int> v(1'000'000, 1);
  std::for_each(std::execution::par, v.begin(), v.end(), [](int& x){ x += 2; });
}

A minimal thread pool for tasks

A thread pool amortizes thread creation by keeping a fixed set of workers that pull tasks from a queue. Push packaged tasks into the queue, then wait on their futures. This pattern scales well for many short tasks and keeps control centralized.

#include <vector>
#include <thread>
#include <future>
#include <queue>
#include <mutex>
#include <condition_variable>
#include <functional>

class thread_pool {
  std::vector<std::jthread> workers;
  std::mutex m;
  std::condition_variable_any cv;
  std::queue<std::function<void()>> q;

public:
  explicit thread_pool(std::size_t n = std::thread::hardware_concurrency()) {
    for (std::size_t i = 0; i < n; ++i) {
      workers.emplace_back([this](std::stop_token st){
        for (;;) {
          std::function<void()> job;
          {
            std::unique_lock lk(m);
            cv.wait(lk, st, [this]{ return !q.empty(); });
            if (st.stop_requested()) return;
            job = std::move(q.front()); q.pop();
          }
          job();
        }
      });
    }
  }

  ~thread_pool() {
    for (auto& w : workers) w.request_stop();
    cv.notify_all();
  }

  template <class F, class... A>
  auto submit(F&& f, A&&... a) {
    using R = std::invoke_result_t<F, A...>;
    std::packaged_task<R()> task(std::bind(std::forward<F>(f), std::forward<A>(a)...));
    auto fut = task.get_future();
    {
      std::scoped_lock lk(m);
      q.emplace([t = std::move(task)]() mutable { t(); });
    }
    cv.notify_one();
    return fut;
  }
};

Pipelines, batching, and backpressure

Real systems often form pipelines: stage A parses, stage B transforms, stage C writes. Use bounded queues between stages to prevent overload. When a queue fills, block producers or shed load to maintain latency targets. Batching small tasks reduces overhead and cache misses.

Executors and senders & receivers

Various proposals define executors and senders & receivers, which standardize how work is submitted and how completion is signaled. Toolkits and libraries already provide similar abstractions. The direction favors explicit scheduling and typed completion. When these interfaces land in your toolchain you can adapt pools and algorithms to them with small wrappers.

Concurrency in C++ rewards discipline: pick simple primitives, keep regions small, publish state carefully, and prefer tasks over manual thread choreography for most designs.