Lesson 20-C++ Concurrency and Parallelism

Threads

Basic Concepts

  • Thread: A thread is the smallest unit of program execution flow. A standard C++ program has at least one main thread.
  • Multithreading: The ability to run multiple threads simultaneously within a single program.
  • Synchronization: Methods to ensure multiple threads share data correctly and safely.
  • Mutex: A mechanism to protect shared resources from simultaneous access by multiple threads.
  • Condition Variable: A mechanism for inter-thread communication, used to block a thread until a certain condition is met.
  • Atomic Operations: Indivisible operations that execute correctly even in a multithreaded environment.
  • Future and Promise: A way to handle the results of asynchronous operations.

Creating and Managing Threads

#include <iostream>
#include <thread>

void thread_function(int id) {
    std::cout << "Thread " << id << " is running." << std::endl;
}

int main() {
    std::thread t1(thread_function, 1);
    std::thread t2(thread_function, 2);

    t1.join();
    t2.join();

    return 0;
}

Problems with Shared Data Between Threads

  • Invariant: A statement that is always true about a specific data structure, e.g., in a doubly linked list, for two adjacent nodes A and B, A’s next pointer must point to B, and B’s previous pointer must point to A. Sometimes, for convenience, the invariant is temporarily broken during updates to complex data structures, such as when deleting a node N from a doubly linked list: first, N’s previous node points to N’s next node (invariant broken), then N’s next node points to the previous node, and finally N is deleted (invariant restored).
  • When one thread modifies shared data, it can break the invariant. If another thread accesses the data at that time, the invariant may be permanently broken—this is a race condition.
  • If the order of thread execution does not affect the result, it is a benign race that can be ignored. The concern is race conditions caused by broken invariants.
  • The C++ standard defines data race as a specific type of race condition involving concurrent modification of a single object. Data races cause undefined behavior.
  • Race conditions require one thread to be executing while another accesses the same data block. Problems are hard to reproduce, so extensive and complex operations are needed during programming to avoid race conditions.

Using Mutex

#include <iostream>
#include <thread>
#include <mutex>

std::mutex mtx;

void thread_function(int id) {
    std::lock_guard<std::mutex> lock(mtx); // Automatically unlocks
    std::cout << "Thread " << id << " is running." << std::endl;
}

int main() {
    std::thread t1(thread_function, 1);
    std::thread t2(thread_function, 2);

    t1.join();
    t2.join();

    return 0;
}

Using Condition Variable

#include <iostream>
#include <thread>
#include <mutex>
#include <condition_variable>

std::mutex mtx;
std::condition_variable cv;
bool ready = false;

void wait_for_signal() {
    std::unique_lock<std::mutex> lock(mtx);
    cv.wait(lock, []{return ready;}); // Wait until ready becomes true
    std::cout << "Processing data..." << std::endl;
}

void set_ready() {
    {
        std::lock_guard<std::mutex> lock(mtx);
        ready = true;
    }
    cv.notify_one(); // Notify one waiting thread
}

int main() {
    std::thread waiter(wait_for_signal);
    std::this_thread::sleep_for(std::chrono::seconds(1));
    std::thread setter(set_ready);

    setter.join();
    waiter.join();

    return 0;
}

Using Atomic Operations

#include <iostream>
#include <thread>
#include <atomic>

std::atomic<int> counter(0);

void increment_counter() {
    for (int i = 0; i < 100000; ++i) {
        ++counter;
    }
}

int main() {
    std::thread t1(increment_counter);
    std::thread t2(increment_counter);

    t1.join();
    t2.join();

    std::cout << "Counter value: " << counter.load() << std::endl;

    return 0;
}

Using Future and Promise

#include <iostream>
#include <thread>
#include <future>

int expensive_computation(int x) {
    std::this_thread::sleep_for(std::chrono::seconds(2));
    return x * x;
}

int main() {
    std::promise<int> promise;
    std::future<int> future = promise.get_future();

    std::thread t([&promise](){
        promise.set_value(expensive_computation(10));
    });

    std::cout << "Future result: " << future.get() << std::endl;
    t.join();

    return 0;
}

Thread Local Storage (TLS)

#include <iostream>
#include <thread>

thread_local int thread_id = 0;

void thread_function(int id) {
    thread_id = id;
    std::cout << "Thread " << thread_id << " has local ID " << thread_id << std::endl;
}

int main() {
    std::thread t1(thread_function, 1);
    std::thread t2(thread_function, 2);

    t1.join();
    t2.join();

    return 0;
}

Exception Handling

#include <iostream>
#include <thread>
#include <exception>

void throw_exception() {
    throw std::runtime_error("Exception from thread!");
}

int main() {
    try {
        std::thread t(throw_exception);
        t.join();
    } catch (const std::exception& e) {
        std::cout << "Caught exception: " << e.what() << std::endl;
    }

    return 0;
}

Passing Parameters to Thread Functions

Parameters are passed to callable objects or functions by simply providing additional arguments to the std::thread constructor. Parameters are copied by default into memory storage accessible by the new execution thread, even if the function expects references.

1. Default arguments are ignored


#include <thread>
void f(int i = 1) { std::cout << i << std::endl; }

int main() {
  std::thread t{f, 42};  // std::thread t{f} would fail because default arguments are ignored
  t.join();
}
// Prints 42
  1. Reference types are also ignored; use std::ref for references
#include <cassert>
#include <thread>

void f(int& i) { ++i; }
int main() {
  int i = 1;
  std::thread t{f, std::ref(i)};
  t.join();
  assert(i == 2);
}

If creating a thread for a non-static member function of an instance, the first parameter is a pointer to the member function, the second is a pointer to the instance, and subsequent parameters are the function’s arguments

#include <iostream>
#include <thread>

class A {
 public:
  void f(int i) { std::cout << i; }
};

int main() {
  A a;
  std::thread t1{&A::f, &a, 42};  // Calls a->f(42)
  std::thread t2{&A::f, a, 42};   // Copy-constructs tmp_a, then calls tmp_a.f(42)
  t1.join();
  t2.join();
}
// Inside a class, starting a thread with a non-static member function uses `this` as the second parameter.
// mProcessTs = std::thread(&Dvr::process_ts, this);

To pass move-only type parameters to a thread function, use std::move


void f(std::unique_ptr<int> p) { std::cout << *p; }
int main()
{
    std::unique_ptr<int> p = std::make_unique<int>(42);
    std::thread t{f, std::move(p)};
    t.join();
}

Transferring Thread Ownership

std::thread is a move-only type, cannot be copied, only moved to transfer ownership, but ownership cannot be transferred to a joinable thread.
Looking at the std::thread source makes this clear:

// Copy constructor and copy assignment are deleted
thread(const thread&) = delete;  
thread& operator=(const thread&) = delete;
// For move assignment, if the previous object is joinable(), std::terminate() is called. So ownership cannot be transferred to a joinable thread.
// This makes sense: if the previous object is joinable(), it is running; reassigning a running thread to another running thread is problematic, so std::terminate() is called.

thread& operator=(thread&& __t)
{
    if (joinable())
        std::terminate();
    swap(__t);
    return *this;
}
#include <thread>
#include <utility>
#include <cassert>
  
void f() {}
void g() {}
  
int main()
{
    std::thread a{f};
    std::thread b = std::move(a);
    assert(!a.joinable()); // Moving a to b makes thread a non-joinable
    assert(b.joinable());  // thread b is joinable
    a = std::thread{g}; //
    assert(a.joinable());
    assert(b.joinable());
    // Error: cannot transfer ownership to a joinable thread; a already has an associated thread (running g()), so std::terminate() is called.
    // a = std::move(b); //【1】
    a.join();
    a = std::move(b); // Now a has finished running, it is not joinable()
    assert(a.joinable());
    assert(!b.joinable());
    a.join();
}

/**
If 【1】 is uncommented, output:
terminate called without an active exception
Aborted (core dumped)
*/

Move operations also apply to containers that support moving

#include <algorithm>
#include <thread>
#include <vector>
#include <functional>
#include <iostream>

int main()
{
    std::vector<std::thread> v;
    for (int i = 0; i < 10; ++i)
    {
        v.emplace_back([] (int id) {std::cout << " " << id << " ";}, i);
    }
    std::for_each(std::begin(v), std::end(v), std::mem_fn(&std::thread::join));
    return 0;
}

// Prints 0  2  1  3  4  5  6  7  8  9

Advanced Concurrency Programming

Advanced concurrency programming involves more complex multithreading patterns and techniques, including but not limited to advanced synchronization mechanisms, thread pools, concurrent containers, and asynchronous programming models.

Advanced Synchronization Mechanisms

Read-Write Locks

Read-write locks allow multiple readers to access a resource simultaneously but only one writer. This is very useful when read operations are much more frequent than writes.

#include <iostream>
#include <thread>
#include <shared_mutex>

std::shared_timed_mutex rw_mutex;
int shared_data = 0;

void read_data() {
    std::shared_lock<std::shared_timed_mutex> lock(rw_mutex);
    std::cout << "Reading data: " << shared_data << std::endl;
}

void write_data(int new_data) {
    std::unique_lock<std::shared_timed_mutex> lock(rw_mutex);
    shared_data = new_data;
    std::cout << "Writing data: " << shared_data << std::endl;
}

int main() {
    std::thread t1(read_data);
    std::thread t2(write_data, 42);
    std::thread t3(read_data);

    t1.join();
    t2.join();
    t3.join();

    return 0;
}

Semaphores

A semaphore is a synchronization mechanism used to control access to shared resources by multiple threads.

#include <iostream>
#include <thread>
#include <semaphore.h>

sem_t semaphore;

void consume_resource() {
    sem_wait(&semaphore);
    std::cout << "Resource consumed" << std::endl;
    sem_post(&semaphore);
}

void produce_resource() {
    sem_post(&semaphore);
    std::cout << "Resource produced" << std::endl;
    sem_wait(&semaphore);
}

int main() {
    sem_init(&semaphore, 0, 1);

    std::thread t1(consume_resource);
    std::thread t2(produce_resource);

    t1.join();
    t2.join();

    sem_destroy(&semaphore);

    return 0;
}

Thread Pool

A thread pool is a design pattern that manages a group of pre-created threads to improve responsiveness and resource utilization.

#include <iostream>
#include <vector>
#include <thread>
#include <queue>
#include <functional>
#include <mutex>
#include <condition_variable>

class ThreadPool {
public:
    ThreadPool(size_t num_threads) {
        for (size_t i = 0; i < num_threads; ++i) {
            workers.emplace_back([this] {
                for (;;) {
                    std::function<void()> task;
                    {
                        std::unique_lock<std::mutex> lock(queue_mutex);
                        condition.wait(lock, [this] { return stop || !tasks.empty(); });
                        if (stop && tasks.empty())
                            return;
                        task = std::move(tasks.front());
                        tasks.pop();
                    }
                    task();
                }
            });
        }
    }

    template <class F, class... Args>
    auto enqueue(F&& f, Args&&... args) -> std::future<typename std::result_of<F(Args...)>::type> {
        using return_type = typename std::result_of<F(Args...)>::type;

        auto task = std::make_shared<std::packaged_task<return_type()>>(
            std::bind(std::forward<F>(f), std::forward<Args>(args)...));

        std::future<return_type> res = task->get_future();
        {
            std::unique_lock<std::mutex> lock(queue_mutex);

            // don't allow enqueueing after stopping the pool
            if (stop)
                throw std::runtime_error("enqueue on stopped ThreadPool");

            tasks.emplace([task]() { (*task)(); });
        }
        condition.notify_one();
        return res;
    }

    ~ThreadPool() {
        {
            std::unique_lock<std::mutex> lock(queue_mutex);
            stop = true;
        }
        condition.notify_all();
        for (std::thread& worker : workers)
            worker.join();
    }

private:
    std::vector<std::thread> workers;
    std::queue<std::function<void()>> tasks;

    std::mutex queue_mutex;
    std::condition_variable condition;
    bool stop;
};

void thread_pool_example() {
    ThreadPool pool(4); // Create a thread pool with 4 threads

    for (int i = 0; i < 10; ++i) {
        pool.enqueue([i] {
            std::cout << "Task " << i << " executed by thread " << std::this_thread::get_id() << std::endl;
        });
    }
}

int main() {
    thread_pool_example();
    return 0;
}

Concurrent Containers

Atomic Containers

The C++ standard library provides atomic containers like std::atomic<T> to implement thread-safe data structures.

#include <iostream>
#include <thread>
#include <atomic>

std::atomic<int> atomic_counter(0);

void increment_counter() {
    for (int i = 0; i < 100000; ++i) {
        atomic_counter.fetch_add(1, std::memory_order_relaxed);
    }
}

int main() {
    std::thread t1(increment_counter);
    std::thread t2(increment_counter);

    t1.join();
    t2.join();

    std::cout << "Counter value: " << atomic_counter.load() << std::endl;

    return 0;
}

Concurrent Queue

A concurrent queue is a thread-safe data structure suitable for multi-producer multi-consumer scenarios.

#include <iostream>
#include <queue>
#include <mutex>
#include <condition_variable>
#include <thread>

template<typename T>
class ConcurrentQueue {
public:
    void push(T value) {
        std::unique_lock<std::mutex> lock(mtx);
        queue.push(value);
        lock.unlock();
        cond.notify_one();
    }

    T pop() {
        std::unique_lock<std::mutex> lock(mtx);
        cond.wait(lock, [this] { return !queue.empty(); });
        T value = queue.front();
        queue.pop();
        return value;
    }

private:
    std::queue<T> queue;
    std::mutex mtx;
    std::condition_variable cond;
};

void producer(ConcurrentQueue<int>& q) {
    for (int i = 0; i < 10; ++i) {
        q.push(i);
        std::this_thread::sleep_for(std::chrono::milliseconds(100));
    }
}

void consumer(ConcurrentQueue<int>& q) {
    for (int i = 0; i < 10; ++i) {
        std::cout << "Consumed: " << q.pop() << std::endl;
        std::this_thread::sleep_for(std::chrono::milliseconds(200));
    }
}

int main() {
    ConcurrentQueue<int> q;

    std::thread prod(producer, std::ref(q));
    std::thread cons(consumer, std::ref(q));

    prod.join();
    cons.join();

    return 0;
}

Asynchronous Programming

Asynchronous Tasks

Use std::async to launch asynchronous tasks and std::future to retrieve results.

#include <iostream>
#include <future>
#include <chrono>

int expensive_computation(int x) {
    std::this_thread::sleep_for(std::chrono::seconds(2));
    return x * x;
}

int main() {
    std::future<int> future_result = std::async(std::launch::async, expensive_computation, 10);

    std::cout << "Future result: " << future_result.get() << std::endl;

    return 0;
}

Advanced Exception Handling

In multithreaded environments, exception handling becomes more complex. It must be ensured that exceptions are caught in the correct context.

#include <iostream>
#include <thread>
#include <future>

int throw_exception() {
    throw std::runtime_error("Exception from thread!");
}

int main() {
    std::future<int> future = std::async(std::launch::async, throw_exception);

    try {
        future.get();
    } catch (const std::exception& e) {
        std::cout << "Caught exception: " << e.what() << std::endl;
    }

    return 0;
}

Atomic Operations

An atomic operation is an indivisible operation that cannot be interrupted by other operations during execution. This means that in a multithreaded environment, once an atomic operation starts, it will complete without being interrupted by another thread.

Example Code

#include <iostream>
#include <atomic>
#include <thread>

std::atomic<int> counter(0);

void increment_counter() {
    for (int i = 0; i < 100000; ++i) {
        ++counter;
    }
}

int main() {
    std::thread t1(increment_counter);
    std::thread t2(increment_counter);

    t1.join();
    t2.join();

    std::cout << "Counter value: " << counter.load() << std::endl;

    return 0;
}

Atomic Variables

An atomic variable is a special variable type that supports atomic operations. In C++, std::atomic is a template class used to define atomic variables of various types.

Example Code

#include <iostream>
#include <atomic>
#include <thread>

std::atomic<int> atomic_var(0);

void update_atomic_var() {
    atomic_var.store(42, std::memory_order_release);
}

int main() {
    std::thread t(update_atomic_var);

    t.join();

    std::cout << "Atomic variable value: " << atomic_var.load(std::memory_order_acquire) << std::endl;

    return 0;
}

Atomic Compare and Swap (CAS)

Atomic compare-and-swap is a common atomic operation often used to implement optimistic locking. CAS attempts to update a variable from an old value to a new value, succeeding only if the current value matches the old value.

Example Code

#include <iostream>
#include <atomic>
#include <thread>

std::atomic<int> shared_value(0);

void cas_example() {
    int expected = 0;
    int desired = 1;
    bool success = shared_value.compare_exchange_strong(expected, desired, std::memory_order_acq_rel);
    if (success) {
        std::cout << "CAS succeeded, new value: " << desired << std::endl;
    } else {
        std::cout << "CAS failed, current value: " << shared_value.load() << std::endl;
    }
}

int main() {
    std::thread t(cas_example);

    t.join();

    return 0;
}

More Atomic Operations

Besides simple addition and compare-exchange, std::atomic provides more atomic operations such as fetch_add, exchange, compare_exchange_weak, etc.

Example Code

#include <iostream>
#include <atomic>
#include <thread>

std::atomic<int> shared_value(0);

void fetch_add_example() {
    int result = shared_value.fetch_add(1, std::memory_order_relaxed);
    std::cout << "Fetch add result: " << result << ", new value: " << shared_value.load() << std::endl;
}

int main() {
    std::thread t(fetch_add_example);

    t.join();

    return 0;
}

Concurrent Containers

C++11 introduced several concurrent containers in the standard library, primarily thread-safe versions of <unordered_map> and <unordered_set>. These containers are based on std::unordered_map and std::unordered_set and provide finer-grained atomicity and locking mechanisms.

  • std::unordered_map
  • std::unordered_set
  • std::shared_timed_mutex (used to implement concurrent containers)

Concurrent Container Example

Here are some common concurrent containers:

Concurrent version of std::unordered_map:

  • std::unordered_map itself is not thread-safe but can be made thread-safe with external locking.
  • Using std::shared_timed_mutex enables finer-grained lock control.

Concurrent version of std::unordered_set:

  • Similarly, std::unordered_set requires external mechanisms for thread safety.

Concurrent containers provided by third-party libraries:

  • Intel TBB (Threading Building Blocks): Provides concurrent_unordered_map and concurrent_unordered_set.
  • Microsoft Visual C++: Provides concurrent_unordered_map and concurrent_unordered_set.

Here is a simple example using concurrent_unordered_map from the Intel TBB library:

#include <tbb/concurrent_unordered_map.h>
#include <iostream>
#include <thread>

int main() {
    tbb::concurrent_unordered_map<int, int> myMap;

    // Multiple threads insert data concurrently
    std::thread t1([&myMap](){
        for (int i = 0; i < 1000; ++i) {
            myMap[i] = i * 2;
        }
    });

    std::thread t2([&myMap](){
        for (int i = 1000; i < 2000; ++i) {
            myMap[i] = i * 2;
        }
    });

    t1.join();
    t2.join();

    // Iterate and print results
    for (auto& pair : myMap) {
        std::cout << "Key: " << pair.first << ", Value: " << pair.second << std::endl;
    }

    return 0;
}

This code demonstrates how to use tbb::concurrent_unordered_map to store and retrieve key-value pairs while allowing concurrent operations from multiple threads.

Thread-Safe Containers

Vector

  • Vector is an early Java-provided thread-safe container that ensures thread safety by using the synchronized keyword on every public method.
  • Due to low performance (lock acquisition on every call), it is generally not recommended in high-concurrency scenarios.

Hashtable

  • Hashtable is similar to Vector, a thread-safe container for key-value storage.
  • It uses internal synchronized locks for thread safety.
  • Note that Hashtable does not allow null keys or values.

Collections.synchronizedList/Collections.synchronizedMap

  • The Collections utility class provides synchronizedList and synchronizedMap methods to convert any List or Map into a thread-safe container.
  • This approach requires manually passing a lock object to the constructor; all operations on the container must be protected by this lock.

ConcurrentHashMap

  • ConcurrentHashMap is a high-performance, thread-safe hash map introduced in Java 5.
  • It achieves thread safety through segment locking, offering better concurrency performance than Hashtable and synchronizedMap.
  • Supports concurrent reads; write operations lock specific segments.

CopyOnWriteArrayList/CopyOnWriteArraySet

  • CopyOnWriteArrayList and CopyOnWriteArraySet are thread-safe containers using the Copy-On-Write (COW) strategy.
  • This means that when a thread modifies the list, a new copy is created, leaving the original list unchanged, so reads require no locking.
  • Suitable for read-heavy, write-light scenarios; write operations can be expensive due to array copying.

ConcurrentSkipListMap/ConcurrentSkipListSet

  • ConcurrentSkipListMap and ConcurrentSkipListSet are thread-safe containers based on skip lists.
  • They provide ordered mapping and sets with good concurrency performance.
  • Suitable for scenarios requiring ordered storage and frequent concurrent read/write operations.

Concurrent Queue

A concurrent queue is a queue that supports efficient concurrent access in a multithreaded environment. Typically, such a queue must support thread-safe enqueue and dequeue operations.

Implementation Ideas

  • Use std::mutex and std::condition_variable to synchronize thread operations.
  • Use std::unique_lock to lock the mutex.
  • Use std::atomic<bool> to indicate whether the queue is empty.

Example Code

#include <queue>
#include <mutex>
#include <condition_variable>
#include <atomic>
#include <thread>

template<typename T>
class ConcurrentQueue {
private:
    std::queue<T> queue_;
    mutable std::mutex mutex_;
    std::condition_variable cond_;
    bool stop_ = false;

public:
    void push(const T& value) {
        std::lock_guard<std::mutex> lock(mutex_);
        queue_.push(value);
        cond_.notify_one(); // Notify waiting threads
    }

    T pop() {
        std::unique_lock<std::mutex> lock(mutex_);
        while (queue_.empty() && !stop_) {
            cond_.wait(lock); // Wait until data is available or stop flag is set
        }
        if (!stop_) {
            T front = queue_.front();
            queue_.pop();
            return front;
        } else {
            throw std::runtime_error("Queue is stopped");
        }
    }

    void stop() {
        {
            std::lock_guard<std::mutex> lock(mutex_);
            stop_ = true;
        }
        cond_.notify_all(); // Notify all waiting threads
    }
};

void producer(ConcurrentQueue<int>& queue) {
    for (int i = 0; i < 10; ++i) {
        queue.push(i);
        std::this_thread::sleep_for(std::chrono::milliseconds(100));
    }
}

void consumer(ConcurrentQueue<int>& queue) {
    while (true) {
        try {
            int value = queue.pop();
            std::cout << "Consumed: " << value << std::endl;
        } catch (const std::runtime_error& e) {
            std::cout << "Consumer exiting: " << e.what() << std::endl;
            break;
        }
    }
}

int main() {
    ConcurrentQueue<int> queue;

    std::thread prod(producer, std::ref(queue));
    std::thread cons(consumer, std::ref(queue));

    prod.join();
    queue.stop();
    cons.join();

    return 0;
}

Concurrent Hash Table

A concurrent hash table is a hash table that supports efficient concurrent access in a multithreaded environment. Typically, it must support thread-safe insertion, lookup, and deletion.

Implementation Ideas

  • Use std::unordered_map as the underlying data structure.
  • Use std::shared_timed_mutex for fine-grained lock control.
  • Use shared locks for read operations; exclusive locks for write operations.

Example Code

#include <unordered_map>
#include <shared_mutex>
#include <thread>
#include <iostream>

template<typename K, typename V>
class ConcurrentHashMap {
private:
    std::unordered_map<K, V> map_;
    std::shared_timed_mutex mutex_;

public:
    void insert(const K& key, const V& value) {
        std::unique_lock<std::shared_timed_mutex> lock(mutex_);
        map_[key] = value;
    }

    V find(const K& key) {
        std::shared_lock<std::shared_timed_mutex> lock(mutex_);
        auto it = map_.find(key);
        if (it != map_.end()) {
            return it->second;
        }
        return V(); // Return default-constructed value
    }

    void remove(const K& key) {
        std::unique_lock<std::shared_timed_mutex> lock(mutex_);
        map_.erase(key);
    }
};

void insert_data(ConcurrentHashMap<int, int>& map) {
    for (int i = 0; i < 10; ++i) {
        map.insert(i, i * 2);
        std::this_thread::sleep_for(std::chrono::milliseconds(100));
    }
}

void find_data(ConcurrentHashMap<int, int>& map) {
    for (int i = 0; i < 10; ++i) {
        int value = map.find(i);
        std::cout << "Found: " << value << std::endl;
        std::this_thread::sleep_for(std::chrono::milliseconds(50));
    }
}

int main() {
    ConcurrentHashMap<int, int> map;

    std::thread insert_thread(insert_data, std::ref(map));
    std::thread find_thread(find_data, std::ref(map));

    insert_thread.join();
    find_thread.join();

    return 0;
}

Thread Pool

A thread pool is a very important concept in modern multithreaded programming. It effectively manages a group of worker threads to handle submitted tasks. Below, we will detail the design principles of a thread pool, task scheduling mechanisms, and the startup and shutdown process of a thread pool, with corresponding code examples.

Thread Pool Design Principles

The main purpose of a thread pool is to reuse pre-created threads instead of creating new threads each time a task needs execution. This reduces the overhead of thread creation and destruction, improving system responsiveness and throughput.

Core Components

  • Thread Pool: The core component managing threads and tasks.
  • Task Queue: Used to store pending tasks.
  • Worker Threads: The actual threads executing tasks.

Design Points

  • Thread Count: The number of threads in the pool should be determined based on system resources and task characteristics.
  • Task Queue Size: Determines how many unprocessed tasks can be buffered.
  • Task Scheduling Strategy: Defines how tasks are selected from the queue and assigned to idle worker threads.
  • Thread Management: Includes thread creation, destruction, activation, and sleep.

Task Scheduling

Task scheduling is the process of selecting appropriate tasks from the task queue and assigning them to idle worker threads. Common scheduling strategies include:

  • First In First Out (FIFO): Schedule tasks in the order they enter the queue.
  • Last In First Out (LIFO): Prioritize recently added tasks.
  • Priority Scheduling: Select tasks based on priority.

Thread Pool Startup and Shutdown

  • Startup: Initialize the thread pool, create a certain number of worker threads, and prepare to accept tasks.
  • Shutdown: Stop accepting new tasks and wait for currently executing tasks to complete.

Example Code Below is a simple thread pool implementation example, including startup, task scheduling, and shutdown.

Thread Pool Class Definition

#include <queue>
#include <thread>
#include <functional>
#include <mutex>
#include <condition_variable>
#include <atomic>
#include <future>

class ThreadPool {
private:
    std::queue<std::function<void()>> tasks_; // Task queue
    std::mutex mutex_; // Mutex
    std::condition_variable cond_; // Condition variable
    std::vector<std::thread> threads_; // Thread pool
    std::atomic<bool> stop_ = false; // Stop flag

    void workerThread() {
        while (!stop_) {
            std::function<void()> task;
            {
                std::unique_lock<std::mutex> lock(mutex_);
                cond_.wait(lock, [this] { return stop_ || !tasks_.empty(); });
                if (stop_ && tasks_.empty())
                    return;
                task = std::move(tasks_.front());
                tasks_.pop();
            }
            task();
        }
    }

public:
    ThreadPool(size_t numThreads) {
        for (size_t i = 0; i < numThreads; ++i) {
            threads_.emplace_back([this] { workerThread(); });
        }
    }

    template<typename Func, typename... Args>
    auto enqueue(Func&& func, Args&&... args)
        -> std::future<typename std::result_of<Func(Args...)>::type> {
        using ReturnType = typename std::result_of<Func(Args...)>::type;
        auto task = std::make_shared<std::packaged_task<ReturnType()>>(
            std::bind(std::forward<Func>(func), std::forward<Args>(args)...));

        std::future<ReturnType> res = task->get_future();
        {
            std::unique_lock<std::mutex> lock(mutex_);
            if (stop_)
                throw std::runtime_error("enqueue on stopped ThreadPool");
            tasks_.emplace([task]() { (*task)(); });
        }
        cond_.notify_one();
        return res;
    }

    ~ThreadPool() {
        {
            std::unique_lock<std::mutex> lock(mutex_);
            stop_ = true;
        }
        cond_.notify_all();
        for (auto& thread : threads_)
            thread.join();
    }
};

Usage Example

#include <iostream>
#include <chrono>

void task(int id) {
    std::cout << "Task " << id << " started" << std::endl;
    std::this_thread::sleep_for(std::chrono::seconds(1));
    std::cout << "Task " << id << " finished" << std::endl;
}

int main() {
    ThreadPool pool(4); // Create a thread pool with 4 worker threads

    for (int i = 0; i < 10; ++i) {
        pool.enqueue(task, i);
    }

    return 0;
}

Code Analysis

Thread Pool Class Definition:

  • The ThreadPool class contains basic components like task queue, mutex, condition variable, etc.
  • The workerThread member function implements the worker loop, continuously fetching and executing tasks from the queue.
  • The enqueue member function is used to submit tasks to the thread pool.
  • The destructor safely shuts down the thread pool.

Usage Example:

  • Create a thread pool with 4 worker threads.
  • Submit 10 tasks to the thread pool.

Types of Locks in C++ Multithreading

Locks between threads include: mutex, condition lock (condition variable), spin lock, read-write lock, recursive lock. Generally, lock functionality is inversely proportional to performance. We generally do not use recursive locks (the C++ standard library provides std::recursive_mutex).

The previous locks have been introduced; here we mainly focus on spin locks.

Below, we compare the principles of mutex and spin lock, which is very helpful for truly understanding spin locks.

Assume we have a computer with two processor cores, core1 and core2. Two threads, T1 and T2, are running on core1 and core2 respectively, sharing a resource.

First, we explain how a mutex works. A mutex is a sleep-waiting lock. Suppose thread T1 acquires the mutex and is running on core1. At this time, thread T2 also wants to acquire the mutex (pthread_mutex_lock), but since T1 is using it, T2 is blocked. When T2 is blocked, it is placed in a waiting queue, and processor core2 handles other tasks instead of busy-waiting. That is, the processor does not idle due to thread blocking; it processes other tasks.

A spin lock is different; it is a busy-waiting lock. That is, if T1 is using the spin lock and T2 also requests it, T2 will definitely not get it. Unlike a mutex, the processor core2 running T2 will continuously loop checking if the lock is available (spin lock request) until it acquires the lock.

From the name “spin lock,” we can see that if a thread wants to acquire a spin lock in use, it will keep occupying the CPU requesting the lock, preventing the CPU from doing anything else until it gets the lock—this is the meaning of “spin.”

When blocking occurs, a mutex allows the CPU to handle other tasks; a spin lock makes the CPU continuously loop requesting the lock. From the comparison of the two meanings, we can see that a “spin lock” is more CPU-intensive.

Share your love