Multithreading uses shared memory. Results from each thread can easily be shared with one another, without the need of explicit communication.
Shared memory allows data to be stored only in one place, reducing the memory footprint. Each worker does not need a copy of the data it is working on.
Each thread has access to the local runtime of the process in memory, including all libraries and pieces of code, instead of each thread having a copy of the local process. This significantly reduces memory overhead, and more importantly the latency, of this pattern.
Latency of utilising multiple threads is usually the fastest of all the parallel paradigms covered in this book, except for hardware level parallelism (hardware SIMD instructions).
If a language has a multithreading implementation (like Julia or C#), it is usually the easiest approach to leverage to allow a program to execute in parallel.
Disadvantages
Multithreading can only be scaled up to the size of one machine, the number of threads limited by the number of logical cores on a CPU.
Shared memory introduces the danger of race conditions occurring in your code, in some cases requiring the need for atomics, mutexes and semaphores, drastically increasing the complexity of some code.
Some algorithms will need a complete redesign to effectively use the multithreading paradigm.
Latency of spinning up additional threads and coordinating execution across all threads incurs a significant overhead cost. Unless the computation can be effectively parallelised, using multithreading may even slow down one’s code.
Execution in parallel necessitates an unpredictable order of execution, which often times greatly impacts the ability of the compiler and the runtime to pre-empt which memory is needed often and therefore has worse cache management than a serial approach. This can be mitigated, but usually requires an extension.
Existing routines or libraries used by your code may not be “thread-safe”, which means they should not be executed concurrently. If two parallel pieces of code run something that is not thread-safe, it could lead to unpredictable results at best, memory corruption and runtime errors at worse. Additional care by the developers must be taken to ensure that using multithreading is safe. Sometimes, if the non thread-safe code is essential, this paradigm may not be available.
Experiments that rely on a deterministic and predictable outcome, such as ones that use random number generation with a seeded pseudo random number generator often require intervention to make sure that results can be duplicated. This may involve re-writing large parts of the code base to ensure reproducibility.
Each thread usually requires its own stack and memory space in order to perform calculations. Additionally, used by each thread is often not allowed to overlap due to the risk of race conditions, and so more memory is usually required.