Understanding the Differences Between Parallelism and Concurrency

Parallelism and concurrency are two related but distinct concepts in computer science.

Parallelism involves executing multiple tasks simultaneously, which is particularly useful for CPU-bound tasks that can be divided into smaller sub-tasks. For instance, a video rendering program can leverage parallelism to render several frames of a video concurrently. This helps to complete the task faster and more efficiently by utilizing multiple processing units such as CPU cores. Another example is a financial simulation program that uses parallelism to run several simulations concurrently, allowing traders to quickly evaluate various scenarios and make informed decisions.

In a parallel system, multiple processors or cores are used to perform multiple tasks simultaneously, which improves the performance of a single large task by distributing the workload across several processors or cores. The workload is divided into smaller, independent units of work that can be executed in parallel on different processors or cores. Parallelism is typically used to increase the performance of tasks that require high levels of computation and can be divided.

Concurrency involves making progress on multiple tasks simultaneously, which is particularly useful for I/O-bound tasks that require waiting for data from external sources, such as databases or network connections. An example of this is a web server that can handle multiple requests concurrently, using concurrency to keep the system responsive by interleaving work. By using coroutines and event loops, the program can manage multiple tasks without blocking the main thread, allowing it to handle more requests and scale better.

In a concurrent system, multiple tasks can be started and executed independently, but they may not be executed simultaneously. The goal of concurrency is to enhance the responsiveness and throughput of a system by allowing it to handle several requests or events simultaneously. Concurrency is often used to improve the efficiency and scalability of systems by taking advantage of modern hardware and software architectures, allowing developers to create responsive and fast systems that can handle numerous tasks simultaneously.

In practice, the terms parallelism and concurrency are often used interchangeably, and many systems use a combination of both parallelism and concurrency to achieve optimal performance. However, it's important to understand the difference between the two concepts in order to effectively design and optimize systems for specific use cases.

Python asyncio as an example of concurrency:

import asyncio

async def task(name, seconds):
    print(f"Task {name} started.")
    await asyncio.sleep(seconds)
    print(f"Task {name} finished after {seconds} seconds.")

async def main():
    await asyncio.gather(
        task("A", 3),
        task("B", 2),
        task("C", 1)
    )

asyncio.run(main())

This code defines a coroutine function task that takes in a name and the number of seconds to sleep before printing a message that the task has been completed. The main function uses the asyncio.gather function to run three tasks concurrently, each with a different name and sleep time.

When executed, this program will output the messages of all three tasks interwoven, indicating that the tasks are running concurrently. Since the tasks are I/O-bound and not CPU-bound, concurrency is an appropriate technique to improve the responsiveness of the program. By utilizing coroutines and event loops, the program can manage multiple tasks without blocking the main thread, allowing it to handle more requests and scale better.

Python multiprocessing as the example of... you know what.

import time
from multiprocessing import Pool

def task(n):
    print(f"Task {n} started.")
    time.sleep(2)
    print(f"Task {n} finished after 2 seconds.")

with Pool(3) as p:
    p.map(task, [1, 2, 3])

This code defines a function task that takes in a single argument n and sleeps for 2 seconds before printing a message that the task has completed. The with Pool(3) as p: block creates a Pool object with 3 worker processes and uses the map function to apply the task function to a list of arguments [1, 2, 3].

When executed, this program will output the messages of all three tasks, indicating that they are running in parallel on different CPU cores. Since the tasks are CPU-bound and can be broken down into smaller sub-tasks, parallelism is an appropriate technique to improve the performance of the program. By utilizing multiple processing units, such as CPU cores, the program can complete the work faster and more efficiently.

Summary

Concurrency is the ability to make progress on multiple tasks at the same time, useful for I/O-bound tasks. Parallelism is the simultaneous execution of multiple tasks, useful for CPU-bound tasks. Concurrency allows multiple tasks to run independently, while parallelism distributes workload across multiple processors. Both techniques can be used together and understanding their differences is important for optimizing software performance and scalability.

tl;dr;

Concurrency is useful for I/O-bound tasks that involve waiting for external data.
Parallelism is useful for CPU-bound tasks that can be broken down into smaller sub-tasks.
Concurrency allows multiple tasks to run independently and make progress on them concurrently.
Parallelism involves the simultaneous execution of multiple tasks using multiple processing units.
Both techniques can be used together, and understanding their differences is important for optimizing software performance and scalability.

Maciej Marzęta Software Engineer Blog