February 17, 2025
Concurrency and parallelism
I wanted to explore the differences between concurrency and parallelism. I know Concurrency ≠parallelism but I wanted to refresh my memory and visualize each term. They refer to executing multiple tasks, but they represent different approaches.
- Concurrency is about managing multiple tasks by interleaving their execution.
- Parallelism is the ability to execute independent tasks of a program in the same instant of time.
Tasks communicate with each other by using a shared memory or by message passing. It's about switching between tasks really quicky. Concurrent tasks (or modules) come in two different kinds: processes and threads.
- Threads are the smallest units of execution within a process. They share the same memory space and can communicate with each other.
- Processes are independent execution units within their own memory space. Each process has its own memory. A process can contain one or more threads.
💡Example of concurrency
- a chef preparing multiple dishes by switching between chopping, stirring, and plating.
- A distributed system processing emails while handling database requests.
There are three main ways to achieve concurrency:
Distributed
Multiple independent processes with no shared memory, communicating only via message passing.
Multi-Threaded
Multiple threads share memory, so require locks. Generally, a single process can have multiple threads.
Event Queue
Only a single thread exists! A single loop reads from the event queue and invokes the handlers.
Unlike concurrency, parallelism is about separating tasks into smaller parts to be performed independently and simultaneously using more than one processor. It's when tasks actually run at the same time, without just giving the illusion of it. The two most common ways to achieve parallelism:
Data Parallelism
Split data into chunks and process them simultaneously (e.g., rendering frames in a video).
Task Parallelism
Execute different tasks at the same time (e.g., downloading files while compressing others).
💡Example of parallelism
- multiple chefs each working on different dishes.
- Streaming music and texting a friend at the same time on your phone.
JavaScript, and anything built on it, like React, is single-threaded. This means that it can only execute one task at a time. However, JavaScript can simulate concurrency by using non-blocking asynchronous operations using an . This is done by using callbacks, promises, and async/await.
- Multiprogramming refers to running multiple programs concurrently on a single computer.
- Multithreading refers to a single program executing multiple threads concurrently.
Parallelism speeds up tasks by leveraging multiple processors to improve performance. Concurrency keeps systems responsive by managing multiple tasks efficiently. Distributed systems rely on concurrency to handle thousands of requests and scale.
I/O bound tasks like reading files, network requests, and database queries can be parallelized and can benefit from multithreading. They benefit more from concurrency than parallelism, while CPU-bound tasks like image processing, video encoding, and data analysis can be parallelized and benefit more from true parallelism.
- In
React, Web Workers can be used for CPU-intensive tasks. Web Workers run scripts in background threads. Promise/async/await can be used for asynchronous tasks. Javahas built-in support for multithreading using the Thread class or ExecutorService.Pythonhas the threading module for multithreading and the multiprocessing module for parallelism.