Asynchronous programming

9 July 2026

Author: Jan Kynčl

Recently, I had my first interview for a Rust internship. I don't want to write here about how it went or if I got the job (maybe in a separate post :D). The only thing you need to know to understand the reason why I started to dig into this topic is that I got melted by the question: how does Tokio, an asynchronous runtime in Rust, work? Of course, I had a pretty poor idea of how it works.

So I fumbled, and I think they kind of expected it. They actually told me that I know how to use it but have no idea how it works. But it's kind of obvious that someone going for an internship won't know this, isn't it? I mean, how many JavaScript junior devs know how the event loop works and stuff like that?

Well, anyway, after the interview, I asked myself, "Well, why don't I just learn it?" There's, of course, the argument, "I don't need to know how a car works to drive it," and that's valid in some cases, but when you are doing something like systems architecture, it's kind of essential to know where you want to use asynchronous code and where it's actually really stupid.

With that, my journey to understand async Rust started, and to be honest, it still continues. Really, it's a really complex topic, and it took me like a month of reading stuff to understand just the tip of the iceberg.

So, of course, a disclaimer: I'm learning this stuff, and if I write something wrong, please contact me and I'll be happy to learn from it and change it. Also, this post is primarily focusing on async Rust, but there are a lot of parallels with other languages, and well, Rust code is still running on the same computer architecture as your JS code.

What is actually async (async/await)

So, let's start with the basics. While we write code, we mostly do it sequentially, with tasks happening line by line. However, sometimes there are tasks that require waiting, like waiting for a response from a server, where we would have to wait on that exact line where we asked for the response.

This is kind of bad because if we leave it like this, it will be blocking the application, and we probably want to do other things, like rendering a GUI and stuff like that. That's where asynchronous code comes in. Instead of waiting for a specific task to finish, we can say, "Hey, start this task and continue with the other instructions." At that moment, your asynchronous runtime takes over and handles this task along with the root task (more about this later). When we need the result of this task, we can later await the result in the code and work with it.

This system is called async/await, and it's probably the only one you've encountered. There are other systems, like Zig's, but to be honest, I have no idea how it works, and I don't have a reason to learn it when there's a chance that a new update could bring a different system.

Parallelism VS Concurrency

Before I started this research, I thought, "Asynchronous programming? Well, the other task is running on another thread, right?" This is partially true. Yes, asynchronous programming can and will use more threads, but it doesn't mean you can't run async code on a single-threaded CPU. WHAT SORCERY IS THAT!? Not sorcery, just a really clever concept. That's why it's really important to understand what parallelism and concurrency actually are.

Parallelism is running multiple tasks at the same time using multiple threads. You could say one thread will run one task. This can be seen inside Figure 1. Then how is it possible to run asynchronous code on a single-threaded CPU? Here comes concurrency.

Instead of running a task on each thread like parallelism, concurrency will swap between multiple tasks on one thread so fast you can't even see it with the human eye. This way, we can create the illusion of running thousands of programs at the same time, even though you are actually running X programs on X threads at any given moment. That's why your CPU can run so many apps at the same time. And if you add the fact that modern CPUs are able to use multi-threading, turning one physical core into two virtual threads, you realize how powerful your CPU actually is. Graph, how concurrent workflow looks, is seeable in Figure 2.

Figure 1: A parallel workflow, where work happens on Task A and Task B independently²

Figure 2: A concurrent workflow, switching between Task A and Task B²

Asynchronous Runtime

As I hinted at before, the runtime is probably the most important part of asynchronous programming. But what is it? Well, you could say it's something of an engine for this whole async thing. It handles what is running, communication with the operating system (requesting resources, like threads), how to handle the await keyword, and so on. Even though it's so essential, there isn't one runtime that rules them all. It's kind of funny how a lot of programming languages have the exact same or a very similar way of writing code, even though there's so much happening differently in the background. Anyways, most likely every major one will, and pretty much does, some kind of concurrency and parallelism.

Tokio and asynchronous runtimes in rust

Unlike other programming languages, Rust doesn't have its own official asynchronous runtime. Instead of making one, they rely on the community to write their own. To be honest, I think it's kind of weird. Don't get me wrong. Making it possible to choose your own runtime is great and useful, like using a runtime that is optimized for embedded systems, for example, makes total sense, but why doesn't the Rust Foundation make its own general-purpose asynchronous runtime?

One of the main priorities that the Rust team has is backward compatibility. That means if the Rust team chose a specific architecture for an asynchronous runtime, they would be stuck with it forever and would never be able to swap it out. That's not the only issue, though. There's also the problem with the runtime itself. We haven't invented a runtime that works perfectly as a backend for a web server, a desktop application, and embedded microcontroller software at the same time (yet :D).

With that in mind, you realize that making one runtime to rule them all would cause issues or lower performance in specific scenarios, which would kind of defeat the main purpose of a standard asynchronous runtime.

Tokio

Even though I just wrote about how there's no standard, I kind of lied. Tokio is a third-party asynchronous runtime that is used in 99% of asynchronous projects, and you will see it a lot of the time. However, there are some situations where it doesn't make sense to use it. Tokio was primarily made for I/O-bound tasks, like web servers. But for tasks where you do heavy CPU-bound computations, like a hash bruteforcer, you won't get any performance advantage whatsoever, and you might even hit a bottleneck caused by Tokio. In that situation, you should use Tokio's spawn_blocking, which we'll cover more in the chapter Blocking threads.

At the start of the app, Tokio will request a number of threads from the operating system, usually the same number as your CPU cores, and initialize the root task. This is the main function in your code. From this root task, we spawn other tasks that will run other operations asynchronously.

It is important to mention that Tokio is really modular. This way, you can use just the parts of Tokio that you actually need. If you want all the features, you need to add the Tokio crate to your Cargo.toml like this: tokio = { version = "1.0", features = ["full"] }.

For people who have never touched Tokio, here's a simple example of utilizing it:

use std::time::Duration;

// This is the default macro to use Tokio as a runtime 
// with its default configuration.
// You can make your own macro with a different configuration, but
// most applications are fine with the default settings.
// Also, it turns the main() function into the root task.
#[tokio::main]
async fn main() {
    let moving_value = String::from("Random value");

    // This will spawn a new task inside the runtime
    let task = tokio::task::spawn(async move {
        // In Rust, async functions themselves are lazy, meaning 
        // if we don't await them, they won't even start. 
        // JavaScript, for example, is eager, just calling 
        // the function without the await keyword will still execute it.
        async_function().await;

        // This won't execute at all and will even 
        // trigger a warning from the compiler.
        async_function();

        let _moving = moving_value;
    });

    // WARNING: This won't work because the value was moved to 
    // another task. It needs to be cloned or use specific types.
    // More on this in the chapter "Sharing data safely".
    // println!("{moving_value}");

    println!("This will execute sooner than the other task");

    // Awaiting the future from task::spawn 
    // will return Result<T, JoinError>.
    // In this example, T is ().
    task.await;
}

async fn async_function() {
    tokio::time::sleep(Duration::from_secs(1)).await;
    println!("Hello from other task");
}

That is why Tokio is called an M:N scheduler, where we have M threads and N tasks, and Tokio tries to juggle these tasks so that they all get their share of the CPU, while utilizing work stealing. This means if thread A finishes all its tasks while thread B is overloaded, thread A will "steal" tasks from B's queue to keep the CPU fully utilized.

The juggling between tasks is kind of complicated, but I'll try to explain it simply:

At startup: Creating the thread pool and initializing the root task. This task will be placed inside one of the free threads.
Spawning: When hitting task::spawn, it creates a new task and adds it to the task queue.
Execution: Inside the task queue, the runtime pops a task and puts it onto an available thread whenever it can.
Completion: When the task is finished, it returns the result.

If you still struggle to understand how it works under the hood, I highly recommend this video.

Rayon

Rayon is one of the coolest libraries you can find in the whole Rust ecosystem. Its main purpose is to convert sequential computations into parallel ones. While we could talk about how lightweight it is and how it guarantees data-race-free execution, the most insane thing about Rayon is how incredibly convenient it is to implement.

For example, look at this sequential code:

fn sum_of_squares(input: &[i32]) -> i32 {
    input.iter()
         .map(|i| i * i)
         .sum()
}

Just by changing iter() into par_iter(), we can turn that sequential computation into a parallel one:

use rayon::prelude::*;

fn sum_of_squares(input: &[i32]) -> i32 {
    input.par_iter()
         .map(|i| i * i)
         .sum()
}

Even though it feels like pure magic, you shouldn't use it blindly everywhere. You still need to consider the nature of your computations and the number of items you are working with. If you are sorting an array with a small number of values, you will actually see slower results.

When your program starts, Rayon creates a fixed thread pool that usually matches your CPU's logical core count. Managing this pool requires a task queue, tracking data chunks, and coordinating threads. This management creates CPU overhead. If the task is too small, the overhead ends up being larger than the actual computation itself.

Rayon works by breaking a collection down into smaller chunks, distributing those chunks across its thread pool, and merging the results back together when finished. If you are sorting or iterating over a small array, like one with less than a few thousand elements, the time it takes Rayon to split the data and coordinate the threads can take longer than just running it sequentially.

If your code is just doing simple math, like adding or multiplying numbers that take a single clock cycle, sequential execution will be the better choice because it doesn't have to handle all the things we just talked about.

So, when should you use Rayon? Usually for tasks that are CPU-heavy. For example, hashing multiple chunks of data, processing images, or parsing complex strings.

Tokio and Rayon

Even though Rayon and Tokio both have a work-stealing feature, they are built for really different use cases. Basically, Tokio is for I/O-bound operations and Rayon is for CPU-bound ones.

There's thread and thread

In computer science, the word "thread" is notorious for causing confusion, at least for me :D. The tech world uses the exact same word to describe multiple concepts that relate to each other but mean entirely different things. Depending on whether you are talking about hardware, the operating system, or your code, a "thread" behaves in a completely different way.

Physical/Hardware thread

This type of thread is the simplest to understand because it's tied directly to your physical computer parts. Every modern CPU has physical cores, and traditionally, every core has its own thread. These are called physical or hardware threads, and you can think of them as the actual physical workers in your computer. Each physical thread can only focus on doing one single task at a time.

There is also a technology called multithreading (or Hyper-Threading, depending on your CPU), which is a massive topic on its own. But to keep it simple: while a core naturally has one physical thread, multithreading splits it into two logical threads. The operating system treats these logical threads exactly the same as physical ones, effectively doubling your CPU's thread count, which is super great.

Os/Software thread

If you open your Task Manager right now, you might see that a single application is using thousands of threads. How is that possible when your CPU only has a few of these hardware threads?

Simply put, your Task Manager is only telling you half the truth. The thousands of threads you see there aren't physical threads, but OS (Operating System) threads.

When an app launches, it essentially asks the operating system, "Hey, give me some threads to work with." The OS doesn't hand over the actual physical CPU threads. Instead, it creates virtual software threads for the program. The operating system uses a component called a scheduler to manage all these thousands of software threads, juggling them so that each one gets a tiny fraction of time on the actual physical CPU to do its work.

(Tokio) Working threads

When you launch an app using the #[tokio::main] macro, Tokio immediately looks at your hardware, sees how many logical CPU cores you have, and requests exactly that many OS/software threads from your system. These are called Worker Threads.

Their entire job is to sit permanently on your CPU cores and execute your async tasks. They act as highly optimized, non-stop loops. As long as your code is truly asynchronous and uses .await keywords to yield control, these worker threads never go to sleep. They just instantly jump from one task to another entirely inside your application's memory space, executing code.

(Tokio) Blocking threads

But what happens if a task doesn't yield? Let’s say you run a heavy synchronous operation, like a massive std::fs::read_to_string() loop or a heavy math computation, inside a standard Tokio task. It will completely freeze that specific Worker Thread loop. Since Tokio only creates enough worker threads to match your cores, losing just one thread means a huge percentage of your entire application’s performance instantly vanishes. Any other async tasks assigned to that thread won't execute until the synchronous operation finishes.

To protect against this happening, Tokio maintains a separate, hidden backup pool of Blocking Threads (which can dynamically scale up to 512 threads by default). When you wrap a synchronous, heavy function in tokio::task::spawn_blocking, Tokio recognizes that this task is one of those heavy workloads. It moves it completely off the worker threads and assigns it to a Blocking Thread. This safely forces the Operating System to handle the heavy, frozen thread, keeping the core async engine completely open and responsive.

Task

An asynchronous Task (the thing you create when calling tokio::spawn) is not a thread. It is often referred to as a "green thread" or a lightweight software task. While this is true in the Tokio world, the word "task" is often used just to describe a block of code that needs to be executed. So, of course, Rayon makes them too, but you don't actually have to explicitly say, "Hey, make a new task now."

Unlike standard OS threads, which are quite expensive (somewhere around 1–2 MB of memory), Tokio tasks are really cheap, just a few hundred bytes, which means you can spawn thousands of them with ease.

CPU-bound and I/O-Bound tasks

To know whether a job belongs in a lightweight standard Task or needs a heavy Blocking thread, you have to evaluate the nature of the workload:

I/O-Bound Tasks (Input/Output-Bound Tasks): These tasks spend 99% of their life waiting. Waiting for a database query, a network packet, a sleep timer, etc. Because they spend their time waiting, they naturally hit .await points and yield control, making them perfect for standard Tokio tasks. Thousands of them can share a single worker thread cleanly.
CPU-Bound Tasks: These tasks spend 100% of their life working. They are calculating hashes, compressing ZIP files, parsing massive JSON chunks, etc. They never pause to wait for anything external, which means they never naturally yield. If you dump a CPU-bound task straight into a standard task, it will hijack the worker thread and starve your system. This is exactly the job for Rayon, or if you are using Tokio, you should put this inside the Blocking pool, assuming you don't want to freeze the whole app.

More allocated threads doesn't mean better

A beginner might think, "Hey, why don't we just create more threads for better performance?" Actually, this is a really bad idea. First of all, allocating an OS thread is expensive. Secondly, this causes context switching and cache thrashing to happen more often.

OS scheduler and time slicing

If we can have thousands of software threads running on a computer but only a tiny handful of physical hardware threads, how does our computer run so many applications? The simple answer is: the OS scheduler.

The OS scheduler is a part of your operating system that uses a technique called Time Slicing to make sure every app gets its fair share of the CPU.

It will pick up a software thread, drop it onto a physical CPU core, let it run for a tiny fraction of a millisecond, stop it, save exactly what it was doing, remove it from the core, and drop the next software thread in line onto that same core. This process is called a context switch, and it happens practically all the time, creating a small performance overhead.

A context switch takes real CPU cycles and physical time. The CPU has to pause execution, store all current hardware registers, update program counters, and talk directly to the OS kernel. If you write an application that creates dozens of unnecessary, heavy threads, your CPU will spend a massive chunk of its raw power just performing context switches instead of actually executing your application's logic.

Because this rotation happens thousands of times every single second, our human eyes can't see the tiny pauses. To us, it feels like everything is running simultaneously.

Cache thrashing

Your physical CPU cores don't read data directly from your system RAM because RAM is way too slow. Instead, the core relies on ultra-fast, local memory layers built right into the chip, called the L1, L2, and L3 caches. When your thread is running a loop, it pulls all its relevant variables right into that local cache so it can operate at maximum speed.

If the OS scheduler suddenly forces a context switch to a completely different application, that new thread will completely wipe out the local cache to fit its own data. When the OS finally switches back to your original thread, its hot cache is completely gone (resulting in a cache miss). The physical CPU core has to completely stall and freeze, waiting for your data to be slowly re-fetched all the way from your system RAM modules.

By utilizing tools like Rayon, which strictly limits its thread pool to match your exact core count, you guarantee that your application never forces the OS scheduler to perform these internal, self-destructive context switches.

Sharing data safely

While working on asynchronous code, we pretty much always need a way of sharing data between tasks. In some languages, like JavaScript, this is a really simple task because the whole ecosystem handles it for us, but in Rust, this can be quite a design challenge. Why? Simply because Rust gives us full power over the process, allowing us to handle this situation in the most effective and secure way possible.

This doesn't mean Rust is always the faster and more reliable option when it comes to this. It can be, but it requires more knowledge about the situation so that you can write more performant code. Basically, in good hands, it can be really fast and generally solid code, but at the same time, it can shoot you in the foot.

How to share values in Rust

When we need to share or pass values into a task, it requires moving the value or passing a reference to it. In some cases, cloning the value is okay, but passing a reference to a new task isn't possible because of lifetimes. So, how can we share large amounts of data that cannot be cloned?

Arc

Arc, which stands for Atomically Reference Counted, is our way of sharing references between tasks. Instead of cloning the actual data, we just clone a new reference (a smart pointer) that can be safely moved into a new task. This way, we can share the same data across multiple tasks. However, as you can guess, the data wrapped inside an Arc is immutable by default and cannot be modified after creation.

Even though Arc doesn't allow mutation on its own, it is still incredibly important. Other structures that do allow mutation rely on Arc to be safely moved into new tasks. That's why you will mostly see Arc as a wrapper for other structures, or used for data structures that are quite large, making traditional cloning too expensive or impossible.

Use example:

use std::sync::Arc;

#[tokio::main]
async fn main() {
    // Creates Arc<Config>
    // Adding the mut keyword here is unnecessary because 
    // Arc doesn't allow mutation anyway
    let config = Arc::new(Config { something: String::from("Initial") });
    
    // Cloning the Arc so that we can move the clone into a new task
    // Using Arc::clone(&config) is preferred over config.clone() 
    // for readability
    let config_clone = Arc::clone(&config); 
    let task = tokio::task::spawn(async move {
        // We can access the data here safely
        println!("Config inside task: {}", config_clone.something);
    });

    // WARNING: This is not possible because Arc only 
    // provides read-only access to its inner data
    // config.something = String::from("something");

    let _ = task.await;
}

// Just imagine a really large Config struct
#[derive(Default)]
struct Config {
    something: String,
}

Mutex

While Arc is great for sharing read-only data, we will likely need to modify that data at some point. That's where Mutexes come in. Unlike an Arc alone, where we can only read the data, a Mutex requires us to lock it before we can even access its contents. Locking a Mutex returns a MutexGuard. With this guard, we gain the ability to both read from and write to the data wrapped inside.

That sounds like a no-brainer upgrade to Arc, right? Well, actually, no. When we hold a MutexGuard, we can modify the data, but it is also the only place where the data can be accessed. This means only one task can use this value at a time, while all other tasks must wait until that specific guard is dropped (dies).

This locking behavior can introduce a new issue: if we lock the data in one specific task and the guard lives too long, all other tasks are blocked from accessing it. They have to wait until the guard dies, which, in some buggy scenarios, might be never. This can drastically slow down your app or even freeze the entire application. We call this a deadlock, and it's probably the most common issue beginners run into.

Usage example:

use tokio::sync::Mutex;
use std::sync::Arc;

#[tokio::main]
async fn main() {
    // Creates Arc<Mutex<Config>>
    let config = Arc::new(Mutex::new(Config { 
        something: String::from("Initial") 
    }));
    
    // Cloning the Arc so that we can move it into a new task
    let config_clone = Arc::clone(&config);
    let task = tokio::task::spawn(async move {
        // We lock the mutex inside the spawned task
        let mut lock = config_clone.lock().await;
        lock.something = String::from("Changed by task");
        // The guard automatically drops here when 
        // 'lock' goes out of scope
    });

    // To avoid deadlocks, we often use a dedicated block 
    // scoped like this.
    // This ensures the guard dies immediately after we are done,
    // allowing other tasks to access the data.
    {
        let mut lock = config.lock().await;
        lock.something = String::from("Changed by main");
    } // The main thread's lock drops right here!

    let _ = task.await;
}

// Just imagine a really large Config struct
#[derive(Default)]
struct Config {
    something: String,
}

RwLock

As I explained in the Mutex section, read and write operations have the same priority. While in some cases this is great, we might often find ourselves needing to read a value frequently, while writing to it is rarely necessary. That's why we have RwLock (Reader-Writer Lock), which is designed exactly for this situation. It allows either N readers and no writers, or 1 writer and no readers. This means you still cannot read and write at the exact same time, and you must wait until the opposing guard is dropped to continue. Because of this, there is still a possibility of a deadlock, just like with Mutexes.

Use example:

use tokio::sync::RwLock;
use std::sync::Arc;

#[tokio::main]
async fn main() {
    // Creates Arc<RwLock<Config>>
    let config = Arc::new(RwLock::new(Config { 
        something: String::from("Initial") 
    }));
    
    // Cloning the Arc so that we can move it into a new task
    let config_clone = Arc::clone(&config);
    let task = tokio::task::spawn(async move {
        // Multiple tasks can hold a read lock at the same time
        let read_lock = config_clone.read().await;
        println!("Reading inside task: {}", read_lock.something);
    });

    // Opening a scope to demonstrate safe locking without deadlocks
    {
        // Creating a read guard in the main thread
        let read_lock = config.read().await;
        println!("Reading inside main: {}", read_lock.something);
    } // The read lock drops right here!

    {
        // Creating a write guard to modify the data
        let mut write_lock = config.write().await;
        write_lock.something = String::from("Updated value");
    } // The write lock drops right here!

    let _ = task.await;
}

// Just imagine a really large Config struct
struct Config {
    something: String,
}

Semaphore

While performing asynchronous operations, we may need to limit access to certain tasks, resources, or APIs. For this job, we use a Semaphore, which can limit the number of concurrent permits available for a specific resource.

Example:

use tokio::sync::Semaphore;
use std::sync::Arc;

#[tokio::main]
async fn main() {
    // We usually need to share the semaphore between multiple tasks,
    // which is why we wrap it in an Arc.
    let semaphore = Arc::new(Semaphore::new(3));

    let semaphore_clone = semaphore.clone();
    let task = tokio::task::spawn(async move {
        // This will wait until it acquires a permit.
        // If no permits are available, it will asynchronously wait
        // until one is released.
        let _a_permit = semaphore_clone.acquire().await.unwrap();
    }); // _a_permit goes out of scope here and automatically 
        // returns back to the semaphore pool.

    // This will ask: "Hey, give me a permit NOW!" 
    // If there are none left inside the semaphore, it won't wait 
    // and returns an Err immediately.
    let _permit_attempt = semaphore.try_acquire();

    let _ = task.await;
}

Channels

As you might guess from the name, a channel is a communication structure made of two parts: a Sender and a Receiver. The Sender sends values, and the Receiver receives them. I know that’s the most basic explanation I could possibly give, but honestly, it really is that simple.

Tokio provides a few different types of channels depending on how many senders and receivers you need to coordinate. Let's look at the two most common.

Broadcast Channel (Multi-producer, Multi-consumer)

This channel allows you to have multiple senders and multiple receivers. When a message is sent, every active receiver gets a copy of it.

Example:

use tokio::sync::broadcast;

#[tokio::main]
async fn main() {
    // Creating the channel requires specifying the maximum number of
    // messages it can hold in its queue at once (capacity).
    // In this example, it's 16.
    let (tx, mut rx1) = broadcast::channel(16);
    let mut rx2 = tx.subscribe();

    let handle1 = tokio::spawn(async move {
        assert_eq!(rx1.recv().await.unwrap(), 10);
        assert_eq!(rx1.recv().await.unwrap(), 20);
    });

    let handle2 = tokio::spawn(async move {
        assert_eq!(rx2.recv().await.unwrap(), 10);
        assert_eq!(rx2.recv().await.unwrap(), 20);
    }); 

    tx.send(10).unwrap();
    tx.send(20).unwrap();
    
    // A handy macro to await all tasks concurrently
    tokio::join!(handle1, handle2);
}

MPSC Channel (Multi-Producer, Single-Consumer)

Similar to a broadcast channel, an MPSC channel gives you senders and receivers. However, MPSC strictly allows only one receiver. On the flip side, you can call tx.clone() as many times as you want to have multiple background tasks all feeding data into that single receiver.

Example:

use tokio::sync::mpsc;

#[tokio::main]
async fn main() {
    // When creating the channel, you must specify its capacity.
    // In this example, it can hold up to 100 messages in the queue.
    let (tx, mut rx) = mpsc::channel(100);

    let task = tokio::spawn(async move {
        for i in 0..10 {
            if let Err(_) = tx.send(i).await {
                println!("Receiver dropped!");
                return;
            }
        }
    }); // tx is dropped here when the task ends.

    // The loop will automatically end when all Senders (tx) are dropped,
    // meaning rx.recv() will return None.
    while let Some(i) = rx.recv().await {
         println!("got = {}", i);
    }

    let _ = task.await;
}

Sources

To be honest, there are even more sources that I used but simply forgot or can't find anymore. I'm sorry about that, but this was a really long learning journey, and my goal was to actually learn the concepts, not just collect quotes for a blog post. Additionally, a lot of this information was gathered simply by building things through trial and error.