Async/Await: An Honest Teardown of State Machines, Poll, and Executors

Part 9’s thread model rests on a silent premise: threads are cheap. True for hundreds of connections; bankrupt at a hundred thousand — each thread carries a megabyte-scale stack, each switch costs a kernel reschedule, and a million concurrent connections means terabytes of stack for sockets that spend 99% of their time waiting on the network. That’s the classic C10k problem, and the reason async exists: make waiting nearly free. This part doesn’t pile up APIs — it disassembles async until you could rebuild it by hand: what an async fn actually compiles into, why a Future sits motionless until driven, what really happens inside an executor’s loop, and finally the selection rule that’s worth the whole ticket.

What an async fn compiles into

Start with the pit every beginner steps into — which happens to be the doorway to understanding:

async fn fetch(name: &str, ms: u64) -> String {
    tokio::time::sleep(std::time::Duration::from_millis(ms)).await;
    format!("{name} done after {ms}ms")
}

let f = fetch("lazy", 10);
println!("future created — nothing has happened yet");
println!("{}", f.await);   // the function body starts executing only here

On the line that calls fetch, not one line of the body executes. Calling an async fn doesn’t “run the function” — it constructs a state machine and hands it back: that’s the Future. The compiler rewrites your body into an enum-plus-match machine: every .await becomes a “suspended” state, local variables become fields of that state, and the return value is the terminal state.

This design explains everything. Laziness: building the machine isn’t starting it, so “created a Future but never awaited it” costs nothing — and it’s the classic bug clippy warns about (let _ = fetch(...) does exactly nothing). Cheapness: a suspended task occupies only its state fields — tens to hundreds of bytes, not a megabyte-scale stack — so a hundred thousand suspended tasks cost tens of megabytes. And no magic: the state machine is a direct application of Part 4’s enums and Part 7’s trait contracts; you are fully capable of writing one yourself. So let’s write one.

A hand-written Future, then a hand-written executor

The Future trait’s core is a single method:

use std::future::Future;
use std::pin::Pin;
use std::task::{Context, Poll};

struct Countdown { remaining: u32 }

impl Future for Countdown {
    type Output = String;
    fn poll(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Self::Output> {
        if self.remaining == 0 {
            Poll::Ready(String::from("liftoff"))
        } else {
            self.remaining -= 1;
            println!("polled: {} to go", self.remaining);
            cx.waker().wake_by_ref();   // "call me when there's news"
            Poll::Pending
        }
    }
}

poll is the only verb in the whole system, and its answer is binary: Ready(v) — done, here’s the goods; Pending — not ready, but you may not sit on the CPU while waiting: you must arrange “wake me when there’s news” through the waker inside cx, then immediately yield control. The entire contract between executor and Future is one sentence: Pending means “don’t call me; I’ll call you.”

And the executor? It’s just a loop. Here’s a toy version — thirty lines, and it genuinely runs:

use std::sync::Arc;
use std::task::{Wake, Waker, Context, Poll};
use std::pin::pin;

struct Spin;
impl Wake for Spin { fn wake(self: Arc<Self>) {} }

fn block_on<F: Future>(fut: F) -> F::Output {
    let waker = Waker::from(Arc::new(Spin));
    let mut cx = Context::from_waker(&waker);
    let mut fut = pin!(fut);
    loop {
        match fut.as_mut().poll(&mut cx) {
            Poll::Ready(v) => return v,
            Poll::Pending => std::hint::spin_loop(),  // real runtimes park the thread here
        }
    }
}

println!("{}", block_on(Countdown { remaining: 3 }));
// polled: 2 to go / polled: 1 to go / polled: 0 to go / liftoff

The toy differs from tokio in exactly one place — but that place is the entire engineering: a real executor parks the thread on Pending, sleeping on epoll/kqueue/IOCP until some waker is fired by an I/O completion and the task re-enters the queue. The cost of waiting drops from CPU time to zero. The name block_on is honest too: it dams the “async world” onto the current thread until a result comes out — at the foundation of every async edifice sits exactly such a loop.

Runtimes: enter tokio, and concurrency is not parallelism

The standard library deliberately ships only the parts — Future, Waker — and no executor. That’s Rust’s old philosophy (the language provides mechanism, the ecosystem provides implementation), and the ecosystem’s de facto standard is tokio. Getting started takes two symbols:

#[tokio::main]
async fn main() {
    let start = std::time::Instant::now();
    let (a, b) = tokio::join!(fetch("a", 300), fetch("b", 500));
    println!("{a} | {b} | elapsed: {:?}", start.elapsed());
    // a done after 300ms | b done after 500ms | elapsed: 501ms
}

#[tokio::main] expands to “build a runtime and hand it main’s Future to block_on.” And the join! result deserves ten seconds of staring: the two Futures total 501ms, not 800ms — they advance alternately on the same thread: while a is suspended waiting out its 300ms, b runs. Concurrency is not parallelism: the first is structure, the second is execution. To actually occupy multiple cores, hand tasks to the worker pool with tokio::spawn — but note that spawn demands 'static; Part 3’s two hats and Part 9’s thread rules keep applying in async-land, unchanged to the letter.

The selection rule: who does the waiting picks the tool

One rule, two branches:

• CPU-bound → threads. When cores are the bottleneck, async helps not at all — however cheap the state machine, it can’t make computation faster. Image processing, compression, scientific computing: Part 9’s scoped threads and channels, plus the ecosystem’s rayon (a three-line change for data parallelism), are the answer.
• I/O-bound → async. When the bottleneck is waiting — servers, proxies, crawlers, anything babysitting thousands of connections at once — async shrinks per-connection cost from a megabyte thread stack to bytes of state. This is tokio’s territory.

And async-land’s most famous own-goal, worth remembering at the highest alert level: calling a blocking API inside async code — std::thread::sleep, synchronous file I/O, certain database drivers — stalls the entire worker thread, starving the hundreds or thousands of tasks riding on it. In the state-machine model, “suspending” can only happen at .await; a blocking call doesn’t suspend, it falls asleep on the spot. The fix is routing blocking work through its proper door: tokio::task::spawn_blocking — the runtime’s dedicated pool for blocking work, the door between the two worlds.

Practice, then Part 11

1. Run this part’s laziness experiment: create the Future, print, then await. Then write let _ = fetch("x", 1);, run the program, and confirm nothing happens — nail “an un-awaited Future is air” into your intuition.
2. Upgrade the toy block_on to two tasks: poll two Countdowns in turn until both are Ready. You’ve just written a minimal concurrent executor by hand — “concurrency needs no threads” now has a concrete shape for you.
3. Use tokio::join! to “download” three simulated tasks at once (sleep for different milliseconds) and confirm total time ≈ the longest, not the sum; then switch one to tokio::spawn and read the compiler’s 'static requirement, making sure you understand why.

Part 11 opens the door deliberately saved for last: unsafe. The five things only unsafe can do, how to draw the boundary of a “safe abstraction,” the counterintuitive fact that unsafe does not turn off the borrow checker, and why seasoned Rustaceans write less and less of it — not out of fear, but because the ecosystem has already sealed those boundaries for you.