I have the opposite experience, working in embedded (C, not Rust...). Building a synchronous API on top of an async one is hell, and making a blocking API asynchronous is easy.
If you want blocking code to run asynchronously, just run it on another task. I can write an api that queues up the action for the other thread to take, and some functions to check current state. Its easy.
To build a blocking API on top of an async one, I now need a lot of cross thread synchronization. For example, nimBLE provides an async bluetooth interface, but I needed a sync one. I ended up having my API calls block waiting for a series of FreeRTOS task notifications from the code executing asynchronously in nimBLE's bluetooth task. This was a mess of thousands of lines of BLE handling code that involved messaging between the threads. Each error condition needed to be manually verified that it sends an error notification. If a later step does not execute, either through library bug or us missing an error condition, then we are deadlocked. If the main thread continues because we expect no more async work but one of the async functions are called, we will be accessing invalid memory, causing who knows what to happen, and maybe corrupting the other task's stack. If any notification sending point is missed in the code, we deadlock.
"If you want blocking code to run asynchronously, just run it on another task."
This highlights one of the main disconnects between "async advocates" and "sync advocates", which is, when we say something is blocking, what, exactly, is it blocking?
If you think in async terms, it is blocking your entire event loop for some executor, which stands a reasonable chance of being your only executor, which is horrible, so yes, the blocking code is the imposition. Creating new execution contexts is hard and expensive, so you need to preserve the ones you have.
If you think in sync terms, where you have some easy and cheap ability to spawn some sort of "execution context", be it a Haskell spark, an Erlang or Go cheap thread, or even are just in a context where a full OS thread doesn't particularly bother you (a situation more people are in than realize it), then the fact that some code is blocking is not necessarily a big deal. The thing it is blocking is cheap, I can readily get more, and so I'm not so worried about it.
This creates a barrier in communication where the two groups don't quite mean the same thing by "sync" and "async".
I'm unapologetically on Team Sync because I have been programming in contexts where a new Erlang process or goroutine is very cheap for many years now, and I strongly prefer its stronger integration with structured programming. I get all the guarantees about what has been executed since when that structured programming brings, and I can read them right out of the source code.
“This highlights one of the main disconnects between "async advocates" and "sync advocates", which is, when we say something is blocking, what, exactly, is it blocking?”
When I have work on a cooperatively scheduled executor for optimal timing characteristics. Sending work/creating a task on a preemptive executor is _expensive_. Furthermore, if that blocking work includes some device drivers with interactions with hardware peripherals, I can’t reasonable place that work on a new executor without invalidating hardware timing requirements.
Threads and executors can be infeasible or impossible to spawn. I have 1 preemptive priority and the rest are cooperative on bare metal. I can eat the suboptimal scheduling overhead with a blocking API/RTOS or I need the async version of things.
You can have more than one executor in some implementations (you know you do if you can keep more than one CPU busy), but in all the implementations you're certainly causing problems if you sync-block an executor for long periods of time, that's not what the system is for. By contrast, you want to open a thread/goroutine/Erlang process/Haskell spark/execution context in a threaded language and have it just bang away on numeric computation for a few days? Usually that's just fine, though if you've got a green-threading implementation, double-check first. Most green thread implementations seem to go through a period where that is not fine, but then eventually it is; Go for instance passed through that quite a few versions ago.
To add: This issue isn't that troublesome in languages like JavaScript where the runtime is event driven out-of-the-box (synchrounous APIs are considered the exception and suffixed with 'Sync' by convention), but it will become difficult to reason about in languages like rust and python where an event driven runtime is optionally choosen by the developer. Because now it's not enough to conceptually grep all awaiting calls to reason about any blocking code, but you'd also need to know if any other synchronous calls are implicitly awaiting in an underlying layer.
I like your take, and I propose there is another kind of division: function "coloring". An async "effect" type (e.g. futures, or even just 'async function') signals to programmers that there is some concurrency stuff going on around the place(s) you use it and that we need to talk about how we're going to handle it. i.e. rather than a performance issue, it's a correctness/safety/semantics issue.
> it is blocking your entire event loop for some executor
In the browser, only. Which is where all this nonsense began.
Browser didn't have threads (Win16) so they invented a crappy
cooperative scheme with callbacks, which became gussied up as "async" and now has a cult following. It was all a hack designed to make a terrible runtime environment usable. And now we have a priesthood telling us this is how to program, and if you disagree then you must be stupid.
You're correct that we should be using CSP (Erlang, Go, Occam, Actors).
It's not completely to blame on the browsers. We had the concept of green threads [1] well before that, albeit for equally dubious reasons: OS threads were believed to be too slow, and so green threads were composed onto OS threads. It seems a better choice here would be to try and improve OS threads, but the speed at which various program languages and libraries develop well outstrips anything we could achieve with the kernel, and so maybe this was inevitable :)
On the topic of improving OS threads, there are scheduler activations and similar mechanisms in some microkernels. Basically sending events for all hardware things and context switches, making user level threads as flexible as kernel threads. Unfortunately they haven't become mainstream.
Note that Erlang processes are essentially green threads within OS thread, but the Erlang process manager is preemptive (unlike most green thread managers). There is room for improvement in both directions.
Making an asynchronous task into a synchronous task is easy in exactly one scenario: when there is no pre-existing event loop you need to integrate with, so the actual thing the synchronous task needs to do is create the event loop, spin it until it's empty, and then continue on its merry way. Fall off this happy path, and everything is, as you say, utterly painful.
In the opposite direction, it's... always easy (if not entirely trivial). Spin a new thread for the synchronous task, and when it's done, post the task into the event loop. As long as the event loop is capable of handling off-thread task posting, it's easy. The complaint in the article that oh no, the task you offload has to be Send is... confusing to me, since in practice, you need that to be true of asynchronous tasks anyways on most of the common async runtimes.
This response just highlights how large the difference between different domains using the same construct is, and why that makes it impossible for people to agree on basically anything.
Rust async is, to a significant degree, designed to also work in environments that are strictly single-threaded. So "Spin a new thread" is just an operation you do not have. Everything that's weird and unwieldy about the design follows from that, and from how it also requires you to be able to allocate almost nothing, and to let the programmer manage the allocations.
I have pointed it out before that the overall design is probably made worse for general use by how it accommodates these cases.
I have the same experience, I like splitting my embedded C microcontroller peripheral drivers into 3 layers:
- header files with registers addresses and bitmasks
- asynchronous layer that starts transactions or checks transaction state or register interrupt handler called when transaction changes states
- top, RTOS primitives powered, blocking layer which encapsulates synchronization problems and for example for UART offers super handy API like this:
status uart_init(int id, int baudrate)
status uart_write(int id, uint8_t* data, int data_len, int timeout_ms)
status uart_read(int id, uint8_t* buf, int buf_len, int timeout_ms, int timeout_char_ms)
Top, blocking API usually covers 95% use cases where business logic code just want to send and receive something and not reinvent the synchronization hell
> If you want blocking code to run asynchronously, just run it on another task
What kind of embedded work are you doing exactly? Linux "soft" embedded or MMUless embedded? I don't have infinite NVIC priority levels to work with here... I can't just spin up another preemptively scheduled (blocking) task without eating a spare interrupt and priority level.
Otoh, I can have as many cooperatively scheduled (async) tasks as I want.
Also, at least in Rust, it's trivial to convert nonblocking to blocking. You can use a library like pollster or embassy-futures.
MMUless embedded with FreeRTOS. For example, at one point, we did not want connecting over TCP to block our command handler, so created a task that waited for a notification, and connected on that task, and went back to waiting for another notification. Though we ended up combining some tasks' responsibilities to reduce the amount of total stack space we needed.
You can spawn async task open a channel and wait for async task to push to it from blocking context, channels can have efficient ways to wait for the message. This is fairly easy to do in rust
The trick with synchronization/parallelism systems is to only communicate over a known yield point this is normally done via queues. It is the only way you get deterministic behavior from your sub-systems or multi-threaded environments.
> There are a substantial number of async crates out there that run on top of tokio. They use tokio’s primitives, tokio’s executor, and tokio’s I/O semantics. Because of this, they rely on tokio’s runtime to be running in the background. If you try the above strategy for a crate that relies on tokio, it will fail at runtime with a panic.
This is the worst design choice in the broader Rust ecosystem right now. It’s insane that people are exporting public functions from packages that rely on their caller running the right thing in the background. Imagine a library that didn’t work if you didn’t create a global mutable variable for it to modify, and that variable didn’t even need to be passed into the library explicitly.
Tokio should have a marker type that’s Copy and returned from functions that start the runtime, and require a parameter of that type when calling functions which expect a runtime to already exist.
Yes this is backwards incompatible. It should be done anyway.
If the value is Copy that value will continue to be usable after the runtime has been stopped and dropped, defeating the purpose.
If the value borrows from the runtime to prevent that, futures that hold on to that value stop being `'static`, which brings its own problems, chief among them being breaking `tokio::spawn()` and causing a self-reference (runtime contains futures which contain borrows of itself).
If the value holds on to (a refcount of) the runtime internally to prevent the runtime from being dropped until the value has been dropped, that will prevent programs from exiting as long as even one future is still running on the runtime, which is undesirable.
You don’t actually need it to be a reference; the runtime could generate a random int64 and then pass that around as a “runtime id” which could be checked when used.
You’re correct that my proposal doesn’t solve every problem with Tokio. I hope someone more knowledgeable than me can write a proposal that solves all the problems.
Checked where? In global mutable state you mentioned that has to be created beforehand in the same way you mention? I don't really see how this changes much other than making everything more verbose from the extra argument.
It would be nice for someone to come to with a new solution to solve everything, but I don't think it's as simple as just "use an id and then tidy up some remaining details". No one has come up with something more compelling yet because those details are where the fundamentally are part of the problem is. When the hard problem is figuring out semantics that will allow exposing a better API, proposing an API and then hoping someone finds a way to make it work under the hood is just backwards.
The verbosity is the point! I want functions that interact with a specific expectation of global state to clearly mark that in their type, and the least complicated way to do that is my proposal. I hope someone can invent something more compelling, but my proposal is still an improvement of the status quo.
I think the toplevel commenter was looking for something that could be checked at compile time. The "random int64" idea is identical to "some type that implements Copy", and would require runtime verification, which is not really an improvement on the current state of affairs.
It would prevent callers from not knowing the function depends on a specific runtime. That’s a huge improvement in my opinion.
I expect something along the lines of the GhostCell paper could be used to implement this, to be honest, but I think it would be more verbose than necessary.
(Really, we need an effects system, and then Tokio could just be an effect that contains, but is different from, async)
Does anyone realistically shut down the Tokio runtime once it's been started? It wouldn't be a problem if the runtime is immortal once created; the runtime reference becomes more of a token demonstrating that, yes, this is running under Tokio.
Though I don't think this is such a huge problem. Yes, it's a runtime crash that could potentially be eliminated at compile-time, but realistically that will only happen once during development, and it will be an easy fix.
If I'm writing an async program (that is, with an async main()), then no, I'll never be shutting my runtime down.
But if I already have a non-async program, but need to use an async library, then yes, I'll probably spin up the runtime before using the library, and shut it down after I'm done using it. Unless of course I expect to be using that library for the full run-time of my program, in which case it'll exist for the duration.
> I'll probably spin up the runtime before using the library, and shut it down after I'm done using it.
Hopefully the library did not leave any background tasks running that you can't ask it to stop :) Raymond Chen has a lot of enlightening horror stories about multi-threaded COM (and multi-threaded programming in general) on his blog that are applicable not only to Windows, but in general context as well.
Well on the compatibility side of things, tokio has a TON of issues as well. I remember when tokio 1.0 came about, I was forced into a full psychopath mode cause Actix and rusoto were using different and incompatible versions of tokio. And I had to dig 200 layers deep to figure out why something simple such as copying a file to s3 wasn't working....
What is the difference between code that blocks waiting for I/O and code that performs a lengthy computation? To the runtime or scheduler, these are very different. But to the caller, maybe it does not matter why the code takes a long time to return, only that it does.
Async only solves one of these two cases.
I’d like to draw an analogy here to ⊥ “bottom” in Haskell. It’s used to represent a computation that does not return a value. Why doesn’t it return a value? Maybe because it throws an exception (and bubbles up the stack), or maybe because it’s in an infinite loop, or maybe it’s just in a very long computation that doesn’t terminate by the time the user gets frustrated and interrupts the program. From a certain perspective, sometimes you don’t care why ⊥ doesn’t return, you just care that it doesn’t return.
Same is often true for blocking calls. You often don’t care whether a call is slow because of I/O or whether it is slow because of a long-running computation. Often, you just care whether it is slow or how slow it is.
(And obviously, sometimes you do care about the difference. I just think that the “blocking code is a leaky abstraction” is irreparably faulty, as an argument.)
> To the caller of that specific function, nothing.
And that's what makes async code, not the blocking code, a leaky abstraction. Because abstraction, after all, is about distracting oneself from the irrelevant details.
That isn't my understanding of a leaky abstraction. An abstraction leaks when it's supposed to always behave in some certain, predictable way, but in practice, sometimes it doesn't. When does an async function not behave the way it's supposed to?
My understanding of a leaky abstraction is that the abstraction itself leaks out details of its design or underlying implementation and requires you to understand them. What you seem to describe is a bug, edge case, or maybe undefined behaviour?
For example, an ORM is a leaky abstraction over an RDBMS and SQL because you inevitably have to know details about your RDBMS and specific SQL dialect to work around shortcomings in the ORM, and also to understand how a query might perform (e.g will it be a join or an N+1?).
I don't really think that the async or blockingness is the leak, but that the time taken to process is not defined in either case, and you can leak failure criteria either way by not holding to that same time.
People can build to your async process finishing in 10ms, but if suddenly it takes 1s, it fails
It's better to think of "async" as indicating that a code will do something that blocks, and we're allowing our process to manage its blocking (via Futures) instead of the operating system (via a context switch mid-thread.)
I would argue a few things:
First: You need to be aware, in your program, of when you need to get data outside of your process. This is, fundamentally, a blocking operation. If your minor refactor / bugfix means that you need to add "async" a long way up the stack, does this mean that you goofed on assuming that some kind of routine could work only with data in RAM?
Instead: A non-async function should be something that you are confident will only work with the data that you have in RAM, or only perform CPU-bound operations. Any time you're writing a function that could get data from out of process, make it async.
I want to restate what you’ve said from a different perspective:
If you write a lot of pure functions, in a Functional Core, Imperative Shell manner, then only the imperative part has to deal with any of the async parts. Yes it’s the topmost part of the code, but there is nothing to “infect” with it which isn’t already destined to be.
It’s when you try to write imperative code like there are no consequences for doing so that the consequences show up en masse and can be confused as symptoms of entirely different problems.
In many application use cases, that is an implementation detail that should not be a concern of higher levels. The ramifications may not even be known by the people at higher levels.
Take something very common like cryptographic hashing, if you use something like node.js you really don't want to block the main thread calculating an advanced bcrypt hash. It also meets all of your requirements that data not come from outside ram and is very CPU bound.
Obviously, if you are directly calling this hashing algorithm you should know, however, the introduction of a need to hash is completely unpredictable.
> Take something very common like cryptographic hashing, if you use something like node.js you really don't want to block the main thread calculating an advanced bcrypt hash. It also meets all of your requirements that data not come from outside ram and is very CPU bound.
I guess I implied quick operations when I said "CPU bound." (IE, calculating a square root, string manipulation, (de)serialization...)
I haven't done hashing in Node.js. I assume that the bcrypt API is async and calls into a native library?
You could make the proposition that sequential code is inherently asynchronous in modern operating systems, because the kernel inherently abstracts the handling of blocking/unblocking your process.
Yes and no. From a practical point of view considering sync code as async is useless as sync and async code have very different usability characteristics and it is useful to have different names for the two domains.
On the other hand, form a more abstract, theoretical level, it is important to know that there is almost always an event loop at some point deep in the stack and sync code is just sugar over async and you can always transform from one to the other.
Yes and no, the concept that is being communicated is that your program inherently is asynchronous how-ever the context switching/block/unblocking is abstracted away from you so you can focus on what is important to you - which is solving your problem. Once you communicate that concept then it makes it a lot easier for people to create new mental models that better reflects reality. Then they can structure their programs that can take advantage of it.
I would say that's a leaky abstraction. (The OS hiding the details of async by blocking your threads and context switching, although this is how many programs operate.)
The problem comes when you have a critical section where you need to hold a mutex (lock): If you're working with "proper" async code, you can safely call any non-async code from within the lock. (C# enforces this, because a lock statement can not contain the await keyword.)
When any method you call can block on IO, synchronous programming (IE, using the OS to become async), can leak and make you hold your mutex longer than you should.
---
That being said, many of us build our careers programming threaded code that relies on the OS to block. So at this point we're splitting hairs.
When most people talk about async code it is with single threaded (hence the need not to block the event executor) applications (web-browsers, nodejs ect.). How-ever on the other end of the delirium when people talk about mutex's (locks) they're talking about multiple threadeds working in parallel which you inherently need to consider if the code you're calling is going to either direct or indirect acquire the same lock or like you pointed out calls a api that blocks. In the single threaded environment the only cognitive load you need to worry about is one or two yield points, how-ever in a parallel system your main concern is mutex's, race-conditions, or dead-locks.
"asynchronous code does not require the rest of your code to be synchronous" fails the smell test.
Many APIs are shipping async-only. You can't stick calls to those APIs in the middle of your sync code in many cases (unless you have an event loop implementation that is re-entrant). So... you gotta convert your sync code to async code.
I am curious as to what a principled definition of a "blocking" function would be. I suppose it's a call to something that ends up calling `select` or the like. Simply saying "this code needs to wait for something" is not specific enough IMO (sometimes we're just waiting for stuff to move from RAM to registers, is that blocking?), but I'm sure someone has a clear difference.
If you care about this problem enough, then you probably want to start advocating for effect system-like annotations for Rust. Given how Rust has great tooling already in the language for ownership, it feels like a bit of tweaking could get you effects (if you could survive the eternal RFC process)
On the other hand, it might flag a code smell to you in that you’re injecting I/O into a code path that had no I/O before which forces you to either make annotations making that explicit for the future or to realize that that I/O call could be problematic. There’s something nice about knowing whether or not functions you’re invoking have I/O as a side effect.
> I am curious as to what a principled definition of a "blocking" function would be.
It's one where the OS puts your (kernel) thread/task to sleep and then (probably) context switches to another thread/task (possibly of another process), before eventually resuming execution of yours after some condition has been satisfied (could be I/O of some time, could be a deliberate wait, could be several other things).
OS threads can be put to sleep for many reasons, or they can be preempted for no explicit reason at all.
On such reason could be accessing memory that is currently not paged in. This could be memory in the rext section, memory mapped from the executable you are running.
I doubt that you would want to include such context switches in your "blocking" definition, as it makes every function blocking, rendering the taxonomy useless.
That seems a necessary but not sufficient condition, since a pre-emptively multitasking OS may do this after almost any instruction.
Not only that, but any OS with virtual memory will almost certainly context-switch on a hard page fault (and perhaps even on a soft page fault, I don't know). So it would seem that teading memory is sufficient to be "blocking", by your criterion.
1) I deliberately left it open to including page faults by design. If you do not understand that reading memory might lead to your process blocking, you have much to learn as a programmer. However, I would concede that that this is not really part of the normal meaning of "blocking", despite the result being very similar.
2) I did not intend to include preemption. I should reword it to make it clear that the "condition" must be something of the process' own making (e.g. reading from a file or a socket) rather than simply an arbitrary decision by the kernel.
This feels mostly right to me. I think that you get into interesting things in the margins (is a memory read blocking? No, except when it is because it's reading from a memory-mapped file!) that make this definition not 100% production ready.
But ultimately if everyone in the stack agrees enough on a definition of blocking, then you can apply annotations and have those propagate.
But this isn’t hemming to the definition brought up by GP. “I will now, of my own accord, sleep so the OS scheduler takes over” is fairly precise. And it’s different from both just doing an operation that takes time… and different from the program doing an operation that, incidentally, the OS sees and then forces a sleep due to some OS-internal abstraction
But you think about this too much and you can easily get to “well now none of my code is blocking”, because of how stuff is implemented. Or, more precisely, “exactly one function blocks, and that function is select()” or something.
For example, the fact that code actually takes time to execute ... this is an abstraction that should almost certainly leak.
The fact that some data you want is not currently available ... whether you want to obscure that a wait is required or not is up for debate and may be context dependent.
"I want to always behave as though I never have to block" is a perfectly fine to thing to say.
"Nobody should ever have to care about needing to block" is not.
> this is an abstraction that should almost certainly leak.
that's not a leak, that is the abstraction for procedural/imperative code, or at least an important part of it. essentially: "each step occurs in order and the next step doesn't happen until the former step is finished"
That's not a particularly interesting description for multithreaded code. The procedural/imperative element of it may be true of any given thread's execution, but with the right user-space or kernel scheduler, you may or may not care what the precise sequencing of things is across threads. This is true whether you're using lightweight user-space threads ("fibers") or kernel threads.
of course, but in that case it's not the base imperative abstraction that's leaky - it's the multi-thread/multi-core abstraction built on top of it - it's almost as easy as multiple threads happening in parallel. until it isn't.
the multi-thread abstraction attempts to be: "hey, you know how that imperative model works? now we can have more than one of these models (we'll call them threads!) running in parallel. neat, huh?"
it's things like memory barriers and race conditions, etc. that are the "leaks". but it's the threading/multi-core abstractions that are leaky not (back to the article) the "blocking" code.
i can write single-thread "blocking" code all day long that does all kinds of interesting non-trivial things and never have to worry about leaks wrt. blocking, in-order execution, etc. even in the presence of out-of-order execution processors and optimizing compilers, VMs, multi-user OS'es, etc. effects are observably identical to the naive model.
the author didn't do a good job of clearly defining anything, but i bristle at the idea that the basic "blocking" code abstraction whats leaky - it's the async, threads, event-driven models, etc. that are (necessarily) a bit leaky when they break the basic imperative blocking model or require accommodation of it.
good points. i'd add, though, that priority inversion is specifically a leakage of the blocking nature of some parts of the imperative code into the thread model. similarly, the implications of locks for real time code (something i work on) is another example of blocking leaking into a thread model.
The author seems to reinforce the original point, that the async paradigm ends up working best in an all-or-nothing deal. Whether the difficulties in interfacing the paradigms should be attributed to the blocking part or the async part does not really matter for the practical result: if calling blocking code from async code is awkward and your project's main design is async, you're going to end up wanting to rewrite blocking code as async code.
the author is misusing the "leaky abstraction" idea. in the section "what's in a leak" a rather muddy argument is made that a leak is that which forces you to bend your program to accommodate it. so `leak => accommodation`.
then it immediately conflates the need for that accommodation (difficulty of calling one type of code with another) with "leakiness". so essentially "calling blocking => more accommodation" (e.g. from event loops that shouldn't block).
- that's logically incorrect, even in the author's own argument (A => B, B, therefore A is the affirming the consequent fallacy).
- that's not what a "leaky abstraction" is. not everything that doesn't fit perfectly or is hard is due a "leaky abstraction". rather it's when the simplistic model (abstraction) doesn't always hold in visible ways.
a "leaky abstraction" to "blocking" code might be if code didn't actually block or execute in-order in all situations. until you get to things like spinlocks and memory barriers this doesn't happen. but that's more a leak caused by naively extending the abstraction to multi-core/multi-thread use, not the abstraction of blocking code itself.
i love the passion for `async` from this fellow, but i think he's reacting to "async is a leaky abstraction" as if it were a slur or insult, while misunderstanding what it means. then replies with a "oh yeah, well your construct is twice as leaky" retort.
I actually think what the author is arguing is that async formalizes the idea of structuring your code to yield execution when it needs to "block". So block is leaky because it lacks a formal definition outside of an async annotated world. I find myself sympathetic to that take, despite my frustration with Rust's implementation of async, and the ecosystem/language shortcomings, etc.
I don't think the author's understanding of what a "leaky abstraction" is jives with how I've ever used it or heard it used. To me, a leaky abstraction is when there's hidden behavior underneath that makes the observed behavior when using the abstraction to change in ways that aren't intuitive based on the API contract.
For example, "everything is a file" on UNIX is a leaky abstraction: read() and write() can behave in wildly different ways, and have significantly different constraints, depending on what the file descriptor refers to (regular file, character device, socket, memfd, etc.).
Hell, C stdio/unistd functions themselves are leaky abstractions even when we're only considering regular files on disk. For example, even if your open()+write()+close() succeeds, the data may have not yet hit the disk, and might be lost if the machine crashes right after close() returns. And in some cases an fsync() might not even be enough, if the disk lies to the OS about the state of its own buffers and cache.
Put another way, abstractions are about hiding (hopefully) unnecessary implementation details from the person using your abstraction. An abstraction is leaky when those hidden details turn out to actually be important, and the person using your abstraction needs to understand them anyway.
The "Dependency Dog" sidebar block that has a mini-rant about AppKit on macOS is further evidence that the author doesn't know what a leaky abstraction is. You might not like AppKit, but every windowing system has canonical APIs to make windows/surfaces and draw things and define behavior and handle input. You just... have to use them, or some abstraction built on top of them. That doesn't make them leaky. (Also, pretty much every UI toolkit out there has restrictions on multi-threading; seems like a weird thing to complain about.)
The author's use of "blocking" was also a little confusing. To me, "blocking" means making a call that's going to sit and spin while waiting on I/O (or some other slow/contended resource), but the author uses it to mean "a function that is not declared async". The opposite of async is sync, not blocking.
Rust doesn’t have a way in the type system to tell if a function that isn’t declared async is going to block or not. Their point is that any of those functions could block, and having them block wouldn’t be an obvious breaking change, and so you need to know about their internals to know if they’re okay to call.
The author has a very strange understanding of the idea of a "leaky abstraction". AppKit requiring you to call methods on the main thread is just not an abstraction at all, leaky or otherwise.
I can't buy this premise. The leaky abstraction is that your selected libraries are attempting to be as efficient and as fast as possible while also trying to be fully general. As a result you get the full abstraction layer necessary to do this, which, as you've noted, for any one project is too much.
The complication you seem to be experiencing is other projects have come to rely on these libraries and their particular abstractions and so you've actually got a dependency management problem due to the design of the "core" libraries here or due to a lack of language features to /constrain/ all of them. A critical point of abstraction totally lacking in this ecosystem.
In any case if you look at any language that has contexts that can be externally canceled you should realize this actual solution must be implemented at a higher level than these "async primitives."
Articles like this remind me how Kotlin is probably 10 times
easier and concise than Rust and C++.
Kind of off topic but has anyone noticed difference in code quality and documentation from projects in C++ / Rust / Kotlin / JavaScript libraries that have the same features?
Recently was working with Seastar which is Tokio / Kotlin Coroutine equivalent in C++ and seems crazy how far behind it is from Kotlin Coroutines. The documentation the example, the boiler plate etc. Don't think it is the language themselves but the language culture. Seems like C++ libraries are just designed to be more difficult. Less documentation, no comments, flat folder structures with each file having with 1000s of lines of code. They expect you to go read through source code etc.
There's a conflation going on here which is a blocker, shall we say, for understanding the two sides of this coin. The antonym of async is not blocking, it is synchronous. The antonym of blocking I/O is evented I/O.
Stackless coroutines à la Rust's async are a perfectly reasonable control flow primitive to use in 'blocking' contexts, which properly aren't blocking at all since they aren't doing I/O, I'm thinking of generator patterns here.
It's also entirely possible, and not even difficult, to use evented I/O in synchronous function-calls-function style code. You poll. That's the whole deal, somewhere in the tick, you check for new data, if it's there, you eat it. This does not require async.
I found the "what if the real abstraction leak was the friends we made along the way" portion of the argument. Unconvincing. When people complain about async leaking, what they mean is that they would like to be able to use libraries without have to start a dummy tokio in the background. That's what's getting peas in the mashed potatoes, that's what annoys people.
"Abstraction leak" is a sort of idiomatic way to complain about this. Focusing on the slogan does nothing to address the substance, and "have you considered... that, a function, might take a while, and perhaps that, that is an 'abstraction leak'? eh, eh?" is. Well it probably seemed clever when it was being drafted.
The author must have specified in the title: "Blocking code IN RUST is a leaky abstraction".
I am so happy to be using Go for most programs!
It has one of the best concurrency implementations that I have ever seen (Erlang permitting ...). And for most of my programs, the garbage collection is not a problem. It's the opposite: simplifies programming incredibly.
No blocking versus async problems. No libraries for async are needed. Just concurrency (which has its own problems, but this is another story ...).
> Dependency Dog: If you want to see a good example of a leaky abstraction, consider AppKit. Not only is AppKit thread-unsafe to the point where many functions can only safely be called on the main() thread, it forces you into Apple’s terrifying Objective-C model. Basically any program that wants to have working GUI on macOS now needs to interface in Apple’s way, with basically no alternatives.
Uhm, every OS that I've developed a GUI for has a similar limitation. The "only call this thing from this thread" is a well-known thread safety pattern. It might not be the pattern that this author prefers, but it is an industry standard pattern for UI.
I agree with your comment. Every UI framework I have used (AppKit, Swing, JavaFX, etc.) is not thread-safe. It requires all UI code in the main thread or, in the case of JavaFX, a dedicated UI thread (not necessarily the main thread, but it must be a single thread where all UI operations occur).
Even outside of UI, use-from-one-thread-only is perfectly valid and acceptable. The issue at that point is with the tools (e.g. C), not the API. If the API provides thread-safe entry points, that's because it is taking on the responsibility of synchronizing. Maybe that's fine for a particular API, but it shouldn't be expected for everything. It's an added layer of complexity, hidden from the programmer, so expect it to be absent in more optimized cases.
Technically in JavaFX you can work with UI objects off the main thread, for instance to load or build up UI in parallel. The single thread requirement only kicks in when UI is attached to a live window (stage).
> Some of this criticism is valid. async code is a little hard to wrap your head around, but that’s true with many other concepts in Rust, like the borrow checker and the weekly blood sacrifices. Many popular async libraries are explicitly tied to heavyweight crates like tokio and futures, which aren’t good picks for many types of programs. There are also a lot of language features that need to be released for async to be used without an annoying amount of Boxing dynamic objects.
Yes. So don't do it
It is entirely possible to write asynchronous code without using asyc/await.
async/await is a fun paradigm for garbage collected systems. It is a horrible mess in Rust
There are other options than async/await and threads.
In my heart that is my main objection to async/await in Rust. It has "poisoned the well" for asynchronous programming. Where people think it is async/await or threads
We have been doing asynchronous programming for many decades.
I posit that while more programmer friendly async abstractions (such as in Go) are good in the short-term, ideally async should be far less scary even in the lower levels. Programmers should shy away from async not because it is hard, but when they have carefully considered if the architecture can afford to be sync instead. If they aren't doing async, it's likely that they rely on other code that should be handling async well.
Alan Kay said (paraphrased roughly) that programmers use multithreading when they don't understand how to write programs as the state machines that they are. Then you get all the surprise and suspicion about distributed locking (featured on HN today!) and whatnot. There is much essential complexity to be tackled here. If robust software is a priority (it really should be), we need to be deliberate in whether async is exposed at a given layer of abstraction. We shouldn't be awed into writing subpar programs.
Was Kay thinking of local code or distributed code? For local I can see his point. But once we start talking to networks or even really hard drives you need scatter-gather semantics. Otherwise you’re running lots of threads, which undercuts his point, or you’re writing your own multitasking state machine which is just a poor implementation of concurrency where you’re responsible for all the bugs and deadlocks, not just some of them.
(I should caveat what I'm about to say that I'm primarily concerned about writing robust and highly performant programs, and while I believe it should be a focus broadly, it's a practical niche.)
That's the thing, though. It's arguably even more important for distributed code. If we abstract away the state machine too much, it becomes difficult to reason about the code precisely because of abstraction. The complexity that was present explicitly in the state machine will just cause confusing behavior in the abstracted version. Using lightweight threads or another high level abstraction that approximates blocking code will allow getting a program out faster, but lower quality at that. Two examples to illustrate my point: first, you mention scatter-gather, but the base concept is orthogonal to sync/async. However, I/O is characteristically async, and therefore the underlying mechanisms are async anyways. Second, io_uring is showing that async can be good for performance while not being a difficult interface.
Sync code makes things easier to reason about but also kinda not. I think the big issues with async are that the OS hasn't done a good job allocating responsibility of async interfaces and the fundamental difficulty. The former makes async seem less efficient than it could be, which is true in that a sync-over-async interface is better than an async-over-sync-over-async interface, but we should have async interfaces accessible. The latter probably feeds into a bias to not even touch async where mixing async and sync would be the best blend of performance and programmability.
I was debating writing a blog post like "Blocking IO Considered Harmful" or something cheeky like that.
I'm not as familiar with Rust or C as other people, but I've done a lot of Java, and JDBC has made my life more difficult than it needs to be due to its semantics. It has been hard for me to get decent thread utilization with it because of all the blocking it does behind the scenes.
I've moved most of my SQL stuff to Vert.x SQL, which has given me better performance overall, and I can do most of it with a single thread.
At least in Java land, it's trivial to convert a non-blocking call into a blocking one (in Vert.x you can do myFuture.toCompletionStage().toCompletableFuture().get()), but it's pretty hard to efficiently go the other direction. I really wish JDBC would create something like "executeAsync" that returns a CompletableFuture or something similar.
In defense of Java blocking calls, in the 1990s to the 2010's the industry mostly considered Asynchronous programming as coordinating multiple threads over know yield points - such as using atomic locks, or mutexs. The main push by the industry was all about abstractions and OOP, and moving away from static C/C++ programming languages to dynamic programming languages such as Java/Python/Visual Basic or C#.
It is only until recently with the popularity of Node.js, Python and the explosion of the web that the developer have to deal with single threaded languages that blocking calls are a significant impact to the users experience. This push for Async in the web has had a cascade effect to system programming languages where old time developers are only first learning about it.
Could you clarify the "static C/C++" vs "dynamic Java" distinction? I'm used to thinking of "static" and "dynamic" in the context of type systems, but do the terms have overloaded meanings in concurrency/parallelism?
The push for async as we know it started in .NET, initially with F# and then popularized by C#, with the pattern then adopted by JS, Swift, Python, Rust and other languages. The implementations, however, are wildly different.
I thought F# was a derivative of OCamel that focused exclusively on immutable data structure and functional programming, instead of mutable/imperative languages.
True, one great thing that did come out from it is that the language needs to natively support asynchronous programming with syntactical sugar otherwise you get an explosion of libraries/frameworks that solve the problem with their own problems. How-ever this is a chicken/egg problem, did everyone really need async or did it come about because the proliferation of frameworks to solve this common pattern?
I can’t use the Vert.x SQL lib for MSSQL because named instances and Kerberos auth aren’t supported, so for a good deal of web calls I use the MSSQL JDBC inside executeBlocking Vert.x blocks.
However, recently I was thinking of utilizing a Vert.x worker verticle with a pool of Java virtual threads that could serve my JDBC calls. Has anyone tried anything like this?
It seems like the magic for the author is in the annotation. There's still two kinds of functions "fearlessly run in the UI thread" and "must be run in a worker" and async is great in that it greatly expands the number of functions
in that first pool but it's effectively the same thing as the [blocking] annotation but the other way and not totally complete. Because there are async-safe but not async functions.
So to me it seems like it all leaks because in all cases what's desired is some external signal about how the function behaves. A "badly written" async function can stall the loop and that isn't conveyed only by the async tag. It's rare to do this but you can and there might even be times where this makes sense[1]. But it happens that by convention async and ui-thread-safe line up but there's a secret other axis that async is getting mixed up in.
[1] Say running a separate event loop in a background thread used for CPU intensive tasks that occasionally perform IO and you want to make the most use of your thread's time slice.
This is some bullshit. And it doesn't help that everyone is using async to mean different things.
Blocking/synchronous system calls are not friendly to userspace async runtimes. Explicit async runtimes are a pain in the ass. The best solution imo is a blocking userspace interface that maps into the system however it likes. Go and Haskell do this. Erlang and probably all BEAM languages do too. Nim and Zig let you choose.
Runtimes like Go and Haskell can map onto code that makes blocking system calls for compatibility reasons when needed, through explicit request to the runtime, or detecting that a green thread has blocked.
I’ve always thought of async await as an ergonomic relaxation (as in constraints) of green threads. People had more problems doing green threads right than I’ve seen with async. Do some of the remaining problems look exactly like green thread problems? Absolutely. But it’s a step toward something more hygienic.
I realise I'm picking one sentence out of the middle of this, but this is the one thing I don't feel I understand about async:
"But well-formed Futures complete quickly"
Does this mean people writing async code don't ever write code which takes a while to compute?
I write lots of code which takes 10s of milliseconds, or 100s of milliseconds, or hours. And none of these feel like a good fit for async, as even a 10s of milliseconds piece of work would be enough to start to limit throughput and add unacceptable lag to other connections?
This is fine in a multi-threaded server, one thread per connection, you make a difficult request, the server spins for half a second before responding to you -- but it doesn't really fit in a async program?
The same problem comes up at the end with the proposed `#[blocking]` solution. There is no formal definition of blocking outside of async. It's a colloquial term meaning "takes a relatively longer time to finish". Async is the formalization of making a distinction between grinding the cpu and yielding until something else wakes you up and tells you to keep going.
So I guess a well formed Future always adds yield points between hot loops? And then you just trust the runtime to manage execution context such that your yields are noops if there's no contention for execution time and/or to avoid needless context switching.
This sounds like the same kind of problem the Linux kernel is trying to get rid of, and we used to live with back in Windows 3. It’s very hard to put in enough yield points to never block for too long in a complex algorithm, and I was so happy when I stopped having to do that myself!
True, but some problems just require that long to get the answer the user wants!
Some problems take 60 CPU years, but those you don’t expect the user to actively wait while you distribute their task and wait for the results, you send them an email in 3 months or so! :)
What I like about async and promises is there’s just enough bookkeeping to keep people from violating Keenighan’s law by too wide a margin. A little goes a long way, especially when any process could be handling several requests at once and make better use of the dead space between steps.
But a friend recently showed me how coroutines work in Kotlin and from my rather extensive experience helping people debug code that’s exceeded their Kernighan threahold, this seems a lot less footgun than nodejs. It’s far too easy to inject unhandled promise rejection in Nodejs. I think await should be the default not the exception, and Kotlin is closer to that.
Then that creates an invisible yield point which messes with your “is this a critical section” sense. Suddenly anything may yield and your naive
pt.x = getX()
pt.y = getY()
turns into the concurrency hell wrt pt’s consistency.
I think await should be the default not the exception, and Kotlin is closer to that.
I somewhat agree in that it should be an await/nowait keyword pair required at async calls. Not realistic though even in typescript, considering that Promise{} is just a continuation shell that may be passed around freely and doesn’t represent a specific function’s asynchronicity.
The biggest problem with the Node/Javascript solution is that if I call a function and don’t need the result, if that function is changed to async (or I missed that it always was) end up with bugs because we didn’t await the result.
When I say things like this people typically claim I’m bringing up some hypothetical, so let me be clear: I’m tired of fixing this bug. Not in my code (mostly), but in the code base. It happens too often both in a numerical sense and in a sense that it’s so stupid that it happens at all. It’s how I started thinking about this problem in the first place. The solution we have isn’t working very well.
All abstractions are leaky, but some are useful. You can't have a perfect abstraction. Your abstraction is always a compromise of various goals you want to achieve.
> Asynchronous code does not require the rest of your code to be asynchronous. I can’t say the same for blocking code.
Well, you can also say that "file systems are a leaky abstractions. Sector-addressing disk access does not require the rest of your code to access disk sectors directly. I can't say the same for the code that uses files".
First of all, asynchronous code does require the rest of your code to be asynchronous: because, as you've yourself said, blocking code doesn't mesh well with the asynchronous. Second, when your are trying to work with two different levels of abstractions at the same time, of course you will find the presence of the higher-level abstraction (the blocking code, the file system, etc) to be inconvenient: the higher-level abstraction has to make some opinionated choices, or it would not be an abstraction but merely an interface.
So yeah, I guess blocking code is a leaky abstraction, just like file systems, TCP streams, garbage-collected memory, and sequential processor instruction execution. It's still a higher-level abstraction though.
Async/await has been a game-changer. You can tell that this is the case by how much shorter and more reliable code with async/await is compared to alternatives (e.g. which rely on callbacks).
Callbacks are a leaky abstraction IMO because they force you to write a lot of additional logic to handle results in the correct sequence. Obtaining the result of multiple long-running function calls on the same line is insanely messy with callbacks.
I've found that async/await helps to write code which stays close to the business domain. You can design it with multiple layers which take care of a distinct level of abstraction so you can understand what's happening just by reading the main file top to bottom. You don't have to jump around many files to read the functions to understand what they are doing and how their results are combined.
Async/await makes the concurrency aspect as succinct as possible.
That said, I've seen many people writing code with async/await as though they were writing using callbacks... Some people haven't adapted correctly to it. If your async/await code doesn't look radically different from how you used to write code, then you're probably doing something seriously wrong. Ensure that you familiarize yourself with the async/await utility functions of your language like (in JS) Promise.all, Promise.race... Notice how try-catch works with the await keyword...
> compared to alternatives (e.g. which rely on callbacks).
This is a browser-centric view, which makes no sense in the context of Rust (which runs on a proper OS with threads and absolutely no need for callbacks).
Really frustrating that Rust became infected with the async disease.
This article is pure nonsense fwiw.
It says: you can always block on the Future returned by an async function.
Ugh. Ok, but let's just not have any async at all please, and then there's be no need to add that code and no behind the scenes context switching done.
Async usage is overblown in some cases, but not in others. The mistake of async Rust is not that it's unnecessary, just that it's being pushed for general purpose usage. Which is probably impractical for any async abstraction, or for any abstraction in general.
block_on and spawn_blocking are tokio's async abstractions, there is no "blocking code's abstraction" that can leak.
What the article intentionally omits is that I can defer io by getting a handler and polling it in a select loop. And right after that, I can also poll my mpsc queue if I want to. This is how "blocking code" has worked for the last few decades.
> Asynchronous code does not require the rest of your code to be asynchronous.
Seems like in many languages the async keyword is viral. And that is the most pointless work I've ever done, where I've to go around and mark everything async just because of one async function. So this is a strange statement to me.
Does the author not know what a "leaky abstraction" is? The first time they referred to memory leaks I thought is a clever joke nested in a jab a hypothetical coder who didn't know, but then in the end they double down in a clearly non-joking way. Then even make a chart showing that calling a normal function from an async function is trivial, that is the opposite of a leaky abstraction.
An abstraction is a "leaky abstraction" when it forces some external condition or behavior onto code (ex: calling code that uses the network doesn't work when the network is disconnected). In this case the hurdles of calling async code from normal code, and this author tries to hand waive that with talk of dependencies.
There are real benefit and drawbacks to async code and styles. Even if I don't think it is a good abstraction I want to discuss it on the real engineering terms of its actual merits and drawbacks. This article is talking past the engineering decisions and seems like it didn't mean to.
It is easy but dangerous to call a non-async function from an async function (since the non-async function could block, forcing all your async code to block on this function).
It is hard but safe to call an async function from a non-async function.
Note that this is language specific; if Rust had “async”, “pure”, and “blocking” as three different function colors, then we would have a very different design. Or, for example, in JavaScript:
It is easy and safe to call a non-async function from an async function, since JavaScript exposes no blocking primitives.
It is impossible to call an async function from a non-async function, since there is no way to block on the result.
Doesn’t change the core issue, which is that if you want to do multiple operations simultaneously that take time to execute, you’re going to need to write stuff to handle that which you wouldn’t otherwise. Whether it’s channel piping or async awaiting is semantics.
Perhaps (?) it wouldn't be an issue for correctness in most languages, but it would certainly add a performance and debuggability cost. It could be useful in some niche domains, probably.
You can perform file I/O in JavaScript synchronously. The localStorage API in the browser is synchronous (and in Node via the --experimental-webstorage option), and of course requiring a CommonJS module is also synchronous (and there are many other sync filesystem APIs in Node as a sibling comment pointed out).
You just can't perform network I/O synchronously. Although a network attached file system allows for both network and file I/O technically, but that's a really pedantic point.
Yeah I should've said there's no Node API for making synchronous HTTP requests (unless you count executing a child process synchronously). Even the older http.request API used in Node prior to the introduction of fetch is async and accepted a callback. Browsers have all sorts of deprecated foot guns though (like the synchronous mode of XMLHttpRequest).
> Frankly, I don’t think async code is leaky at all, and the ways that it does leak are largely due to library problems.
Huh? One could argue the exact same in opposite direction - blocking code isn't leaky at all, just wrap it in a Future. And if doing that is difficult because of !Send data, then those are just "library problems". You could always just use a threadsafe library instead, right?
I am surprised by some of the comments claiming to find it very hard to make an async API out of a sync one.
I write a hobby kernel with a profoundly asynchronous I/O system. Since it's nice to have synchronous I/O in some places (reading in data in response to a page fault, for example, or providing the POSIX read/write functions), I have to turn it into synchronous I/O in some cases.
Asynchronous I/O notifies its completion in a configurable-per-I/O-packet (called "IOP") way. The default is to signal an event object. You can wait on an event object and when it's signalled, you're woken up. The result is that synchronously sending an IOP is hardly any more work than:
I would be surprised to hear of any system where you can't do similar. Surely any environment has at least semaphores available, and you can just have the async I/O completion signal that semaphore when it's done?
I don't think the problem comes so much from async v. sync, it comes from the implementation language/framework.
If you were to wait_for_event (or its equivalent) in the node main thread you will stall the 1 thread handling requests.
This whole problem is not just async/sync, it is the environment you live in.
What we should be questioning is why high level languages in high level frameworks make the programmer even consider the distinction at all.
If I am handling an http request, the async nature of this is an optimization because threads (or worse, processes, I once easily dos'ed an apache system that created a new process for each request) are too expensive at scale.
It should not concern the programmer whether a thread is launched for each request or we are using cooperative multitasking (async). The programmer/application needs that information from that call before that one piece can move forward.
The lower level async-as-state-machine (or coroutine or whatever) should be more standard, but that's an argument for another day. For now, you're right and the opaque threading model is basically what Go does. On the other hand, it's not suitable for Rust because in some ways Rust is lower level. C# went and did it the Rust way, which I personally appreciate but I understand fits into the problem you're describing.
just document your api's thread safety and let me handle it. don't make me deal with this async/await bullshit. multithreaded programming is hard enough as it is.
Godël covers how it’s leaky abstraction all the way down. If you consider a specific chunk of useful async code it still doesn’t compose enough of system to do much of use; it can’t answer many questions alone or any about itself. We never just have Map() but other functions. Its constant composition of symbolic logic we vaguely agree means something relative to an electrical system of specific properties.
A useful system is “blocked” on continuous development of that system.
These posts are too specific to code, miss the forest for a tree. A more generalized pedantry (like electrical state of systems rather than dozens of syntax sugars to simulate the same old Turing machine) and more frequent spot checks for stability and correctness relative to the ground truth would be great.
Way too much circumlocution around code. Map is not the terrain. I for one am having blast training a state generator that acts like a REPL and (for example) infers how to generate Firefox from a model of its code (among other things).
Excited about the potential to peel away layers of abstraction by generalizing the code with automated assistants.
I have the opposite experience, working in embedded (C, not Rust...). Building a synchronous API on top of an async one is hell, and making a blocking API asynchronous is easy.
If you want blocking code to run asynchronously, just run it on another task. I can write an api that queues up the action for the other thread to take, and some functions to check current state. Its easy.
To build a blocking API on top of an async one, I now need a lot of cross thread synchronization. For example, nimBLE provides an async bluetooth interface, but I needed a sync one. I ended up having my API calls block waiting for a series of FreeRTOS task notifications from the code executing asynchronously in nimBLE's bluetooth task. This was a mess of thousands of lines of BLE handling code that involved messaging between the threads. Each error condition needed to be manually verified that it sends an error notification. If a later step does not execute, either through library bug or us missing an error condition, then we are deadlocked. If the main thread continues because we expect no more async work but one of the async functions are called, we will be accessing invalid memory, causing who knows what to happen, and maybe corrupting the other task's stack. If any notification sending point is missed in the code, we deadlock.
"If you want blocking code to run asynchronously, just run it on another task."
This highlights one of the main disconnects between "async advocates" and "sync advocates", which is, when we say something is blocking, what, exactly, is it blocking?
If you think in async terms, it is blocking your entire event loop for some executor, which stands a reasonable chance of being your only executor, which is horrible, so yes, the blocking code is the imposition. Creating new execution contexts is hard and expensive, so you need to preserve the ones you have.
If you think in sync terms, where you have some easy and cheap ability to spawn some sort of "execution context", be it a Haskell spark, an Erlang or Go cheap thread, or even are just in a context where a full OS thread doesn't particularly bother you (a situation more people are in than realize it), then the fact that some code is blocking is not necessarily a big deal. The thing it is blocking is cheap, I can readily get more, and so I'm not so worried about it.
This creates a barrier in communication where the two groups don't quite mean the same thing by "sync" and "async".
I'm unapologetically on Team Sync because I have been programming in contexts where a new Erlang process or goroutine is very cheap for many years now, and I strongly prefer its stronger integration with structured programming. I get all the guarantees about what has been executed since when that structured programming brings, and I can read them right out of the source code.
“This highlights one of the main disconnects between "async advocates" and "sync advocates", which is, when we say something is blocking, what, exactly, is it blocking?”
When I have work on a cooperatively scheduled executor for optimal timing characteristics. Sending work/creating a task on a preemptive executor is _expensive_. Furthermore, if that blocking work includes some device drivers with interactions with hardware peripherals, I can’t reasonable place that work on a new executor without invalidating hardware timing requirements.
Threads and executors can be infeasible or impossible to spawn. I have 1 preemptive priority and the rest are cooperative on bare metal. I can eat the suboptimal scheduling overhead with a blocking API/RTOS or I need the async version of things.
> If you think in async terms, it is blocking your entire event loop for some executor
I think I'm mis-understanding this. async doesn't block your entire event loop. Other events continue to be processed. Trivial example
Did I misunderstand your meaning of "blocking the entire event loop"?I think GP means that a synchronous blocking call in an async runtime would block and bring the executor (event loop) to a global halt.
Think
VS (no error handling to bring the point better across).Just to confirm, whilenot-dev had it correct.
You can have more than one executor in some implementations (you know you do if you can keep more than one CPU busy), but in all the implementations you're certainly causing problems if you sync-block an executor for long periods of time, that's not what the system is for. By contrast, you want to open a thread/goroutine/Erlang process/Haskell spark/execution context in a threaded language and have it just bang away on numeric computation for a few days? Usually that's just fine, though if you've got a green-threading implementation, double-check first. Most green thread implementations seem to go through a period where that is not fine, but then eventually it is; Go for instance passed through that quite a few versions ago.
To add: This issue isn't that troublesome in languages like JavaScript where the runtime is event driven out-of-the-box (synchrounous APIs are considered the exception and suffixed with 'Sync' by convention), but it will become difficult to reason about in languages like rust and python where an event driven runtime is optionally choosen by the developer. Because now it's not enough to conceptually grep all awaiting calls to reason about any blocking code, but you'd also need to know if any other synchronous calls are implicitly awaiting in an underlying layer.
Your code is wrong.
This is blocking synchronous code in JavaScript:
I like your take, and I propose there is another kind of division: function "coloring". An async "effect" type (e.g. futures, or even just 'async function') signals to programmers that there is some concurrency stuff going on around the place(s) you use it and that we need to talk about how we're going to handle it. i.e. rather than a performance issue, it's a correctness/safety/semantics issue.
> it is blocking your entire event loop for some executor
In the browser, only. Which is where all this nonsense began. Browser didn't have threads (Win16) so they invented a crappy cooperative scheme with callbacks, which became gussied up as "async" and now has a cult following. It was all a hack designed to make a terrible runtime environment usable. And now we have a priesthood telling us this is how to program, and if you disagree then you must be stupid.
You're correct that we should be using CSP (Erlang, Go, Occam, Actors).
It's not completely to blame on the browsers. We had the concept of green threads [1] well before that, albeit for equally dubious reasons: OS threads were believed to be too slow, and so green threads were composed onto OS threads. It seems a better choice here would be to try and improve OS threads, but the speed at which various program languages and libraries develop well outstrips anything we could achieve with the kernel, and so maybe this was inevitable :)
[1]: https://en.wikipedia.org/wiki/Green_thread
On the topic of improving OS threads, there are scheduler activations and similar mechanisms in some microkernels. Basically sending events for all hardware things and context switches, making user level threads as flexible as kernel threads. Unfortunately they haven't become mainstream.
Note that Erlang processes are essentially green threads within OS thread, but the Erlang process manager is preemptive (unlike most green thread managers). There is room for improvement in both directions.
Making an asynchronous task into a synchronous task is easy in exactly one scenario: when there is no pre-existing event loop you need to integrate with, so the actual thing the synchronous task needs to do is create the event loop, spin it until it's empty, and then continue on its merry way. Fall off this happy path, and everything is, as you say, utterly painful.
In the opposite direction, it's... always easy (if not entirely trivial). Spin a new thread for the synchronous task, and when it's done, post the task into the event loop. As long as the event loop is capable of handling off-thread task posting, it's easy. The complaint in the article that oh no, the task you offload has to be Send is... confusing to me, since in practice, you need that to be true of asynchronous tasks anyways on most of the common async runtimes.
This response just highlights how large the difference between different domains using the same construct is, and why that makes it impossible for people to agree on basically anything.
Rust async is, to a significant degree, designed to also work in environments that are strictly single-threaded. So "Spin a new thread" is just an operation you do not have. Everything that's weird and unwieldy about the design follows from that, and from how it also requires you to be able to allocate almost nothing, and to let the programmer manage the allocations.
I have pointed it out before that the overall design is probably made worse for general use by how it accommodates these cases.
I have the same experience, I like splitting my embedded C microcontroller peripheral drivers into 3 layers:
- header files with registers addresses and bitmasks
- asynchronous layer that starts transactions or checks transaction state or register interrupt handler called when transaction changes states
- top, RTOS primitives powered, blocking layer which encapsulates synchronization problems and for example for UART offers super handy API like this:
status uart_init(int id, int baudrate)
status uart_write(int id, uint8_t* data, int data_len, int timeout_ms)
status uart_read(int id, uint8_t* buf, int buf_len, int timeout_ms, int timeout_char_ms)
Top, blocking API usually covers 95% use cases where business logic code just want to send and receive something and not reinvent the synchronization hell
> If you want blocking code to run asynchronously, just run it on another task
What kind of embedded work are you doing exactly? Linux "soft" embedded or MMUless embedded? I don't have infinite NVIC priority levels to work with here... I can't just spin up another preemptively scheduled (blocking) task without eating a spare interrupt and priority level.
Otoh, I can have as many cooperatively scheduled (async) tasks as I want.
Also, at least in Rust, it's trivial to convert nonblocking to blocking. You can use a library like pollster or embassy-futures.
MMUless embedded with FreeRTOS. For example, at one point, we did not want connecting over TCP to block our command handler, so created a task that waited for a notification, and connected on that task, and went back to waiting for another notification. Though we ended up combining some tasks' responsibilities to reduce the amount of total stack space we needed.
You can spawn async task open a channel and wait for async task to push to it from blocking context, channels can have efficient ways to wait for the message. This is fairly easy to do in rust
The trick with synchronization/parallelism systems is to only communicate over a known yield point this is normally done via queues. It is the only way you get deterministic behavior from your sub-systems or multi-threaded environments.
To build a blocking API on an asynchronous API you make a place for the result to go, start the async API, then poll on the result appearing. Right?
> There are a substantial number of async crates out there that run on top of tokio. They use tokio’s primitives, tokio’s executor, and tokio’s I/O semantics. Because of this, they rely on tokio’s runtime to be running in the background. If you try the above strategy for a crate that relies on tokio, it will fail at runtime with a panic.
This is the worst design choice in the broader Rust ecosystem right now. It’s insane that people are exporting public functions from packages that rely on their caller running the right thing in the background. Imagine a library that didn’t work if you didn’t create a global mutable variable for it to modify, and that variable didn’t even need to be passed into the library explicitly.
Tokio should have a marker type that’s Copy and returned from functions that start the runtime, and require a parameter of that type when calling functions which expect a runtime to already exist.
Yes this is backwards incompatible. It should be done anyway.
If the value is Copy that value will continue to be usable after the runtime has been stopped and dropped, defeating the purpose.
If the value borrows from the runtime to prevent that, futures that hold on to that value stop being `'static`, which brings its own problems, chief among them being breaking `tokio::spawn()` and causing a self-reference (runtime contains futures which contain borrows of itself).
If the value holds on to (a refcount of) the runtime internally to prevent the runtime from being dropped until the value has been dropped, that will prevent programs from exiting as long as even one future is still running on the runtime, which is undesirable.
You don’t actually need it to be a reference; the runtime could generate a random int64 and then pass that around as a “runtime id” which could be checked when used.
You’re correct that my proposal doesn’t solve every problem with Tokio. I hope someone more knowledgeable than me can write a proposal that solves all the problems.
> which could be checked and used
Checked where? In global mutable state you mentioned that has to be created beforehand in the same way you mention? I don't really see how this changes much other than making everything more verbose from the extra argument.
It would be nice for someone to come to with a new solution to solve everything, but I don't think it's as simple as just "use an id and then tidy up some remaining details". No one has come up with something more compelling yet because those details are where the fundamentally are part of the problem is. When the hard problem is figuring out semantics that will allow exposing a better API, proposing an API and then hoping someone finds a way to make it work under the hood is just backwards.
The verbosity is the point! I want functions that interact with a specific expectation of global state to clearly mark that in their type, and the least complicated way to do that is my proposal. I hope someone can invent something more compelling, but my proposal is still an improvement of the status quo.
I think the toplevel commenter was looking for something that could be checked at compile time. The "random int64" idea is identical to "some type that implements Copy", and would require runtime verification, which is not really an improvement on the current state of affairs.
It would prevent callers from not knowing the function depends on a specific runtime. That’s a huge improvement in my opinion.
I expect something along the lines of the GhostCell paper could be used to implement this, to be honest, but I think it would be more verbose than necessary.
(Really, we need an effects system, and then Tokio could just be an effect that contains, but is different from, async)
Does anyone realistically shut down the Tokio runtime once it's been started? It wouldn't be a problem if the runtime is immortal once created; the runtime reference becomes more of a token demonstrating that, yes, this is running under Tokio.
Though I don't think this is such a huge problem. Yes, it's a runtime crash that could potentially be eliminated at compile-time, but realistically that will only happen once during development, and it will be an easy fix.
If I'm writing an async program (that is, with an async main()), then no, I'll never be shutting my runtime down.
But if I already have a non-async program, but need to use an async library, then yes, I'll probably spin up the runtime before using the library, and shut it down after I'm done using it. Unless of course I expect to be using that library for the full run-time of my program, in which case it'll exist for the duration.
> I'll probably spin up the runtime before using the library, and shut it down after I'm done using it.
Hopefully the library did not leave any background tasks running that you can't ask it to stop :) Raymond Chen has a lot of enlightening horror stories about multi-threaded COM (and multi-threaded programming in general) on his blog that are applicable not only to Windows, but in general context as well.
Well on the compatibility side of things, tokio has a TON of issues as well. I remember when tokio 1.0 came about, I was forced into a full psychopath mode cause Actix and rusoto were using different and incompatible versions of tokio. And I had to dig 200 layers deep to figure out why something simple such as copying a file to s3 wasn't working....
What is the difference between code that blocks waiting for I/O and code that performs a lengthy computation? To the runtime or scheduler, these are very different. But to the caller, maybe it does not matter why the code takes a long time to return, only that it does.
Async only solves one of these two cases.
I’d like to draw an analogy here to ⊥ “bottom” in Haskell. It’s used to represent a computation that does not return a value. Why doesn’t it return a value? Maybe because it throws an exception (and bubbles up the stack), or maybe because it’s in an infinite loop, or maybe it’s just in a very long computation that doesn’t terminate by the time the user gets frustrated and interrupts the program. From a certain perspective, sometimes you don’t care why ⊥ doesn’t return, you just care that it doesn’t return.
Same is often true for blocking calls. You often don’t care whether a call is slow because of I/O or whether it is slow because of a long-running computation. Often, you just care whether it is slow or how slow it is.
(And obviously, sometimes you do care about the difference. I just think that the “blocking code is a leaky abstraction” is irreparably faulty, as an argument.)
>What is the difference between code that blocks waiting for I/O and code that performs a lengthy computation?
To the caller of that specific function, nothing. To the entire program, the difference is that other, useful CPU work can be done in the meantime.
This might not matter at all, or it might be the difference between usable and impractically slow software.
> To the caller of that specific function, nothing.
And that's what makes async code, not the blocking code, a leaky abstraction. Because abstraction, after all, is about distracting oneself from the irrelevant details.
That isn't my understanding of a leaky abstraction. An abstraction leaks when it's supposed to always behave in some certain, predictable way, but in practice, sometimes it doesn't. When does an async function not behave the way it's supposed to?
My understanding of a leaky abstraction is that the abstraction itself leaks out details of its design or underlying implementation and requires you to understand them. What you seem to describe is a bug, edge case, or maybe undefined behaviour?
For example, an ORM is a leaky abstraction over an RDBMS and SQL because you inevitably have to know details about your RDBMS and specific SQL dialect to work around shortcomings in the ORM, and also to understand how a query might perform (e.g will it be a join or an N+1?).
I don't really think that the async or blockingness is the leak, but that the time taken to process is not defined in either case, and you can leak failure criteria either way by not holding to that same time.
People can build to your async process finishing in 10ms, but if suddenly it takes 1s, it fails
Yes, you understand exactly the point I’m making.
Async is worse than leaky it often goes viral because many other parts need to be async to call it.
It's better to think of "async" as indicating that a code will do something that blocks, and we're allowing our process to manage its blocking (via Futures) instead of the operating system (via a context switch mid-thread.)
I would argue a few things:
First: You need to be aware, in your program, of when you need to get data outside of your process. This is, fundamentally, a blocking operation. If your minor refactor / bugfix means that you need to add "async" a long way up the stack, does this mean that you goofed on assuming that some kind of routine could work only with data in RAM?
Instead: A non-async function should be something that you are confident will only work with the data that you have in RAM, or only perform CPU-bound operations. Any time you're writing a function that could get data from out of process, make it async.
I want to restate what you’ve said from a different perspective:
If you write a lot of pure functions, in a Functional Core, Imperative Shell manner, then only the imperative part has to deal with any of the async parts. Yes it’s the topmost part of the code, but there is nothing to “infect” with it which isn’t already destined to be.
It’s when you try to write imperative code like there are no consequences for doing so that the consequences show up en masse and can be confused as symptoms of entirely different problems.
In many application use cases, that is an implementation detail that should not be a concern of higher levels. The ramifications may not even be known by the people at higher levels.
Take something very common like cryptographic hashing, if you use something like node.js you really don't want to block the main thread calculating an advanced bcrypt hash. It also meets all of your requirements that data not come from outside ram and is very CPU bound.
Obviously, if you are directly calling this hashing algorithm you should know, however, the introduction of a need to hash is completely unpredictable.
> Take something very common like cryptographic hashing, if you use something like node.js you really don't want to block the main thread calculating an advanced bcrypt hash. It also meets all of your requirements that data not come from outside ram and is very CPU bound.
I guess I implied quick operations when I said "CPU bound." (IE, calculating a square root, string manipulation, (de)serialization...)
I haven't done hashing in Node.js. I assume that the bcrypt API is async and calls into a native library?
You could make the proposition that sequential code is inherently asynchronous in modern operating systems, because the kernel inherently abstracts the handling of blocking/unblocking your process.
Sure, but doesn't that remove the usefulness of the words?
Yes and no. From a practical point of view considering sync code as async is useless as sync and async code have very different usability characteristics and it is useful to have different names for the two domains.
On the other hand, form a more abstract, theoretical level, it is important to know that there is almost always an event loop at some point deep in the stack and sync code is just sugar over async and you can always transform from one to the other.
Yes and no, the concept that is being communicated is that your program inherently is asynchronous how-ever the context switching/block/unblocking is abstracted away from you so you can focus on what is important to you - which is solving your problem. Once you communicate that concept then it makes it a lot easier for people to create new mental models that better reflects reality. Then they can structure their programs that can take advantage of it.
I would say that's a leaky abstraction. (The OS hiding the details of async by blocking your threads and context switching, although this is how many programs operate.)
The problem comes when you have a critical section where you need to hold a mutex (lock): If you're working with "proper" async code, you can safely call any non-async code from within the lock. (C# enforces this, because a lock statement can not contain the await keyword.)
When any method you call can block on IO, synchronous programming (IE, using the OS to become async), can leak and make you hold your mutex longer than you should.
---
That being said, many of us build our careers programming threaded code that relies on the OS to block. So at this point we're splitting hairs.
When most people talk about async code it is with single threaded (hence the need not to block the event executor) applications (web-browsers, nodejs ect.). How-ever on the other end of the delirium when people talk about mutex's (locks) they're talking about multiple threadeds working in parallel which you inherently need to consider if the code you're calling is going to either direct or indirect acquire the same lock or like you pointed out calls a api that blocks. In the single threaded environment the only cognitive load you need to worry about is one or two yield points, how-ever in a parallel system your main concern is mutex's, race-conditions, or dead-locks.
"asynchronous code does not require the rest of your code to be synchronous" fails the smell test.
Many APIs are shipping async-only. You can't stick calls to those APIs in the middle of your sync code in many cases (unless you have an event loop implementation that is re-entrant). So... you gotta convert your sync code to async code.
I am curious as to what a principled definition of a "blocking" function would be. I suppose it's a call to something that ends up calling `select` or the like. Simply saying "this code needs to wait for something" is not specific enough IMO (sometimes we're just waiting for stuff to move from RAM to registers, is that blocking?), but I'm sure someone has a clear difference.
If you care about this problem enough, then you probably want to start advocating for effect system-like annotations for Rust. Given how Rust has great tooling already in the language for ownership, it feels like a bit of tweaking could get you effects (if you could survive the eternal RFC process)
On the other hand, it might flag a code smell to you in that you’re injecting I/O into a code path that had no I/O before which forces you to either make annotations making that explicit for the future or to realize that that I/O call could be problematic. There’s something nice about knowing whether or not functions you’re invoking have I/O as a side effect.
> I am curious as to what a principled definition of a "blocking" function would be.
It's one where the OS puts your (kernel) thread/task to sleep and then (probably) context switches to another thread/task (possibly of another process), before eventually resuming execution of yours after some condition has been satisfied (could be I/O of some time, could be a deliberate wait, could be several other things).
OS threads can be put to sleep for many reasons, or they can be preempted for no explicit reason at all.
On such reason could be accessing memory that is currently not paged in. This could be memory in the rext section, memory mapped from the executable you are running.
I doubt that you would want to include such context switches in your "blocking" definition, as it makes every function blocking, rendering the taxonomy useless.
That seems a necessary but not sufficient condition, since a pre-emptively multitasking OS may do this after almost any instruction.
Not only that, but any OS with virtual memory will almost certainly context-switch on a hard page fault (and perhaps even on a soft page fault, I don't know). So it would seem that teading memory is sufficient to be "blocking", by your criterion.
1) I deliberately left it open to including page faults by design. If you do not understand that reading memory might lead to your process blocking, you have much to learn as a programmer. However, I would concede that that this is not really part of the normal meaning of "blocking", despite the result being very similar.
2) I did not intend to include preemption. I should reword it to make it clear that the "condition" must be something of the process' own making (e.g. reading from a file or a socket) rather than simply an arbitrary decision by the kernel.
Would signaling, say, a semaphore count? That could cause another higher priority realtime process to wake-up and block the signaler forever.
I think the canonical meaning of "blocking code" is more of the "I know it when I see it" variety than an useful and rigorous definition.
This feels mostly right to me. I think that you get into interesting things in the margins (is a memory read blocking? No, except when it is because it's reading from a memory-mapped file!) that make this definition not 100% production ready.
But ultimately if everyone in the stack agrees enough on a definition of blocking, then you can apply annotations and have those propagate.
> except when it is because it's reading from a memory-mapped file
Where "memory mapped file" includes your program executable. Or any memory if you have swap space available.
And any operations can be "blocking" if your thread is preempted which can happen at basically any point.
So yes, everything is blocking. It is just shades of grey.
But this isn’t hemming to the definition brought up by GP. “I will now, of my own accord, sleep so the OS scheduler takes over” is fairly precise. And it’s different from both just doing an operation that takes time… and different from the program doing an operation that, incidentally, the OS sees and then forces a sleep due to some OS-internal abstraction
But you think about this too much and you can easily get to “well now none of my code is blocking”, because of how stuff is implemented. Or, more precisely, “exactly one function blocks, and that function is select()” or something.
For reference, there are 2 lists of such functions in signal(7).
The first list is for syscalls that obey `SA_RESTART` - generally, these are operations on single objects - read, lock, etc.
The second list is for syscalls that don't restart - generally, these look for an event on any of a set of objects - pause, select, etc.
Sometimes, things need to leak.
For example, the fact that code actually takes time to execute ... this is an abstraction that should almost certainly leak.
The fact that some data you want is not currently available ... whether you want to obscure that a wait is required or not is up for debate and may be context dependent.
"I want to always behave as though I never have to block" is a perfectly fine to thing to say.
"Nobody should ever have to care about needing to block" is not.
> this is an abstraction that should almost certainly leak.
that's not a leak, that is the abstraction for procedural/imperative code, or at least an important part of it. essentially: "each step occurs in order and the next step doesn't happen until the former step is finished"
That's not a particularly interesting description for multithreaded code. The procedural/imperative element of it may be true of any given thread's execution, but with the right user-space or kernel scheduler, you may or may not care what the precise sequencing of things is across threads. This is true whether you're using lightweight user-space threads ("fibers") or kernel threads.
of course, but in that case it's not the base imperative abstraction that's leaky - it's the multi-thread/multi-core abstraction built on top of it - it's almost as easy as multiple threads happening in parallel. until it isn't.
the multi-thread abstraction attempts to be: "hey, you know how that imperative model works? now we can have more than one of these models (we'll call them threads!) running in parallel. neat, huh?"
it's things like memory barriers and race conditions, etc. that are the "leaks". but it's the threading/multi-core abstractions that are leaky not (back to the article) the "blocking" code.
i can write single-thread "blocking" code all day long that does all kinds of interesting non-trivial things and never have to worry about leaks wrt. blocking, in-order execution, etc. even in the presence of out-of-order execution processors and optimizing compilers, VMs, multi-user OS'es, etc. effects are observably identical to the naive model.
the author didn't do a good job of clearly defining anything, but i bristle at the idea that the basic "blocking" code abstraction whats leaky - it's the async, threads, event-driven models, etc. that are (necessarily) a bit leaky when they break the basic imperative blocking model or require accommodation of it.
good points. i'd add, though, that priority inversion is specifically a leakage of the blocking nature of some parts of the imperative code into the thread model. similarly, the implications of locks for real time code (something i work on) is another example of blocking leaking into a thread model.
The author seems to reinforce the original point, that the async paradigm ends up working best in an all-or-nothing deal. Whether the difficulties in interfacing the paradigms should be attributed to the blocking part or the async part does not really matter for the practical result: if calling blocking code from async code is awkward and your project's main design is async, you're going to end up wanting to rewrite blocking code as async code.
the author is misusing the "leaky abstraction" idea. in the section "what's in a leak" a rather muddy argument is made that a leak is that which forces you to bend your program to accommodate it. so `leak => accommodation`.
then it immediately conflates the need for that accommodation (difficulty of calling one type of code with another) with "leakiness". so essentially "calling blocking => more accommodation" (e.g. from event loops that shouldn't block).
- that's logically incorrect, even in the author's own argument (A => B, B, therefore A is the affirming the consequent fallacy).
- that's not what a "leaky abstraction" is. not everything that doesn't fit perfectly or is hard is due a "leaky abstraction". rather it's when the simplistic model (abstraction) doesn't always hold in visible ways.
a "leaky abstraction" to "blocking" code might be if code didn't actually block or execute in-order in all situations. until you get to things like spinlocks and memory barriers this doesn't happen. but that's more a leak caused by naively extending the abstraction to multi-core/multi-thread use, not the abstraction of blocking code itself.
i love the passion for `async` from this fellow, but i think he's reacting to "async is a leaky abstraction" as if it were a slur or insult, while misunderstanding what it means. then replies with a "oh yeah, well your construct is twice as leaky" retort.
I actually think what the author is arguing is that async formalizes the idea of structuring your code to yield execution when it needs to "block". So block is leaky because it lacks a formal definition outside of an async annotated world. I find myself sympathetic to that take, despite my frustration with Rust's implementation of async, and the ecosystem/language shortcomings, etc.
I don't think the author's understanding of what a "leaky abstraction" is jives with how I've ever used it or heard it used. To me, a leaky abstraction is when there's hidden behavior underneath that makes the observed behavior when using the abstraction to change in ways that aren't intuitive based on the API contract.
For example, "everything is a file" on UNIX is a leaky abstraction: read() and write() can behave in wildly different ways, and have significantly different constraints, depending on what the file descriptor refers to (regular file, character device, socket, memfd, etc.).
Hell, C stdio/unistd functions themselves are leaky abstractions even when we're only considering regular files on disk. For example, even if your open()+write()+close() succeeds, the data may have not yet hit the disk, and might be lost if the machine crashes right after close() returns. And in some cases an fsync() might not even be enough, if the disk lies to the OS about the state of its own buffers and cache.
Put another way, abstractions are about hiding (hopefully) unnecessary implementation details from the person using your abstraction. An abstraction is leaky when those hidden details turn out to actually be important, and the person using your abstraction needs to understand them anyway.
The "Dependency Dog" sidebar block that has a mini-rant about AppKit on macOS is further evidence that the author doesn't know what a leaky abstraction is. You might not like AppKit, but every windowing system has canonical APIs to make windows/surfaces and draw things and define behavior and handle input. You just... have to use them, or some abstraction built on top of them. That doesn't make them leaky. (Also, pretty much every UI toolkit out there has restrictions on multi-threading; seems like a weird thing to complain about.)
The author's use of "blocking" was also a little confusing. To me, "blocking" means making a call that's going to sit and spin while waiting on I/O (or some other slow/contended resource), but the author uses it to mean "a function that is not declared async". The opposite of async is sync, not blocking.
Rust doesn’t have a way in the type system to tell if a function that isn’t declared async is going to block or not. Their point is that any of those functions could block, and having them block wouldn’t be an obvious breaking change, and so you need to know about their internals to know if they’re okay to call.
The author has a very strange understanding of the idea of a "leaky abstraction". AppKit requiring you to call methods on the main thread is just not an abstraction at all, leaky or otherwise.
I can't buy this premise. The leaky abstraction is that your selected libraries are attempting to be as efficient and as fast as possible while also trying to be fully general. As a result you get the full abstraction layer necessary to do this, which, as you've noted, for any one project is too much.
The complication you seem to be experiencing is other projects have come to rely on these libraries and their particular abstractions and so you've actually got a dependency management problem due to the design of the "core" libraries here or due to a lack of language features to /constrain/ all of them. A critical point of abstraction totally lacking in this ecosystem.
In any case if you look at any language that has contexts that can be externally canceled you should realize this actual solution must be implemented at a higher level than these "async primitives."
Articles like this remind me how Kotlin is probably 10 times easier and concise than Rust and C++.
Kind of off topic but has anyone noticed difference in code quality and documentation from projects in C++ / Rust / Kotlin / JavaScript libraries that have the same features?
Recently was working with Seastar which is Tokio / Kotlin Coroutine equivalent in C++ and seems crazy how far behind it is from Kotlin Coroutines. The documentation the example, the boiler plate etc. Don't think it is the language themselves but the language culture. Seems like C++ libraries are just designed to be more difficult. Less documentation, no comments, flat folder structures with each file having with 1000s of lines of code. They expect you to go read through source code etc.
There's a conflation going on here which is a blocker, shall we say, for understanding the two sides of this coin. The antonym of async is not blocking, it is synchronous. The antonym of blocking I/O is evented I/O.
Stackless coroutines à la Rust's async are a perfectly reasonable control flow primitive to use in 'blocking' contexts, which properly aren't blocking at all since they aren't doing I/O, I'm thinking of generator patterns here.
It's also entirely possible, and not even difficult, to use evented I/O in synchronous function-calls-function style code. You poll. That's the whole deal, somewhere in the tick, you check for new data, if it's there, you eat it. This does not require async.
I found the "what if the real abstraction leak was the friends we made along the way" portion of the argument. Unconvincing. When people complain about async leaking, what they mean is that they would like to be able to use libraries without have to start a dummy tokio in the background. That's what's getting peas in the mashed potatoes, that's what annoys people.
"Abstraction leak" is a sort of idiomatic way to complain about this. Focusing on the slogan does nothing to address the substance, and "have you considered... that, a function, might take a while, and perhaps that, that is an 'abstraction leak'? eh, eh?" is. Well it probably seemed clever when it was being drafted.
The author must have specified in the title: "Blocking code IN RUST is a leaky abstraction".
I am so happy to be using Go for most programs!
It has one of the best concurrency implementations that I have ever seen (Erlang permitting ...). And for most of my programs, the garbage collection is not a problem. It's the opposite: simplifies programming incredibly.
No blocking versus async problems. No libraries for async are needed. Just concurrency (which has its own problems, but this is another story ...).
> Dependency Dog: If you want to see a good example of a leaky abstraction, consider AppKit. Not only is AppKit thread-unsafe to the point where many functions can only safely be called on the main() thread, it forces you into Apple’s terrifying Objective-C model. Basically any program that wants to have working GUI on macOS now needs to interface in Apple’s way, with basically no alternatives.
Uhm, every OS that I've developed a GUI for has a similar limitation. The "only call this thing from this thread" is a well-known thread safety pattern. It might not be the pattern that this author prefers, but it is an industry standard pattern for UI.
I agree with your comment. Every UI framework I have used (AppKit, Swing, JavaFX, etc.) is not thread-safe. It requires all UI code in the main thread or, in the case of JavaFX, a dedicated UI thread (not necessarily the main thread, but it must be a single thread where all UI operations occur).
Even outside of UI, use-from-one-thread-only is perfectly valid and acceptable. The issue at that point is with the tools (e.g. C), not the API. If the API provides thread-safe entry points, that's because it is taking on the responsibility of synchronizing. Maybe that's fine for a particular API, but it shouldn't be expected for everything. It's an added layer of complexity, hidden from the programmer, so expect it to be absent in more optimized cases.
Technically in JavaFX you can work with UI objects off the main thread, for instance to load or build up UI in parallel. The single thread requirement only kicks in when UI is attached to a live window (stage).
IBM made a similar decision before the larger Java community thought it was anything like a good idea.
WPF was the same IIRC
That's how winforms did it; and I suspect that's how the underlying Windows OS needs it done too.
yup!
Asynchronous code is the bees knees
Async/await is a horrible fit for Rust.
> Some of this criticism is valid. async code is a little hard to wrap your head around, but that’s true with many other concepts in Rust, like the borrow checker and the weekly blood sacrifices. Many popular async libraries are explicitly tied to heavyweight crates like tokio and futures, which aren’t good picks for many types of programs. There are also a lot of language features that need to be released for async to be used without an annoying amount of Boxing dynamic objects.
Yes. So don't do it
It is entirely possible to write asynchronous code without using asyc/await.
async/await is a fun paradigm for garbage collected systems. It is a horrible mess in Rust
It's the only real solution if you need tens of thousands of concurrent tasks. Real threads use too much memory.
That is a false dichotomy.
There are other options than async/await and threads.
In my heart that is my main objection to async/await in Rust. It has "poisoned the well" for asynchronous programming. Where people think it is async/await or threads
We have been doing asynchronous programming for many decades.
Or you use something like goroutines or Elixir processes where green threads are preemptively scheduled.
I posit that while more programmer friendly async abstractions (such as in Go) are good in the short-term, ideally async should be far less scary even in the lower levels. Programmers should shy away from async not because it is hard, but when they have carefully considered if the architecture can afford to be sync instead. If they aren't doing async, it's likely that they rely on other code that should be handling async well.
Alan Kay said (paraphrased roughly) that programmers use multithreading when they don't understand how to write programs as the state machines that they are. Then you get all the surprise and suspicion about distributed locking (featured on HN today!) and whatnot. There is much essential complexity to be tackled here. If robust software is a priority (it really should be), we need to be deliberate in whether async is exposed at a given layer of abstraction. We shouldn't be awed into writing subpar programs.
Was Kay thinking of local code or distributed code? For local I can see his point. But once we start talking to networks or even really hard drives you need scatter-gather semantics. Otherwise you’re running lots of threads, which undercuts his point, or you’re writing your own multitasking state machine which is just a poor implementation of concurrency where you’re responsible for all the bugs and deadlocks, not just some of them.
(I should caveat what I'm about to say that I'm primarily concerned about writing robust and highly performant programs, and while I believe it should be a focus broadly, it's a practical niche.)
That's the thing, though. It's arguably even more important for distributed code. If we abstract away the state machine too much, it becomes difficult to reason about the code precisely because of abstraction. The complexity that was present explicitly in the state machine will just cause confusing behavior in the abstracted version. Using lightweight threads or another high level abstraction that approximates blocking code will allow getting a program out faster, but lower quality at that. Two examples to illustrate my point: first, you mention scatter-gather, but the base concept is orthogonal to sync/async. However, I/O is characteristically async, and therefore the underlying mechanisms are async anyways. Second, io_uring is showing that async can be good for performance while not being a difficult interface.
Sync code makes things easier to reason about but also kinda not. I think the big issues with async are that the OS hasn't done a good job allocating responsibility of async interfaces and the fundamental difficulty. The former makes async seem less efficient than it could be, which is true in that a sync-over-async interface is better than an async-over-sync-over-async interface, but we should have async interfaces accessible. The latter probably feeds into a bias to not even touch async where mixing async and sync would be the best blend of performance and programmability.
I was debating writing a blog post like "Blocking IO Considered Harmful" or something cheeky like that.
I'm not as familiar with Rust or C as other people, but I've done a lot of Java, and JDBC has made my life more difficult than it needs to be due to its semantics. It has been hard for me to get decent thread utilization with it because of all the blocking it does behind the scenes.
I've moved most of my SQL stuff to Vert.x SQL, which has given me better performance overall, and I can do most of it with a single thread.
At least in Java land, it's trivial to convert a non-blocking call into a blocking one (in Vert.x you can do myFuture.toCompletionStage().toCompletableFuture().get()), but it's pretty hard to efficiently go the other direction. I really wish JDBC would create something like "executeAsync" that returns a CompletableFuture or something similar.
In defense of Java blocking calls, in the 1990s to the 2010's the industry mostly considered Asynchronous programming as coordinating multiple threads over know yield points - such as using atomic locks, or mutexs. The main push by the industry was all about abstractions and OOP, and moving away from static C/C++ programming languages to dynamic programming languages such as Java/Python/Visual Basic or C#.
It is only until recently with the popularity of Node.js, Python and the explosion of the web that the developer have to deal with single threaded languages that blocking calls are a significant impact to the users experience. This push for Async in the web has had a cascade effect to system programming languages where old time developers are only first learning about it.
Could you clarify the "static C/C++" vs "dynamic Java" distinction? I'm used to thinking of "static" and "dynamic" in the context of type systems, but do the terms have overloaded meanings in concurrency/parallelism?
The push for async as we know it started in .NET, initially with F# and then popularized by C#, with the pattern then adopted by JS, Swift, Python, Rust and other languages. The implementations, however, are wildly different.
I thought F# was a derivative of OCamel that focused exclusively on immutable data structure and functional programming, instead of mutable/imperative languages.
https://www.microsoft.com/en-us/research/publication/the-f-a...
It was also pushed by the C10K movement.
True, one great thing that did come out from it is that the language needs to natively support asynchronous programming with syntactical sugar otherwise you get an explosion of libraries/frameworks that solve the problem with their own problems. How-ever this is a chicken/egg problem, did everyone really need async or did it come about because the proliferation of frameworks to solve this common pattern?
I can’t use the Vert.x SQL lib for MSSQL because named instances and Kerberos auth aren’t supported, so for a good deal of web calls I use the MSSQL JDBC inside executeBlocking Vert.x blocks.
However, recently I was thinking of utilizing a Vert.x worker verticle with a pool of Java virtual threads that could serve my JDBC calls. Has anyone tried anything like this?
> I really wish JDBC would create something like "executeAsync" that returns a CompletableFuture or something similar.
Relevant: https://github.com/jasync-sql/jasync-sql#getting-started
Yeah, vert.x SQL is similar, which is what I am using.
Does your issue (which can be solved with JDBC creating executeAsync that returns a CompletableFuture) go away with virtual threads?
Probably for someone competent, though I had trouble getting the performance I wanted out of that as well when I tried it.
I’m not sure if my issues come from the use of blocking APIs at the lower level, or if I’m just doing something dumb. Neither would surprise me.
It seems like the magic for the author is in the annotation. There's still two kinds of functions "fearlessly run in the UI thread" and "must be run in a worker" and async is great in that it greatly expands the number of functions in that first pool but it's effectively the same thing as the [blocking] annotation but the other way and not totally complete. Because there are async-safe but not async functions.
So to me it seems like it all leaks because in all cases what's desired is some external signal about how the function behaves. A "badly written" async function can stall the loop and that isn't conveyed only by the async tag. It's rare to do this but you can and there might even be times where this makes sense[1]. But it happens that by convention async and ui-thread-safe line up but there's a secret other axis that async is getting mixed up in.
[1] Say running a separate event loop in a background thread used for CPU intensive tasks that occasionally perform IO and you want to make the most use of your thread's time slice.
This is some bullshit. And it doesn't help that everyone is using async to mean different things.
Blocking/synchronous system calls are not friendly to userspace async runtimes. Explicit async runtimes are a pain in the ass. The best solution imo is a blocking userspace interface that maps into the system however it likes. Go and Haskell do this. Erlang and probably all BEAM languages do too. Nim and Zig let you choose.
Runtimes like Go and Haskell can map onto code that makes blocking system calls for compatibility reasons when needed, through explicit request to the runtime, or detecting that a green thread has blocked.
This is the right answer! Preemptively scheduled n:m green threads >>>> any async design.
I’ve always thought of async await as an ergonomic relaxation (as in constraints) of green threads. People had more problems doing green threads right than I’ve seen with async. Do some of the remaining problems look exactly like green thread problems? Absolutely. But it’s a step toward something more hygienic.
I realise I'm picking one sentence out of the middle of this, but this is the one thing I don't feel I understand about async:
"But well-formed Futures complete quickly"
Does this mean people writing async code don't ever write code which takes a while to compute?
I write lots of code which takes 10s of milliseconds, or 100s of milliseconds, or hours. And none of these feel like a good fit for async, as even a 10s of milliseconds piece of work would be enough to start to limit throughput and add unacceptable lag to other connections?
This is fine in a multi-threaded server, one thread per connection, you make a difficult request, the server spins for half a second before responding to you -- but it doesn't really fit in a async program?
The same problem comes up at the end with the proposed `#[blocking]` solution. There is no formal definition of blocking outside of async. It's a colloquial term meaning "takes a relatively longer time to finish". Async is the formalization of making a distinction between grinding the cpu and yielding until something else wakes you up and tells you to keep going.
So I guess a well formed Future always adds yield points between hot loops? And then you just trust the runtime to manage execution context such that your yields are noops if there's no contention for execution time and/or to avoid needless context switching.
This sounds like the same kind of problem the Linux kernel is trying to get rid of, and we used to live with back in Windows 3. It’s very hard to put in enough yield points to never block for too long in a complex algorithm, and I was so happy when I stopped having to do that myself!
Definitely not going to solve a c10k problem with 10ms pauses, that’s for sure.
True, but some problems just require that long to get the answer the user wants!
Some problems take 60 CPU years, but those you don’t expect the user to actively wait while you distribute their task and wait for the results, you send them an email in 3 months or so! :)
What I like about async and promises is there’s just enough bookkeeping to keep people from violating Keenighan’s law by too wide a margin. A little goes a long way, especially when any process could be handling several requests at once and make better use of the dead space between steps.
But a friend recently showed me how coroutines work in Kotlin and from my rather extensive experience helping people debug code that’s exceeded their Kernighan threahold, this seems a lot less footgun than nodejs. It’s far too easy to inject unhandled promise rejection in Nodejs. I think await should be the default not the exception, and Kotlin is closer to that.
How do they work in Kotlin? If it’s just:
Then that creates an invisible yield point which messes with your “is this a critical section” sense. Suddenly anything may yield and your naive turns into the concurrency hell wrt pt’s consistency.I think await should be the default not the exception, and Kotlin is closer to that.
I somewhat agree in that it should be an await/nowait keyword pair required at async calls. Not realistic though even in typescript, considering that Promise{} is just a continuation shell that may be passed around freely and doesn’t represent a specific function’s asynchronicity.
The biggest problem with the Node/Javascript solution is that if I call a function and don’t need the result, if that function is changed to async (or I missed that it always was) end up with bugs because we didn’t await the result.
When I say things like this people typically claim I’m bringing up some hypothetical, so let me be clear: I’m tired of fixing this bug. Not in my code (mostly), but in the code base. It happens too often both in a numerical sense and in a sense that it’s so stupid that it happens at all. It’s how I started thinking about this problem in the first place. The solution we have isn’t working very well.
All abstractions are leaky, but some are useful. You can't have a perfect abstraction. Your abstraction is always a compromise of various goals you want to achieve.
> Blocking code is a leaky abstraction
> Asynchronous code does not require the rest of your code to be asynchronous. I can’t say the same for blocking code.
Well, you can also say that "file systems are a leaky abstractions. Sector-addressing disk access does not require the rest of your code to access disk sectors directly. I can't say the same for the code that uses files".
First of all, asynchronous code does require the rest of your code to be asynchronous: because, as you've yourself said, blocking code doesn't mesh well with the asynchronous. Second, when your are trying to work with two different levels of abstractions at the same time, of course you will find the presence of the higher-level abstraction (the blocking code, the file system, etc) to be inconvenient: the higher-level abstraction has to make some opinionated choices, or it would not be an abstraction but merely an interface.
So yeah, I guess blocking code is a leaky abstraction, just like file systems, TCP streams, garbage-collected memory, and sequential processor instruction execution. It's still a higher-level abstraction though.
Async/await has been a game-changer. You can tell that this is the case by how much shorter and more reliable code with async/await is compared to alternatives (e.g. which rely on callbacks).
Callbacks are a leaky abstraction IMO because they force you to write a lot of additional logic to handle results in the correct sequence. Obtaining the result of multiple long-running function calls on the same line is insanely messy with callbacks.
I've found that async/await helps to write code which stays close to the business domain. You can design it with multiple layers which take care of a distinct level of abstraction so you can understand what's happening just by reading the main file top to bottom. You don't have to jump around many files to read the functions to understand what they are doing and how their results are combined.
Async/await makes the concurrency aspect as succinct as possible.
That said, I've seen many people writing code with async/await as though they were writing using callbacks... Some people haven't adapted correctly to it. If your async/await code doesn't look radically different from how you used to write code, then you're probably doing something seriously wrong. Ensure that you familiarize yourself with the async/await utility functions of your language like (in JS) Promise.all, Promise.race... Notice how try-catch works with the await keyword...
> compared to alternatives (e.g. which rely on callbacks).
This is a browser-centric view, which makes no sense in the context of Rust (which runs on a proper OS with threads and absolutely no need for callbacks).
I work a lot with Node.js on the server too.
Really frustrating that Rust became infected with the async disease. This article is pure nonsense fwiw. It says: you can always block on the Future returned by an async function. Ugh. Ok, but let's just not have any async at all please, and then there's be no need to add that code and no behind the scenes context switching done.
Async usage is overblown in some cases, but not in others. The mistake of async Rust is not that it's unnecessary, just that it's being pushed for general purpose usage. Which is probably impractical for any async abstraction, or for any abstraction in general.
block_on and spawn_blocking are tokio's async abstractions, there is no "blocking code's abstraction" that can leak.
What the article intentionally omits is that I can defer io by getting a handler and polling it in a select loop. And right after that, I can also poll my mpsc queue if I want to. This is how "blocking code" has worked for the last few decades.
> Asynchronous code does not require the rest of your code to be asynchronous.
Seems like in many languages the async keyword is viral. And that is the most pointless work I've ever done, where I've to go around and mark everything async just because of one async function. So this is a strange statement to me.
Does the author not know what a "leaky abstraction" is? The first time they referred to memory leaks I thought is a clever joke nested in a jab a hypothetical coder who didn't know, but then in the end they double down in a clearly non-joking way. Then even make a chart showing that calling a normal function from an async function is trivial, that is the opposite of a leaky abstraction.
An abstraction is a "leaky abstraction" when it forces some external condition or behavior onto code (ex: calling code that uses the network doesn't work when the network is disconnected). In this case the hurdles of calling async code from normal code, and this author tries to hand waive that with talk of dependencies.
There are real benefit and drawbacks to async code and styles. Even if I don't think it is a good abstraction I want to discuss it on the real engineering terms of its actual merits and drawbacks. This article is talking past the engineering decisions and seems like it didn't mean to.
maybe I am just being a pedantic jerk.
What I think this comes down to, is that in Rust:
It is easy but dangerous to call a non-async function from an async function (since the non-async function could block, forcing all your async code to block on this function).
It is hard but safe to call an async function from a non-async function.
Note that this is language specific; if Rust had “async”, “pure”, and “blocking” as three different function colors, then we would have a very different design. Or, for example, in JavaScript:
It is easy and safe to call a non-async function from an async function, since JavaScript exposes no blocking primitives.
It is impossible to call an async function from a non-async function, since there is no way to block on the result.
Or just use fibers and avoid icky futures.
Doesn’t change the core issue, which is that if you want to do multiple operations simultaneously that take time to execute, you’re going to need to write stuff to handle that which you wouldn’t otherwise. Whether it’s channel piping or async awaiting is semantics.
It's not just semantics. With synchronous code, the blocking aspects are invisible in when reading code:
With some async code, the fact that there will be a wait and then a continuation is visible from just reading the code.Is having the 'await' keyword really necessary?
What if every time I call an async function the interpreter would automatically "await" for its result?
Perhaps (?) it wouldn't be an issue for correctness in most languages, but it would certainly add a performance and debuggability cost. It could be useful in some niche domains, probably.
This is the opposite in JavaScript (and similar single-threaded languages) where sync code is atomic and normally can't do I/O.
You can perform file I/O in JavaScript synchronously. The localStorage API in the browser is synchronous (and in Node via the --experimental-webstorage option), and of course requiring a CommonJS module is also synchronous (and there are many other sync filesystem APIs in Node as a sibling comment pointed out).
You just can't perform network I/O synchronously. Although a network attached file system allows for both network and file I/O technically, but that's a really pedantic point.
> You just can't perform network I/O synchronously.
Sure you can, you just shouldn't ever do it because it blocks the UI: https://developer.mozilla.org/en-US/docs/Web/API/XMLHttpRequ...
Yeah I should've said there's no Node API for making synchronous HTTP requests (unless you count executing a child process synchronously). Even the older http.request API used in Node prior to the introduction of fetch is async and accepted a callback. Browsers have all sorts of deprecated foot guns though (like the synchronous mode of XMLHttpRequest).
node.js has a whole plethora of synchronous IO methods. they're actually really nice. or have I misunderstood your meaning?
Node uses libuv[1] for all its IO operations[2]. The sync versions still use libuv, but block the main thread until they receive the result.
[1] https://libuv.org [2] https://nodejs.org/en/learn/asynchronous-work/overview-of-bl...
They mean a viral abstraction.
> Frankly, I don’t think async code is leaky at all, and the ways that it does leak are largely due to library problems.
Huh? One could argue the exact same in opposite direction - blocking code isn't leaky at all, just wrap it in a Future. And if doing that is difficult because of !Send data, then those are just "library problems". You could always just use a threadsafe library instead, right?
I am surprised by some of the comments claiming to find it very hard to make an async API out of a sync one.
I write a hobby kernel with a profoundly asynchronous I/O system. Since it's nice to have synchronous I/O in some places (reading in data in response to a page fault, for example, or providing the POSIX read/write functions), I have to turn it into synchronous I/O in some cases.
Asynchronous I/O notifies its completion in a configurable-per-I/O-packet (called "IOP") way. The default is to signal an event object. You can wait on an event object and when it's signalled, you're woken up. The result is that synchronously sending an IOP is hardly any more work than:
I would be surprised to hear of any system where you can't do similar. Surely any environment has at least semaphores available, and you can just have the async I/O completion signal that semaphore when it's done?I don't think the problem comes so much from async v. sync, it comes from the implementation language/framework.
If you were to wait_for_event (or its equivalent) in the node main thread you will stall the 1 thread handling requests.
This whole problem is not just async/sync, it is the environment you live in.
What we should be questioning is why high level languages in high level frameworks make the programmer even consider the distinction at all.
If I am handling an http request, the async nature of this is an optimization because threads (or worse, processes, I once easily dos'ed an apache system that created a new process for each request) are too expensive at scale.
It should not concern the programmer whether a thread is launched for each request or we are using cooperative multitasking (async). The programmer/application needs that information from that call before that one piece can move forward.
The lower level async-as-state-machine (or coroutine or whatever) should be more standard, but that's an argument for another day. For now, you're right and the opaque threading model is basically what Go does. On the other hand, it's not suitable for Rust because in some ways Rust is lower level. C# went and did it the Rust way, which I personally appreciate but I understand fits into the problem you're describing.
just document your api's thread safety and let me handle it. don't make me deal with this async/await bullshit. multithreaded programming is hard enough as it is.
Godël covers how it’s leaky abstraction all the way down. If you consider a specific chunk of useful async code it still doesn’t compose enough of system to do much of use; it can’t answer many questions alone or any about itself. We never just have Map() but other functions. Its constant composition of symbolic logic we vaguely agree means something relative to an electrical system of specific properties.
A useful system is “blocked” on continuous development of that system.
These posts are too specific to code, miss the forest for a tree. A more generalized pedantry (like electrical state of systems rather than dozens of syntax sugars to simulate the same old Turing machine) and more frequent spot checks for stability and correctness relative to the ground truth would be great.
Way too much circumlocution around code. Map is not the terrain. I for one am having blast training a state generator that acts like a REPL and (for example) infers how to generate Firefox from a model of its code (among other things).
Excited about the potential to peel away layers of abstraction by generalizing the code with automated assistants.