I was a bit nervous about this trend when it started picking up because I care about Pyodide (Python running via WebAssembly) and libraries that use PyO3 might not work in Pyodide.
Thankfully that's now been mostly solved - you can compile and publish WASM builds of Rust or C extensions on PyPI now and a Pyodide can then use them.
I'm kind of surprised to hear that was a concern because my perception is that PyO3 is more analogous to wrapping native C code in Python libraries (which is pretty well established since long before either Rust or WASM), and Pyodide seems more like something for use in browsers rather on the server-side. When I looked into this earlier this year, trying to run Python via WASM on the server side with Pyodide had a bunch of hoops I needed to jump through, and it ended up being a lot simpler to use a build of Python from one of the core Python contributors that directly targeted WASM.
This is a nice party trick! I use it to provide easy installation of software that is written in rust, but is primarily used by Python users. (e.g.: Biology tools) They can do `pip install <name>`, since some people prefer this over downloading executables/installers. Bonus: Maturin/PyO3 can build "manylinux" binaries automatically, which helps with the Linux ABI diaspora.
But does it work everywhere Python works? My main issue with this Rust move has always been compatibility. Python can be embedded and ran in a heck of a lot of places. What's the story when libraries that I may want to depend on are actually implemented in Rust and my target doesn't/can't handle the toolchain and there's no build target?
Python doesn't even work everywhere Python works. I am really starting to like and lean towards languages that compile to a binary instead of scripting languages like Python, Ruby, PHP.
It's nice that, as long as it's compiled for the correct architecture, I don't have to install the language to run the application.
It won't work in those cases. Do you have explicit examples? I suspect this won't come up much as Rust has, IME, much broader target support. (Including platforms without a GPOS)
I can't come up with any cases where cpython would compile that rustc can't target. There are some more niche interpreters that can run on embedded stuff with no rustc support, but they are also subsets of python so it isn't like you can just pull a wheel file and expect it to work
It does, and rustc nostd already supports a lot of places cpython can't run, but the other way around is just non existent. Rustc can't and will likely never support ti dsps because they own the compiler and a long is 48bits. Dreamcast and other things with weird data sizes also are out.
They’re plugging away at it. A major milestone was getting unwinding to work. I’d say they’re getting quite close and it’s getting better and better. It’s usable for Linux kernel compilation I believe which unlocks Rust to be used for the non-driver components in the kernel.
I'm using Flet for a major project, which includes iroh, and it's been hellish going because the only way was to use the Kotlin build for mobile support, where I've had to be patching things all over to get something reasonable. Burned a lot of time on it and it still occasionally breaks, while the Python parts have been generally solid. Initially I looked into converting the Rust to Python, but that's a non-starter because of the semantics.
The Python cryptography library switched out C for Rust which invalidated some platforms[0]. 2021 story, so not sure if anything has changed.
...Since the Gentoo Portage package manager indirectly depends on cryptography, ""we will probably have to entirely drop support for architectures that are not supported by Rust"". He listed five architectures that are not supported by upstream Rust (alpha, hppa, ia64, m68k, and s390) and an additional five that are supported but do not have Gentoo Rust packages (mips, 32-bit ppc, sparc, s390x, and riscv).
It's too "clunky" for AI, like you stumble too much over the words when reading
If we take the first sentence that could have been from a German person, where such language structures aren't that uncommon.
I myself write a lot like that, and always have written like that, often in way worse ways, like sentences which never, not even after docent of words, end, and have levels of sub-sentence nesting no normal thinking human would want to read. (<- yes that was intentional. I tend to write more concise today.)
Through there are indicators of AI usage in pretty much every single sentence.
But at the same time the writing is too bad, for most current AI agents.
Like reading it is hard, in ways AI tends to not mess up. There are missing filler and glue words. Including some cases where I think they are probably grammatically require (through school was too long ago, so idk.). Sometimes it looks like a different language construct was used (like bracket or -- hyphen inlays) which then where search+replaced with commas.
So I really wonder what that is?
- a lower quality LLM Model?
- AI written text modified by hand leading to it having the worst of both worlds?
- or a human who has self learned English as a second language using AI and believes this is how normal English speakers write ...
and the last option is kinda frightening, because sooner or later we will have exactly that: People who write like AI "but worse" and likely do not even realize what they are doing.
---
EDIT/PS: Side note; Rust in Python is a decent match. PyO3 is far from perfect but decently usable. Spreading more awareness about it is nice. This comment is only about the writing structure not the content of the blog post.
In practice it is easier to support rust packages on web assembly than C or C++ packages since Rust cross compilation is in better shape. Of course packages that do IO can be a challenge but problematic IO is also more common in C libraries than Rust libraries.
Tokio doesn't work yet but epoll-based support for it is in progress.
It’s time to start figuring out how to block domains on hacker news. This slop is getting out of control. Who can read this? The first few sentences already induced a migraine.
They’re not running Rust they’re running machine code compiled with the Rust compiler. Like many other Python libraries such as numpy are running machine code compiled with C++, C or Fortran. That was always the case for Python and is its main selling point, it’s a slow but easy to write glue language that can make faster native code scriptable.
Also the article reeks of AI slop, it’s just trying to sell you a Rust course.
One main difference is that pyo3 is safer and easier to use to build said modules and have them work correctly without issues. Thus modules which hadn’t been worth it before to build in this manner suddenly become so
There’s a whole legacy of wrapper interface generators like SWIG that are way more powerful than PyO3, QT is e.g. fully mapped to Python using a very powerful wrapper generator (SIP). PyO3 is just a really simple interface generator in comparison, look at what SWIG can do, or SIP, what PyO3 does is trivial in comparison.
I was a bit nervous about this trend when it started picking up because I care about Pyodide (Python running via WebAssembly) and libraries that use PyO3 might not work in Pyodide.
Thankfully that's now been mostly solved - you can compile and publish WASM builds of Rust or C extensions on PyPI now and a Pyodide can then use them.
Here's the WASM build of the Rust-including Pydantic-core package for example: https://pypi.org/project/pydantic_core/#pydantic_core-2.49.0...
It wasn't all that easy to make it work: https://blog.pyodide.org/posts/rust-pyo3-support-in-pyodide/
I'm kind of surprised to hear that was a concern because my perception is that PyO3 is more analogous to wrapping native C code in Python libraries (which is pretty well established since long before either Rust or WASM), and Pyodide seems more like something for use in browsers rather on the server-side. When I looked into this earlier this year, trying to run Python via WASM on the server side with Pyodide had a bunch of hoops I needed to jump through, and it ended up being a lot simpler to use a build of Python from one of the core Python contributors that directly targeted WASM.
Yes, Pyodide is almost entirely used in web browsers. I want to be able to run existing code that uses libraries like Pyodide-core in a browser.
This is a nice party trick! I use it to provide easy installation of software that is written in rust, but is primarily used by Python users. (e.g.: Biology tools) They can do `pip install <name>`, since some people prefer this over downloading executables/installers. Bonus: Maturin/PyO3 can build "manylinux" binaries automatically, which helps with the Linux ABI diaspora.
`pip install zig` works today.
But does it work everywhere Python works? My main issue with this Rust move has always been compatibility. Python can be embedded and ran in a heck of a lot of places. What's the story when libraries that I may want to depend on are actually implemented in Rust and my target doesn't/can't handle the toolchain and there's no build target?
Python doesn't even work everywhere Python works. I am really starting to like and lean towards languages that compile to a binary instead of scripting languages like Python, Ruby, PHP.
It's nice that, as long as it's compiled for the correct architecture, I don't have to install the language to run the application.
It won't work in those cases. Do you have explicit examples? I suspect this won't come up much as Rust has, IME, much broader target support. (Including platforms without a GPOS)
I can't come up with any cases where cpython would compile that rustc can't target. There are some more niche interpreters that can run on embedded stuff with no rustc support, but they are also subsets of python so it isn't like you can just pull a wheel file and expect it to work
For what it’s worth the GCC backend for rustc is making progress and rustc more generally supports a lot of embedded stuff already
It does, and rustc nostd already supports a lot of places cpython can't run, but the other way around is just non existent. Rustc can't and will likely never support ti dsps because they own the compiler and a long is 48bits. Dreamcast and other things with weird data sizes also are out.
How’s that GCC backend coming along? I could look it up, but I bet others would like to hear about it too.
https://blog.antoyo.xyz/rustc_codegen_gcc-progress-report-43
They’re plugging away at it. A major milestone was getting unwinding to work. I’d say they’re getting quite close and it’s getting better and better. It’s usable for Linux kernel compilation I believe which unlocks Rust to be used for the non-driver components in the kernel.
This is so cool. That's going to be amazing when it lands.
It’s already in rustup for what it’s worth. Not sure what “when it lands” means but I’m sure it’s an ever moving goalpost.
I'm using Flet for a major project, which includes iroh, and it's been hellish going because the only way was to use the Kotlin build for mobile support, where I've had to be patching things all over to get something reasonable. Burned a lot of time on it and it still occasionally breaks, while the Python parts have been generally solid. Initially I looked into converting the Rust to Python, but that's a non-starter because of the semantics.
-flet.dev
The Python cryptography library switched out C for Rust which invalidated some platforms[0]. 2021 story, so not sure if anything has changed.
[0] https://news.ycombinator.com/item?id=26097153> reach for, a Rust extension does the work
ai; dr, sorry
it probably is AI written
but it's a bit strange
It's too "clunky" for AI, like you stumble too much over the words when reading
If we take the first sentence that could have been from a German person, where such language structures aren't that uncommon.
I myself write a lot like that, and always have written like that, often in way worse ways, like sentences which never, not even after docent of words, end, and have levels of sub-sentence nesting no normal thinking human would want to read. (<- yes that was intentional. I tend to write more concise today.)
Through there are indicators of AI usage in pretty much every single sentence.
But at the same time the writing is too bad, for most current AI agents.
Like reading it is hard, in ways AI tends to not mess up. There are missing filler and glue words. Including some cases where I think they are probably grammatically require (through school was too long ago, so idk.). Sometimes it looks like a different language construct was used (like bracket or -- hyphen inlays) which then where search+replaced with commas.
So I really wonder what that is?
- a lower quality LLM Model?
- AI written text modified by hand leading to it having the worst of both worlds?
- or a human who has self learned English as a second language using AI and believes this is how normal English speakers write ...
and the last option is kinda frightening, because sooner or later we will have exactly that: People who write like AI "but worse" and likely do not even realize what they are doing.
---
EDIT/PS: Side note; Rust in Python is a decent match. PyO3 is far from perfect but decently usable. Spreading more awareness about it is nice. This comment is only about the writing structure not the content of the blog post.
How much of the existing PyPI ecosystem do you think could realistically work this way without package authors doing anything specifically for WASM?
In practice it is easier to support rust packages on web assembly than C or C++ packages since Rust cross compilation is in better shape. Of course packages that do IO can be a challenge but problematic IO is also more common in C libraries than Rust libraries.
Tokio doesn't work yet but epoll-based support for it is in progress.
It’s time to start figuring out how to block domains on hacker news. This slop is getting out of control. Who can read this? The first few sentences already induced a migraine.
Wait til you see the first diagram
They’re not running Rust they’re running machine code compiled with the Rust compiler. Like many other Python libraries such as numpy are running machine code compiled with C++, C or Fortran. That was always the case for Python and is its main selling point, it’s a slow but easy to write glue language that can make faster native code scriptable.
Also the article reeks of AI slop, it’s just trying to sell you a Rust course.
One main difference is that pyo3 is safer and easier to use to build said modules and have them work correctly without issues. Thus modules which hadn’t been worth it before to build in this manner suddenly become so
There’s a whole legacy of wrapper interface generators like SWIG that are way more powerful than PyO3, QT is e.g. fully mapped to Python using a very powerful wrapper generator (SIP). PyO3 is just a really simple interface generator in comparison, look at what SWIG can do, or SIP, what PyO3 does is trivial in comparison.
Why not just run C inside of Python?
For the same reasons someone might prefer Rust to C in general; safety, ergonomics, etc. If you want to write C in Python, nothing is stopping you.
You can do that too. I generally prefer rust as it's a nice lang to work with.
yeah, Rust is eating C/C++...
Because you want to use other existing libraries that are written in Rust.
This is for WASM, but you can also do it to use a Qt frontend via Python's PySide6 with a Rust backend, avoiding the need of Rust bindings for Qt.