Excellent writeup, thanks for posting. I was involved with efforts to achieve similar things in Windows (Direct Composition, etc.). In practice, it was enlightening to debug various apps and frameworks and discover just how long repainting can take and how many edge cases there are around slow apps, hung apps, and so on. Who knows, maybe someone there still has some of the malicious test apps I wrote (using the language of this article, apps which acknowledge the sync request and then take multiple seconds to actually paint, etc., etc.).
It's definitely a hard problem space that's basically incompatible with how most GUI apps and frameworks worked from 1985-2015.
>> malicious test apps I wrote (using the language of this article, apps which acknowledge the sync request and then take multiple seconds to actually paint, etc., etc.)
This is one area where Wayland should do better. It's the compositors job to resize windows and IMHO draw their decorations. The compositor has a frame to paint and it can't have to wait on applications. Unfortunately some toolkits and DEs have decided that client side decorations should still be a thing.
Also with this new partitioning of responsibility is my main annoyance: The DE needs to remember window placement for all my apps, since they are not allowed to know their environment under Wayland (for good reasons).
>The compositor has a frame to paint and it can't have to wait on applications.
This is the source of the visual artifacts this article is trying to prevent, however. Sure, you probably don't want to block resize for multiple seconds, but in general the compositor is very responsive. The app may not be. If you just let the window chrome resize as fast as you get, what do you do with the rest of the window? Leave it transparent? Draw a default black or white area? Both are very ugly and very noticeable in practice.
This is the part of the problem space I know the most about - for a number of years I owned window chrome on Windows (don't blame me for the 1px border and way-too-subtle shadows, but after being overruled by PM/design, yes, I was responsible for implementing them).
As far as custom decorations - that is a lost battle. Companies and apps want their special design language and will simply not build for your product/operating system/compositor/whatever if you don't give them that kind of support. Twitter and Facebook both want their specific shade of blue and their specific font. Adobe is....well, let's not talk about Adobe. Browser tabs in the title bar are a P0 requirement these days and anyone who doesn't support that will be laughed out of the room. Etc., etc.
> Also with this new partitioning of responsibility is my main annoyance: The DE needs to remember window placement for all my apps, since they are not allowed to know their environment under Wayland (for good reasons).
This might eventually get fixed, above is what I believe is the current proposal for handling this issue and letting apps do some amount of positioning without exposing things that a program shouldn't know about. That said this has had multiple proposals over the past 5-6 years at least and none have managed to make it all the way through. If you go through the previous ones (ext-placement and i forget the others) and ignore the angry messages involved it turns out that it's a very difficult problem to deal with in a way that isn't just a free-for-all with apps either not knowing about monitor placement, or having to handle so much detail about the displays that nothing will ever act consistently.
That said, recent discussion on that latest one does look promising so maybe it'll finally happen.
That isn't really what I was hoping for. IMHO it is wasteful for every application to remember its window placement and restore it. Not only does that require additional code in every app, but if there is some kind of policy change there is no way to make it consistent. In other words, at the compositor level one could use the previous position as a default and have an option to do something different if another window is already there. It's simply not the applications job to place itself on the desktop. There are probably use case for such self positioning functionality, but restoring to the last position is not one of them.
The application side of it is more for things like the multi-window setup of things like GIMP so that windows that are "docked" next to each other will stay that way past restarts. That's one of the reasons that the newer proposals are doing things with relative positioning between a zone or main window rather than allowing applications to place themselves randomly on whatever monitor or space that they want, interrupting whatever workflow is going on (which actually allows for security issues, i.e. a window pretending to be a password prompt putting itself on top of a browser or something to confuse the user). This also allows for new windows from the application to request that they're positioned next to any others so that related things stay together. This also apparently helps in a few cases where a single "application" to the user is actually multiple separate programs that get run by a main interface. Not as common in new software today but it used to be one of the ways that a lot of older software worked and there's still a decent amount out there apparently that are maintained that way.
Keep in mind that this is also a request by the application, not a requirement of the compositor to obey it. If there's not sufficient space where the application requests things then the compositor can just ignore it and do what it believes makes sense.
IMO, the neatest trick that the DWM does is avoiding jitter/lag between the mouse cursor and window when dragging a window.
Sadly, DXGI flip model seems to have reintroduced some issues -- it is difficult to avoid the kind of jank shown in this article when resizing a window drawn using flip mode presentation. Which is unfortunate, since it's the highest performing mode.
It's kinda interesting that macOS doesn't even care about this problem anymore. When moving or resizing windows on macOS the hardware mouse cursor slightly trails ahead of the window, it's not very noticeable because of the 120Hz refresh rate (meaning the framebuffer surface spends less time waiting for presentation, thus reducing rendering lag) - OTH the lag is still visible, and this sort of minor visual imperfection is quite un-Apple-like (at least for 'old Apple').
It's a bit similar to how high resolution displays make fancy subpixel font rendering tricks obsolete (e.g. high display refresh rates making tricks to hide presentation latency obsolete). Sometimes hardware progress makes complex software solutions obsolete, but usually still with one or another tradeoff.
> It's a bit similar to how high resolution displays make fancy subpixel font rendering tricks obsolete
Personally, I don't think so—subpixel font rendering still has value even in the densest displays, simply because it triples the available horizontal resolution.
Unfortunately a low of newer displays (especially OLED ones) don't have regular horizontal RGB/BGR layouts anymore. I have seen both vertical subpixel layouts as well as triangle layouts or other funky subpixel arrangements. Even if you configure/update your software to those different subpixel layouts the result is not going to be as good.
That trick is fun, but I wish Windows did a better job of synchronizing the visibility of the cursor sprite versus the presentation of the frames where the cursor is rendered in the frame. It's impossible for me to not notice the brief period where the cursor is missing!
I think it's cute. They're doing "disable hardware cursor, enable software cursor" instead of the other way around. Seeing that happen is somehow fun for me.
I think they do both simultaneously, it's just that the cursor sprite/plane is overlaid during scan out and the software cursor is rendered in the next frame to be rendered by the compositor. The cursor sprite/plane can turn on or off pretty much mid-frame, whereas the frame that's about to be rendered could be two frames away from the one that's about to be scanned out. What you actually need to do is track when the first frame with the software cursor is about to be presented, then turn off the cursor sprite/plane. I think technically speaking all of the information needed to implement this is present somewhere, but I'm sure it's quite complicated to actually implement, possibly made worse by interface layering issues :)
For me, the quickest way to tell that I'm running on Wayland is seeing if multi-lingual keyboard input is broken. I have right-alt-while-pressed set to switch to a Greek layout (because who doesn't math?), and it reliably works on X11 but breaks in all sorts of weird ways under Wayland (sometimes it simply fails to enable/disable, sometimes it shows a notification as if it changed but switches back when input actually happens, ...).
On the bright side I'm finally learning how Vim's digraphs work.
Both Xorg and Wayland compositors use libinput, so it's probably more to do with lack of decent configuration support in your DE or WM. You can always configure libinput manually. Sway just lets you add libinput configs inline inside your sway config.
[edit]
This is probably not relevant (assuming correct wayland libinput config), since this is not where mult-lingual input transformations live. libinput just handles the physical keyboard mapping and behaviour.
On more careful reading of the parent this sounds more like a buggy input method editor.. or maybe an issue with switching between X and wayland apps.
NO! my mistake, I latched onto keyboard input and didn't think about multi-lingual. I'm guessing you mean things like pinyin, I think that's a dbus thing, which is still a thing on wayland but I don't really know anything about this... I'll get my coat.
That still boils down to a Wayland design issue, since Wayland requires each individual DE/WM to get support for such features while X only needs them in Xorg.
More like do a thing and do it once. If every WM and DE needs to do it, you have some that don't, or you have issues that are handled correctly by some DEs and not others, etc.
Snark aside, it's still orthogonal to my question. I wasn't questioning whether it should be done centralized, but why it's supposed to be part of the display server protocol/compositor.
Because the "display" server is also responsible for window management and input. Does it have to be this way? Perhaps not - but seperating window management and input is also not trivial.
Note that more complex input methods do somewhat bypass the display server and communicate via dbus instead.
Did you miss the do it well part? Nearly every unix tool does more than one thing, the alternative would be a usability nightmare.
* grep, which primarily does pattern matching has dozens of file traversal related flags that could be handled by calling it from find
* find, a tool supposed to find files for some reason has dozens of flags related to executing applications which could probably be done by using xargs
* did you know that xargs can do pattern matching and replacement on its input string? there are probably dozens of unix tools that are specialized for that
X11 eventually punted the whole thing to each individual application. For input methods like Fcitx or IBus to work reliably on modern X11 programs, you need to set envvars like the following [0]:
That seems weird complaint about Wayland, Wayland requires every server to support these features, just like X11 also requires every server to support them? Sure, there weren't that many X11 server implementations so maybe it was not so apparent, but as a protocol they seem pretty much equal in this regard.
The criticism is that Wayland duplicates a lot of effort in multiple compositors. The compositor does work that happens in Xorg and in the WM. Whereas lots of common functionality gets implemented in Xorg that WMs and DEs don't need to duplicate.
It's not a weird complaint when on X11 when something is added it generally works for everyone whereas for Wayland the different implementators are still squabbling how to accomplish basic functionality people expect from their desktop computer.
Or in other words, your comparison is like saying food and shit are basically the same thing because both are made up of similar elements.
But wayland merges more components into one server than X11 did. Therefore, there is a need for more wayland server then there was a need for X11 servers.
Multilingual keyboard input is such a mess haha. I need to type a lot of Chinese using pinyin and still haven't found a reasonably polished implementation on Wayland (Sway) that works on both Xwayland and true Wayland applications. So I reverted to i3 with ibus.
Another way to check if you're running on Wayland is to start some 3D accelerated application and see if the screen is all flickery. Apparently Wayland has some issue with not generating a frame sync that causes all variety of problems, but usually lots of flicker.
Really? My laptop has my national keyboard but I switch to USA for programming because of {} [] and all the other characters that can be typed with minimal use of extra keys. My fingers know the keys and the switch combination, maybe windows + space. I didn't expect Wayland not to have such basic functionality after so many years. But frames must be perfect, no tearing (is that the word?)
It's possible that the "permanently switch to next layout" works reliably, but not the "switch while held" (which I do because I'm selecting single symbols at a time)?
Have you tried using a compose key? I lost my custom mappings when switching from X to Wayland, but they work fine in both.
I did lose my custom mappings though, but I only needed them when I was in emacs and obviously there's already a command for inserting weird stuff, so I just added a binding for it.
I use the compose key for compose stuff. It's a huge pain to actually write and install custom compositions so I don't bother anymore, just use the defaults (which is mostly annoying due to missing keypad variants, e.g. ± must be typed using the top of the keyboard).
It looks like the default en_US.UTF-8/Compose includes mappings of the form:
<dead_greek> <a> : "α"
but to use that I'd have to figure out how to map a key to `<dead_greek>`, and keyboard mappings that aren't in the standard checkboxes are such a pain.
I'm guessing that maybe I forgot to restart the machine to make sure everything got to read it. Or that whatever was broken long ago got fixed.
BTW, I can't reach my backup right now, but this seems like a good start to build up custom mappings in case anyone gets interested in this,
https://github.com/kragen/xcompose
Even with Xorg toolkit support has degraded quite a bit. For GTK it doesn't work by default and you need to set GTK_IM_MODULE=xim in your environment. Qt on the other hand picks up ~/.XCompose by default but truncates the compose result to one character.
Huge respect to any dev who goes out of their way to fix flickering/glitching/black regions when resizing or moving windows, I honestly can't stand it when I see it because I know it can be solved with effort.
Hoping this makes Xwayland Emacs much smoother to resize. It currently resizes at 0.5 fps for me on KDE.
If you're wondering why I don't use the native Wayland build of Emacs, it's because it is massively more laggy on HiDPI screens than the X11 version. I reported the issue upstream, probably won't be fixed for a decade when they'll decide to port to GTK4 and its hardware accelerated rendering backend. It took me a long while to notice that the typing lag I was experiencing was not because of native compilation or single-threading, but pgtk being a bit weak at rendering 3840x2160 pixels at 60 fps. In fact, it was not until I tried the Xwayland build that I experienced how much faster Emacs can feel.
That’s actually something you don’t want nowadays. A random app like xeyes should not be able to know mouse position at all times, for privacy reasons. (Unless you very explicitly gave xeyes super-extra permission to do that.)
We’ve long stopped living in a world where you should need to fully trust every piece software that you run as your user on your computer. And even probably trustworthy software can go bad at an update due to supply-chain attacks.
Even you don't want this, you only want it sometimes, for some apps. Which is exactly what they said.
I mean, would you like VSCode tracking your mouse movements across the entire desktop and your keypresses and then sending them off to Microsoft? Probably not, so we're all in agreement.
I don't use VSCode or other user hostile programs like it so I don't care what kind of anti-features it enables. I don't want my actually useful tools hobbled in order to deal with such programs.
xz wasn’t “user hostile”, and so weren’t countless other pieces of software affected by supply chain attacks. Nothing is hobbled if you can give it explicit permission (which you may well do on xeyes).
The days where you downloaded your software from the sunsite or tsx11.ai.mit.edu FTP servers and could be confident that it and all its dependencies were trustworthy are unfortunately gone for a very long time.
The xz-utils hack didn't care about the window system at all. It also hasn't actually caused any known damage and I'm sure if the american three letter agencies cared the perpetrator would have been dealt with by now. Project takeovers of that kind is not something regular users need to be worried about because the cost of pulling one off is too righ to waste on petty crime. It's yet another boogeyman and scaring people into giving up their computing freedom for "security" migh as well have been the goal of the operation.
> The xz-utils hack didn't care about the window system at all
It's an example, demonstrating that the mentality of "I don't use bad software" doesn't really work.
> giving up their computing freedom for "security" might as well have been the goal of the operation
How could you possibly reasonably argue that you're losing "computing freedom" because now you have the power to deny or allow applications from doing things? You're literally gaining freedom - that's not something you were able to do before. Now, you have the freedom to deny applications accessing something you don't think they need.
Did the POSIX security model (users, file permissions) also impact your “computing freedom”? Is your CPU constantly running in kernel mode? Do you have all physical memory mapped into all address spaces, and as read/write/executable? Are you allowing all software to listen on outward ports without any packet filter?
It really is something I want. But most people do live in your described world. That's the smartphone and "run's every javascript application sent to my web browser automatically" kind of computing security model. But there do exist personal behaviors of desktop operating system use where you can actually trust the applications installed and not put up walls between everything. And then things like keyboard/mouse sharing work, windows return to their places, and screen readers work. Those are all very important to me. For them I'm willing to browse with javascript temp-whitelist-only and many other such tedious things.
What I'm trying to get across is that the need for that kind of intense security model, every process a threat, is not intrinsic to modern computing.
But what’s the limiting factor of doing the sane and safe thing by default?
The most popular operating systems all do that (ios and android), and they have carved out safe APIs for all of that to work. You can’t patch up a Swiss cheese after the fact.
Is it hard to create standard APIs in a bazaar style of development? Yeah. But that doesn’t mean that it’s not the correct approach.
I think their concern is valid, it's difficult to do something which is both secure AND not limited at the same time.
Sure Android and iOS are secure but in practice they kind of suck for making anything non-standard which limits creativity and freedom.
Can we have both a secure and extendable system? Maybe but none of them exist yet. I'm really worrying that Linux mainstream distros will become like Android or iOS.
What use is a safe API when it makes the entire system impossible to use by a significant fraction of the population (ie wayland and the visually disabled)? It's been a decade+ and none of the waylands have managed to support screen readers yet.
Are we to just throw out that whole class of people and tell them, "You don't get to use linux desktop computers anymore when X11 support is dropped". As someone with retinas that are progressively tearing apart, who already uses text to speech for many things, this is incredibly disheartening. I really don't want to have to switch to the Apple ecosystem.
Let’s not fool ourselves, linux’s accessibility (and anything else) support was lackluster to begin with. Android and ios is far far superior on every count from an accessibility perspective and interestingly they have a sane security model.
This has absolutely nothing to do with the technology, it’s just that there is no standard protocol for one more thing simply because accessibility experts don’t happen to do some free work that will be de facto accepted by multiple different vendors. Comparatively, apple or google can just declare that this new API is the way to go, and support it natively from the de facto frameworks of the platform (and probably paid for accessibility experts along the way).
No one wants to use android and iOS for serious desktop work though. Like it's cool when your only interaction with the device is consuming content, definitely not for creating.
You might be living in a bubble. The majority of the population doesn’t have a PC, most people use a smartphone as their only general purpose computer. And while you may not run blender on your phone and render a full-time movie with ray-tracing there, there is absolutely no fundamental limitation, it just so happens to primarily target portable devices, not beasts of a machine with 4 video cards. This functionality requires zero special permission, neither does photoshop (of which there are multiple mobile-versions), or digital painting which, etc.
You would be surprised how many content creator gets by with a single ipad.
The security model is not the only factor, but it is one very important factor. Why do I have to open Moebius sync to keep syncthing synchronization running? Why is the whole landscape build under the assumption that a cloud storage is more trustworthy than local storage?
> But requiring user permissions for apps to do shady shit is a good thing.
> Why do I have to open Moebius sync to keep syncthing synchronization running
Because it’s a mobile OS and every single spent CPU cycle is a detriment to battery life? There is absolutely nothing in the security model that would prevent it from running - but it is essential that processes have a “structured” lifetime.
E.g. compare how much more graceful android is in low-memory situations, asking apps to serialize their state and then stopping the last used one. Linux oomkiller will just reap my whole display manager for some reason.
Sigh. Okay, but Wayland doesn't work this way because it is a Desktop software. I don't understand the complaints here - we're upset that we, the user, are empowered to give and remove permissions from applications?
Nobody is turning Linux into iOS. But iOS DOES have some good ideas. It's good, for example, that for an app to access your photos library they have to ask. I know for a fact you prefer that to the app just opening your photos without your knowledge and doing whatever they want with them.
Similarly, I see no reason why Chrome should be able to read the display output and keyboard inputs of my graphical password manager. It should ask me.
> You would be surprised how many content creator gets by with a single ipad.
Can you name one professional software developer?
Probably, you can. But I don't want to limit myself to that sub standard environment. I love my iPad for some activities, for others iOS is just impractical.
The limiting factor is all the use cases that have not been invented yet. Screen sharing would have never been a thing if we started out with Wayland-like restrictions.
> A random app like xeyes should not be able to know mouse position at all times, for privacy reasons. (Unless you very explicitly gave xeyes super-extra permission to do that.)
The thing about xeyes isn't that its privacy invasive, is that it shows that xeyes knows what you're doing in other clients.
Got a gnome terminal root shell open? That's a privilege escalation method for any other client running on the desktop under Xorg. This itself, isn't really a problem, but chained with other attacks could be (e.g. browser escape).
> Got a gnome terminal root shell open? That's a privilege escalation method for any other client running on the desktop under Xorg. This itself, isn't really a problem, but chained with other attacks could be (e.g. browser escape).
Unless you're sandboxed up the ass, Wayland won't save you when that browser escape happens. Something I did 20 years ago to a friend as a prank that still works today on a typical Linux desktop with Wayland; wrap sudo to log the users password the next time they use it. I didn't use a browser exploit for that, but it can easily be done if you have write access to the user's environment however that happened. Wayland won't protect you from that sort of thing unless you're willing to commit to extensive sandboxing.
No, not really. Modern APIs usually don't even let you access that kind of information without additional privileges. Some older toolkits have functions that are supposed to do this, but it doesn't work everywhere.
It works fine on Windows and macOS's desktop APIs in 2024. This is a basic "desktop app" expected feature - if you aren't able to implement fully something like AutoHotkey or AppleScript macros, you aren't really a proper desktop platform.
You can implement all those things, just differently now.
Relatedly, now random apps can't record the whole screen. This is a good thing, now they get explicit user permission and go through xdg. I don't know how anyone could be opposed to this - it's objectively more empowering for you, the user.
Yes, it's empowering because your "rights model" is one where you don't have rights. You don't know what applications are using what data, and you also can't stop it. Is that empowering? I think no.
But if you wanna argue Chrome should be able to read all your keyboard inputs whenever it wants be my guest. I can't fathom why people want that type of setup.
Windows' pointer position is available to even the most limited GUI application; it is not protected information. For win32 at least, I am not familiar enough to say that about the newer app packages which are much more locked down, but I would be surprised.
They queried that info based on a specific implementation of a framework. If that framework is implemented under Wayland, then through the Wayland APIs it will only get rudimentary info on mouse position (e.g. only when it’s in focus and the mouse is over it).
App doesn't have to be in focus. It receives pointer events over its own surfaces (plus in some additional cases like drags and grabs). It doesn't receive any information about pointer position otherwise, nor its global coordinates (at least not without extra protocols).
Is it possible that one could write essentially a transparent app, which is always in focus and just passes click events through the transparent layer?
You can write a transparent app, but its the compositor that decides whether it's in focus and your app will either pass both hovers and clicks to the window behind (not staying in focus for long and not getting any pointer events even when it still is), or it will consume them all.
Why? I want to have some applications that can always see the mouse cursor like, xeyes. Because that allows me to implement a better customized desktop environment.
Then give that piece of code extra-special permissions. As I’ve said in another comment, the days where you downloaded your software from the sunsite or tsx11.ai.mit.edu FTP servers and could be confident that it and all its dependencies were trustworthy are unfortunately gone for a very long time.
"stdlib" was shorthand for "the standard library of a given language" and it came across poorly.
I did a quick google before I asked you that question, and it looks like wayland is one of the few exceptions to completely exposing current cursor position. Most "stdlibs" do expose it.
I will not refrain, you're the only one confused by it.
Just google it. You are incorrect and you're not pumped about it, I get it. The horse is fully beat to death. If you want to discuss further, my responses will probably just be lmgtfy links.
I find their point pedantic, but I tend to think it is correct. The "standard library" of a language is the library that comes with the language itself, usually across all platforms, and that rarely contains anything UI-wise beyond the very basics (e.g. text/terminal I/O like printing and reading lines).
I do agree that probably no-one was confused, though.
It's clearly you who's confusing "standard library" with "UI toolkit". Some standard libraries come with a toolkit, like Java, but that's an exception rather than rule.
Not that it makes a huge difference in the discussion, but if you want to be smug about it then at least make sure you're right first.
And an often under appreciated tenet of security — even a “good” software can be exposed to “bad” data, and you only need a bug (especially a memory bug, which is exceedingly common because linux userspace can’t get rid of c for the life of it) to have arbitrary code executed.
Like, your pdf reader is surely not evil, but do you trust every single pdf file you open?
I expect my PDF reader to be secure. If the PDF format is too complex to implement safely then the renderer should be sandboxed in the reader itself instead of preventing me from scripting using xdotool and similar.
And unless you fully sandbox your PDF reader then an exploit is going to have access to your user directory without any display server involvement anyway. X11 vs. Wayland doesn't even come into the picture.
That severy limits the usability and even functionality that programs can implement. If you want a phone os then go use one but don't make Desktop Linux into one.
Nice results. It's kind of wild that the "before" behavior would be considered acceptable/shippable by anyone. That kind of jank should be a show stopper.
Linux desktop is full of such jank. Fortunately, thanks to contributors like OP, the jank is gradually getting fixed. The whole point of Wayland is to eliminate a whole class of jank (screen tearing) that was essentially unfixable with X.
And yet, empirically, jank is far more common on Wayland than X11. With X11, on the rare case I noticed, it was usually just a matter of "okay, which toolkit or driver do I need to look up the vsync environment variable for?"
I have no opposition to Wayland in theory; my concerns are entirely practical and unignorable.
In my experience Wayland has been a better experience than X11, I have two monitors with different refresh rates and I could never eliminate tearing with X11 whilst Wayland works as expected.
Wayland does some things well; X11 does some things well. Unfortunately Wayland is not a straightforward improvement. At some point they climbed the abstraction ivory tower and lost sight of the system they were supposed to implement.
I have not noticed any tearing issues with X11 for a long time, with or without multiple monitors. Were you using X.Org graphics drivers or external vendor blobs?
I had that issue with both AMD and Nvidia with the open source and proprietary drivers respectively. If I ran both at 60Hz it was fine, but my primary monitor is 144Hz and I couldn’t make 144Hz and 60Hz work well under X11.
For me, fractional scaling on a HiDPI screen is broken on X11, but works well in Wayland. Here's an example from yesterday: I was playing Factorio full-screen, and some of the GUI elements were too large and wouldn't fit it in the screen. I found a Factorio setting that enabled Wayland, and it instantly fixed the problem. Kudos to Factorio for supporting Wayland! You wouldn't think it was needed for a full screen game, but it was.
This does not make sense at all as an argument really.
I think your argument is about specific implementations of WM.
While the argument of "I deal with X11-based WMs because it's fine when I don't care about security at all" may be valid in very narrow cases (such as air-gapped systems), the argument more generally is pretty weak.
Its not surpising that x11 based WMs, such as the almighty [awesomeWM](https://github.com/awesomeWM/awesome), have more features implemented than, for instance, [jay](https://github.com/mahkoh/jay) due to the enormous time it has had to develop (though I am _very_ excited to see `jay` develop more fully, and expect it to be well used by the more tech-savy devs).
However, some WMs in the Wayland space are doing quite well on that front. I recently had some substantial problems arise in my system which (surprisingly to me, but perhaps some are getting used to this) would have been prevented by using a memory safety language for my WM, so I have made the switch to (for better or worse) only ever consider Wayland+Rust WMs. In this space, [niri](https://github.com/YaLTeR/niri) is actually quite good, and to the point - it is developing correctly _and very quickly_. So, any issues on some WM not implementing some desired feature are quickly disappearing.
IIRC, all the major 'gateway' linux distros, such as Ubuntu or Fedora, are all on Wayland by default now - so I don't imagine x11 will stay relevant much longer.
> With X11, on the rare case I noticed, it was usually just a matter of "okay, which toolkit or driver do I need to look up the vsync environment variable for?"
Heh? That hasn’t been true on even the very first POC wayland compositor, Weston. I mean, it used to crash from time to time in the very early days, but visual artifacts? I don’t remember any, besides the occasional xwayland app (which is literally an X app running inside wayland).
I don't think that's an honest representation of what happened. Wayland relied on implicit sync by design, it wasn't "a bug" but a choice. The fact that in the end there were valid reasons to start moving towards explicit sync besides of lacking Nvidia implementation is irrelevant to the fact that Nvidia did not implement a correct driver for the existing requirements of the platform.
It relied on implicit sync as far as I know by accident. But since it worked for the popular configuration, no one dug deeper that support for accelerated drawing without multi-frame lag of manually calling commit on surfaces had gaps in design.
Implicit sync worked by design. There were years of discussions whether it's a good idea to rely on it and how exactly explicit sync would actually look like, with protocols such as zwp_linux_explicit_synchronization_v1 (eventually deprecated) being worked on almost a decade ago already, but it's definitely not an "accident". Explicit sync offers several advantages, but implicit sync is how DRM was designed and what Wayland consciously relied on, and what Nvidia disregarded because it would be rather challenging to implement it in their driver.
The ecosystem slowly moves towards explicit sync now, but Nvidia was supposed to provide a driver already before that was happening and they didn't comply with platform's requirements at date, resulting in user-facing issues. With this one, they just got lucky that the consensus happens to move towards what they had already asserted.
Explicit Sync, yes. The reason why I consider previous implicit sync a bug is that it was essentially unfounded expectation - nothing guaranteed you'd get a magic mutex in the sky that synchronized renders for you, and in fact, you do not want one for maximum performance. Lack of explicit sync also essentially meant common multi-gpu setups would be broken.
I was not saying it's an nvidia thing, just that significant majority of devices used by wayland proponents had undeclared implicit sync issues that were invalidated when faced with a driver that didn't have them.
> And yet, empirically, jank is far more common on Wayland than X11
It depends on the type of jank you're talking about. It's wholly disingenuous to characterize Wayland as janky but x11 as not - the jank inherent to x11 is what made the original Xorg developers start making Wayland in the first place. Empirically, if x11 was perfectly fine there would be no motivation to design a successor.
x11 gets the first-mover advantage of a lot of implementations and a straightforward design goal, but that's about it. It's not secure enough to recommend as a serious alternative to Mac and Windows users, and it's too slow and unopinionated to present a friendly face to new users. Features like 1:1 trackpad gestures, genuine HDR pipelines and a locked compositor framerate are all getting to the point that regular consumers expect them as default.
If you want to keep using x11, it's unlikely someone is going to take it away from you. But it's on track for depreciation and hasn't been actively developed in years. Recommending it as a panacea to new users is a bad idea.
> It's not secure enough to recommend as a serious alternative to Mac and Windows users
The security risk of X11 is theoretical, not practical. Yes, X11 programs can maliciously keylog each other, but this just isn't a thing that actually happens. And even if you do start installing random malware from the internet like a classic windows user, Wayland isn't going to prevent you from screwing yourself anyway. To actually be safe while installing and running malicious applications you need extensive sandboxing. Wayland can be one part of that sandboxing but is useless without the rest (to prevent the malware from stealing user files including credentials, using LD_PRELOAD hacks or similar to keylog other applications anyway, etc), and no distro suitable for recommending to Windows/MacOS newbs has the rest of the requisite sandboxing. The sandboxing touted by Wayland advocates is very esoteric and without all that sandboxing, a newb using Wayland has to exercise just as much caution when downloading software as if he were using X11.
I am saying that X11 users getting hit with keyloggers is extremely rare if it happens at all. It's trivial to make such a keyloggers but that doesn't mean it's common for the programs users install to do it.
And yes, X11 makes that sandboxing useless, but that sandboxing isn't in play anyway because we're talking about noobs from Windows.
I think it is dangerous to feel yourself as superior. UNIX veterans can also be victim of supply chain attack, either from their distro package manager, a language module manager, a malicious flatpak with too much access, a trojaned appimage from a supposedly reputable vendor, etc.
Superior? All I'm doing is sticking to programs in my distro's repos. Such programs including malicious keyloggers is unheard of. And if a maintainer turns evil or let's something slip through by accident, the damage they could do is hardly mitigated by Wayland. Thankfully, this virtually never happens
In principle I am also vulnerable to something like a RCE zero day in Firefox turning an otherwise trusty program into malware which exploits X11's open nature, but again, this sort of thing actually happening is unheard of.
I'm not superior, I'm just trying to keep a realistic grip on the threats I face. Modern security culture is fixated on what is theoretically possible, I care more about what is actually likely.
> the damage they could do is hardly mitigated by Wayland.
I disagree. How do you hijack interprocess communication on a Wayland device? I can tell you in very certain steps how to manipulate an x11 client but outside hijacking /dev/ I can't imagine a similar attack on Wayland.
But the core framework of every x11 server relies on a loop that does poll HID connected to your machine for events. You can have an x11 server running with no windows connected and your keypresses will still be broadcast to any program that knows where to look.
That's exactly the same as Wayland, isn't it? Every Wayland compositor also polls HIDs connected to your machine for events. And if your Wayland compositor has permission to open those HIDs when running as your user account, so does any other program you run, regardless of Wayland or X.
There isn't a real attack using it yet, only because attacking Desktop Linux is a really unprofitable endeavor (considering the marketshare, the ROI must be very low).
> To actually be safe while installing and running malicious applications you need extensive sandboxing
FWIW, X11 is unsandboxable unless you run a second X server on top of your current server [0]. Which is fine, but you need to consider that most, if not all sandboxing solutions on Linux that "newbs" use, like Flatpak, do not employ such technique when running sandboxed X11 applications.
The "security by default" behavior of Wayland limits the possible attack surface a lot, without requiring the end user to understand all the nitty details involved.
Why is X11 unsandboxable? A similar but reverse approach to Xwayland, something like waylandX could be used to be part of the overall sandbox approach to run untrusted applications. That would have the advantage that the severe restrictions and feature degradations of wayland are only applied to those untrusted sandboxed applications, not everything.
Ultimately, X11 opens up everything. What you suggest (WaylandX) is essentially allow-by-default.
When this is the case and there is a supply chain attack, what you think is a trusted application (and therefore not running under "WaylandX") can very well keylog you or take screenshots of your desktop without your consent.
In a deny-by-default model ala Wayland, applications will have to ask for permissions before they can do something considered to be privileged.
You're not telling me anything I don't already know and haven't already explained. X11 keyloggers are trivial, and virtually never seen in the wild. X11 makes sandboxing impossible, but that doesn't matter because I'm not going to waste my time on something like Qubes anyway, and newbs from Windows aren't being directed to setups like that either. They're all installing Mint or Ubuntu where the security of Wayland is nullified by the absense of sandboxing.
>newbs from Windows aren't being directed to setups like that either. They're all installing Mint or Ubuntu where the security of Wayland is nullified by the absense of sandboxing.
This cannot be more further from the truth. Amongst the newcomers, it is rather popular nowadays for them to use Flatpak-bundled apps, especially with the rise of SteamOS (the Deck essentially) lots of Linux newcomers are in fact first exposed to Flatpak and running untrusted executables in a sandbox.
And the most prominent "untrusted executable" today to those newcomers has to be Bottles, which is a nice GUI wrapper for Wine and is sandboxed (if you enable wine-wayland, of course).
>I wouldn't trust flatpak enough to run a truly untrusted executable. I am sure flatpak's isolation is full of holes unrelated to windowing.
As compared to running untrusted programs completely naked?
>But I don't think a game purchased through steam counts as untrusted.
Bottles is there for people to run any Win32 program, not just Steam games. And I shouldn't have to tell you how many malicious Win32 programs there are.
Just google the criticisms of flatpak from a security perspective. They're out there.
Containerization on Linux was never intended to be a security feature for totally untrusted, malicious code. It's isolation for trusted code. If your scenario relies on securely running untrusted executables in a Linux container you are doing stupid things.
I am well aware of the weak points of Flatpak. But are you suggesting that running applications in a container is not more secure than running an executable completely naked?
You see: If you want absolute security, for sure, go for a full-fledged VM! Or run something like QubesOS. It is a completely reasonable decision.
However, malice certainly has degrees, and the "mildly malicious" programs most likely cannot take advantage of sandbox escaping exploits. If Flatpak can stop 95% of all attacks (relative to running a program completely without sandboxing), that is already a win in my book.
But I will note again that X11 is a big hole (as in, almost a complete free-for-all) for sandbox escaping in Flatpak.
You seem to think a lot of things that aren't security boundaries are security boundaries. There have been VM escapes too. VMs are not for running untrusted OS images you get from end users.
I am not sure where you got the impression of me talking about only "Steam games". Bottles allows you to run any Win32.
And besides that, "these threats are off in fantasy land" is an invalid defense in my opinion, considering the (quite sophisticated) XZ Utils backdoor happened not too long ago! Like I said, if such an attack towards X11 hasn't been deployed in the wild, it can only suggest such endeavor is unprofitable, not because the threats are fantastical.
XZ utils backdoor could have exploited X11, but didn't. And the most common Wayland configurations wouldn't have protected people from the backdoored utility; only extremely paranoid and therefore esoteric setups might have.
> And the most common Wayland configurations wouldn't have protected people from the backdoored utility
If the attacker decided to backdoor an utility and make use of X11, it is most likely the backdoored utility will listen to keyboard events, read the bitmaps of other X11 clients.
And there's nothing that can stop the backdoor from doing so on X11...
Anyways, if you are saying the Wayland security policies are unneeded because there hasn't been an attack on X11 (this is the fundamental disagreement between us), consider the following: You don't install doors in your premise, because there hasn't been a case of burglary in your neighborhood?
> x11 gets the first-mover advantage of a lot of implementations and a straightforward design goal, but that's about it. It's not secure enough to recommend as a serious alternative to Mac and Windows users, and it's too slow and unopinionated to present a friendly face to new users.
I've been recommending it as a serious alternative for years, and it's always presented a friendly face.
> Features like 1:1 trackpad gestures, genuine HDR pipelines and a locked compositor framerate are all getting to the point that regular consumers expect them as default.
I have never heard anyone not already a Linux user comment on even one of those as a problem.
>I have never heard anyone not already a Linux user comment on even one of those as a problem.
Those specific words are uttered by Linux users, true. However, Linux beginners _do_ notice some X11 issues, it's just that very frequently they only know "something" is off but not why they feel so.
From anecdotal experience, touchpad gestures are actually something my friend complained so YMMV. We ended up making a file in /etc/X11/xorg.conf.d/ to configure the synaptics driver. Another experience had to do with screen tearing, I helped them fix it by installing a compositor.
To be fair, my anecdotal coworker evidence with Wayland has been "the Webex client doesn't work" with the ten-second remedy of showing them how to set their default desktop back to X11.
And once all the Wayland jank is fixed it's going to be just as "crusty" and "full of hacks" and "unmaintainable" as X is now and the cycle is going to repeat again :|
I assume you know this, but just to be sure: The "before" state only happened in a fallback codepath when using a legacy application, i.e. an X11 app in the modern Wayland-based desktop. When using modern apps, or when using the entire desktop in legacy X11 mode, this doesn't happen.
Most legacy X11 apps in active use are actually games which tend to be fullscreen and not resized, so it wouldn't happen that often for many users.
(That said, sure, I can also respect the stance as I tend to place a premium on glitch-freeness too.)
My window manager had the “before” behavior, but I complained it was defective and they gave it to me free. Not sure how much it would have cost beforehand, but I can’t complain after a 100% discount, right?
"Jank" is still functional. And, to a first order approximation, nobody cares.
Even Windows generally switches down to software rendering when resizing. And on mobile devices, nobody resizes.
No pointer warp, however, is a failure (CAD packages all have significant issues on Wayland). Lack of multilingual support/accessibility is a failure. Lack of screenshots/screencasts is a failure. Lack of support for the BSD lineages is a failure. etc.
People are still bitching up a storm because Wayland still (Pipewire does screen sharing using DBUS(!) for example) hasn't fixed basic things while DeadRat is shoving it down everybody's throat by dropping X11 support.
The Wayland devs aren't wrong about the security implications of this kind of stuff. However, they're also not giving anybody solutions, either.
One big issue is that Wayland devlopment is so slow that the entire space moved forward and destroyed a bunch of assumptions that Wayland is based around.
> Lack of support for the BSD lineages is a failure.
Not true. FreeBSD and OpenBSD have had various wlroots compositors such as Sway available for some time. e.g [1] Some people have even been experimenting with KDE-wayland on FreeBSD since 2021.
Wayland support on any OS is not binary because there is no single layer like Xorg. It's a matter of individual compositors and their components being ported to the OS, which is a matter of popularity, and the BSDs are always at a disadvantage in that respect, so they lag behind, same as other software. Nevertheless, they are definitely gaining support. Then again this distinction is only technical, for functional support new DEs/WMs would always have needed to be ported regardless of display architecture, the only case it would not is for an Xorg drop-in, which defeats the purpose.
> Lack of screenshots/screencasts is a failure.
... but there are many functional screencapture apps, and even browser support, I use this pretty much every week. I think you might be operating on out of date info. I'd highly recommend giving it another try, the Wayland ecosystem has come quite far over the last decade.
Wayland unfortunately is a mess that is hard to implement so we get a balkanized situation where not only everything often boils down to Big Important Process in the middle (that actually integrates more things than X.Org used to), the set of features it supports are way more balkanized and most importantly, specific to that one Big Important Process as they can't be viably separated out.
And even then you have to deal with a mix because you have to work through two different unsynced possibly broken in weird ways connections (Wayland + D-Bus).
This results in how last week I couldn't screenshare under Wayland, and had zero chances to figure it out - and to make it worse since I had some important state and no time to play with reboots, I couldn't fix it.
XOrg was predominantly used as a useless middleman between a compositor and a window manager communicating with each other. Just because the wayland compositor is “one big important process” doesn’t mean that the whole architecture is more complicated.
Multiple IPCs make the whole stuff way more complex, there is a reason why we have a big monolith as browsers and not some `curl | htmlRender | jsInterpreter` Rube Goldberg machine — the unix philosophy is not the pan ultimate design, it works in some cases and is utterly broken in others.
X11 != the lowest common denominator implementation of X11. Nor was everyone using it with big heavy compositors (honestly, the only reason I used a compositor for years was intel removing sync circuitry from their iGPUs).
And Wayland did not stop requiring multiple IPCs, in fact it mandates them as several features require passing data by secondary channels, not just for Portals et al - some don't even have any described way to reliably pass the information that I have seen (like how to pass around cookie to let one application activate window of another? Or maybe the spec is such a mess that I'm looking at completely wrong place when I tried to fix ability to reliably switch between applications without extending compositor).
And yes, the architecture is more complicated in practice, otherwise it might have reached parity with what X11 did after as many years - like input method support. Unfortunately it's so broken that you have multiple versions of it in practice, it requires extra IPC in practice, and at least in my experience just does not work.
I'm using Debian so I can't say how well this applies to FreeBSD:
You need to have `xdg-desktop-portal-wlr` or equivalent package installed (should be depended upon by sway package really), and also the gtk one for some other stuff wlroots doesn't implement like notifications I think (don't worry it doesn't drag in all of gnome).
This is technically the only thing you should need to do beyond picking a wayland compatible screen capture program or browser - However it's possible the dbus variables are broken if you wrote your own sway config from scratch (guilty)... Depending on your OS, the default config should do this automatically, on Debian it includes /etc/sway/config.d/50-systemd-user.conf, I can't say for FreeBSD. So point is if your config omits the dbus-update-activation-environment command, you probably need to include it in your config.
Those environment variables are set by sway, but this line imports them into dbus (is my limited understanding) so that programs call the correct xdg portal backend.
> The Wayland devs aren't wrong about the security implications of this kind of stuff.
But they are wrong. Their security model assumes an ecosystem of untrusted apps when we already had something far superior: distributions with vetted packages.
Wayland is like bolting down your furniture because if you let in random strangers from the street they might steal your stuff. Instead of adding obstacles that make my own life harder I prefer to keep better company.
Vetted packages are great, but they aren't the end of all security problems. Good security needs defense-in-depth.
It's rare but not unheard of for someone to be able to sneak malicious code into a vetted package. It's extremely common for vetted packages to have security vulnerabilities that could be exploited.
I don't want someone who finds a vulnerability in a fart app to be able to escalate that to attack other apps on my computer.
I trust my accountant with a lot of sensitive data but I don't give them the keys to my house. I trust a friend with my house keys to water plants while I'm gone, but I don't give them the password to my bank account.
What good security needs more than anything is proportionality. After all the only criterium you care about is how secure your computer is then you should simply turn it off because nothing else is going to beat that.
> It's rare but not unheard of for someone to be able to sneak malicious code into a vetted package. It's extremely common for vetted packages to have security vulnerabilities that could be exploited.
Ok, when is the last time you or anyone you know personally sustained any nontrivial damage because of such an event? You can make up hypotheticals to scare people all you want but the simple fact is that no, people are not actually in any more danger on their computer than they are just being alive.
Hmm. Even with the fix it didn't look very smooth even if it is a lot better. I wonder if that is due to the recording, or due to the mouse settings (mouse poll rate and acceleration settings are very impactful on smoothness), due to X or the toolkit, or due to it not being as good as it can be.
I think issue is the screen capture tool or a framerate mismatch in transcoding. The video is 59fps, but the mouse cursor only updates every few frames. Usually it's every other frame, but sometimes it takes three frames or more to update, and sometimes it updates in two or more consecutive frames. On most of the frames where the cursor moves, the window size also changes.
Excellent writeup, thanks for posting. I was involved with efforts to achieve similar things in Windows (Direct Composition, etc.). In practice, it was enlightening to debug various apps and frameworks and discover just how long repainting can take and how many edge cases there are around slow apps, hung apps, and so on. Who knows, maybe someone there still has some of the malicious test apps I wrote (using the language of this article, apps which acknowledge the sync request and then take multiple seconds to actually paint, etc., etc.).
It's definitely a hard problem space that's basically incompatible with how most GUI apps and frameworks worked from 1985-2015.
>> malicious test apps I wrote (using the language of this article, apps which acknowledge the sync request and then take multiple seconds to actually paint, etc., etc.)
This is one area where Wayland should do better. It's the compositors job to resize windows and IMHO draw their decorations. The compositor has a frame to paint and it can't have to wait on applications. Unfortunately some toolkits and DEs have decided that client side decorations should still be a thing.
Also with this new partitioning of responsibility is my main annoyance: The DE needs to remember window placement for all my apps, since they are not allowed to know their environment under Wayland (for good reasons).
>The compositor has a frame to paint and it can't have to wait on applications.
This is the source of the visual artifacts this article is trying to prevent, however. Sure, you probably don't want to block resize for multiple seconds, but in general the compositor is very responsive. The app may not be. If you just let the window chrome resize as fast as you get, what do you do with the rest of the window? Leave it transparent? Draw a default black or white area? Both are very ugly and very noticeable in practice.
This is the part of the problem space I know the most about - for a number of years I owned window chrome on Windows (don't blame me for the 1px border and way-too-subtle shadows, but after being overruled by PM/design, yes, I was responsible for implementing them).
As far as custom decorations - that is a lost battle. Companies and apps want their special design language and will simply not build for your product/operating system/compositor/whatever if you don't give them that kind of support. Twitter and Facebook both want their specific shade of blue and their specific font. Adobe is....well, let's not talk about Adobe. Browser tabs in the title bar are a P0 requirement these days and anyone who doesn't support that will be laughed out of the room. Etc., etc.
> Also with this new partitioning of responsibility is my main annoyance: The DE needs to remember window placement for all my apps, since they are not allowed to know their environment under Wayland (for good reasons).
https://gitlab.freedesktop.org/wayland/wayland-protocols/-/m...
This might eventually get fixed, above is what I believe is the current proposal for handling this issue and letting apps do some amount of positioning without exposing things that a program shouldn't know about. That said this has had multiple proposals over the past 5-6 years at least and none have managed to make it all the way through. If you go through the previous ones (ext-placement and i forget the others) and ignore the angry messages involved it turns out that it's a very difficult problem to deal with in a way that isn't just a free-for-all with apps either not knowing about monitor placement, or having to handle so much detail about the displays that nothing will ever act consistently.
That said, recent discussion on that latest one does look promising so maybe it'll finally happen.
That isn't really what I was hoping for. IMHO it is wasteful for every application to remember its window placement and restore it. Not only does that require additional code in every app, but if there is some kind of policy change there is no way to make it consistent. In other words, at the compositor level one could use the previous position as a default and have an option to do something different if another window is already there. It's simply not the applications job to place itself on the desktop. There are probably use case for such self positioning functionality, but restoring to the last position is not one of them.
The application side of it is more for things like the multi-window setup of things like GIMP so that windows that are "docked" next to each other will stay that way past restarts. That's one of the reasons that the newer proposals are doing things with relative positioning between a zone or main window rather than allowing applications to place themselves randomly on whatever monitor or space that they want, interrupting whatever workflow is going on (which actually allows for security issues, i.e. a window pretending to be a password prompt putting itself on top of a browser or something to confuse the user). This also allows for new windows from the application to request that they're positioned next to any others so that related things stay together. This also apparently helps in a few cases where a single "application" to the user is actually multiple separate programs that get run by a main interface. Not as common in new software today but it used to be one of the ways that a lot of older software worked and there's still a decent amount out there apparently that are maintained that way.
Keep in mind that this is also a request by the application, not a requirement of the compositor to obey it. If there's not sufficient space where the application requests things then the compositor can just ignore it and do what it believes makes sense.
IMO, the neatest trick that the DWM does is avoiding jitter/lag between the mouse cursor and window when dragging a window.
Sadly, DXGI flip model seems to have reintroduced some issues -- it is difficult to avoid the kind of jank shown in this article when resizing a window drawn using flip mode presentation. Which is unfortunate, since it's the highest performing mode.
It's kinda interesting that macOS doesn't even care about this problem anymore. When moving or resizing windows on macOS the hardware mouse cursor slightly trails ahead of the window, it's not very noticeable because of the 120Hz refresh rate (meaning the framebuffer surface spends less time waiting for presentation, thus reducing rendering lag) - OTH the lag is still visible, and this sort of minor visual imperfection is quite un-Apple-like (at least for 'old Apple').
It's a bit similar to how high resolution displays make fancy subpixel font rendering tricks obsolete (e.g. high display refresh rates making tricks to hide presentation latency obsolete). Sometimes hardware progress makes complex software solutions obsolete, but usually still with one or another tradeoff.
> It's a bit similar to how high resolution displays make fancy subpixel font rendering tricks obsolete
Personally, I don't think so—subpixel font rendering still has value even in the densest displays, simply because it triples the available horizontal resolution.
Unfortunately a low of newer displays (especially OLED ones) don't have regular horizontal RGB/BGR layouts anymore. I have seen both vertical subpixel layouts as well as triangle layouts or other funky subpixel arrangements. Even if you configure/update your software to those different subpixel layouts the result is not going to be as good.
But if we continued using subpixel rendering then 1080p and 1440p displays wouldn't look as terrible, and we can't have that /s
It's pretty noticeable on older apple hardware running newer macOS. Definitely a lot of compositing performance regression in Sequoia specifically.
That trick is fun, but I wish Windows did a better job of synchronizing the visibility of the cursor sprite versus the presentation of the frames where the cursor is rendered in the frame. It's impossible for me to not notice the brief period where the cursor is missing!
I think it's cute. They're doing "disable hardware cursor, enable software cursor" instead of the other way around. Seeing that happen is somehow fun for me.
I think they do both simultaneously, it's just that the cursor sprite/plane is overlaid during scan out and the software cursor is rendered in the next frame to be rendered by the compositor. The cursor sprite/plane can turn on or off pretty much mid-frame, whereas the frame that's about to be rendered could be two frames away from the one that's about to be scanned out. What you actually need to do is track when the first frame with the software cursor is about to be presented, then turn off the cursor sprite/plane. I think technically speaking all of the information needed to implement this is present somewhere, but I'm sure it's quite complicated to actually implement, possibly made worse by interface layering issues :)
For me, the quickest way to tell that I'm running on Wayland is seeing if multi-lingual keyboard input is broken. I have right-alt-while-pressed set to switch to a Greek layout (because who doesn't math?), and it reliably works on X11 but breaks in all sorts of weird ways under Wayland (sometimes it simply fails to enable/disable, sometimes it shows a notification as if it changed but switches back when input actually happens, ...).
On the bright side I'm finally learning how Vim's digraphs work.
Both Xorg and Wayland compositors use libinput, so it's probably more to do with lack of decent configuration support in your DE or WM. You can always configure libinput manually. Sway just lets you add libinput configs inline inside your sway config.
[edit]
This is probably not relevant (assuming correct wayland libinput config), since this is not where mult-lingual input transformations live. libinput just handles the physical keyboard mapping and behaviour.
On more careful reading of the parent this sounds more like a buggy input method editor.. or maybe an issue with switching between X and wayland apps.
Is (simple, so e.g. non-CJK) multilingual input a libinput thing? I thought it was a libxkbcommon thing.
NO! my mistake, I latched onto keyboard input and didn't think about multi-lingual. I'm guessing you mean things like pinyin, I think that's a dbus thing, which is still a thing on wayland but I don't really know anything about this... I'll get my coat.
That still boils down to a Wayland design issue, since Wayland requires each individual DE/WM to get support for such features while X only needs them in Xorg.
But why are management and handling of language-dependent input methods supposed to be in Xorg?
What happened to separation of concerns?
"Do one thing and do it well"?
More like do a thing and do it once. If every WM and DE needs to do it, you have some that don't, or you have issues that are handled correctly by some DEs and not others, etc.
We're still talking about Linux, are we?
Snark aside, it's still orthogonal to my question. I wasn't questioning whether it should be done centralized, but why it's supposed to be part of the display server protocol/compositor.
Because the "display" server is also responsible for window management and input. Does it have to be this way? Perhaps not - but seperating window management and input is also not trivial.
Note that more complex input methods do somewhat bypass the display server and communicate via dbus instead.
> "Do one thing and do it well"?
Did you miss the do it well part? Nearly every unix tool does more than one thing, the alternative would be a usability nightmare.
* grep, which primarily does pattern matching has dozens of file traversal related flags that could be handled by calling it from find
* find, a tool supposed to find files for some reason has dozens of flags related to executing applications which could probably be done by using xargs
* did you know that xargs can do pattern matching and replacement on its input string? there are probably dozens of unix tools that are specialized for that
X11 eventually punted the whole thing to each individual application. For input methods like Fcitx or IBus to work reliably on modern X11 programs, you need to set envvars like the following [0]:
[0]: https://wiki.archlinux.org/title/Fcitx5#IM_modulesThat seems weird complaint about Wayland, Wayland requires every server to support these features, just like X11 also requires every server to support them? Sure, there weren't that many X11 server implementations so maybe it was not so apparent, but as a protocol they seem pretty much equal in this regard.
The criticism is that Wayland duplicates a lot of effort in multiple compositors. The compositor does work that happens in Xorg and in the WM. Whereas lots of common functionality gets implemented in Xorg that WMs and DEs don't need to duplicate.
It's not a weird complaint when on X11 when something is added it generally works for everyone whereas for Wayland the different implementators are still squabbling how to accomplish basic functionality people expect from their desktop computer.
Or in other words, your comparison is like saying food and shit are basically the same thing because both are made up of similar elements.
But wayland merges more components into one server than X11 did. Therefore, there is a need for more wayland server then there was a need for X11 servers.
Xorg can use libinput but is not limited to that.
Neither are Wayland compositors.
Hear, hear.
KDE just elected improving the Input story in its bi-annual community wide goals election: https://kde.org/goals/
Multilingual keyboard input is such a mess haha. I need to type a lot of Chinese using pinyin and still haven't found a reasonably polished implementation on Wayland (Sway) that works on both Xwayland and true Wayland applications. So I reverted to i3 with ibus.
For what it's worth, the latest 1.10 release of sway introduced support for IME popups. Maybe that would help
Another way to check if you're running on Wayland is to start some 3D accelerated application and see if the screen is all flickery. Apparently Wayland has some issue with not generating a frame sync that causes all variety of problems, but usually lots of flicker.
Really? My laptop has my national keyboard but I switch to USA for programming because of {} [] and all the other characters that can be typed with minimal use of extra keys. My fingers know the keys and the switch combination, maybe windows + space. I didn't expect Wayland not to have such basic functionality after so many years. But frames must be perfect, no tearing (is that the word?)
It's possible that the "permanently switch to next layout" works reliably, but not the "switch while held" (which I do because I'm selecting single symbols at a time)?
Have you tried using a compose key? I lost my custom mappings when switching from X to Wayland, but they work fine in both.
I did lose my custom mappings though, but I only needed them when I was in emacs and obviously there's already a command for inserting weird stuff, so I just added a binding for it.
I use the compose key for compose stuff. It's a huge pain to actually write and install custom compositions so I don't bother anymore, just use the defaults (which is mostly annoying due to missing keypad variants, e.g. ± must be typed using the top of the keyboard).
It looks like the default en_US.UTF-8/Compose includes mappings of the form:
but to use that I'd have to figure out how to map a key to `<dead_greek>`, and keyboard mappings that aren't in the standard checkboxes are such a pain.> It's a huge pain to actually write and install custom compositions
Could you say more about this?
IME, it's just a matter of adding lines to ~/.XCompose — is there something I'm missing?
Is ~/.XComposed picked up anymore? At least it wasn't early on, and I just gave up on keeping it around.
I guess it's up to each Wayland compositor, which calls for inconsistency :/
My understanding is that generally, Wayland uses XKB, which respects ~/.XCompose
But I think there are specific exceptions still, such as this Chromium bug: https://issues.chromium.org/issues/40272818
Any chance you typo'd the file name then, like you did here?
Tried it again and it works!
I'm guessing that maybe I forgot to restart the machine to make sure everything got to read it. Or that whatever was broken long ago got fixed.
BTW, I can't reach my backup right now, but this seems like a good start to build up custom mappings in case anyone gets interested in this, https://github.com/kragen/xcompose
If you use Xorg sure. If you use wayland it can't be done.
I think you're mistaken. You can use .XCompose on Wayland (modulo certain broken cases, perhaps).
The Wayland book recommends XKB: https://wayland-book.com/seat/xkb.html
It does come down to the libraries used by a given app, though (see sibling comment).
Even with Xorg toolkit support has degraded quite a bit. For GTK it doesn't work by default and you need to set GTK_IM_MODULE=xim in your environment. Qt on the other hand picks up ~/.XCompose by default but truncates the compose result to one character.
compose key on wayland is why I don't use wayland :D
Huge respect to any dev who goes out of their way to fix flickering/glitching/black regions when resizing or moving windows, I honestly can't stand it when I see it because I know it can be solved with effort.
Hoping this makes Xwayland Emacs much smoother to resize. It currently resizes at 0.5 fps for me on KDE.
If you're wondering why I don't use the native Wayland build of Emacs, it's because it is massively more laggy on HiDPI screens than the X11 version. I reported the issue upstream, probably won't be fixed for a decade when they'll decide to port to GTK4 and its hardware accelerated rendering backend. It took me a long while to notice that the typing lag I was experiencing was not because of native compilation or single-threading, but pgtk being a bit weak at rendering 3840x2160 pixels at 60 fps. In fact, it was not until I tried the Xwayland build that I experienced how much faster Emacs can feel.
I like to run xeyes and see if they move when I interact with the test application. It's not better but they're as charming as they always were.
Also consider running an xwayland app under gamescope which smooths out some issues.
That’s actually something you don’t want nowadays. A random app like xeyes should not be able to know mouse position at all times, for privacy reasons. (Unless you very explicitly gave xeyes super-extra permission to do that.)
We’ve long stopped living in a world where you should need to fully trust every piece software that you run as your user on your computer. And even probably trustworthy software can go bad at an update due to supply-chain attacks.
> That’s actually something you don’t want nowadays.
No, that is something you don't want. I and many others, do want this functionality.
Even you don't want this, you only want it sometimes, for some apps. Which is exactly what they said.
I mean, would you like VSCode tracking your mouse movements across the entire desktop and your keypresses and then sending them off to Microsoft? Probably not, so we're all in agreement.
I don't use VSCode or other user hostile programs like it so I don't care what kind of anti-features it enables. I don't want my actually useful tools hobbled in order to deal with such programs.
xz wasn’t “user hostile”, and so weren’t countless other pieces of software affected by supply chain attacks. Nothing is hobbled if you can give it explicit permission (which you may well do on xeyes).
The days where you downloaded your software from the sunsite or tsx11.ai.mit.edu FTP servers and could be confident that it and all its dependencies were trustworthy are unfortunately gone for a very long time.
The xz-utils hack didn't care about the window system at all. It also hasn't actually caused any known damage and I'm sure if the american three letter agencies cared the perpetrator would have been dealt with by now. Project takeovers of that kind is not something regular users need to be worried about because the cost of pulling one off is too righ to waste on petty crime. It's yet another boogeyman and scaring people into giving up their computing freedom for "security" migh as well have been the goal of the operation.
> The xz-utils hack didn't care about the window system at all
It's an example, demonstrating that the mentality of "I don't use bad software" doesn't really work.
> giving up their computing freedom for "security" might as well have been the goal of the operation
How could you possibly reasonably argue that you're losing "computing freedom" because now you have the power to deny or allow applications from doing things? You're literally gaining freedom - that's not something you were able to do before. Now, you have the freedom to deny applications accessing something you don't think they need.
Did the POSIX security model (users, file permissions) also impact your “computing freedom”? Is your CPU constantly running in kernel mode? Do you have all physical memory mapped into all address spaces, and as read/write/executable? Are you allowing all software to listen on outward ports without any packet filter?
It really is something I want. But most people do live in your described world. That's the smartphone and "run's every javascript application sent to my web browser automatically" kind of computing security model. But there do exist personal behaviors of desktop operating system use where you can actually trust the applications installed and not put up walls between everything. And then things like keyboard/mouse sharing work, windows return to their places, and screen readers work. Those are all very important to me. For them I'm willing to browse with javascript temp-whitelist-only and many other such tedious things.
What I'm trying to get across is that the need for that kind of intense security model, every process a threat, is not intrinsic to modern computing.
But what’s the limiting factor of doing the sane and safe thing by default?
The most popular operating systems all do that (ios and android), and they have carved out safe APIs for all of that to work. You can’t patch up a Swiss cheese after the fact.
Is it hard to create standard APIs in a bazaar style of development? Yeah. But that doesn’t mean that it’s not the correct approach.
I think their concern is valid, it's difficult to do something which is both secure AND not limited at the same time.
Sure Android and iOS are secure but in practice they kind of suck for making anything non-standard which limits creativity and freedom.
Can we have both a secure and extendable system? Maybe but none of them exist yet. I'm really worrying that Linux mainstream distros will become like Android or iOS.
Safe and sane for you. Not for everyone.
What use is a safe API when it makes the entire system impossible to use by a significant fraction of the population (ie wayland and the visually disabled)? It's been a decade+ and none of the waylands have managed to support screen readers yet.
Are we to just throw out that whole class of people and tell them, "You don't get to use linux desktop computers anymore when X11 support is dropped". As someone with retinas that are progressively tearing apart, who already uses text to speech for many things, this is incredibly disheartening. I really don't want to have to switch to the Apple ecosystem.
Let’s not fool ourselves, linux’s accessibility (and anything else) support was lackluster to begin with. Android and ios is far far superior on every count from an accessibility perspective and interestingly they have a sane security model.
This has absolutely nothing to do with the technology, it’s just that there is no standard protocol for one more thing simply because accessibility experts don’t happen to do some free work that will be de facto accepted by multiple different vendors. Comparatively, apple or google can just declare that this new API is the way to go, and support it natively from the de facto frameworks of the platform (and probably paid for accessibility experts along the way).
No one wants to use android and iOS for serious desktop work though. Like it's cool when your only interaction with the device is consuming content, definitely not for creating.
You might be living in a bubble. The majority of the population doesn’t have a PC, most people use a smartphone as their only general purpose computer. And while you may not run blender on your phone and render a full-time movie with ray-tracing there, there is absolutely no fundamental limitation, it just so happens to primarily target portable devices, not beasts of a machine with 4 video cards. This functionality requires zero special permission, neither does photoshop (of which there are multiple mobile-versions), or digital painting which, etc.
You would be surprised how many content creator gets by with a single ipad.
Android and iOS are limited operating systems for limited devices. They don't scale up to office workloads.
Certainly not because of their security model, lol. It's because they're phones.
But requiring user permissions for apps to do shady shit is a good thing. Cannot fathom why people are against that.
The security model is not the only factor, but it is one very important factor. Why do I have to open Moebius sync to keep syncthing synchronization running? Why is the whole landscape build under the assumption that a cloud storage is more trustworthy than local storage?
> But requiring user permissions for apps to do shady shit is a good thing.
> Why do I have to open Moebius sync to keep syncthing synchronization running
Because it’s a mobile OS and every single spent CPU cycle is a detriment to battery life? There is absolutely nothing in the security model that would prevent it from running - but it is essential that processes have a “structured” lifetime.
E.g. compare how much more graceful android is in low-memory situations, asking apps to serialize their state and then stopping the last used one. Linux oomkiller will just reap my whole display manager for some reason.
So that's another reason it doesn't scale to office workloads
Sigh. Okay, but Wayland doesn't work this way because it is a Desktop software. I don't understand the complaints here - we're upset that we, the user, are empowered to give and remove permissions from applications?
Nobody is turning Linux into iOS. But iOS DOES have some good ideas. It's good, for example, that for an app to access your photos library they have to ask. I know for a fact you prefer that to the app just opening your photos without your knowledge and doing whatever they want with them.
Similarly, I see no reason why Chrome should be able to read the display output and keyboard inputs of my graphical password manager. It should ask me.
In what way or form? Citation needed.
> You would be surprised how many content creator gets by with a single ipad.
Can you name one professional software developer?
Probably, you can. But I don't want to limit myself to that sub standard environment. I love my iPad for some activities, for others iOS is just impractical.
Pretty sure this specific thing is under similar restrictions on macOS.
The limiting factor is all the use cases that have not been invented yet. Screen sharing would have never been a thing if we started out with Wayland-like restrictions.
> A random app like xeyes should not be able to know mouse position at all times, for privacy reasons. (Unless you very explicitly gave xeyes super-extra permission to do that.)
The thing about xeyes isn't that its privacy invasive, is that it shows that xeyes knows what you're doing in other clients.
Got a gnome terminal root shell open? That's a privilege escalation method for any other client running on the desktop under Xorg. This itself, isn't really a problem, but chained with other attacks could be (e.g. browser escape).
> Got a gnome terminal root shell open? That's a privilege escalation method for any other client running on the desktop under Xorg. This itself, isn't really a problem, but chained with other attacks could be (e.g. browser escape).
Unless you're sandboxed up the ass, Wayland won't save you when that browser escape happens. Something I did 20 years ago to a friend as a prank that still works today on a typical Linux desktop with Wayland; wrap sudo to log the users password the next time they use it. I didn't use a browser exploit for that, but it can easily be done if you have write access to the user's environment however that happened. Wayland won't protect you from that sort of thing unless you're willing to commit to extensive sandboxing.
Isn’t is possible to get something like mouse position from almost any stdlib of most languages?
No, not really. Modern APIs usually don't even let you access that kind of information without additional privileges. Some older toolkits have functions that are supposed to do this, but it doesn't work everywhere.
It works fine on Windows and macOS's desktop APIs in 2024. This is a basic "desktop app" expected feature - if you aren't able to implement fully something like AutoHotkey or AppleScript macros, you aren't really a proper desktop platform.
You can implement all those things, just differently now.
Relatedly, now random apps can't record the whole screen. This is a good thing, now they get explicit user permission and go through xdg. I don't know how anyone could be opposed to this - it's objectively more empowering for you, the user.
You think enforcing the one true rights model to everyone is more empowering? We have obviously different definitions.
Yes, it's empowering because your "rights model" is one where you don't have rights. You don't know what applications are using what data, and you also can't stop it. Is that empowering? I think no.
But if you wanna argue Chrome should be able to read all your keyboard inputs whenever it wants be my guest. I can't fathom why people want that type of setup.
Can’t you get the cursor position from the windows c++ stdlib? Or like pyautogui? Or the Java stdlib?
Windows' pointer position is available to even the most limited GUI application; it is not protected information. For win32 at least, I am not familiar enough to say that about the newer app packages which are much more locked down, but I would be surprised.
They queried that info based on a specific implementation of a framework. If that framework is implemented under Wayland, then through the Wayland APIs it will only get rudimentary info on mouse position (e.g. only when it’s in focus and the mouse is over it).
App doesn't have to be in focus. It receives pointer events over its own surfaces (plus in some additional cases like drags and grabs). It doesn't receive any information about pointer position otherwise, nor its global coordinates (at least not without extra protocols).
Is it possible that one could write essentially a transparent app, which is always in focus and just passes click events through the transparent layer?
You can write a transparent app, but its the compositor that decides whether it's in focus and your app will either pass both hovers and clicks to the window behind (not staying in focus for long and not getting any pointer events even when it still is), or it will consume them all.
Hopefully only if the given app has focus.
Why? I want to have some applications that can always see the mouse cursor like, xeyes. Because that allows me to implement a better customized desktop environment.
Then give that piece of code extra-special permissions. As I’ve said in another comment, the days where you downloaded your software from the sunsite or tsx11.ai.mit.edu FTP servers and could be confident that it and all its dependencies were trustworthy are unfortunately gone for a very long time.
stdlib? Nope, a definite nope!
I won't expect stdlib.h to provide me a magical get_cursor_pos() function in any way.
GUI toolkits? (This includes the Win32 GUI.) Maybe.
It's not a universal thing though -- for example, you may receive mouse position _only_ when the cursor is in your window.
"stdlib" was shorthand for "the standard library of a given language" and it came across poorly.
I did a quick google before I asked you that question, and it looks like wayland is one of the few exceptions to completely exposing current cursor position. Most "stdlibs" do expose it.
Please refrain from using the term "stdlib", it has a specific meaning and I have trouble understanding what do you mean by it.
Are you referring to the native GUI toolkit of a platform (e.g. Winforms, Cocoa) or something else?
I will not refrain, you're the only one confused by it.
Just google it. You are incorrect and you're not pumped about it, I get it. The horse is fully beat to death. If you want to discuss further, my responses will probably just be lmgtfy links.
I find their point pedantic, but I tend to think it is correct. The "standard library" of a language is the library that comes with the language itself, usually across all platforms, and that rarely contains anything UI-wise beyond the very basics (e.g. text/terminal I/O like printing and reading lines).
I do agree that probably no-one was confused, though.
Check out the link a few comments down.
https://en.wikipedia.org/wiki/Standard_library
> In computer programming, a standard library is the library made available across implementations of a programming language.
You should show me that, among the mainstream programming languages, their standard library will provide get_cursor_pos().
[dead]
It's clearly you who's confusing "standard library" with "UI toolkit". Some standard libraries come with a toolkit, like Java, but that's an exception rather than rule.
Not that it makes a huge difference in the discussion, but if you want to be smug about it then at least make sure you're right first.
You literally contradicted yourself.
And an often under appreciated tenet of security — even a “good” software can be exposed to “bad” data, and you only need a bug (especially a memory bug, which is exceedingly common because linux userspace can’t get rid of c for the life of it) to have arbitrary code executed.
Like, your pdf reader is surely not evil, but do you trust every single pdf file you open?
I expect my PDF reader to be secure. If the PDF format is too complex to implement safely then the renderer should be sandboxed in the reader itself instead of preventing me from scripting using xdotool and similar.
And unless you fully sandbox your PDF reader then an exploit is going to have access to your user directory without any display server involvement anyway. X11 vs. Wayland doesn't even come into the picture.
It shouldn’t complicate the program itself, everything should be sandboxed by default.
And they should simply not have access to my home folder, it should be given access to a specific file only it is about to read.
That severy limits the usability and even functionality that programs can implement. If you want a phone os then go use one but don't make Desktop Linux into one.
I would argue that it is better; it works regardless of this fix and on all compositors.
Nice results. It's kind of wild that the "before" behavior would be considered acceptable/shippable by anyone. That kind of jank should be a show stopper.
Linux desktop is full of such jank. Fortunately, thanks to contributors like OP, the jank is gradually getting fixed. The whole point of Wayland is to eliminate a whole class of jank (screen tearing) that was essentially unfixable with X.
And yet, empirically, jank is far more common on Wayland than X11. With X11, on the rare case I noticed, it was usually just a matter of "okay, which toolkit or driver do I need to look up the vsync environment variable for?"
I have no opposition to Wayland in theory; my concerns are entirely practical and unignorable.
In my experience Wayland has been a better experience than X11, I have two monitors with different refresh rates and I could never eliminate tearing with X11 whilst Wayland works as expected.
Wayland does some things well; X11 does some things well. Unfortunately Wayland is not a straightforward improvement. At some point they climbed the abstraction ivory tower and lost sight of the system they were supposed to implement.
I have not noticed any tearing issues with X11 for a long time, with or without multiple monitors. Were you using X.Org graphics drivers or external vendor blobs?
I had that issue with both AMD and Nvidia with the open source and proprietary drivers respectively. If I ran both at 60Hz it was fine, but my primary monitor is 144Hz and I couldn’t make 144Hz and 60Hz work well under X11.
For me, fractional scaling on a HiDPI screen is broken on X11, but works well in Wayland. Here's an example from yesterday: I was playing Factorio full-screen, and some of the GUI elements were too large and wouldn't fit it in the screen. I found a Factorio setting that enabled Wayland, and it instantly fixed the problem. Kudos to Factorio for supporting Wayland! You wouldn't think it was needed for a full screen game, but it was.
This does not make sense at all as an argument really.
I think your argument is about specific implementations of WM. While the argument of "I deal with X11-based WMs because it's fine when I don't care about security at all" may be valid in very narrow cases (such as air-gapped systems), the argument more generally is pretty weak.
Its not surpising that x11 based WMs, such as the almighty [awesomeWM](https://github.com/awesomeWM/awesome), have more features implemented than, for instance, [jay](https://github.com/mahkoh/jay) due to the enormous time it has had to develop (though I am _very_ excited to see `jay` develop more fully, and expect it to be well used by the more tech-savy devs).
However, some WMs in the Wayland space are doing quite well on that front. I recently had some substantial problems arise in my system which (surprisingly to me, but perhaps some are getting used to this) would have been prevented by using a memory safety language for my WM, so I have made the switch to (for better or worse) only ever consider Wayland+Rust WMs. In this space, [niri](https://github.com/YaLTeR/niri) is actually quite good, and to the point - it is developing correctly _and very quickly_. So, any issues on some WM not implementing some desired feature are quickly disappearing.
IIRC, all the major 'gateway' linux distros, such as Ubuntu or Fedora, are all on Wayland by default now - so I don't imagine x11 will stay relevant much longer.
> With X11, on the rare case I noticed, it was usually just a matter of "okay, which toolkit or driver do I need to look up the vsync environment variable for?"
This is not something you should ever have to do
Heh? That hasn’t been true on even the very first POC wayland compositor, Weston. I mean, it used to crash from time to time in the very early days, but visual artifacts? I don’t remember any, besides the occasional xwayland app (which is literally an X app running inside wayland).
Yeah I have experienced all sorts of flickering and artifacts on Wayland, especially on nvidia. Recently it has gotten a lot better though.
FWIW, Nvidia problems on Wayland were Nvidia's problems, not Wayland's. Wayland was just a victim of Nvidia's bugs and lacking implementations.
Except for tearing, which turned out to be a combination of Mesa/DRM bugs and Wayland naively assuming the behaviour of those bugs.
Then nVidia decided to switch to GBM/EGL way of doing things and it turned out everyone had incorrect assumptions...
I don't think that's an honest representation of what happened. Wayland relied on implicit sync by design, it wasn't "a bug" but a choice. The fact that in the end there were valid reasons to start moving towards explicit sync besides of lacking Nvidia implementation is irrelevant to the fact that Nvidia did not implement a correct driver for the existing requirements of the platform.
It relied on implicit sync as far as I know by accident. But since it worked for the popular configuration, no one dug deeper that support for accelerated drawing without multi-frame lag of manually calling commit on surfaces had gaps in design.
Implicit sync worked by design. There were years of discussions whether it's a good idea to rely on it and how exactly explicit sync would actually look like, with protocols such as zwp_linux_explicit_synchronization_v1 (eventually deprecated) being worked on almost a decade ago already, but it's definitely not an "accident". Explicit sync offers several advantages, but implicit sync is how DRM was designed and what Wayland consciously relied on, and what Nvidia disregarded because it would be rather challenging to implement it in their driver.
The ecosystem slowly moves towards explicit sync now, but Nvidia was supposed to provide a driver already before that was happening and they didn't comply with platform's requirements at date, resulting in user-facing issues. With this one, they just got lucky that the consensus happens to move towards what they had already asserted.
You mean this https://www.phoronix.com/news/NVIDIA-EGL-Wayland-1.15 and the multy year story that I can guess from the comments in there?
Explicit Sync, yes. The reason why I consider previous implicit sync a bug is that it was essentially unfounded expectation - nothing guaranteed you'd get a magic mutex in the sky that synchronized renders for you, and in fact, you do not want one for maximum performance. Lack of explicit sync also essentially meant common multi-gpu setups would be broken.
Androids graphics stack and drivers also use explicit sync, it isn't just a Nvidia thing.
I was not saying it's an nvidia thing, just that significant majority of devices used by wayland proponents had undeclared implicit sync issues that were invalidated when faced with a driver that didn't have them.
> And yet, empirically, jank is far more common on Wayland than X11
It depends on the type of jank you're talking about. It's wholly disingenuous to characterize Wayland as janky but x11 as not - the jank inherent to x11 is what made the original Xorg developers start making Wayland in the first place. Empirically, if x11 was perfectly fine there would be no motivation to design a successor.
x11 gets the first-mover advantage of a lot of implementations and a straightforward design goal, but that's about it. It's not secure enough to recommend as a serious alternative to Mac and Windows users, and it's too slow and unopinionated to present a friendly face to new users. Features like 1:1 trackpad gestures, genuine HDR pipelines and a locked compositor framerate are all getting to the point that regular consumers expect them as default.
If you want to keep using x11, it's unlikely someone is going to take it away from you. But it's on track for depreciation and hasn't been actively developed in years. Recommending it as a panacea to new users is a bad idea.
> It's not secure enough to recommend as a serious alternative to Mac and Windows users
The security risk of X11 is theoretical, not practical. Yes, X11 programs can maliciously keylog each other, but this just isn't a thing that actually happens. And even if you do start installing random malware from the internet like a classic windows user, Wayland isn't going to prevent you from screwing yourself anyway. To actually be safe while installing and running malicious applications you need extensive sandboxing. Wayland can be one part of that sandboxing but is useless without the rest (to prevent the malware from stealing user files including credentials, using LD_PRELOAD hacks or similar to keylog other applications anyway, etc), and no distro suitable for recommending to Windows/MacOS newbs has the rest of the requisite sandboxing. The sandboxing touted by Wayland advocates is very esoteric and without all that sandboxing, a newb using Wayland has to exercise just as much caution when downloading software as if he were using X11.
> Yes, X11 programs can maliciously keylog each other, but this just isn't a thing that actually happens.
Are you really saying keyloggers do not exist in the wild???
> Wayland can be one part of that sandboxing but is useless without the rest
Yes but that is the point, and if you turn it the other way around X11 usually makes the rest of the sandboxing useless.
I am saying that X11 users getting hit with keyloggers is extremely rare if it happens at all. It's trivial to make such a keyloggers but that doesn't mean it's common for the programs users install to do it.
And yes, X11 makes that sandboxing useless, but that sandboxing isn't in play anyway because we're talking about noobs from Windows.
I think it is dangerous to feel yourself as superior. UNIX veterans can also be victim of supply chain attack, either from their distro package manager, a language module manager, a malicious flatpak with too much access, a trojaned appimage from a supposedly reputable vendor, etc.
Superior? All I'm doing is sticking to programs in my distro's repos. Such programs including malicious keyloggers is unheard of. And if a maintainer turns evil or let's something slip through by accident, the damage they could do is hardly mitigated by Wayland. Thankfully, this virtually never happens
In principle I am also vulnerable to something like a RCE zero day in Firefox turning an otherwise trusty program into malware which exploits X11's open nature, but again, this sort of thing actually happening is unheard of.
I'm not superior, I'm just trying to keep a realistic grip on the threats I face. Modern security culture is fixated on what is theoretically possible, I care more about what is actually likely.
> the damage they could do is hardly mitigated by Wayland.
I disagree. How do you hijack interprocess communication on a Wayland device? I can tell you in very certain steps how to manipulate an x11 client but outside hijacking /dev/ I can't imagine a similar attack on Wayland.
If you aren't sandboxes you can edit the user's environment to swap out their programs with backdoored copies.
There's no rule that an X11 server has to send all keypresses to all processes.
But the core framework of every x11 server relies on a loop that does poll HID connected to your machine for events. You can have an x11 server running with no windows connected and your keypresses will still be broadcast to any program that knows where to look.
That's exactly the same as Wayland, isn't it? Every Wayland compositor also polls HIDs connected to your machine for events. And if your Wayland compositor has permission to open those HIDs when running as your user account, so does any other program you run, regardless of Wayland or X.
> Yes, X11 programs can maliciously keylog each other, but this just isn't a thing that actually happens.
A simple search leads me to this: https://github.com/anko/xkbcat
There isn't a real attack using it yet, only because attacking Desktop Linux is a really unprofitable endeavor (considering the marketshare, the ROI must be very low).
> To actually be safe while installing and running malicious applications you need extensive sandboxing
FWIW, X11 is unsandboxable unless you run a second X server on top of your current server [0]. Which is fine, but you need to consider that most, if not all sandboxing solutions on Linux that "newbs" use, like Flatpak, do not employ such technique when running sandboxed X11 applications.
The "security by default" behavior of Wayland limits the possible attack surface a lot, without requiring the end user to understand all the nitty details involved.
[0]: https://wiki.archlinux.org/title/Bubblewrap#Sandboxing_X11
Why is X11 unsandboxable? A similar but reverse approach to Xwayland, something like waylandX could be used to be part of the overall sandbox approach to run untrusted applications. That would have the advantage that the severe restrictions and feature degradations of wayland are only applied to those untrusted sandboxed applications, not everything.
Ultimately, X11 opens up everything. What you suggest (WaylandX) is essentially allow-by-default.
When this is the case and there is a supply chain attack, what you think is a trusted application (and therefore not running under "WaylandX") can very well keylog you or take screenshots of your desktop without your consent.
In a deny-by-default model ala Wayland, applications will have to ask for permissions before they can do something considered to be privileged.
You're not telling me anything I don't already know and haven't already explained. X11 keyloggers are trivial, and virtually never seen in the wild. X11 makes sandboxing impossible, but that doesn't matter because I'm not going to waste my time on something like Qubes anyway, and newbs from Windows aren't being directed to setups like that either. They're all installing Mint or Ubuntu where the security of Wayland is nullified by the absense of sandboxing.
>newbs from Windows aren't being directed to setups like that either. They're all installing Mint or Ubuntu where the security of Wayland is nullified by the absense of sandboxing.
This cannot be more further from the truth. Amongst the newcomers, it is rather popular nowadays for them to use Flatpak-bundled apps, especially with the rise of SteamOS (the Deck essentially) lots of Linux newcomers are in fact first exposed to Flatpak and running untrusted executables in a sandbox.
And the most prominent "untrusted executable" today to those newcomers has to be Bottles, which is a nice GUI wrapper for Wine and is sandboxed (if you enable wine-wayland, of course).
I wouldn't trust flatpak enough to run a truly untrusted executable. I am sure flatpak's isolation is full of holes unrelated to windowing.
But I don't think a game purchased through steam counts as untrusted.
>I wouldn't trust flatpak enough to run a truly untrusted executable. I am sure flatpak's isolation is full of holes unrelated to windowing.
As compared to running untrusted programs completely naked?
>But I don't think a game purchased through steam counts as untrusted.
Bottles is there for people to run any Win32 program, not just Steam games. And I shouldn't have to tell you how many malicious Win32 programs there are.
Just google the criticisms of flatpak from a security perspective. They're out there.
Containerization on Linux was never intended to be a security feature for totally untrusted, malicious code. It's isolation for trusted code. If your scenario relies on securely running untrusted executables in a Linux container you are doing stupid things.
I am well aware of the weak points of Flatpak. But are you suggesting that running applications in a container is not more secure than running an executable completely naked?
You see: If you want absolute security, for sure, go for a full-fledged VM! Or run something like QubesOS. It is a completely reasonable decision.
However, malice certainly has degrees, and the "mildly malicious" programs most likely cannot take advantage of sandbox escaping exploits. If Flatpak can stop 95% of all attacks (relative to running a program completely without sandboxing), that is already a win in my book.
But I will note again that X11 is a big hole (as in, almost a complete free-for-all) for sandbox escaping in Flatpak.
You seem to think a lot of things that aren't security boundaries are security boundaries. There have been VM escapes too. VMs are not for running untrusted OS images you get from end users.
I'm done with this thread, have a nice day.
Has there ever been a single instance of a game from Steam including an X11 keyloggers? These threats are off in fantasy land.
I am not sure where you got the impression of me talking about only "Steam games". Bottles allows you to run any Win32.
And besides that, "these threats are off in fantasy land" is an invalid defense in my opinion, considering the (quite sophisticated) XZ Utils backdoor happened not too long ago! Like I said, if such an attack towards X11 hasn't been deployed in the wild, it can only suggest such endeavor is unprofitable, not because the threats are fantastical.
XZ utils backdoor could have exploited X11, but didn't. And the most common Wayland configurations wouldn't have protected people from the backdoored utility; only extremely paranoid and therefore esoteric setups might have.
> And the most common Wayland configurations wouldn't have protected people from the backdoored utility
If the attacker decided to backdoor an utility and make use of X11, it is most likely the backdoored utility will listen to keyboard events, read the bitmaps of other X11 clients.
And there's nothing that can stop the backdoor from doing so on X11...
Anyways, if you are saying the Wayland security policies are unneeded because there hasn't been an attack on X11 (this is the fundamental disagreement between us), consider the following: You don't install doors in your premise, because there hasn't been a case of burglary in your neighborhood?
> x11 gets the first-mover advantage of a lot of implementations and a straightforward design goal, but that's about it. It's not secure enough to recommend as a serious alternative to Mac and Windows users, and it's too slow and unopinionated to present a friendly face to new users.
I've been recommending it as a serious alternative for years, and it's always presented a friendly face.
> Features like 1:1 trackpad gestures, genuine HDR pipelines and a locked compositor framerate are all getting to the point that regular consumers expect them as default.
I have never heard anyone not already a Linux user comment on even one of those as a problem.
>I have never heard anyone not already a Linux user comment on even one of those as a problem.
Those specific words are uttered by Linux users, true. However, Linux beginners _do_ notice some X11 issues, it's just that very frequently they only know "something" is off but not why they feel so.
From anecdotal experience, touchpad gestures are actually something my friend complained so YMMV. We ended up making a file in /etc/X11/xorg.conf.d/ to configure the synaptics driver. Another experience had to do with screen tearing, I helped them fix it by installing a compositor.
To be fair, my anecdotal coworker evidence with Wayland has been "the Webex client doesn't work" with the ten-second remedy of showing them how to set their default desktop back to X11.
That only tells us X11 has a first-mover advantage. Wayland supports screencasting too.
>I have never heard anyone not already a Linux user comment on even one of those as a problem
....because Windows and Mac users have not had these problems since 2006?
That's not what he said.
... while introducing lot's of other jank that was solved before.
And once all the Wayland jank is fixed it's going to be just as "crusty" and "full of hacks" and "unmaintainable" as X is now and the cycle is going to repeat again :|
The real jank was the friends we met along the way.
I assume you know this, but just to be sure: The "before" state only happened in a fallback codepath when using a legacy application, i.e. an X11 app in the modern Wayland-based desktop. When using modern apps, or when using the entire desktop in legacy X11 mode, this doesn't happen.
Most legacy X11 apps in active use are actually games which tend to be fullscreen and not resized, so it wouldn't happen that often for many users.
(That said, sure, I can also respect the stance as I tend to place a premium on glitch-freeness too.)
This didn't happen on Hummingbird Exceed or Xorg on Windows. It's not X's fault that rendering is wrong in the encapsulating windowing system.
Nor did I claim it is.
My comment was concerned with making it abundantly clear that this glitch happened in specific scenarios and not just any and all window resizing.
My window manager had the “before” behavior, but I complained it was defective and they gave it to me free. Not sure how much it would have cost beforehand, but I can’t complain after a 100% discount, right?
"Jank" is still functional. And, to a first order approximation, nobody cares.
Even Windows generally switches down to software rendering when resizing. And on mobile devices, nobody resizes.
No pointer warp, however, is a failure (CAD packages all have significant issues on Wayland). Lack of multilingual support/accessibility is a failure. Lack of screenshots/screencasts is a failure. Lack of support for the BSD lineages is a failure. etc.
People are still bitching up a storm because Wayland still (Pipewire does screen sharing using DBUS(!) for example) hasn't fixed basic things while DeadRat is shoving it down everybody's throat by dropping X11 support.
The Wayland devs aren't wrong about the security implications of this kind of stuff. However, they're also not giving anybody solutions, either.
One big issue is that Wayland devlopment is so slow that the entire space moved forward and destroyed a bunch of assumptions that Wayland is based around.
> Lack of support for the BSD lineages is a failure.
Not true. FreeBSD and OpenBSD have had various wlroots compositors such as Sway available for some time. e.g [1] Some people have even been experimenting with KDE-wayland on FreeBSD since 2021.
Wayland support on any OS is not binary because there is no single layer like Xorg. It's a matter of individual compositors and their components being ported to the OS, which is a matter of popularity, and the BSDs are always at a disadvantage in that respect, so they lag behind, same as other software. Nevertheless, they are definitely gaining support. Then again this distinction is only technical, for functional support new DEs/WMs would always have needed to be ported regardless of display architecture, the only case it would not is for an Xorg drop-in, which defeats the purpose.
> Lack of screenshots/screencasts is a failure.
... but there are many functional screencapture apps, and even browser support, I use this pretty much every week. I think you might be operating on out of date info. I'd highly recommend giving it another try, the Wayland ecosystem has come quite far over the last decade.
[1] https://docs.freebsd.org/en/books/handbook/wayland/
Wayland unfortunately is a mess that is hard to implement so we get a balkanized situation where not only everything often boils down to Big Important Process in the middle (that actually integrates more things than X.Org used to), the set of features it supports are way more balkanized and most importantly, specific to that one Big Important Process as they can't be viably separated out.
And even then you have to deal with a mix because you have to work through two different unsynced possibly broken in weird ways connections (Wayland + D-Bus).
This results in how last week I couldn't screenshare under Wayland, and had zero chances to figure it out - and to make it worse since I had some important state and no time to play with reboots, I couldn't fix it.
XOrg was predominantly used as a useless middleman between a compositor and a window manager communicating with each other. Just because the wayland compositor is “one big important process” doesn’t mean that the whole architecture is more complicated.
Multiple IPCs make the whole stuff way more complex, there is a reason why we have a big monolith as browsers and not some `curl | htmlRender | jsInterpreter` Rube Goldberg machine — the unix philosophy is not the pan ultimate design, it works in some cases and is utterly broken in others.
X11 != the lowest common denominator implementation of X11. Nor was everyone using it with big heavy compositors (honestly, the only reason I used a compositor for years was intel removing sync circuitry from their iGPUs).
And Wayland did not stop requiring multiple IPCs, in fact it mandates them as several features require passing data by secondary channels, not just for Portals et al - some don't even have any described way to reliably pass the information that I have seen (like how to pass around cookie to let one application activate window of another? Or maybe the spec is such a mess that I'm looking at completely wrong place when I tried to fix ability to reliably switch between applications without extending compositor).
And yes, the architecture is more complicated in practice, otherwise it might have reached parity with what X11 did after as many years - like input method support. Unfortunately it's so broken that you have multiple versions of it in practice, it requires extra IPC in practice, and at least in my experience just does not work.
If you know how to make screen sharing work under Sway on FreeBSD, please do share. I have not managed to figure it out.
I'm using Debian so I can't say how well this applies to FreeBSD:
You need to have `xdg-desktop-portal-wlr` or equivalent package installed (should be depended upon by sway package really), and also the gtk one for some other stuff wlroots doesn't implement like notifications I think (don't worry it doesn't drag in all of gnome).
This is technically the only thing you should need to do beyond picking a wayland compatible screen capture program or browser - However it's possible the dbus variables are broken if you wrote your own sway config from scratch (guilty)... Depending on your OS, the default config should do this automatically, on Debian it includes /etc/sway/config.d/50-systemd-user.conf, I can't say for FreeBSD. So point is if your config omits the dbus-update-activation-environment command, you probably need to include it in your config.
TL;DR
And include this at the top of your sway.conf Those environment variables are set by sway, but this line imports them into dbus (is my limited understanding) so that programs call the correct xdg portal backend.Remember to restart sway to test this.
Whoops, you will probably need to remove the --systemd flag.
> The Wayland devs aren't wrong about the security implications of this kind of stuff.
But they are wrong. Their security model assumes an ecosystem of untrusted apps when we already had something far superior: distributions with vetted packages.
Wayland is like bolting down your furniture because if you let in random strangers from the street they might steal your stuff. Instead of adding obstacles that make my own life harder I prefer to keep better company.
Vetted packages are great, but they aren't the end of all security problems. Good security needs defense-in-depth.
It's rare but not unheard of for someone to be able to sneak malicious code into a vetted package. It's extremely common for vetted packages to have security vulnerabilities that could be exploited.
I don't want someone who finds a vulnerability in a fart app to be able to escalate that to attack other apps on my computer.
I trust my accountant with a lot of sensitive data but I don't give them the keys to my house. I trust a friend with my house keys to water plants while I'm gone, but I don't give them the password to my bank account.
What good security needs more than anything is proportionality. After all the only criterium you care about is how secure your computer is then you should simply turn it off because nothing else is going to beat that.
> It's rare but not unheard of for someone to be able to sneak malicious code into a vetted package. It's extremely common for vetted packages to have security vulnerabilities that could be exploited.
Ok, when is the last time you or anyone you know personally sustained any nontrivial damage because of such an event? You can make up hypotheticals to scare people all you want but the simple fact is that no, people are not actually in any more danger on their computer than they are just being alive.
The FreeBSD handbook claims that KDE works with Wayland but I have never gotten it to work. X11 still works flawlessly.
[flagged]
Hmm. Even with the fix it didn't look very smooth even if it is a lot better. I wonder if that is due to the recording, or due to the mouse settings (mouse poll rate and acceleration settings are very impactful on smoothness), due to X or the toolkit, or due to it not being as good as it can be.
I think issue is the screen capture tool or a framerate mismatch in transcoding. The video is 59fps, but the mouse cursor only updates every few frames. Usually it's every other frame, but sometimes it takes three frames or more to update, and sometimes it updates in two or more consecutive frames. On most of the frames where the cursor moves, the window size also changes.
Nice! It would be cool for Wine Wayland driver to also fix window resizing (regedit currently is blinking all over when being resized).
Is this really wine on wayland, or is it the default wine on xwayland on wayland?