People started using Sysprof to have insights about the internal states of Webkit, gather data on how long different operations took, and more. Eventually we started hitting some limitations in Sysprof, mostly in the UI itself, such as lack of correlational and visualization features.

Earlier this year a rather interesting enhancement in Sysprof was added: it is now possible to filter the callgraph based on marks. What it means in practice is, it’s now possible to get statistically relevant data about what’s being executed during specific operations of the app.

In parallel to WebKit, recently Mesa merged a patch that integrates Mesa’s tracing routines with Sysprof. This brought data from yet another layer of the stack, and it truly enriches the profiling we can do on apps. We now have marks from the DRM vblank event, the compositor, GTK rendering, WebKit, Mesa, back to GTK, back to the compositor, and finally the composited frame submitted to the kernel. A truly full stack view of everything.

So, what’s the catch here? Well, if you’re an attentive reader, you may have noticed that the marks counter went from this last year:

To this, in March 2025:

And now, we’re at this number:

I do not jest when I say that this is a significant number! I mean, just look at this screenshot of a full view of marks:

Naturally, this is pushing Sysprof to its limits! The app is starting to struggle to handle such massive amounts of data. Having so much data also starts introducing noise in the marks – sometimes, for example, you don’t care about the Mesa marks, or the WebKit marks, of the GLib marks.

Hiding Marks

The most straightforward and impactful improvement that could be done, in light of what was explained above, was adding a way to hide certain marks and groups.

Sysprof heavily uses GListModels, as is trendy in GTK4 apps, so marks, catalogs, and groups are all considered lists containing lists containing items. So it felt natural to wrap these items in a new object with a visible property, and filter by this property, pretty straightforward.

Except it was not

Turns out, the filtering infrastructure in GTK4 did not support monitoring items for property changes. After talking to GTK developers, I learned that this was just a missing feature that nobody got to implementing. Sounded like a great opportunity to enhance the toolkit!

It took some wrestling, but it worked, the reviews were fantastic and now GtkFilterListModel has a new watch-items property. It only works when the the filter supports monitoring, so unfortunately GtkCustomFilter doesn’t work here. The implementation is not exactly perfect, so further enhancements are always appreciated.

So behold! Sysprof can now filter marks out of the waterfall view:

Counters

Another area where we have lots of potential is counters. Sysprof supports tracking variables over time. This is super useful when you want to monitor, for example, CPU usage, I/O, network, and more.

Naturally, WebKit has quite a number of internal counters that would be lovely to have in Sysprof to do proper integrated analysis. So between last year and this year, that’s what I’ve worked on as well! Have a look:

Unfortunately it took a long time to land some of these contributions, because Sysprof seemed to be behaving erratically with counters. After months fighting with it, I eventually figured out what was going on with the counters, and wrote the patch with probably my biggest commit message this year (beat only by few others, including a literal poem.)

Wkrictl

WebKit also has a remote inspector, which has stats on JavaScript objects and whatnot. It needs to be enabled at build time, but it’s super useful when testing on embedded devices.

I’ve started working on a way to extract this data from the remote inspector, and stuff this data into Sysprof as marks and counters. It’s called wkrict. Have a look:

This is far from finished, but I hope to be able to integrate this when it’s more concrete and well developed.

Future Improvements

Over the course of an year, the WebKit project went from nothing to deep integration with Sysprof, and more recently this evolved into actual tooling built around this integration. This is awesome, and has helped my colleagues and other contributors to contribute to the project in ways it simply wasn’t possible before.

There’s still *a lot* of work to do though, and it’s often the kind of work that will benefit everyone using Sysprof, not only WebKit. Here are a few examples:

• Integrate JITDump symbol resolution, which allows profiling the JavaScript running on webpages. There’s ongoing work on this, but needs to be finished.
• Per-PID marks and counters. Turns out, WebKit uses a multi-process architecture, so it would be better to redesign the marks and counters views to organize things by PID first, then groups, then catalogs.
• A new timeline view. This is strictly speaking a condensed waterfall view, but it makes it more obvious the relationship between “inner” and “outer” marks.
• Performance tuning in Sysprof and GTK. We’re dealing with orders of magnitude more data than we used to, and the app is starting to struggle to keep up with it.

Some of these tasks involve new user interfaces, so it would be absolutely lovely if Sysprof could get some design love from the design team. If anyone from the design team is reading this, we’d love to have your help

Finally, after all this Sysprof work, Christian kindly offered me to help co-maintain the project, which I accepted. I don’t know how much time and energy I’ll be able to dedicate, but I’ll try and help however I can!

I’d like to thank Christian Hergert, Benjamin Otte, and Matthias Clasen for all the code reviews, for all the discussions and patience during the influx of patches.

First let me start with a massive thank you for everyone that helped organize the event. It looked extremely well rounded, the kind of well rounded that can only be explained by a lot of work from the organizers. Thank you all!

Preparations

For this GUADEC I did something special: little calendars!

These were 3D printed from a custom model I’ve made, based on the app icon of GNOME Calendar. I brought a small batch to GUADEC, and to my surprise – and joy – they vanished in just a couple of hours in the first day! It was very cool to see the calendars around after that:

Talks

This year I gave two talks:

• GNOME Live Streaming & Outreach
• State of Portals

The first one was rather difficult. On the one hand, streaming GNOME development and interacting with people online for more than 6 years has given me many anecdotal insights about the social dynamics of free software. On the other hand, it was very difficult to materialize these insights and summarize them in form of a talk.

I’ve received good feedback about this talk, but for some reason I still left it feeling like it missed something. I don’t know what, exactly. But if anyone felt energized to try some streaming, goal accomplished I guess?

The second talk was just a regular update on the XDG Desktop Portal project. It was made with the sole intention of preparing territory for the Flatpak & Portals BoF that occurred later. Not much to say about it, it was a technical talk, with some good questions and discussions after that.

As for the talks that I’ve watched, to me there is one big highlight for this GUADEC: Emmanuelle’s “Getting Things Done in GNOME”.

Emmanuele published the contents of this talk in article form recently.

Sometimes, when we’re in the inflection point towards something, the right person with the right sensitivities can say the right things. I think that’s what Emmanuele did here. I think myself and others have already been feeling that the “maintainer”, in the traditional sense of the word, wasn’t a fitting description of how things have been working lately. Emmanuele gifted us with new vocabulary for that: “special interest group”. By the end of GUADEC, we were comfortably using this new descriptor.

Photography

It’s not exactly a secret that I started dipping my toes towards a long admired hobby: photography. This GUADEC was the first time I traveled with a camera and a lens, and actually attempted to document the highlights and impressions.

I’m not going to dump all of it here, but I found Brescia absolutely fascinating and fell in love with the colors and urban vision there. It’s not the most walkable or car-free city I’ve ever been, but there are certain aspects to it that caught my attention!

The buildings had a lovely color palette, sometimes bright, sometimes pastel:

Some of the textures of the material of walls were intriguing:

I fell in love with how Brescia lights itself:

I was fascinated by how many Vespas (and similar) could be found around the city, and decided to photograph them all! Some of the prettiest ones:

The prize for best modeling goes to Sri!

Conclusion

Massive thanks to everyone that helped organize GUADEC this year. It was a fantastic event. It was great to see old friends, and meet new people in there. There were many newcomers attending this GUADEC!

And here’s the GUADEC dinner group photo!

Given the principles of free software, one would think it would be the best possible place to advocate for accessibility. After all, there’s a collection of ideologically motivated individuals trying to craft desktops to themselves and other fellow humans. And yet, when you look at the current state of accessibility on the Linux desktop, you couldn’t possibly call it good, not even sufficient.

It’s a tough situation that’s forcing people who need assistive technologies out of these spaces.

I think accessibility on the Linux desktop is in a particularly difficult position due to a combination of poor incentives and historical factors:

• The dysfunctional state of accessibility on Linux makes it so that the people who need it the most cannot even contribute to it.
• There is very little financial incentive for companies to invest in accessibility technologies. Often, and historically, companies invest just enough to tick some boxes on government checklists, then forget about it.
• Volunteers, especially those who contribute for fun and self enjoyment, often don’t go out of their ways to make the particular projects they’re working on accessible. Or to check if their contributions regress the accessibility of the app.
• The nature of accessibility makes it such that the “functional progression” is not linear. If only 50% of the stack is working, that’s practically a 0%. Accessibility requires that almost every part of the stack to be functional for even the most basic use cases.
• There’s almost nobody contributing to this area anymore. Expertise and domain knowledge are almost entirely lost.

In addition to that, I feel like work on accessibility is invisible. In the sense that most people are simply apathetic to the work and contributions done on this area. Maybe due to the dynamics of social media that often favor negative engagement? I don’t know. But it sure feels unrewarding. Compare:

Now, I think if I stopped writing here, you dear reader might feel that the situation is mostly gloomy, maybe even get angry at it. However, against all odds, and fighting a fight that seems impossible, there are people working on accessibility. Often without any kind of reward, doing this out of principle. It’s just so easy to overlook their effort!

So as we prepare for the Global Accessibility Awareness Day, I thought it would be an excellent opportunity to highlight these fantastic contributors and their excellent work, and also to talk about some ongoing work on GNOME.

If you consider this kind of work important and relevant, and/or if you need accessibility features yourself, I urge you: please donate to the people mentioned here. Grab these people a coffee. Better yet, grab them a monthly coffee! Contributors who accept donations have a button beneath their avatars. Go help them.

Calendar

GNOME Calendar, the default calendaring app for GNOME, has slowly but surely progressing towards being minimally accessible. This is mostly thanks to the amazing work from Hari Rana and Jeff Fortin Tam!

Hari recently wrote about it on Mastodon. In fixing one issue, Hari accidentally fixed at least two other issues. Jeff, as an exemplary product manager and co-maintainer, was the one who noticed and also blogged about these collateral fixes.

If you consider this kind of work important, please consider getting them a coffee!

Hari Rana

@TheEvilSkeleton

Support

Jeff Fortin Tam

@jfft

Website¹

Elevado

Back when I was working on fixing accessibility on WebKitGTK, I found the lack of modern tools to inspect the AT-SPI bus a bit off-putting, so I wrote a little app to help me through. Didn’t think much of it, really.

But the project started getting some attention when Bilal Elmoussaoui contributed to it while testing some accessibility work in GNOME Shell. After that, Matthias Clasen – of GTK fame – and Claire – a new contributor! – started sending some nice patches around.

In preparation for the Global Accessibility Awareness Day we have made the first public release of Elevado! The project is evolving mostly without me these days, and it’s all thanks to these people.

Claire

@qwery

Support

Bilal Elmoussaoui

@bilelmoussaoui

Support

GTK

Of course, almost nothing I’ve mentioned so far would be possible if the toolkit itself didn’t have support for accessibility. Thanks to Emmanuele Bassi GTK4 received an entirely new accessibility backend.

Over time, more people picked up on it, and continued improving it and filling in the gaps. Matthias Clasen and Emmanuele continue to review contributions and keep things moving.

One particular contributor is Lukáš Tyrychtr, who has implemented the Text interface of AT-SPI in GTK. Lukáš contributes to various other parts of the accessibility stack as well!

Emmanuele Bassi

@ebassi

Support

Lukáš Tyrychtr

@tyrylu

Matthias Clasen

@matthiasc

Design

On the design side, one person in particular stands out for a series of contributions on the Accessibility panel of GNOME Settings: Sam Hewitt. Sam introduced the first mockups of this panel in GitLab, then kept on updating it. More recently, Sam introduced mockups for text-to-speech (okay technically these are in the System panel, but that’s in the accessibility mockups folder!).

Please join me in thanking Sam for these contributions!

Sam Hewitt

@snwh

Infrastructure

Having apps and toolkits exposing the proper amount of accessibility information is a necessary first step, but it would be useless if there was nothing to expose to.

Thanks to Mike Gorse and others, the AT-SPI project keeps on living. AT-SPI is the service that receives and manages the accessibility information from apps. It’s the heart of accessibility in the Linux desktop! As far as my knowledge about it goes, AT-SPI is really old, dating back to Sun days.

Samuel Thibault continues to maintain speech-dispatcher and Accerciser. Speech dispatcher is the de facto text-to-speech service for Linux as of now. Accerciser is a venerable tool to inspect AT-SPI trees.

Eitan Isaacson is shaking up the speech synthesis world with libspiel, a speech framework for the desktop. Orca has experimental support for it. Eitan is now working on a desktop portal so that sandboxed apps can benefit from speech synthesis seamlessly!

One of the most common screen readers for Linux is Orca. Orca maintainers have been keeping it up an running for a very long time. Here I’d like to point out that we at Igalia significantly fund Orca development.

I would like to invite the community to share a thank you for all of them!

Eitan Isaacson

@eeejay

Mike Gorse

@mgorse

Samuel Thibault

@sthibaul

… and more!

I tried to reach out to everyone nominally mentioned in this blog post. Some people preferred not to be mentioned. I’m pretty sure I’ve never got to learn about others that are involved in related projects.

I guess what I’m trying to say is, this list is not exhaustive. There are more people involved. If you know some of them, please let me encourage you to pay them a tea, a lunch, a boat trip in Venice, whatever you feel like; or even just reach out to them and thank them for their work.

If you contribute or know someone who contributes to desktop accessibility, and wishes to be here, please let me know. Also, please let me know if this webpage itself is properly accessible!

A Look Into The Future

Shortly after I started to write this blog post, I thought to myself: “well, this is nice and all, but it isn’t exactly robust.” Hm. If only there was a more structured, reliable way to keep investing on this.

Coincidentally, at the same time, we were introduced to our new executive director Steven. With such a blast of an introduction, and seeing Steven hanging around in various rooms, I couldn’t resist asking about it. To my great surprise and joy, Steven swiftly responded to my inquiries and we started discussing some ideas!

Conversations are still ongoing, and I don’t want to create any sort of hype in case things end up not working, but… maaaaaaybe keep in mind that there might be an announcement soon!

Huge thanks to the people above, and to everyone who helped me write this blog post

¹ – Jeff doesn’t accept donations for himself, but welcomes marketing-related business

Let’s take a quick look at what’s new in this release!

New Devices

Boatswain 5.0 comes with support for 2 new device models from Elgato: Stream Deck Plus, and Stream Deck Neo.

Support for Elgato Stream Deck Plus came after the massively successful fundraising campaign from last year. A huge thanks to everyone that contributed to it!

As for Elgato Stream Deck Neo, I tentatively introduced support for it without actually having a device to test, so if there’s anyone out there that can test it, that’d be absolutely appreciated.

Support for Stream Deck Plus was probably the main reason it took so long to release Boatswain 5.0. The entirety of the app was originally written under the assumption that all devices were simply a grid of buttons. Introducing a touchscreen, and dials that act as buttons, required basically rewriting most of the app.

I used this opportunity to make Boatswain able to handle any kind of device, with any kind of layout. Everything is represented as regions in a grid layout. Simple Stream Deck devices just contain a button grid; Stream Deck Plus contains a button grid, a touchscreen, and a dial grid.

Keyboard Shortcuts

The new Keyboard Shortcut action allows executing any keyboard shortcut – or any keyboard event in general – on the desktop. This seems to work better than I could have anticipated!

Under the hood, this action uses the Remote Desktop portal be able to inject input on the desktop. Locally controlling the desktop was probably not on the original goals of the portal, but alas, it fit the use case perfectly!

Paired with folders, Keyboard Shortcuts are very powerful, especially for large and complex software with a large number of shortcuts.

Next Steps

This release might be a little disappointing as it took so long, and yet didn’t come as packed with new features. And yet, this was the largest release of Boatswain, perhaps larger than the initial release even.

I’ve reached a point where I’m mostly satisfied with how the internals work now. So much so that, right after the Boatswain 5.0 release, I was able to split the core logic of the app into an internal library, and hide device-specific details from the rest of the app. This paved the way for adding a testing framework using umockdev, and also will allow adding support for devices from other brands such as Loupedeck. If you have any Stream Deck-like device and wish to see it supported in Boatswain, now’s your chance!

For Boatswain 6, I personally want to focus on 2 major features:

1. Make Boatswain use the new USB portal. One of my goals with Boatswain is to make it a reference app, using the most modern platform features available – and adding missing features if necessary. The USB portal is an obvious choice!
2. Remove X11 support. This might come as a controversial decision, but I don’t personally use X11 anymore, do not support it, and will not work on fixing bugs that only exist there. As such, I think it’s fair to just remove X11 support from the apps that I maintain. Practically speaking, this just means removing --socket=fallback-x11, and users can add back this permission using Flatseal; but please do not expect any kind of support anymore.

Some features that would be lovely to have, but we currently don’t have either because we lack platform support (i.e. portals), or simply because nobody sat down and wrote it:

1. Tracking the current desktop state, such as the focused app, the session idle state, etc. This will be useful for contextual actions.
2. Clipboard integration. In theory people can simulate this using the Keyboard Shortcuts action, but proper clipboard integration will work better and in more cases.
3. Picking and launching apps from the host system. This needs to happen through portals which currently don’t exist.
4. A fancy visual icon editor so that people can create their pretty icons in the app! If any UI designer is reading, please consider yourself assigned to this little project.
5. Support for custom backgrounds in the touchscreen. I did not have time to finish it before the 5.0 release, but it shouldn’t be too hard to add it.
6. A proper testing framework!

Finally, I’d like to thank my Ko-Fi and YouTube supporters for all the patience and for enabling me to do this work. The fundraiser campaign last year was a massive success, and I’m happy to see the all this progress! You all are awesome and I truly appreciate the support.

Keep an eye on this space as there may be more good news in the near future!

But over time we started to notice some limitations in Sysprof. We now have tons of data being collected (yay!) but some types of data analysis were pretty difficult yet. In particular, it was difficult to answer questions like “why does render times increased after 3 seconds?” or “what is the CPU doing during layout?”

In order to answer these questions, I’ve introduced a new feature in Sysprof: filtering by marks.

Hopefully people can use this new feature to provide developers with more insightful profiling data! For example if you spot a slowdown in GNOME Shell, you open Sysprof, profile your whole system, and filter by the relevant Mutter marks to demonstrate what’s happening there.

Here’s a fancier video (with music) demonstrating the new feature:

Enjoy!

This release comes after more than two years since Flatpak 1.14, so it’s pretty packed with new features, bug fixes, and improvements. Let’s have a look at some of the highlights!

USB & Input Devices

Two new features are present in Flatpak 1.16 that improve the handling of devices:

• The new input device permission
• Support for USB listing

The first, while technically still a sandbox hole that should be treated with caution, allows some apps to replace --device=all with --device=input, which has a far smaller surface. This is interesting in particular for apps and games that use joysticks and controllers, as these are usually exported by the kernel under /dev/input.

The second is likely the biggest new feature in the Flatpak release! It allows Flatpak apps to list which USB devices they intend to use. This is stored as static metadata in the app, which is then used by XDG Desktop Portal to notify the app about plugs and unplugs, and eventually request the user for permission.

Using the USB portal, Flatpak apps are able to list the USB devices that they have permission to list (and only them). Actually accessing these USB devices triggers a permission request where the user can allow or deny the app from having access to the device.

Finally, it is possible to forcefully override these USB permissions locally with the --usb and --nousb command-line arguments.

This should make the USB access story fairly complete. App stores like Flathub are able to review the USB permissions ahead of time, before the app is published, and see if they make sense. The portal usage prevents apps from accessing devices behind the user’s back. And users are able to control these permissions locally even further.

Better Wayland integration

Flatpak 1.16 brings a handful of new features and improvements that should deepen its integration with Wayland.

Flatpak now creates a private Wayland socket with the security-context-v1 extension if available. This allows the Wayland compositor to properly identify connections from sandboxed apps as belonging to the sandbox.

Specifically, with this protocol, Flatpak is able to securely tell the Wayland compositor that (1) the app is a Flatpak-sandboxed app, (2) an immutable app id, and (3) the instance id of the app. None of these bits of information can be modified by apps themselves.

With this information, compositors can implement unique policies and have tight control over security.

Accessibility

Flatpak already exposes enough of the accessibility stack for most apps to be able to report their accessible contents. However, not all apps are equal, and some require rather challenging setups with the accessibility stack.

One big example here is the WebKit web engine. It basically pushes Flatpak and portals to their limit, since each tab is a separate process. Until now, apps that use WebKit – such as GNOME Web and Newsflash – were not able to have the contents of the web pages properly exposed to the accessibility stack. That means things like screen readers wouldn’t work there, which is pretty disappointing.

Fortunately a lot of work was put on this front, and now Flatpak has all the pieces of the puzzle to make such apps accessible. These improvements also allow apps to detect when screen readers are active, and optimize for that.

WebKit is already adapted to use these new features when they’re available. I’ll be writing about this in more details in a future series of blog posts.

Progress Reporting

When installing Flatpak apps through the command-line utility, it already shows a nice fancy progress bar with block characters. It looks nice and gets the job done.

However terminals may have support for an OSC escape sequence to report progress. Christian Hergert wrote about it here. Christian also went ahead and introduced support to emitting the progress escape sequence in Flatpak. Here’s an example:

Unfortunately, right before the release, it was reported that this new feature was spamming some terminal emulators with notifications. These terminals (kitty and foot) have since been patched, but older LTS distributions probably won’t upgrade. That forced us to make it opt-in for now, through the FLATPAK_TTY_PROGRESS environment variable.

Ptyxis (the terminal app above) automatically sets this environment variable so it should work out of the box. Users can set this variable on their session to enable the feature. For the next stable release (Flatpak 1.18), assuming terminals cooperate on supporting this feature, the plan is to enable it by default and use the variable for opting out.

Honorable Mentions

I simply cannot overstate how many bugs were fixed in Flatpak in all these releases.

We had 13 unstable releases (the 1.15.X series) until we finally released 1.16 as a stable release. A variety of small memory leaks and build warnings were fixed.

The gssproxy socket is now shared with apps, which acts like a portal for Kerberos authentication. This lets apps use Kerberos authentication without needing a sandbox hole.

Flatpak now tries to pick languages from the AccountsService service, making it easier to configure extra languages.

Obsolete driver versions and other autopruned refs are now automatically removed, which should help keeping things tight and clean, and reducing the installed size.

If the timezone is set through the TZDIR environment variable, Flatpak takes timezone information from there. This should fix apps with the wrong timezone in NixOS systems.

More environment variables are now documented in the man pages.

This is the first stable release of Flatpak that can only be built with Meson. Autotools served us honorably for the past decades, but it was time to move to something more modern. Meson has been a great option for a long time now. Flatpak 1.16 limits itself to require a fairly old version of Meson, which should make it easy to distribute on old LTS distributions.

Finally, the 1.10 and 1.12 series have now reached their end of life, and users and distributions are encouraged to upgrade to 1.16 as soon as possible. During this development cycle, four CVEs were found and fixed, all of these fixes were backported to the 1.14 series, but not all were backported to versions older than that. So if you’re using Flatpak 1.10 or 1.12, be aware that you’re on your own risk.

Future

The next milestone for the platform is a stable XDG Desktop Portal release. This will ship with the aforementioned USB portal, as well as other niceties for apps. Once that’s done, and after a period of receiving bug reports and fixing them, we can start thinking about the next goals for these projects.

These are important parts of the platform, and are always in need of contributors. If you’re interested in helping out with development, issue management, coordination, developer outreach, and/or translations, please reach out to us in the following Matrix rooms:

• flatpak:matrix.org
• xdg-desktop-portals:matrix.org

Acknowledgements

Thanks to all contributors, volunteers, issue reporters, and translators that helped make this release a reality. In particular, I’d like to thank Simon McVittie for all the continuous maintenance, housekeeping, reviews, and coordination done on Flatpak and adjacent projects.

Shadows match the Adwaita style as well, including shadows of unfocused windows. These titlebars continue to follow the system dark and light mode, even when apps don’t.

Should make using legacy apps a little less unpleasant

Not everyone may know this but GNOME has its own browser, Web (a.k.a. Epiphany, or Ephy for the intimates). It’s a fairly old project, descendant of Galeon. It uses the GTK port of WebKit as its web engine.

The recent announcement that WebKit on Linux (both WebKitGTK and WPE WebKit) switched to Skia for rendering brought with it a renewed interest in measuring the performance of WebKit.

And that was only natural; prior to that, WebKit on Linux was using Cairo, which is entirely CPU-based, whereas Skia had both CPU and GPU-based rendering easily available. The CPU renderer mostly matches Cairo in terms of performance and resource usage. Thus one of the big promises of switching to Skia was better hardware utilization and better overall performance by switching to the GPU renderer.

A Note About Cairo

Even though nowadays we often talk about Cairo as a legacy piece of software, there’s no denying that Cairo is really good at what it does. Cairo can and often is extremely fast at 2D rendering on the CPU, specially for small images with simple rendering. Cairo has received optimizations and improvements for this specific use case for almost 20 years, and it is definitely not a low bar to beat.

I think it’s important to keep this in mind because, as tempting as it may sound, simply switching to use GPU rendering doesn’t necessarily imply better performance.

Guesswork is a No-No

Optimizations should always be a byproduct of excellent profiling. Categorically speaking, meaningful optimizations are a consequence of instrumenting the code so much that the bottlenecks become obvious.

I think the most important and practical lesson I’ve learned is: when I’m guessing what are the performance issues of my code, I will be wrong pretty much 100% of the time. The only reliable way to optimize anything is to have hard data about the behavior of the app.

I mean, so many people – myself included – were convinced that GNOME Software was slow due to Flatpak that nobody thought about looking at app icons loading.

Enter the Profiler

Thanks to the fantastic work of Søren Sandmann, Christian Hergert, et al, we have a fantastic modern system profiler: Sysprof.

Sysprof offers a variety of instruments to profile the system. The most basic one uses perf to gather stack traces of the processes that are running. Sysprof also supports time marks, which allow plotting specific events and timings in a timeline. Sysprof also offers extra instrumentation for more specific metrics, such as network usage, graphics, storage, and more.

All these metrics are super valuable when profiling any app, but they’re particularly useful for profiling WebKit.

One challenging aspect of WebKit is that, well, it’s not exactly a small project. A WebKit build can easily take 30~50min. You need a fairly beefy machine to even be able to build a debug build of WebKit. The debug symbols can take hundreds of megabytes. This makes WebKit particularly challenging to profile.

Another problem is that Sysprof marks require integration code. Apps have to purposefully link against, and use, libsysprof-capture to send these marks to Sysprof.

Integrating with Sysprof

As a first step, Adrian brought the libsysprof-capture code into the WebKit tree. As libsysprof-capture is a static library with minimal dependencies, this was relatively easy. We’re probably going to eventually remove the in-tree copy and switch to host system libsysprof-capture, but having it in-tree was enough to kickstart the whole process.

Originally I started sprinkling Sysprof code all around the WebKit codebase, and to some degree, it worked. But eventually I learned that WebKit has its own macro-based tracing mechanism that is only ever implemented for Apple builds.

Looking at it, it didn’t seem impossible to implement these macros using Sysprof, and that’s what I’ve been doing for the past few weeks. The review was lengthy but behold, WebKit now reports Sysprof marks!

Right now these marks cover a variety of JavaScript events, layout and rendering events, and web page resources. This all came for free from integrating with the preexisting tracing mechanism!

This gives us a decent understanding of how the Web process behaves. It’s not yet complete enough, but it’s a good start. I think the most interesting data to me is correlating frame timings across the whole stack, from the kernel driver to the compositor to GTK to WebKit’s UI process to WebKit’s Web process, and back:

But as interesting as it may be, oftentimes the fun part of profiling is being surprised by the data you collect.

For example, in WebKit, one specific, seemingly innocuous, completely bland method is in the top 3 of the callgraph chart:

Why is WebCore::FloatRect::contains so high in the profiling? That’s what I’m investigating right now. Who guessed this specific method would be there? Nobody, as far as I know.

Once this is out in a stable release, anyone will be able to grab a copy of GNOME Web, and run it with Sysprof, and help find out any performance issues that only reproduce in particular combinations of hardware.

Next Plans

To me this already is a game changer for WebKit, but of course we can do more. Besides the rectangular surprise, and one particular slowdown that comes from GTK loading Vulkan on startup, no other big obvious data point popped up. Specially in the marks, I think their coverage is still fairly small compared to what it could have been.

We need more data.

Some ideas that are floating right now:

• Track individual frames and correlate them with Sysprof marks
• Measure top-to-bottom-to-top latency
• Measure input delay
• Integrate with multimedia frames

Perhaps this will allow us to make WebKit the prime web engine for Linux, with top-tier performance, excellent system integration, and more. Maybe we can even redesign the whole rendering architecture of WebKit on Linux to be more GPU friendly now. I can dream high, can’t I?

In any case, I think we have a promising and exciting time ahead for WebKit on Linux!

I’ve acquired the device and it’s already on its way, expected to reach me by the end of January ~ early February due to being an international purchase.

Thanks everyone for your your overwhelming support! This is really exciting.

The goal is US$ 500, which should cover the costs of acquiring the device, and should pay for a few hours of reverse engineering work Boatswain development. Naturally, I’ll also document the USB format that these devices use, so that other developers out there can implement support in their own apps.

This particular model will be more complicated than other Stream Deck devices because it features a touch screen, and 4 activatable knobs. It will require a rather substantial rework of Boatswain so that it can support separate button grids, different button types, finger detection, and more.

I did reach out to Elgato, but they don’t seem interested in giving away one of their devices, or even a devboard, for me to hack on.

Whether or not this goal is reached, I’d like to thank all of the people that supported me on Ko-Fi and GitHub so far. Your support is truly humbling and it allowed me to write Boatswain in the first place, and much more!

Edit: since writing this article, I was let known that there is one other project that already reverse engineered parts of this device. I’ve adjusted the description to mention “Boatswain development” instead of “reverse engineering”.