Infrequently Noted Alex Russell on browsers, standards, and the process of progress. 2026-01-20T00:00:00Z https://infrequently.org Alex Russell [email protected] Naked Power 2026-01-20T00:00:00Z https://infrequently.org/2026/01/naked-power/ Twitter's descent into a gutter of the lowest order has been wrenching:

Elon Musk’s Grok and the Mass Undressing Scandal Elon Musk’s Grok and the Mass Undressing Scandal

As I draft this, a week later, it appears pressure from civil society, investigations by regulators, and outright bans on multiple continents have forced Musk to back down to an uncertain degree.

As this scandal roiled, Twitter's apps have been continuously available in Google's Play and Apple's App Store, marking new lows in moral cowardice and non-enforcement of the duopolist's own policies.

Now we sit on tenterhooks, wondering if the worst has actually passed. What outrage the will the valley's billionaire man-children unleash next? Meanwhile, we brace for this episode to embolden censorious authoritarians keen to suppress a free press and legitimate speech they dislike.

This is the backdrop to Elizabeth Lopatto's must-read denunciation in The Verge:

It is genuinely unbelievable to me that I wasted hours of my actual life on a court case where Apple explained it needed total control of its App Store to protect its users. Total control of the App Store was Apple’s main argument against antitrust enforcement: The company insisted that its monopolistic control of what users could install on their phones was essential to create a walled garden where it could protect children from unsafe content.

Ha! Ha ha ha!!

— Elizabeth Lopatto,
"Tim Cook and Sundar Pichai are cowards, The Verge"

Failure to react to the “everything app” going all-in on abuse of women and girls for weeks reveals the illegitimacy of their mobile monopolies1; anyone pretending otherwise is a fool or a dupe.

We don't need to guess why they sat on their hands.

Acting against Musk's abusive apps might put Apple and Google out of favour with an erratic, power-tripping administration which in turn could impact short-term business prospects. Their stated principles are incompatible with maximizing shareholder value under competitive authoritarianism.

Recall that both firms lent their monopolies on software distribution to ICE, citing the implausible claim that federal agents are a “vulnerable group.” The undressing scandal is the same choice in relief.

Facilitating the unthinkable at the behest of administration allies pays homage to power through obscenity. If they offend Musk…who knows what could happen? So maybe let it play out; let others take the heat. Surely somebody will do something. The internal monologue of the quisling scarcely needs exposition.

And so abuse at scale was amplified through their channels, against their own policies, for weeks.

The duopolist's justifications for monarchical app stores have always been bullshit, top to bottom, stem to stern.

App stores are not sui generis; they're just programs that install other programs, and "apps" are whatever the OS says they are.

As Cory Doctorow observed:

Apps interact with law in precisely the way that web-pages don't. “An app is just a web-page wrapped in enough IP to make it a crime to defend yourself against corporate predation”

It sure looks like Apple and Google failed to protect women and girls in order to preserve the rents they extract from the ecosystems these IP wrappers give them control over.

Gatekeepers like to point out that the wrapper comes with treats — business-critical capabilities and services that OS vendors lock behind proprietary APIs — but this is misdirection.

Web apps could provide safe, privacy-enhancing versions of every capability they currently reserve to native apps, and the duopolists know it. In an earlier era, open platforms chewed up proprietary features and spat out interoperable standards to cover the most useful 80% of that ground safely. Along the way, they published under open licences, dropping the price of commodity features to near-zero. This reduced lock-in for both developers and users which, in turn, forced incumbents to innovate.

Today's gatekeepers are desperate to keep that from happening again. It would upset the entire rentier model.

That's why Apple has worked furiously to keep APIs away from browsers through legal wrangling and subversion of standards. Cryptography and lawyers have also been enlisted to keep other programs-that-install-programs out and a safe, powerful open web at bay. Without those shields, we'd see the deeper failures clearly.

Consider the justifications Apple and its merry band of astroturfers trot out like clockwork to delay browser choice. Cupertino argues it must exclusively control browsers and software distribution to:

  • Ensure device security
  • Prevent frauds and scams
  • Provide a bulwark for privacy
  • Simplify software acquisition and distribution
  • Keep a lid on the most objectionable content

Now we see clearly that protection on the last point comes not from the stores, but from civil society and governments. This provides a template: each justification is an admission; misdirection to cover culpability.

Let's take it from the top.

Stores don't ensure security, runtimes do.

Operating systems and browsers — the platforms that sandbox code and mediate permissions — protect users to the extent they're designed to; app stores are just overwrought “beware of dog“ signs meant to scare off easily intimidated ne'er-do-wells. It's no surprise that whenever app stores are trusted with the role, a trail of harrowing failure follows.

This unearths the lie behind the obfuscation: iOS and Android didn't create stores to deliver unheard of security — iOS 1.0 did that by forbidding unsafe native code, replacing it with a better web2 — the gatekeepers built app stores because their OSes were and are insecure platforms for native apps. When mobile apps were web apps, the presumption of safety reigned. Alas.

Retreating from safety dovetailed with retreats from safe, open, interoperable computing in other ways. It's no coincidence that Apple backed away from adding capabilities to the web at the same moment it realised it could tax native apps extortionately.

App stores are Marketing's answer to a brand-promise problem: what to do about a hole below the water line that Product and Engineering aren't just failing to patch, but are enthusiastically expanding instead?

The whole facade of the duopolist's power hinges on the false claim that stores create security. Without the need to paper over the disaster of carelessly dispensed power tools, none of the rest of the services the stores provide could be justified; certainly not at the ruinous prices they demand.

More recent, chest-thumping pronouncements need to be evaluated in the same light. These aren't heroic explorers of new frontiers, they're embarrassed students bluffing book reports for tomes they didn't read.

Instead of protecting us, app stores reward platform vendors for security failure and foster centralising, anti-Open Source ecosystems. Open societies cannot abide closed platforms that assert ownership of this much of our lives, particularly not when claims of security are based on misrepresentations.

In the narrow conception, the app stores are feckless. Taking a wider view, they're complicit. Enabling, even.

Under a strict definition of “fraud," the track record of app stores is abysmal. Take just one recent example: while loudly proclaiming to protect users from scams, Apple simultaneously facilitated wide-scale app impersonation at the launch of Sora. This failure isn't a one-off, either. Bald-faced imposters are a long-running problem for stores that pretend to both users and developers that they protect from exactly these sorts of scams.

For its part, Google routinely facilitates shocking amounts of ad fraud via Play. Stores also failed to catch clearly fraudulent fronts for sanctioned Russian banks. This is just the tip of the proverbial iceberg.

If we widen the aperture to let in adjacent classes of user abuse, the situation looks immeasurably worse.

Apple's policies purport to disallow use of the ultra-low-friction IAP systems for gambling:

5.3.3 Apps may not use in-app purchase to purchase credit or currency for use in conjunction with real money gaming of any kind.

This text is lawyered to sound like a curb on gambling addiction's worst effects. In reality, it's designed to facilitate the predatory “gambling lite” systems Apple and Google gleefully promote.

For most of the mobile duopoly's existence, the primary revenue driver has been the problematic, gambling-adjacent behaviour of “digital whales” in so-called “casual games.”

And don't imagine the wilfully predatory behaviour is limited to adults. By allowing “bait apps” — even after previous FTC settlements that should have forbidden them — the app stores have shown us the duopolist's true colours. Serial disregard for the financial health of users is literally baked into their model.

This is the rotten core of mobile app stores. Understood in POSIWID terms, they exist to tax casinos that exploit gambling addictions of vulnerable users.

App stores safeguard privacy the way packs of wolves safeguard flocks of sheep.

The only appropriate response to the two-faced, duplicitous claims by Apple and Google towards privacy in recent years is incandescent rage.

I've covered before how Apple's posturing against Facebook is nothing but kayfabe and how Cupertino's privacy arguments regarding alternative browsers are steaming piles of illogical nonsense.

In reality, our privacy problems have been multiplied by Apple and Google.

It was the duopolists that created APIs for persistent background access to your contacts, calendar, location, radios, battery levels, and much else besides. It was the duopolists that then turned around and claimed credit for incrementally curbing the worst abuses of the APIs they themselves handed out like candy. Remember, they added these easily-tracked features knowing full well they would be abused.

How do we know they knew better? Both exposed shocking amounts of information about users to all comers after building browsers that protected from these very risks. Both had past form building web APIs that expanded platform power more thoughtfully. Caution was thrown to the wind by the very folks that now demand credit for remediating tiny patches of the superfund sites they created.

It was the duopolists who handed those APIs to native apps from shady publishers like Facebook with less-than-thoughtful controls. And it was these very companies that failed to police even their mildest policies.

And these same trillion-dollar market-cap firms simultaneously declined to do the one thing that had a chance to dramatically improve privacy: using their incredible lobbying capacity to forcefully call for privacy regulations worth a damn. Instead, they prefer a market structure where they can posture against each other over problems they jointly exacerbate.

And they have got away with it. Their press and product shops are keenly aware reporters don't understand privacy deeply enough to call their bluff, and that so-called privacy experts will clap as loudly for symbolic gestures as for fundamental change.

Humiliatingly for the fourth estate, Cupertino and Mountain View's self-issued privacy participation prizes were never questioned. Indeed, credulous journalists continue to shower them with praise for steps away from the very worst excesses best measured in angstroms.

Recently, Apple have been allowed to take credit for foisting responsibility onto users while Google has faced no sustained questioning for just giving up, having never launched anything at all to structurally curb Android abuses.

Cynics might be inclined to think this was very much the point.

These monopolies on apps-that-install-apps exist to squash competition, not to preserve privacy. Apple's not working furiously to deny iOS users alternative browsers because they might track users, they're misleading anyone who will listen because the web might provide an alternative that doesn't.

You know what the easiest way to get an app is? Clicking a link.

Apple literally pioneered this model with iOS 1.0, only to walk away from it a year later when it chose to carelessly expose overpowered, unsafe-by-default APIs with the hurried introduction of native apps. Throwing away privacy and security made software harder to build and distribute, too, but deposited power over developers with OS gatekeepers. Over time, the power to tax those developers became addictive.

A more secure and privacy preserving model is still possible but the duopolists continue to suppress it. I can't speak out of school about all the ways Android and Play mirrored Apple's underhanded tactics to suppress PWAs, but suffice to say it was a lot.

Industrial-scale suppression of safe, privacy-respecting platforms has been packaged up in florid terms as an advantage for developers. Except developers hate mobile app stores. But you don't have to take my word for it.

Given the choice, developers would do exactly what the gatekeepers do when constructing billing, distribution, and marketing systems: shop around in an open market, based on standards-oriented technologies, and select the best fit for their needs.

This is exactly the model that gave rise to the web and to web search. Discovery for web apps isn't impossible without omnipotent app stores; it isn't even hard. If we can build search engines for web pages, we can also highlight sites that are installable. None of this is magic, and none of it requires a 30% take from the developer's budget.

For the sake of completeness, we should stipulate that an end to app stores would not meaningfully change the content moderation landscape.

We now have a powerful example of this counterfactual thanks to the Twitter/Grok episode. There is no safety to be lost when we replace the gatekeeper's stores with a powerful, open, interoperable web. Mobile app store proprietors stand for nothing but profit and can be counted on only to defend their take. Good riddance.

Before the 2024 US elections, tech titans were well-enough advised to know which way the winds were blowing. But that did not stir them to defend truth, the rule-of-law, or even the employees that enabled their success. Instead, they hurried to capitulate. Today they sponsor coup-excusers, pay vigs, grovel to people they surely loathe, and fund the literal destruction of America's institutions.

This month's failure to stand up for basic decency is just another link in that chain.

Having narrowed the running to two choices, mobile's masters always ask us to consider governing our phones through the authoritarian frame of "who should rule?"

But these aren't our only choices. As Popper retorts, the better question is "How can we so organize political institutions that bad or incompetent rulers can be prevented from doing too much damage?"

This isn't purely a political question, but applies to all of society's power structures. The callous indifference of the app store's billionaire managers (1, 2) when faced with an even moderately difficult call tells us that they cannot be trusted; this was the test, and the mobile overlords failed by their own terms.3

What's left for the rest of us to take on is how we dismantle the mechanisms our misplaced trust helped them build. This will not be easy, and an insightful commenter at The Verge restates the core problem:

This is true and fantastic reporting and why we need to pay for The Verge.

But, it begs the question, what do we do?

Do we opt out of the tech of the modern world to protest? Commitment to values isn’t what we talk about, it’s what we are willing to give up. A key problem is that we don’t have any real competition vs Apple or Google as platforms if we want to exist in the modern world or even have this conversation.

You can’t (easily) read this or participate from a Kobo or Lightphone. Anyone have any suggestions?

I dropped off Twitter and Meta, but I’m running out of options.

— Anonymous commenter,
"Tim Cook and Sundar Pichai are cowards, The Verge, Comments"

We aren't going to get anywhere by throwing our iPhones and Androids into the sea.

Credible, incremental steps that remove power from the gatekeepers are now demanded, and as I have previewed throughout this piece, the open web is that next step. It has all the properties we need to attenuate misgiven power: no single vendor control, based in standards, multiple OSS implementations, and most of all, portability.

The web is an abstraction that holds the power to liberate our computing in-situ, removing superfluous gatekeepers from the loop an increasing fraction of the time. As use of the web grows, so do the prospects for alternatives OSes and hardware ecosystems. They know this, and that's why they're trying to keep the web from winning.

Moving our computing to browsers and web apps won't protect us from Musk, but neither will Apple or Google.4 Now that we know that, we can at least start to claw back at the corrosive power of monopolists in our pockets by building for a future that doesn't depend exclusively on them.

FOOTNOTES

  1. Some folks like to continue to pretend that the mobile duopoly still includes any serious competition for either player. I assume those people are paid to review phones for a living.

    As I outlined in this year's instalment of the Performance Inequality Gap series, the mobile market is actually two distinct markets: iOS for the rich, and Android for the rest. The average price for iPhones is hovering nearly $1K, while the average Android costs $300 new, unlocked. There is no functional competition between these ecosystems, and though they'll never admit it, that's a situation the duopolists are more than comfortable in, even if they don't particularly love it.

  2. ChromeOS introduced the same safe-by-default model to desktop computing several years later. In both cases it was necessary to aggressively expand the web platform's capabilities to support the sorts of applications that users and developers needed, and in both cases that effort was successful. But both Apple (and to a lesser extent, Google) backed away from that strategy when it became clear that fundamentally unsafe, wickedly overpowered platforms were a hit with developers of viral apps. What followed has been 15+ years of papering over the inherent flaws in the model and shifting blame for the mobile platform maker's own predictable (and predicted) failure to keep users safe.

  3. If it always falls to civil society and regulators to protect women and girls from Elon Musk and his Trumpian alliance, what is the point of Tim and Sundar? Of Play and the App Store?

    And if their policies are just fig leaves to justify rent extraction, why should any regulator listen to anything they say?

    These questions should be hair-on-fire in the capitols of still-functioning democracies.

  4. It is not the most offensive thing about this episode by a country mile, but I am driven to distraction by how unbelievably stupid Apple and Google have become.

    Did Tim and Sundar really think that, having sniffed weakness once, Trumpist shake downs would pass them over the next time a pro quo could be extracted for the quid?

    Did they not understand that by participating in oligarchy they signed on to authoritarianism?

    Did they really fail to calculate that capitulation didn't lower their risks, only centralised them?

    This was all predictable. You don't have to look as far as Russia to understand that autocrats grant temporary loans of state power towards undemocratic ends to create leverage for themselves, not the borrower. And whatever the price, autocrats never stay bought.

    Everyone but the smartest people in the room knew that domination is a ladder, and now we're all paying the price.

]]>
The Performance Inequality Gap, 2026 2025-11-24T00:00:00Z https://infrequently.org/2025/11/performance-inequality-gap-2026/

Let's cut to the chase, shall we? Updated network test parameters for 2026 are:

  • 9 Mbps downlink
  • 3 mbps uplink
  • 100 millisecond RTT

Regarding devices, my updated recommendations are the Samsung Galaxy A24 4G (or equivalent) and the HP 14. The goal of these recommendations is to emulate a 75th percentile user experience, meaning a full quarter of devices and networks will perform worse than this baseline.

Plugging these parameters into the updated budget calculator, we can derive critical-path resource thresholds for three and five second page load targets. Per usual, we consider pages built in two styles: JS-light, where only 15% of critical-path bytes are JavaScript, and JS-heavy, comprised of 50% JavaScript:

Time JS-light (MiB) JS-heavy
Total JS Other Total JS Other
3 sec 2.0 0.3 1.7 1.2 0.62 0.62
5 sec 3.7 0.57 3.2 2.3 1.15 1.15

Note: Budgets account for two TLS connections.

Many sites initiate more early connections, reducing time available to download resources. Using four connections cuts the three-second budget by 350 KiB, to 1.5 MiB / 935 KiB. The five-second budget loses nearly half a megabyte, dropping to 3.2 / 1.9 MiB.

It pays to adopt H/2 or H/3 and consolidate connections.

These budgets are extremely generous. Even the target of three seconds is lavish; most sites should be able to put up interactive content much sooner for nearly all users.

Meanwhile, sites are ballooning. The median mobile page is now 2.6 MiB, blowing past the size of DOOM (2.48 MiB) in April. The 75th percentile site is now larger than two copies of DOOM. P90+ sites are more than 4.5x larger, and sizes at each point have doubled over the past decade. Put another way, the median mobile page is now 70 times larger than the total storage of the computer that landed men on the moon.

Median page weights are more than 2.5x larger for mobile sites than a decade ago, and sites at the 75th percentile are now 4x their 2015 weight.
Median page weights are more than 2.5x larger for mobile sites than a decade ago, and sites at the 75th percentile are now 4x their 2015 weight.

An outsized contributor to this bloat comes from growth in JavaScript. Mobile JavaScript payloads have more than doubled since 2015, reaching 680 KiB and 1.3 MiB at P50 and P75 (respectively). This compositional shift exacerbates latent inequality and hurts businesses trying to grow.

When JavaScript grows as a proportion of critical-path resources, the impact of higher CPU cost per byte reduces budgets. This coffin corner effect explains why image and CSS-heavy experiences perform better byte-for-byte than sites built with the failed tools of frontend's lost decade.

Indeed, the latest CrUX data shows not even half of origins have passing Core Web Vitals scores for mobile users. More than 40% of sites still perform poorly for desktop users, and progress in both cohorts is plateauing:

This is a technical and business challenge, but also an ethical crisis. Anyone who cares to look can see the tragic consequences for those who most need the help technology can offer. Meanwhile, the lies, half-truths, and excuses made by frontend's influencer class are in defence of these approaches are, if anything, getting worse.

Through no action of their own, frontend developers have been blessed with more compute and bandwidth every year. Instead of converting that bounty into delightful experiences and positive business results, the dominant culture of frontend has leant into self-aggrandising narratives that venerate failure as success. The result is a web that increasingly punishes the poor for their bad luck while paying developers huge salaries to deliver business-undermining results.

Nobody comes to work wanting to do a bad job, but low-quality results are now the norm. This is a classic case of under-priced externalities created by induced demand from developers and PMs living in a privilege bubble.

The interactive budget calculator has been updated and revised for 2026, allowing you to see the impact of networks, devices, connections, and JavaScript on site performance.
The interactive budget calculator has been updated and revised for 2026, allowing you to see the impact of networks, devices, connections, and JavaScript on site performance.

Embedded in this year's estimates is hopeful news about the trajectory of devices and networks. Compared with early 2024's estimates, we're seeing budget growth of 600+KiB for three seconds, and a full megabyte of extra headroom at five seconds.1

While this is not enough to overcome continued growth in payloads, budgets are now an order of magnitude more generous than those first sketched in 2017. It has never been easier to deliver pages quickly, but we are not collectively hitting the mark.

To get back to a healthy, competitive web, developers will need to apply considerably more restraint. If past is prologue, moderation is unlikely to arise organically. It's also unhelpful to conceive of ecosystem-level failures as personal failings. Yes, today's frontend culture is broken, but we should not expect better while incentives remain misaligned.

Browsers, search engines, and developer tools will need to provide stronger nudges, steering users away from bloated sites where possible, and communicating the problem to decision-makers. This will be unpopular, but it is necessary for the web to thrive.

This series has continually stressed that today's P75 device is yesterday's mid-market Android, and that trend continues.

The explosive smartphone market growth of the mid 2010s is squarely in the rear-view mirror, and so historical Average Selling Prices (ASPs) and replacement dynamics now dominate any discussion of fleet performance.

Hardware upgrade timelines are elongating. Previous estimates of 18 months for replacement on average is now too rosy, with the median smartphone now living longer than two years. P75 devices may be nearly 3 years old, and TechInsights estimates a 23.7% annual replacement rate.

With all of this in mind, we update our target test device to the Samsung Galaxy A24 4G, a mid-2023 release featuring an SoC fabbed on a 6 nm process; a notable improvement over previous benchmark devices.

Readers of this blog are unlikely to have used a phone as slow as the A24 in at least a decade.
Readers of this blog are unlikely to have used a phone as slow as the A24 in at least a decade.

The A24 sold for less than the global Average Selling Price for smartphones at launch ($250 vs. $353). Because that specific model may be hard to acquire for testing, anything based on the MediaTek Helio G99 or Samsung Exynos 1330 will do; e.g.:

Teams that are serious about performance should track the low-end cohort instead, sticking with previously acquired Samsung Galaxy A51's, or any late-model device from the Moto E range.2

For link-accurate network throttling, I recommend spending $3 for Throttly for Android. It supports custom network profiles, allowing you to straightforwardly emulate a 9/3/100 network. DevTools throttling will always be inaccurate, and this is the best low-effort way to correctly condition links on your primary test device.

Desktops are not currently the limiting factor in the ecosystem, but it's still helpful to have physical test devices. Do not spend more than $250 (new) on a low-end test laptop. It should have a Celeron processor, eMMC storage, and run Windows. The last point is not an effort to sell more licences, but rather to represent the nasty effects of defender, NTFS, and parasitic background services on system performance. Something like the HP 14 dq3500nr.

Behold, the HP 14! A Celeron N4500 laptop, sporting a 4-core chip first released in 2021. This CPU packs less than a quarter the cache of a late-model iPhone.
Behold, the HP 14! A Celeron N4500 laptop, sporting a 4-core chip first released in 2021. This CPU packs less than a quarter the cache of a late-model iPhone.

Desktop network throttling remains fraught, and the best solutions are still those from Pat Meenan's 2016 article announcing winShaper.

What we see in our recommended test setups is an echo of the greatest influence of the past decade on smartphone performance: the spread of slow, ever-cheaper Androids with ageing chipsets, riding the cost curve downward, year-on-year.

The explosive growth of this segment drove nearly all market growth between 2011 and 2017. Now that smartphones have reached global saturation, flat sales volumes mirror the long-term trends in desktop device ownership:

At no point in the past dozen years has iOS accounted for more than 20% of new-device shipments. Quarterly fluctuations have pushed that number as high as 25% when new models are released, but the effect never lasts.

Most phones — indeed, most computers — are 24+ month old Androids. This is the result of a price segmented market: a preponderance of smartphones sold for more than $600USD (new, unlocked) are iPhones, and the overwhelming majority of devices sold for less than that are slow Androids.

Source: IDC, Statista, and Counterpoint Research.
Not adjusted for inflation. Full year 2025 ASP is extrapolated from the first several quarters and may be revised upward.

The “i” in “iPhone” stands for “inequality.”

Global ASPs show the low-end isn't just alive-and-well, it's many multiples of high-end device volume. To maintain a global ASP near $370, an outsized number of cheaper Androids must be sold for every $1K (average) iOS device.

To understand how the payload budget estimate is derived, we need to peer deeper into the device and network situation. Despite huge, unpredictable shocks in the market (positive: Reliance Jio; negative: a pandemic), the market trends this series tracks have allowed us to forecast accurately.

75th+ percentile users are almost always on older devices, meaning we don't need to divine what will happen, just remember the recent past.

The properties of today's mobile devices define how our sites run in practice. From the continued prevalence of 4G radios, to the shocking gaps in CPU performance, the reality of the modern web is best experienced through real devices. The next best way to understand it is through data.

Per usual, single and multicore CPU performance charts track four market segments:

  • Fastest iOS device
  • Fastest Android
  • Mid-range Android ($300-350)
  • Low-end Android ($100-150)

The last two cohorts account for more than 2/3 of new device sales:

The performance of JavaScript-based web experiences is heavily correlated with single-core speed, meaning that the P75 device places a hard cap on the amount of JavaScript that is reasonable for any website to rely on.

Depressingly, budget device CPUs have not meaningfully improved since 2022. But the nearly-identical SoCs in each year's device are getting cheaper. Reduced bill-of-materials costs mean declining retail prices for low-spec phones.

Meanwhile, the high end continues to pull away. As previewed in prior instalments of this series, top-end Androids are beginning to close the massive performance lead that Apple's A-series chips have opened up over the past decade. This is largely thanks to Qualcomm and MediaTek finally starting to address the cache-gap I have harped on since 2016:

Source: Wikipedia, vendor documentation, and author's calculations.

Some cache sizings are estimates, particularly in the Android ecosystem, where vendor documentation is lacking. Android SoC vendors have a habit of implementing the smallest values ARM allows for a licensed core design.

The latest iPhone chip (A19 Pro) features truly astonishing amounts of cache. For a sense of scale, the roughly 50 MiB of L1, L2, and L3 cache in an iPhone 17 Pro provides 8.3 MiB of cache per core. This is more than double the per core cache of Intel's latest high-end desktop part, the 285K, which provides a comparatively skimpy 3.3 MiB combined cache per core.

The gobsmacking caches of A-series parts allow Apple to keep the beast fed, leading to fewer stalls and more efficient use of CPU time. This advantage redounds to better battery life because well-fed CPUs can retire work faster, returning to lower-power states sooner.

That it has taken this long for even the top end of the Android ecosystem to begin to respond is a scandal.

Source: GSMArena, Geekbench, and vendor documentation.
Geekbench 6 points per dollar at each price point over time. Prices are MSRP at date of launch.

If there's good news for buyers of low-end devices, it's that performance per dollar continues to improve across the board.

Frustratingly, though, low-end single core performance is still 9x slower than contemporary iPhones, and mid-tier devices remain more than 3.5x slower:

Multicore performance tells a similar tale. As a reminder, this metric is less correlated with web application performance than single-core throughput:

Source: GSMArena, Geekbench.
When Geekbench reports multiple scores for an SoC, recent non-outlier values are selected.

Performance per dollar looks compelling for the lower tier parts, but recall that they are 1/3 to 1/5 the performance:

Source: GSMArena, Geekbench, and vendor documentation.

What makes iPhones so bloody fast, while Androids have languished? Several related factors in Apple's chip design and development strategy provided a commanding lead over the past decade:

  • In-house tuning of all ARM-designed cores, thanks to a long-ago negotiated Architecture licence, leading to aggressive cache sizings.

  • Concentration in fewer SoC SKUs enabled focused yield optimisation.

  • Large, early orders with TSMC secured exclusive access to the latest fabrication nodes.

Android vendors, meanwhile, have spread their SoC development budgets in penny-wise, pound-foolish fashion. Even Google and Samsung's in-house efforts have failed to replicate the virtuous effects of Apple's disciplined CPU designs.

Source: GSMArena, Wikipedia, and vendor documentation.

Feature sizes are a fudge below 10 nanometres, but marketing names usually reflect real increases in transistor density and frequencies, along with reductions in power use. High-end Android and iOS parts have generally been produced on comparable nodes, with Apple's lead lasting less than a year. But that's less than half the story. Android SoC vendors have avoided adding competitively sized caches, dedicating the same mm^2 on die to higher core counts and on-die radios. From a performance perspective, this has been catastrophic:

Source: GSMArena, Wikipedia, and vendor documentation.

Core counts are a headline fixture of device marketing, but even the cheapest phones have featured eight (slow, memory-starved) cores since 2019. Speed comes from other properties; namely appropriate cache sizing, memory latency, frequency scaling, and chip architecture. Apple's core-count restraint and focus on other aspects should have been a lesson to the entire industry long before 2024.

Smaller transistors also allow for higher peak frequencies, giving Apple a perennial advantage thanks to early access to TSMC's latest, smallest, power-sipping processes:

Source: GSMArena, Wikipedia, and vendor documentation.
Maximum advertised frequency of the fastest on-package core.

These trade-offs have allowed Apple to charge a premium for devices which no other vendor can justify:

Source: GSMArena, IDC, Statista, and Counterpoint Research.
Prices are new, unlocked MSRP at launch.

Global ASPs are conservative estimates; some research groups estimate values 10-15% higher, but the trends are consistent. The premium end continues to pull away in both price and performance, dragging ASPs slightly upward. Meanwhile, high-end devices continue to be outsold more than 3:1 by low-end phones that aren't getting faster. Those slow devices are, however, getting increasingly inexpensive, dropping below $100 in the last two years. That's 1/10th the price of the cheapest iPhone with Apple's fastest chip.

Thanks to market trends, the recent-spec iPhones many web developers carry don't even represent the experience of most iOS users.

It may seem incongruous that the ASP of iOS devices is bumping up against the $1K mark while most developed countries experience bouts of slow growth post-pandemic, along with well-documented cost-of-living crises among middle-class buyers.

One possible solution to this riddle is that Android sales remain strong, as Apple is cordoned into the segment of the market it can justify to shareholders with a 30% net margin. Another is growth in the resale market, particularly for iOS devices. Thanks to premium components and better-than-Android software support lifetimes, the longevity of iPhones has created a vibrant and growing market below the $400 price point.

This also helps to explain the flat-to-slightly-declining market for new smartphones, as refurbished devices accelerate past $40BN/yr in sales.

What does this mean? We should expect, and see, higher-than-first-sale volumes of iOS use in various aggregate statistics. Wealth effects have historically explained much of this, but the scale is growing. Refurbishment and resale are now likely to be driving growing discontinuity in the data:

Wikimedia reports more than 40% of mobile site visits come from iOS devices over the past decade, despite global sales ratios never breeching 20% annually.
Wikimedia reports more than 40% of mobile site visits come from iOS devices over the past decade, despite global sales ratios never breeching 20% annually.

Not only is an iPhone 17 Pro not real life in the overall market, it isn't even real life in the iOS market any more.

The situation on desktop is one of overwhelming stasis, modulo the Windows 11 upgrade cycle resulting from Windows 10's EOL at the end of 2025. That has driven an unusually strong one-off cycle of upgrades that will reverberate through the data in coming years.

IDC's recent sales analysis shows that only 20% of 'desktop' devices are fully wired, with performance of the vast majority subject to power and thermal limits arising from battery power.
IDC's recent sales analysis shows that only 20% of 'desktop' devices are fully wired, with performance of the vast majority subject to power and thermal limits arising from battery power.

This impetus to upgrade is cross-pressured by pricing headwinds. Economic uncertainty, tariffs, scrambled component pricing from AI demand for silicon of all sorts, and ever-longer device replacement cycles all mean that new PCs may not provide more than incremental performance gains. As a result, an increase in recent worldwide PC Average Selling Prices from ~$650 in our previous estimate to ~$750 in 2025 (per IDC) may not indicate premiumisation. In a globally connected economy, inflation comes for us all.

Overall, the composition and trajectory of the desktop market remains stable. Despite 2025's device replacement boomlet, IDC predicts stasis in “personal computing device” volumes, with growth bumping along at ±1-2% a year for the next 5 years, and landing just about where things are today. The now-stable baseline of ~410MM devices per year is predicted to be entirely flat into 2030.

Top-line things to remember about desktops are:

  • 80% of “desktop” devices are laptops, tablets, and other battery-powered form-factors. The performance impact of thermal and power envelopes for these devices is drastic; many cores are spun down to low power states most of the time; symmetric multiprocessing is now a datacentre curio.

  • Flaky Wi-Fi, rather than wired Ethernet, is now the last mile to most desktops.

  • Desktops (including laptops) are only 15-18% of web-capable device sales; a pattern that has been stable for a decade.

Put another way: if you spend a majority of your time in front of a computer looking at a screen that's larger than 10", you live in a privilege bubble.

Per previous instalments, we can use Edge's population-level data to understand the evolution of the ecosystem. As of late 2025, the rough breakdown looks like:

Device Tier Fleet % Definition
Low-end 30% Either:
<= 4 cores, or
<= 4GB RAM
Medium 60% HDD (not SSD), or
4-16 GB RAM, or
4-8 cores
High 10% SSD +
> 8 cores +
> 16GB RAM

Compared with the data from early 2024, we see some important shifts:

  • Low-end devices have fallen from ~45% to ~30% of the fleet.

  • Most growth is in the middle tier, growing from 48% to 60%.

  • The high-end is growing slowly.

Because the user base is also growing, it's worth mentioning that the apparent drop in the low-end is a relative change. In absolute terms, the ecosystem is seeing a slower absolute removal of low-spec machines. This matches what we should intuitively expect from incremental growth of the Edge user base, which is heavily skewed to Windows 11.

Older and slower devices likely constitute an even larger fraction of the total market, but may be invisible to us. Indeed, computers with spinning rust HDDs, <= 4GB of RAM, and <= 2 cores still represent millions of active devices in our data. Alarmingly, they have dropped by less than 20% in absolute terms over the past six months. And remember, this isn't even the low end of the low end, as our stats don't include Chromebooks.

Building to the limits of ”feels fine on my MacBook Pro” has never been realistic, but in 2025 it's active malpractice.

The TL;DR for networks for 2026 is that the P75 connection provides 9Mbps of downlink bandwidth, 3Mbps upload, with 100ms of RTT latency.

This 9Mbps down, 3Mbps up configuration represents sizeable uplift from the 2024/2025 guidance of 7.2Mbps down, 1.4Mbps up, with 94ms RTT, but also a correction. 2024's estimate for latency was probably off by ~15%, and should have been set closer to 110ms.

Global or ambitious firms should target networks even slower than 9/3/100 in their design parameters; the previous 5/1/28 “cable” network configuration is still worth building for, but with an upward adjustment to latency (think 5/1/100). Uneven service has the power to negatively define brands, and the way to remain in user's good graces digitally is to build for the P95+ user. Smaller, faster sites that serve this cohort well will stand out as being exceptional, even on the fastest networks and devices.

Looking forward into 2026 and 2027, the hard reality is that networks remain slower than developers expect and will not improve quickly.

Upload speeds, in particular, remain frustratingly slow, as wider upload channels correlate with faster early-session downloads. Only users on the fastest 10% of networks see downlink:uplink bandwidth rising above 2.5:1 ratios, even under ideal conditions.3

Owing to physics, device replacement rates, CAPEX budgets of mobile carriers, inherent variability in mobile networks (vs. fixed-line broadband), and worldwide regulatory divergence, we should expect experiences to be heavily bottlenecked by networks for the foreseeable future.

Previous posts in this series have documented improvements, but as we have been saying since 2021, 4G is a miracle, and 5G is a mirage.

According to the GSMA's 2025 State of Mobile Internet Connectivity report, nominal 5G penetration coverage just crossed the 50% mark in 2025, and the rate of progress is predicted to slow as the attractive economics of dense rural areas give way to rural build outs. But coverage is not the same as service; many users will need newer, more expensive devices to unlock 5G speeds.
According to the GSMA's 2025 State of Mobile Internet Connectivity report, nominal 5G penetration coverage just crossed the 50% mark in 2025, and the rate of progress is predicted to slow as the attractive economics of dense rural areas give way to rural build outs. But coverage is not the same as service; many users will need newer, more expensive devices to unlock 5G speeds.

This will remain true for at least another three years, with bandwidth and latency improving only incrementally. Sites that want to reach their full potential, if only to beat the competition, must build to budgets that are inclusive for folks on the margins. When it comes to web performance, doing well is the same as doing good.

Bandwidth numbers are derived from Cloudflare's incredible Radar dataset.4 Looking at (downlink) bandwidth trends over the past year, we see stasis:

November 2024-November 2025 downlink bandwidth. Note that percentiles are inverted in this chart (we take their P25 as our P75 and vice versa). The trend line is remarkably stable; median and slower connections have not improved over the past year.
November 2024-November 2025 downlink bandwidth. Note that percentiles are inverted in this chart (we take their P25 as our P75 and vice versa). The trend line is remarkably stable; median and slower connections have not improved over the past year.

So where does the improvement in our estimate come from? Looking back to 2024, we see (predicted) gains emerge, but note their small absolute size:

November 2023-November 2024 downlink bandwidth. Connections representing the slowest quartile improved from ~6 to ~9 Mbps over the year, while median downlinks improved from ~12 to ~17 Mbps. They have not moved since.
November 2023-November 2024 downlink bandwidth. Connections representing the slowest quartile improved from ~6 to ~9 Mbps over the year, while median downlinks improved from ~12 to ~17 Mbps. They have not moved since.

The gap between P25 and P75 downlinks was 15 Mbps at the start of 2024 and has grown to 21 Mbps at the end of 2025; an increase of 40%. Meanwhile, bottom quartile are only 28% faster, improving from ~7 to 9 Mbps. In absolute terms, wealthier users saw 3x as much absolute gain.

The performance inequality gap is growing at the network layer too.

Latency across networks (RTTs) is improving somewhat, with a nearly 10% decrease at P75 over the past year, from ~110ms to ~100ms. Small improvements on faster links (P50, P25) look to be in the 5% range:

November 2024-November 2025 RTT. Percentiles align with our usual intuition, with P75 representing the value at which 25% of connections are slower.
November 2024-November 2025 RTT. Percentiles align with our usual intuition, with P75 representing the value at which 25% of connections are slower.

Given the variability at higher percentiles, we'll stick with a 100ms target for the 2026 calendar year, although we should expect slight gains.

Underlying these improvements are datacentre build outs in traditionally underserved regions, undersea cable completions, and cellular backhaul improvements from the 5G build out. Faster client CPUs and radios will also contribute meaningfully and predictably.5

All of these factors will continue to incrementally improve, and we predict another ~5% improvement (to 95ms) at P75 for 2027.

Gains will be modest for both bandwidth and latency over the next few years. The lion's share of web traffic is now carried across mobile networks, meaning that high percentiles represent users feeling the confounding effects of cellular radios, uneven backhaul, coverage gaps, and interference from the built environment. Those factors are hardest and slowest to change, involving the largest number of potential long tent poles.

As discussed in previous instalments, the world's densest emerging markets reached the smartphone tipping point by 2020, displacing feature phones for most adults. More recently, we have approached saturation; the moment at which smartphone sales are dominated by replacements, rather than first-time sales in those markets. Affordability and literacy remain large challenges to get the residually unconnected online. The 2025 GSMA State of Mobile Internet Connectivity report has a full chapter on these challenges (PDF), complete with rich survey data.

What we see in now-stable smartphone shipment volumes primes the pump for improvements in device performance. First-time smartphone users invest less in their devices (relative to income) as the value may be unclear; users shopping for a second phone have clearer expectations and hopes. Having lived the frustration of slow sites and apps, an incrementally larger set of folks will be willing to pay a bit more to add 5G modems to their next phone.

This effect is working its way down the price curve from the ultra-high-end in 2020 to the mid-tier in 2024, but it has yet to reach the low end. Our latest low-end specimen — 2025's Moto E15 — still features a 4G radio. Because of the additional compute requirements that 5G places on SoCs, we will expect to see process node and CPU performance increases as 5G costs fall far enough to impact the low-price band.

Today, devices with otherwise similar specs and a 5G radio still command a considerable premium. The Galaxy A16 5G was introduced at a 20% premium over the 4G version, mirroring the mid-market dynamic from 2022 and 2023 where devices were offered in “regular” and (pricier) 5G versions. It will take a transition down the fab node scale like we saw for mid-market SoCs in 2022 and 2024 to make 5G a low-end table-stakes feature. I'm not holding my breath.

Given the current market for chips of all types, we may be seeing low-spec SoCs produced at 12 nanometres for several more years, reprising the long-term haunting of Android by huge volumes of A53 cores produced at 28 nm from 2012 to 2019.

The budget estimates generated in this series may seem less relevant now that tools like Core Web Vitals provide nuanced, audience-specific metrics that allow us to characterise important UX attributes. But this assumption is flawed thanks to the effects of deadweight losses and the biases inherent in those metrics.

Case-in-point: last year CWV deprecated FID and replaced it with the (better calibrated) INP metric. This predictably dropped the CWV pass rates of sites built on desktop-era JavaScript frameworks like React, Angular, and Ember:

2024's Frontend Sadness Index shows that CWV scores for sites based on legacy JS frameworks like React and Angular not only started off in trouble, but dropped more as INP replaced FID.
2024's Frontend Sadness Index shows that CWV scores for sites based on legacy JS frameworks like React and Angular not only started off in trouble, but dropped more as INP replaced FID.

RUM data, in isolation, undercounts the opportunity costs of slow and bloated experiences. Users have choices, and lost users do not show up in usage-based statistics.

A team I worked with this year saw these effects play out directly in their CrUX data:

Form-factor ratios can show what use statistics for high-growth sites obscure.
Form-factor ratios can show what use statistics for high-growth sites obscure.

This high-profile, fast-growing site added nearly 100 KiB of critical-path JavaScript per month from January to June. The result? A growing share of visits from desktop devices, and proportionally fewer mobile users every month. Once we began to fumigate for dead code and overeager preloading, mobile users returned.

These effects can easily overcome other factors, particularly in the current era of JavaScript bundle hyper-growth:

Median page weights are more than 2.5x larger for mobile sites than a decade ago, and 4x larger at the 75th percentile.
Median page weights are more than 2.5x larger for mobile sites than a decade ago, and 4x larger at the 75th percentile.

Growth in JavaScript bytes over the wire mirrors that of overall content despite the continuing Performance Inequality Gap crisis.
Growth in JavaScript bytes over the wire mirrors that of overall content despite the continuing Performance Inequality Gap crisis.

Perhaps the most important insight I spotted while re-immersing myself in the data for this post were the implications of these charts from the RUM Archive:

The RUM Archive reports that SPAs are, on average, only generating a one (1) soft navigation per hard navigation, undermining the case for SPAs.

The top-line takeaway is chilling: sites that are explicitly designed as SPAs, and which have intentionally opted in to metrics measurement around soft-navigations are seeing one (1) soft-navigation for every full page load on average.

The rinky-dink model we discussed last year for the appropriateness of investing in SPA-based stacks is a harsh master, defining average session performance as the sum of interaction latencies, including initial navigation, divided by the total number of interactions (excluding scrolling):

L avg = latency ( navigation ) + i = 1 I latency ( i ) N

If the RUM Archive's data is directionally correct, at an ecosystem level, N=~2 for both mobile and desktop. Sessions this shallow make a mockery of the idea that we can justify more up-front JavaScript to deliver SPA technology, even on sites with reason to believe it would help.

In private correspondence, Michal Mocny shared an early analysis from data collected via the Soft Navigations API Origin Trial. Unlike the Akamai mPulse data that feeds the RUM Archive, Chromium's data tracks interactions from all sites, not only those that have explicitly opted-in to track soft navigations, providing a much wider aperture. On top-10K origins, Chrome is currently observing values for N between 1.6 and 2.2, depending on how the analysis is run, or 0.8-1.1 additional soft navigations per initial page load.

It's difficult to convey the earth-shattering magnitude of these congruent findings. Under these conditions, the amount of JavaScript a developer can justify up-front to support follow-on in-page navigations is de minimis.6

This should shake our industry to the bone, driving rapid reductions in emitted JavaScript. And yet:

Growth in JavaScript bytes is driving growth in overall payloads.
Growth in JavaScript bytes is driving growth in overall payloads.

This series has three main goals:

  1. Provide a concrete set of page-weight targets for working web developers.

  2. Arm teams with an understanding of how the client-side computing landscape is evolving.

  3. Show how budgets are constructed, giving teams tools to construct their own estimates from their own RUM and market data.

This is not altruism. I want the web to win. I began to raise the alarm about the problems created by a lack of adaptation to mobile's constraints in 2016, and they have played out on the same trend-line I feared. The web is now decisively losing the battle for relevance.

To reverse this trend, I believe several (currently unmet) conditions must be fulfilled:

  • Reaching users via the web must be cost-competitive with other digital channels, meaning that re-engagement features like being on the home screen and in the notification tray must work correctly.

  • The web must deliver the 80/20 set of critical capabilities for most of the important JTBDs in modern computing, but in a webby (safe, privacy-respecting by default) way.

  • Web experiences, on average, have to feel responsive enough that the idea of tapping a link doesn't inspire dread.

But the web is not winning mobile. Apple, Google, and Facebook nearly extinguished the web's potential to disrupt their cosy arrangement. Preventing the web from breaking out — from meaningfully delivering app-like experiences outside an app store — is essential to maintaining dominance. But some are fighting back, and against the odds, it's working.

What's left, then, is the subject of this series. Even if browser competition comes to iOS and competitors deliver the features needed to make the web a plausible contender, the structure of today's sites is an impediment to a future in which users prefer the web.

Most of the world's computing happens on devices that are older and slower than anything on a developer's desk, and connected via networks that contemporary “full-stack” developers don't emulate. Web developers almost never personally experience these constraints, and over frontend's Lost Decade, this has created an out-of-touch privilege bubble that poisons the products of the teams that follow the herd, as well as the broader ecosystem.

That's bleak, but the reason I devote weeks to this research each year isn't to scold. The hope is that actionable targets can help shape choices, and that by continuing to stay engaged with the evolution of the landscape, we can see green shoots sprout.

If we can hold down the rate of increase in critical-path resource growth it will give hardware progress time to overtake our excesses. If we make enough progress in that direction, we might even get back to a place where the web is a contender in the minds of users.

And that's a future worth working for.

FOOTNOTES

  1. I really do try to avoid being an unremitting downer, but the latest device in our low-cost cohort — the Motorola E15 — is not an improvement in CPU benchmarks from last year.

    More worrying, no device in that part of the market has delivered meaningful CPU gains since 2020. That's five years of stasis, and a return to a situation where new low-end devices are half as fast as their mid-tier contemporaries. Even as process node improvements trickle down to the $300-350 price bracket, the low end is left further and further behind.

    As the wider Android ecosystem experienced from 2015-2020, devices with the same specs are getting cheaper, but not better. This allows them to open new markets and sell in massive numbers, helping to prop up overall annual device sales, even as devices last longer and former growth markets (India, Indonesia, etc.) hit smartphone saturation. This is reflected in the low-end models finally sinking below $100USD new, unlocked, at retail. But to hit this price-point, they deliver performance on par with 2019's mid-tier Galaxy A50; a phone whose CPU was fabbed on a smaller process node than today's latest low-end phones.

    Services trying to scale, and anyone trying to build for emerging markets, should be anchoring P90 or P95, not P75. For serious shops, the target has not moved much at all since 2019.

    This reality alone is enough to justify rejection of frameworkist hype-shilling, without even discussing the negative impacts of JS-first stacks on middle-distance team velocity and site maintenance costs.

  2. Because low-end and medium-tier devices were so similar until very recently, this differentiation wasn't necessary. But progress in process nodes does eventually trickle down. The mid-tier began to see improvement away from the (utterly blighted) 28 nm node in 2017, a mere 4 years after the high-end decamped for greener pastures. The low-end, meanwhile, was trapped in that register until 2020, nearly a decade after 28nm was first introduced. Since then, the mid tier has tracked process node scaling with a 2-3 year delay, while the low end has gotten stuck since 2021 at 12 nm.

    The failure to include meaningful amounts of cache in Android SoCs levelled out low-end and medium tier performance until 2023, but transistor shrinkage and architecture improvements at ~$350 are now decoupling the performance of these tiers once again, and we should expect to see them grow further apart in coming years, creating a worrying gap between P90+ and P75 devices.

  3. Cloudflare's worldwide bandwidth data only provides downlink estimates, and so to understand the downlink:uplink bandwidth ratio, I turned instead to their API and queried CF's speed test data, which provide fuller histograms.

    These tests are explicitly run by users, meaning they occur under synthetic conditions and likely with a skew towards best-case network performance. They also report maximums over a session, rather than loaded network behaviour, which explains the divergence between the higher speed values reported there than the more realistic "Internet Quality Index" dataset we use for primary bandwidth and latency analysis.

    The data we can derive from it is therefore much rosier, but it does give us a sense for downlink/uplink ratios (bitrates are in Mbps):

    %-ile Download Upload Ratio
    20th 15 5 3
    25th 25 10 2.5
    30th 10 30 3
    40th 20 50 2.5
    median 85 35 2.4
    60th 125 50 2.5
    70th 220 85 2.6
    75th 280 100 2.8
    80th 345 140 2.5
    90th 525 280 1.9

    Because the data is skewed in an optimistic direction (thanks to usage biases towards wealth, which correlates with high-performance networks), we pick a 3:1 ratio in our global baseline.

    Despite variance in the lower percentiles, it is reasonable to expect tough ratios in the bottom quartile given the build-out properties of various network types. These include:

    • Asymmetries in cable and DSL channel allocations.
    • Explicit frequency/bandwidth allocation in cellular networks.
    • Radio power lopsidedness vs. the base stations they connect to, particularly for battery-powered devices.

    Even new networks like Starlink are spec'd with 10:1 or greater ratios. Indeed, despite being "fast", the author's own home fixed-line connection has a ratio grater than 30:1. We should expect many such discrepancies up and down the wealth spectrum.

    A 4:1 or 5:1 ratio is probably justified, and previous estimates used 5:1 ratios for that reason. Lacking better data, going with 3:1 is a judgement call, and I welcome feedback on the choice.

  4. Why am I relying on Cloudflare's data?

    Google, Microsoft, Amazon, Fastly, Akamai, and others obviously have similar data (at least in theory), but do not publish it in such a useful and queryable way. That said, these estimates are on trend with my priors about the network situation developed from many sources over the years (including non-public datasets).

    There is a chance Cloudflare's data is unrepresentative, but given their CDN market penetration, my primary concern is that their data is too rosy, rather than too generous. Why? Geographic and use-based bias effects.

    The wealthy are better connected and heavier internet users, generating more sessions. Better performance of experiences increases engagement, so we know CF's data contains a bias towards the experiences of the affluent. This potentially blinds us to large fraction of the theoretical TAM and (I think) convincingly argues that we should be taking a P90 value instead of P75. However, we stick with P75 for two reasons:

    • It would be incongruent to cite P90 this year without first introducing it in previous installations.
    • A lack of explicitly triangulating data from the current network environment makes it challenging to judge the magnitude of use-based biases in the data.

    Thankfully, Cloudflare also produces country-level data. We can use this to cabin the scale of potential issues in global data. Here, for instance, are the P75 network situations for a few populous geos that every growth-oriented international brand must consider in descending downlink speed:

    Geo @ P75 Down Up RTT Pop (MM)
    UK 21 7 34 69
    USA 17 5.5 47 340
    Brazil 12 4 60 213
    Global 9 3 100
    Indonesia 6.4 2.1 75 284
    India 6.2 2.1 85 1,417
    Pakistan 4 1.3 130 241
    Nigeria 3.1 1 190 223

    Underweighting the less-affluent is a common bias in tech, and my consulting experience has repeatedly reconfirmed what Tammy Everts writes about when it comes to the opportunities that are available when sites push past performance plateaus.

    There is no such thing as “too fast”, but most teams are so far away from minimally acceptable results that they have never experienced the huge wins on the other side of truly excellent and consistent performance. Entire markets open up when teams expand access through improved performance, and wealthy users convert more too.

    It's this reality that lemon vendors have sold totally inappropriate tools into, and the results remain shockingly poor.

  5. As we mentioned in the last instalment, improvements in mid-tier and low-end mobile SoCs are delivering better network performance independent of protocol and spectrum improvements.

    Modern link-layer cell and Wi-Fi stacks rely heavily on client-side compute for the digital signal processing necessary to implement advanced techniques like MIMO beam forming.

    This makes the device replacement rates doubly impactful, even within radio generations and against fixed channel capacity. As process improvements trickle down (glacially) to mid-tier and low-end SoCs, the radios they contain also get faster processing, improving latency and throughput, ceteris paribus.

  6. The RUM Archive's soft-to-hard navigations ratio and the early data from the Chromium Soft Navigations Origin Trial leave many, many questions unanswered including, but not limited to:

    • What's the distribution?
      • Globally: do some SPA-premised sites have many more or many fewer soft-navigations? Are only a few major sites pushing the ratios up (or down)?
      • Locally: can we characterise user's sessions to understand what fraction trigger many soft-navigations per session?
    • Do other data sources agree?
      • Will the currently-running Origin Trial for Soft Navigations continue to agree as the reach grows?
      • Can other RUM vendors validate or refute these insights?
    • What about in-page changes not triggering URL updates?
      • How should infinite-scrolling be counted?
      • We should expect Chromium histogram data to capture more of this vs. the somewhat explicit instrumentation of mPulse, driving up soft-navigations per hard navigation. Do things stay in sync in these data sets over time?

    Given the scale of the mystery, a veritable stampede of research in the web performance community should follow. I hope to see an explosion of tools to guide teams toward the most appropriate architectures based on comparative data within their verticals, first-party RUM data about session lengths, distribution mono/bi/tri/multi-modality of sessions, and other situated factors.

    The mystery I have flicked at in the past is now hitting us smack in the face. Will we pay attention?

]]>
The App Store Was Always Authoritarian 2025-10-11T00:00:00Z https://infrequently.org/2025/10/the-app-store-was-always-authoritarian/

Eric Prouzet
Eric Prouzet

And now we see it clear, like a Cupertino sunrise bathing Mt. Bielawski in amber: Apple will censor its App Store at the behest of the Trump administration without putting up a fight.

It will twist words into their antipodes to serve the powerful at the expense of the weak.

To better serve autocrats, it will talk out both sides of its mouth in ways it had previously reserved for dissembling arguments against threats to profits, like right-to-repair and browser choice.

They are, of course, linked.

Apple bent the knee for months, leaving many commentators to ask why. But the reasons are not mysterious: Apple wants things that only the government can provide, things that will defend and extend its power to extract rents, rather than innovate. Namely, selective exemption from tariffs and an end to the spectre of pro-competition regulation that might bring about real browser choice.

Over the past few weeks, Tim Apple got a lot what he paid for,1 with the full weight of the US foreign and industrial policy apparatus threatening the EU over DMA enforcement. This has been part of a full-court press from Cupertino. Apple simultaneously threatened the EU while rolling out fresh astroturf for pliant regulators to recline on. This is loud, coordinated, and calculated. But calculated to achieve what? Why is the DMA such a threat to Apple?

Interoperability.

The DMA holds the power to unlock true, safe, interoperability via the web. Its core terms require that Apple facilitate real browser engine choice, and Apple is all but refusing, playing games to prevent powerful and safe iOS browsers and the powerful web applications they facilitate. Web applications that can challenge the App Store.

Unlike tariffs, which present a threat to short-term profits through higher costs and suppression of upgrades in the near term, interoperability is a larger and more insidious boogeyman for Apple. It could change everything.

Apple's profits are less and less attributable to innovation as “services” revenue swells Cupertino's coffers out of all proportion to iPhone sales volume. “Services” is code for rent extraction from captive users and developers. If they could acquire and make safe apps outside the App Store, Apple wouldn't be able to take 30% from an outlandishly large fraction of the digital ecosystem's wealthiest players.

Apple understands browser choice is a threat to its rentier model. The DMA holds the potential for developers to finally access the safe, open, and interoperable web technologies that power most desktop computing today. This is a particular threat to Apple, because its class-leading hardware is perfectly suited to running web applications. All that's missing are browsers that aren't intentionally hobbled. This helps to explain why Apple simultaneously demands control over all browser technology on iOS while delaying important APIs, breaking foundational capabilities, and gaslighting developers about Apple's unwillingness to solve pressing problems.

Keeping capable, stable, high-quality browsers away from iOS is necessary to maintain the App Store's monopoly on the features every app needs. Keeping other software distribution mechanisms from offering those features at a lower price is a hard requirement for Cupertino's extractive business model. The web (in particular, PWAs) present a worst-case scenario.

Unlike alternative app stores that let developers decouple distribution of proprietary apps from Apple's App Store, PWAs further free developers from building for each OS separately, allowing them to deliver apps though a zero-cost platform that builds on standards. And that platform doesn't feature a single choke point. For small developers, this is transformative, and it's why late-stage Apple cannot abide laws that create commercial fairness and enable safe, secure, pro-user alternatives.

This is what Apple is mortgaging its brand (or, if you prefer, soul) to prevent: a world where users have a real choice in browsers.

Horrors.

Apple is loaning its monopoly on iOS software to yet another authoritarian regime without a fight, painting a stark contrast: when profits are on the line, Cupertino will gaslight democratic regulators and defy pro-competition laws with all the $1600/hr lawyers Harvard can graduate. And when it needs a transactional authoritarian's help to protect those same profits, temporarily2 lending its godlike power over iOS to censor clearly protected speech isn't too high a price to ask. Struggle for thee, but not for me.

The kicker is that the only alternative for affected users and developers is Apple's decrepit implementation of web apps; the same platform Cupertino serially knee-caps to deflect competition with its proprietary APIs.

It is no exaggeration to say the tech press is letting democracy down by failing to connect the dots. Why is Apple capitulating? Because Apple wants things from the government. What are those things? We should be deep into that debate, but our reportage and editorial classes cannot grasp that A precedes B. The obvious answers are also the right ones: selective protection from tariffs, defanged prosecution by the DOJ, and an umbrella from the EU's democratic, pro-competition regulation.

The Verge tiptoed ever so close to getting it, quoting letters that former Apple executives sent the company:3

I used to believe that Apple were unequivocally ‘the good guys,’” Hodges writes. “I passionately advocated for people to understand Apple as being on the side of its users above all else. I now feel like I must question that.”

— Wiley Hodges,
"The Verge"

This is a clue; a lead that a more thoughtful press and tech commentariat could use to evaluate the frames the parties deploy to their own benefit.

The tech press is failing to grasp the moral stakes of API access. Again and again they flunk at connecting boring questions of who can write and distribute programs for phones to urgent issues of power over publication and control of devices. By declining to join these threads, they allow the unearned and increasingly indefensible power app stores to proliferate. The urgent question is how that power can be attenuated in the political economy of mobile OSes, or as Popper put it:

We must ask whether…we should not prepare for the worst leaders, and hope for the best. But this leads to a new approach to the problem of politics, for it forces us to replace the question: "Who should rule?" by the new question: "How can we so organize political institutions that bad or incompetent rulers can be prevented from doing too much damage?"

— Karl Popper,
"The Open Society and Its Enemies"

But the tech press does not ask these questions.

Instead of questioning why Apple's OS is so fundamentally insecure that an App Store is necessary, they accept the ever-false idea that iOS has been relatively secure because of the App Store.

Rather than confronting Apple with the reality that it used the App Store to hand out privacy-invading APIs in undisciplined ways to unscrupulous actors, it congratulates Cupertino on the next kayfabe performance. Links between Apple's monopoly over APIs and the growth of monopolies in adjacent sectors are rarely, if ever, questioned. Far too often, the tech press accepts the narrative structure of Apple's marketing, satisfying pangs of journalistic conscience with ineffectual critique of surface features, rather analyses of latent power imbalances.

Nowhere, e.g., in The Verge's coverage of these letters is there a discussion about alternatives to the App Store. Only a few outlets ever press Apple on its suppression of web apps, including failure to add PWA install banners and essential capabilities. It's not an Apple vs. Google horse-race story, and so discussion of power distribution doesn't get coverage.

Settling for occasionally embarrassing Apple into partially reversing its most visibly egregious actions is ethically and morally stunted. Accepting the frame of "who should rule?" that Cupertino reflexively deploys is toxic to any hope of worthwhile technology because it creates and celebrates the idea of kings, leaving us supine relative to the mega-corps in our phones.

This is, in a word, childish.

Adults understand that things are complicated, that even the best intentioned folks get things wrong, or can go astray in larger ways. We build institutions and technologies to protect ourselves and those we love from the worst impacts of those events, and those institutions always model struggles over power and authority. If we are lucky and skilled enough to build them well, the results are balanced systems that attenuate attempts at imposing overt authoritarianism.

In other words, the exact opposite of Apple's infantilising and totalitarian world view.

Instead of debating which wealthy vassals might be more virtuous than the current rulers, we should instead focus on attenuating the power of these monarchical, centralising actors. The DMA is doing this, creating the conditions for interoperability, and through interoperability, competition. Apple know it, and that's why they're willing to pawn their own dignity, along with the rights of fellow Americans, to snuff out the threat.

These are not minor points. Apple has power, and that power comes from its effective monopoly on the APIs that make applications possible on the most important computing platform of our adult lives.

Protecting this power has become an end unto itself, curdling the pro-social narratives Apple takes pains to identify itself with. Any reporter that bothers to do what a scrappy band of web developers have done — to actually read the self-contradictory tosh Apple flings at regulators and legislators around the world — would have been able to pattern match; to see that twisting words to defend the indefensible isn't somehow alien to Apple. It's not even unusual.

But The Verge, 404, and even Wired are declining to connect the dots. If our luminaries can't or won't dig in, what hope do less thoughtful publications with wider audiences have?

Apple's power and profits have made it an enemy of democracy and civic rights at home and abroad. A mealy-mouthed tech press that cannot see or say the obvious is worse than useless; it is an ally in Apple's attempts to obfuscate.

The most important story about smartphones for at least the past decade has been Cupertino's suppression of the web, because that is a true threat to the App Store, and Apple's power flows from the monopolies it braids together. As Cory Doctorow observed:

Apple's story – the story of all centralized, authoritarian technology – is that you have to trade freedom for security. If you want technology that Just Works(TM), you need to give up on the idea of being able to override the manufacturer's decisions. It's always prix-fixe, never a la carte.

This is a kind of vulgar Thatcherism, a high-tech version of her maxim that "there is no alternative." Decomposing the iPhone into its constituent parts – thoughtful, well-tested technology; total control by a single vendor – is posed as a logical impossibility, like a demand for water that's not wet

— Cory Doctorow,
"Plenty of room at the bottom (of the tech stack)"

Doctorow's piece on these outrages is a must-read, as it does what so many in the tech press fail to attempt: connecting patterns of behaviour over time and geography to make sense of Apple's capitulation. It also burrows into the rot at the heart of the App Store: the claim that anybody should have as much power as Apple has arrogated to itself.

We can see clearly now that this micro-authoritarian structure is easily swayed by macro-authoritarians, and bends easily to those demands. As James C. Scott wrote:

I believe that many of the most tragic episodes of state development in the late nineteenth and twentieth centuries originate in a particularly pernicious combination of three elements. The first is the aspiration to the administrative ordering of nature and society, an aspiration that we have already seen at work in scientific forestry, but one raised to a far more comprehensive and ambitious level. “High modernism” seems an appropriate term for this aspiration. As a faith, it was shared by many across a wide spectrum of political ideologies. Its main carriers and exponents were the avant-garde among engineers, planners, technocrats, high-level administrators, architects, scientists, and visionaries.

If one were to imagine a pantheon or Hall of Fame of high-modernist figures, it would almost certainly include such names as Henri Comte de Saint-Simon, Le Corbusier, Walther Rathenau, Robert McNamara, Robert Moses, Jean Monnet, the Shah of Iran, David Lilienthal, Vladimir I. Lenin, Leon Trotsky, and Julius Nyerere. They envisioned a sweeping, rational engineering of all aspects of social life in order to improve the human condition.

— James C. Scott,
"Seeing Like A State"

This is also Apple's vision for the iPhone; an unshakeable belief in its own rightness and transformative power for good. Never mind all the folks that get hurt along the way, it is good because Apple does it. There is no claim more central to the mythos of Apple's marketing wing, and no deception more empowering to abusers of power.4

Apple claims to stand for open societies, but POSIWID shows that to be a lie. It is not just corrupted, but itself has become corrupting; a corrosive influence on the day-to-day exercise of rights necessary for democracy and the rule-of-law to thrive.5

Apple's Le Corbusierian addiction to control has not pushed it into an alliance with those resisting oppression, but into open revolt against efforts that would make the iPhone an asset for citizens exercising their legitimate rights to aid the powerless. It scuttles and undermines open technologies that would aid dissidents. It bends the knee to tyranny because unchecked power helps Cupertino stave off competition, preserving (it thinks) a space for its own messianic vision of technology to lift others out of perdition.

If the consequences were not so dire, it would be tragically funny.

Let's hope our tech press find their nerve, and a copy of “The Open Society and Its Enemies," before we lose the ability to laugh.

I spent a dozen and change years at Google, and my greatest disappointment in leadership over those years was the way the founders coddled the Android team's similarly authoritarian vision.

For the price of a prominent search box on every phone,6 senior leadership (including Sundar) were willing to sow the seeds of the web's obsolescence, handing untold power to Andy Rubin's team of Java zealots. It was no secret that they sought to displace the web as the primary way for users to experience computing, substituting proprietary APIs for open platforms along the way.

With the growth of Android, Play grew in influence, in part as cover for Android's original sins.7 This led to a series of subtler, but no less effective, anti-web tactics that dovetailed with Apple's suppression of web apps on iOS. The back doors and exotic hoops developers must jump through to gain distribution for interoperable apps remains a scandal.

But more than talking about Google and what it has done, we should talk about how we talk about Google. In specific, how the lofty goals of its Search origins were undercut by those anti-social, anti-user failures in Android and Play.

It's no surprise that Google is playing disingenuous games around providing access to competitors regarding web apps on Android, while simultaneously pushing to expand its control over app distribution. The Play team covet what Apple have, and far from exhibiting any self-awareness of their own culpability, are content to squander whatever brand reputation Google may have left in order to expand its power over software distribution.

And nobody can claim that power is being used for good.

Google is not creating moral distance between itself and Apple, or seeking to help developers build PWAs to steer around the easily-censored channels it markets, and totally coincidentally, taxes.8 Google is Apple's collaborator in capitulation. A moral void, trotting out the same, tired tactic of hiding behind Apple's skirt whenever questions about the centralising and authoritarian tendencies of App Store monopolies crop up. For 15 years, Android has been content to pen 1-pagers for knock-off versions of whatever Apple shipped last year, including authoritarian-friendly acquiescence.

Play is now the primary software acquisition channel for most users around the world, and that should cause our tech press to intensify scrutiny of these actions, but that's not how Silicon Valley's wealth-pilled technorati think, talk, or write. The Bay Area's moral universe extends to the wall of the privilege bubble, and no further. We don't talk about the consequences of enshittified, trickle-down tech, or even bother to look hard at it. That would require using Android and…like…eww.

Far from brave truth-telling, the tech press we have today treats the tech the other half (80%) use as a curio; a destination to gawp at on safari, rather than a geography whose residents are just as worthy of dignity and respect as any other. And that's how Google is getting away with shocking acts of despicable cowardice to defend a parallel proprietary ecosystem of gambling, scams, and shocking privacy invasion, but with a fraction of the negative coverage.

And that's a scandal, too.

FOOTNOTES

  1. Does anyone doubt that Tim Apple's wishlist didn't also include a slap-on-the-wrist conclusion to the US vs. Apple?

    And can anyone safely claim that, under an administration as nakedly corrupt as Donald Trump's, Apple couldn't buy off the DOJ? And what might the going rate for such policy pliability be?

    That we have to ask says everything.

  2. It hopes.

  3. I don't know Wiley Hodges, but the tone of his letter is everything I expect from Apple employees attempting to convince their (ex-)bosses: over-the-top praise, verging on hagiography, combined with overt appeals to the brand's value. This is how Apple discusses Apple to Apple, not just the outside world.

    I have no doubt that this sort of sickly sweet presentation is necessary for even moderate criticism to be legible when directed up the chain. Autocracies are not given to debate, and Apple is nothing if not internally autocratic.

    No large western company exerts deeper control over its image, prying deep into the personal lives of its employees in domineering ways to protect its brand from (often legitimate) critique that might undermine the message of the day. Every Apple employee not named "Tim" submits to an authoritarian regime all day, every day. It's no wonder that the demands of power come so easily to the firm. All of this is done to maintain the control that allows Marketing to cast Apple's image in a light that makes it the obvious answer to "who should rule?"

    But as we know, that question is itself the problem.

    Hodges' follow-up post is more open and honest, and that's commendable. Now on the outside, he's visibly grasping with some level of deprogramming. Like the rest of us, he's discovering that Apple is demanding far more trust than its actions can justify. He's rightly disappointed that Apple isn't living up to the implications of its stated commitment to open societies, and that the stated justifications seem uncomfortably ad hoc, if not self-serving.

    This discomfort stems from the difference between principle and PR.

    Principles construct tests with which we must wrestle. Marketing creates frames that cast one party as an unambiguous hero. I've often joked that Apple is a marketing firm with an engineering side gig, and this is never more obvious than in the stark differences between communicated positions and revealed outcomes.

    Reading these posts, I really feel for the guy, and wish him luck in convincing Apple to change course. If (as seems likely) it does not, I would encourage him to re-read that same Human Rights Policy and ask: "is this document a statement of principle or is it marketing collateral?"

  4. The cultish belief that "it is good because we do it" is first and foremost a self-deception. It's so much easier to project confidence in this preposterous proposition when the messenger themselves is a convert.

    The belief that "we should rule" is only possible to sustain among thoughtful people once the question "who should rule?" is deeply engrained. No wonder, then, that the firm works so dang hard to market its singular virtue to the internal, captive audience.

  5. As I keep pointing out, Apple can make different choices. Apple could unblock competing browsers tomorrow. It could fully and adequately fund the Safari team tomorrow. It could implement basic features (like install banners) that would make web apps more viable tomorrow. These are small technical challenges that Apple has used disingenuous rhetoric to blow out of all proportion as it has tried to keep the web at bay. But if Apple wanted to be on the side of the angels, it could easily provide a viable alternative for developers who get edited out of the App Store.

  6. Control over search entry points is the purest commercial analogue in Android to green/blue messages on iOS. Both work to dig moats around commodity services, erecting barriers to switching away from the OS's provider, and both have been fantastically successful in tamping down competition.

  7. It will never cease to be a scandal that Android's singular success metric in the Andy Rubin years was “activations.” The idea that more handsets running Android was success is a direct parallel to Zuckian fantasies about “connection” as an unalloyed good.

    These are facially infantile metrics, but Google management allowed it to continue well past the sell-by date, with predictably horrendous consequences for user privacy and security. Play, and specifically the hot-potato of "GMS Core" (a.k.a., "Play Services") were tasked with covering for the perennially out of date OSes running on client devices. That situation is scarcely better today. At last check, the ecosystem remains desperately fragmented, with huge numbers of users on outdated and fundamentally insecure releases. Google has gone so far as to remove these statistics from its public documentation site to avoid the press asking uncomfortable questions. Insecurity in service of growth is Android's most lasting legacy.

    Like Apple, Andy Rubin saw the web as a threat to his growth ambitions, working to undermine it as a competitor at every step. Some day the full story of how PWAs came to be will be told, but suffice to say, Android's rabid acolytes within the company did everything they could to prevent them, and when that was no longer possible, to slow their spread.

  8. Don't worry, though, Play doesn't tax developers as much as Apple. So Google are the good guys. Right?

    Right?

]]>
11ty Hacks for Fun and Performance 2025-10-04T00:00:00Z https://infrequently.org/2025/10/11ty-hacks-for-fun-and-performance/ This blog really isn't just for beating up on Apple for the way it harms users, the web, standards, and society to maintain power and profits. So here's some fun stuff I've been doing in my 11ty setup to improve page performance.

You know how it gets once you've got a mature 11ty setup: shortcodes proliferate, and some generate output that might depend on JS or CSS.

If your setup is anything like mine, you might also feel conflicted about including some of those resources globally. It's important to have them available for the posts that need them, but it's not great to bring a charting library, e.g., into every page when only 1 in 20 will need it.

This blog has grown many such shortcodes that generate expansions for features like:

Here, for instance, is how a Vimeo embed looks in the markdown of a page, using Nunjucks for Markdown pre-processing:

{% vimeo "VIDEOID", "TITLE" %}

This expands to:

<lite-vimeo 
  videoid="VIDEOID" 
  videotitle="TITLE">
  <a href="http://vimeo.com/VIDEOID">TITLE</a>
</lite-vimeo>

But this also requires script; e.g.:

<script type="module" async
    src="/assets/js/lite-vimeo/lite-vimeo.js">
</script>

And the list keeps growing. To avoid having the weight of these components clogging up every page, a system for selectively pulling in their code would be helpful.

It'd also be grand if we could make sure scripts appear just once, even if shortcodes are invoked multiple times per page. Included code should preferably also be located towards the top of the document.

The 11ty Bundle plugin to the rescue!

Sort of.

At first glance, the 11ty Bundle plugin looks ideal for this, but we need to solve two problems the documentation doesn't cover:

  1. Shortcodes should auto-include the scripts they need, but not over-include them. How can we use Bundle plugin provided shortcodes from within other shortcodes? This is important to avoid having every page needing to remember to include scripts.

  2. How to make this work with templates that use pagination?

The first problem turns out to be very simple because 11ty shortcodes are themselves callable functions. Somewhere in my .eleventy.js configuration, there's now a function like:

function addToBundle(scope, bundle, code) {
  eleventyConfig.getPairedShortcode(bundle)
                .call(scope, code);
}

This works because 11ty's internals are dogmatic about not binding this, allowing tools like shortcodes to be regular 'ole functions that can be invoked from dynamic scopes via call() and apply(). This is awesome!

The Vimeo shortcode then calls addToBundle to make sure that the JS it depends on will get loaded:

eleventyConfig.addShortcode("vimeo",
                            function(id, title="") {
  addToBundle(this, "js", `
    <script type="module" async
        src="proxy.php?url=/assets/js/lite-vimeo/lite-vimeo.js">
    </script>
  `);

  let titleAttr = title ? ` videotitle="${title}" ` : "";
  return `
    <lite-vimeo videoid="${id}" ${titleAttr}>
      <a href="proxy.php?url=http://vimeo.com/${id}">${title}</a>
    </lite-vimeo>
  `;
});

The real code is a tad more complex to handle things like proper escaping, script file versioning, stripping whitespace to avoid markdown issues, etc…but not much. Page templates then include the usual:

<!DOCTYPE html>
<html>
  <head>
    <!-- ... -->

    {% getBundle "js" %}

    <!-- ... -->
  </head>
  <!-- ... -->

And this should be it!

Except it didn't work on my homepage.

Why not? Mostly because the 11ty Bundle plugin was built for sane sites; projects where a single output page's content doesn't need to hoist bundle inputs from across multiple entries. But I'm using pagination, like a mug.

I've got a patch out that addresses this, hackily, and for now I'm using package.json overrides to target that branch. It seems to be working well enough here:

{
  "name": "infrequently.org",
  // ...
  "devDependencies": {
    "@11ty/eleventy": "3.1.2",
    "@11ty/eleventy-plugin-rss": "^2.0.4",
    "@11ty/eleventy-plugin-syntaxhighlight": "^5.0.2",
    // ...
  },
  "overrides": {
    "@11ty/eleventy-plugin-bundle": 
      "https://github.com/slightlyoff/eleventy-plugin-bundle-pagination.git#pagination-aware"
  },
  // ...
}

Now I can build shortcodes that use other shortcodes to bundle code only for the pages that need it, trimming the default JS payload while leaving me free to build and use richer components as necessary.

The one downside of this approach has been an increase in build times.

I've worked to keep full re-builds under five seconds, and incremental builds under a second on my main writing device (a recent Chromebook). The Bundle plugin adds a post-processing phase (via @11ty/eleventy/html-transformer) to builds which tack on two seconds to both scenarios. This blog generates ~1,500 pages in a build, so the per page hit isn't bad, but it's enough to be noticeable.

I will likely spend time getting this trimmed back down in the near future. If you've got a smaller site, I can recommend the bundler-with-generative-shortcodes approach. If your site is much larger, it may be worth adopting if you're already paying the price of a post-processing step. Otherwise, and as ever, it's worth measuring.

Loading code only on the pages that need it is great, but you know what's even better? Only loading code when it's going to actually be needed.

For a lot of pages, it makes sense to load specific widgets only when users scroll down far enough to encounter their content. Normally this is the sort of thing folks lean on big frameworks to handle, but that's not how this blog rolls. Instead, we'll use an IntersectionObserver and a MutationObserver to:

  • Locate scripts that want deferred loading.

  • Use attributes on those elements to identify which elements they should be invoked for.

  • Watch the page scrolling and trigger the code loading when target elements get close enough to the viewport.

Taken together, this reduces code loaded up front that might otherwise contend with above-the-fold resources without sacrificing interactive features further down the page.

Here, for instance, was how some code from a recent blog post that needed charts looked before:

{% js %}
<script type="module" 
    src="/assets/js/d3.min.js"></script>
<script type="module" 
    src="/assets/js/plot.min.js"></script> 

<script type="module">
  // ...
  genFeatureTypePlot("leading", true, {
    caption: "Chromium launches ... ahead of other engines."
  });
  // ...
</script>
{% endjs %}

#### Leading Launches by Year

<div id="leading"></div>

The Bundle plugin makes it simple to write code near the elements that will target it and not worry about duplicates. That's helpful, and the bundle plugin provides many tools for choosing where to output the gathered-up bits, but what I really want is to delay the fetching of those scripts until the user might plausibly benefit from them.

Here's what the revised code looks like:

{% js %}
<script type="io+module" data-for="leading" 
    src="/assets/js/d3.min.js"></script>
<script type="io+module" data-for="leading" 
    src="/assets/js/plot.min.js"></script> 

<script type="io+module" data-for="leading">
  // ...
  genFeatureTypePlot("leading", true, {
    caption: "Chromium launches ... ahead of other engines."
  });
  // ...
</script>
{% endjs %}

#### Leading Launches by Year

<div id="leading"></div>

These scripts will still be hoisted into the <head>, but they won't execute because they are using a type attribute the browser doesn't recognise. The data-for attribute provides an ID of the element to trigger loading of the script. These are enough to build a scroll-based loader with.

Our loader uses a bit of inline'd script at the top of the page to set up an IntersectionObserver to watch scrolling, and a collaborating MutationObserver to identify elements matching this description as the parser creates them.

Here's the meat of that snippet, loaded early in the document as a <script type="module">. Pardon the pidgin JavaScript style; the last thing I want is a JS transpiler as part of builds, so bytes matter:

// Utilities to delay code until the next task
let rAF = requestAnimationFrame;
let doubleRaf = (func) => {
  rAF(()=> { rAF(()=> { func(); }) });
};

// Bookkeeping
let ioScripts = new Set();
let ioScriptsFor = new Map();
let triggerIDs = new Set();

// Record which scripts should wait on elements
let processScript = (s) => {
  if(ioScripts.has(s)) { return; }
  ioScripts.add(s);
  let idFor = s.getAttribute("data-for");
  let isf = ioScriptsFor.get(idFor);
  if(!isf) {
    isf = [];
    ioScriptsFor.set(idFor, isf);
    triggerIDs.add(idFor);
  }
  isf.push(s);
};

// Handle existing scripts before setting up 
// the Mutation Observer
document.querySelectorAll(`script[type="io+module"]`)
    .forEach(processScript);

// Preload element tempalate;
let plt = document.createElement("link");
plt.setAttribute("rel", "modulepreload");

// For a given element, begin loading scripts 
// that were waiting on it.
let triggerScripts = async (id) => {
  let scripts = ioScriptsFor.get(id);
  let head = document.head;
  for(let s of scripts) {
    if(!s.src) { continue; }
    // Get a preload request started
    let pl = plt.cloneNode(true);
    pl.setAttribute("href", s.src);
    head.append(pl);
  }

  for(let s of scripts) {
    // Clone because setting type alone does not 
    // trigger evaluation.
    let sc = s.cloneNode(true); 
    // Set type to an executable value
    sc.setAttribute("type", "module");
    if(sc.src) { // External scripts
      let lp = new Promise((res) => {
        sc.onload = res; 
      });
      head.append(sc);
      await lp; // Serialisation handled above
    } else { // Inline modules
      head.append(sc);
    }
  }
};

let forObs = new IntersectionObserver(
  (entries) => {
    entries.forEach((e) => {
      if(e.intersectionRatio > 0) {
        forObs.unobserve(e.target);
        doubleRaf(() => {
          triggerScripts(e.target.id);
        });
      }
    });
  }, 
  {
    // If we boot far down the page, e.g. via back 
    // button scroll restoration, load eagerly. 
    // Else, watch two screens ahead:
    rootMargin: "1000% 0px 200% 0px",
  }
);

// When new elements are added, watch for scripts 
// with the right type and elements that scripts are 
// waiting on.
let documentMo = new MutationObserver((records) => {
  for(let r of records) {
    if(!r.addedNodes) { continue; }
    for(let n of r.addedNodes) {
      if(n.nodeType === 1) { // Elements only
        if((n.tagName === "SCRIPT") &&
           (n.type === "io+module")) {
          processScript(n);
        }
        // If we find elements in the watch list, 
        // observe their scrolling relative to 
        // the viewport
        if(n.id && triggerIDs.has(n.id)) {
          forObs.observe(n);
        }
      }
    }
  }
});

documentMo.observe(document.documentElement, { 
  childList: true, 
  subtree: true,
});

That's the whole job done in just 110 lines of modern platform code, including utility functions and 20 lines of comments.

A few small tricks of note:

  • <link rel="modulepreload" href="..."> allows us to get network requests started without waiting on previous scripts to download, parse, and evaluate.

  • Limiting support to modules enables us to get async, but ordered, execution semantics.

  • The early querySelectorAll() ensures hoisted script blocks that occur before the inline'd script are handled correctly.

As this was enough for the moment, I haven't implemented a few obvious improvements:

  • This technique could be combined with shortcodes-calling-the-bundler to create shortcodes that dynamically load their code based on scroll position and only on pages that need them.

  • The id-based system is a bit fugly and can easily be upgraded to use any simple CSS selector that matches() supports.

It's a small thing, but I do try to optimise the CSS selectors used on this blog, as it's element-heavy and doesn't encapsulate much of the style-recalculation time work with Shadow DOM.

It was something of a surprise, then, to find that use of the @11ty/eleventy-plugin-syntax-highlight module's use of some basic styles cargo-culted many years ago was tanking style recalculation performance. How? Slow attribute rules:

code[class*="language-"],
pre[class*="language-"] {
	color: #f8f8f2;
	background: none;
	text-shadow: 0 1px rgba(0, 0, 0, 0.3);
  /* ... */
}

Thanks to the prevalence of inline <code> blocks in my writing, the Selector Stats panel showed a fair few slow-path misses.

Thankfully, the fix is simple. In the CSS I switched to faster whole-attribute selectors:

code[highlighted],
pre[highlighted] {
	color: #f8f8f2;
	background: none;
	text-shadow: 0 1px rgba(0, 0, 0, 0.3);
  /* ... */
}

Next, I used the configuration options available in the plugin (which I only figured out from reading the source) to specify those attributes be added to the <pre>s and <code>s generated by the syntax highlighter:

import syntaxHighlight from 
    "@11ty/eleventy-plugin-syntaxhighlight";

// ...

eleventyConfig.addPlugin(syntaxHighlight, {
  // For faster CSS selectors
  preAttributes: { highlighted: "highlighted" },
  codeAttributes: { highlighted: "highlighted" },
});

Now the browser doesn't have to attempt a slow substring search every time it encounters one of these elements.

There's more to do in terms of selector optimisation on this site, but this was a nice quick win.

The history is boring, but suffice to note that the <img> helpers this site uses predate the official 11ty Image transform plugin, and are tuned to generate URLs that work with Netlify's Image CDN. This means I've also been responsible for generating my own previews for images that haven't loaded yet, and devising strategies for displaying and animating them.

This has been, by turns, fun and frustrating.

The system relies on (you guessed it) shortcodes which both produce a list of images without previews and consume cached preview data to inline the scaled-down versions. Historically this worked by using sharp to generate low-res WebP (then AVIF) base64-encoded values that got blurred. I played a bit with BlurHash and ThumbHash, but the need for a <canvas> element was unattractive.

A better answer would have relied on CSS custom paint, but between Safari being famously rekt (and representing large fraction of this blog's readership), and the Paint Worklet context missing the ImageData() constructor, it never felt like a workable approach.

But as of this year, there's a new kid in town: Lean Rada's badass CSS-only LQIP approach.

That system is now implemented, which does a lot to shrink the HTML payloads of pages, as well as speeding up raster of the resulting image previews. This is visible in detailed traces, where the layout phase no longer has to wait for a background threads to synchronously decode image literals.

It took a few weekends of playing around to get it going correctly, as the code linked from Lean's blog post is not what he's using now. The colourspace conversion code they're using is also inaccurate, and so attempts to replace it with color-space produced visually incorrect results. Using the exact code they do for RGB-to-Lab conversion is necessary to generate the correct effect, and dialling in those differences was time-consuming.

Happy to make the code I use for this available upon request, but it's not amazing, and you really should go read Lean's blog post for yourself. It's a masterpiece.

I recently added markdown-it-abbr to my configuration to make some technical writing a bit more accessible. Across a few posts, this ended up with a lot of repetition for terms like “W3C”, “IETF”, etc.

This was both a bit time-consuming and error-prone. What if, I wondered, it were possible to centralise them?

Turns out it's trivial! My setup pre-processes markdown as Nunjucks (via markdownTemplateEngine: "njk"), which makes the full range of directives available, including…include.

This means I can just create a single file with commonly used acronyms and include it from every page; the physical location is _includes/acronyms.md:

{% include "acronyms.md" %}

This doesn't improve performance, but has been hugely helpful for consistency.

Putting it all together, what do we get?


On a page which previously saw contention between scripts for charts and above-the-fold fonts and initial layout work, the wins have been heartening on a low-spec test bench:


The long tent pole is now Netlify's pokey connection setup and TTFB, and I'm content (if not happy) with that.

]]>
Apple's Antitrust Playbook 2025-09-25T00:00:00Z https://infrequently.org/2025/09/apples-antitrust-playbook/

Contents

This blog is failing on several levels. First, September 2025 is putting the “frequent” in “infrequently”, much to my chagrin. Second, my professional mission is to make a web that's better for everyone, not to tear Apple down.

But just when I thought I was out, they pull me back in.

Over the past 24 hours, Cupertino has played the hits from its iPhone-era anti-user, anti-business, anti-rule-of-law power trip. To recap:

Make no mistake about what's going on: Apple is claiming that the EU is forcing Apple to adopt interpretations of the DMA that no other party has, which the EC itself has not backed, and which are “forcing” Apple to avoid shipping features in the EU.

Or, put another way, Apple wants to launder the consequences of its own anticompetitive, anti-user choices through a credulous tech press. The goal is to frame regulators for Apple's own deeds.

If this smells like bullshit, that's because it is.

Apple tried similar ploys last year when it sprung a blatant attempt to kill PWAs on the EU as a price for having the temerity to regulate. The plan, it seemed then, was to adopt an exotic reading of the DMA and frame the EC for killing PWAs.

What Apple's attempting now is even more brazen, particularly after 16+ months of serial non-compliance since the DMA came into force (h/t OWA).

Remember that:

  • Apple's browser, and by extension every iOS browser, fails on basic security and privacy controls, putting users at risk constantly. Apple serially misrepresents this situation to anyone who will listen.

  • Cupertino has constructed a legal (PDF) and technical thicket that prevents competitors from building better browsers, including:

    • Geofencing choice to the EU.

    • Forcing users to download separate browsers to access better versions (should they appear).

    • Withholding PWA and Push Notification APIs from competitors.

    • Providing a busted SDK that included multiple show-stopping bugs.

    • Making the process of acquiring and using the necessary entitlements for development nearly impossible.

    • Gaslighting the EC and the press at every turn; simultaneously claiming it is complying, while complaining loudly that it has to do work to implement the wholly unnecessary hurdles to compatibility that Apple itself created.

  • Apple have used every opportunity to rubbish both the spirit and letter of the law, spreading outlandish claims about user harms. These claims keep getting rejected whenever they're weighed by regulators and independent third parties.

All of this to prevent the web from threatening the App Store, via the DMA.

Add today's shake-down, and we see an assault on the rule of law. Tim's appearance in The Oval bearing gaudy-ass trinkets makes sense as a kickback for services rendered against the idea of laws. Or at least laws that bind those with power the way they constrain everyone else.

This is not a one-off or a fluke.

The name of the game is delay, and Apple is using the same tactics it has practised to maintain profits through outrageous attacks on the right-to-repair and sensible anti-e-waste regulation. When you're the incumbent, delay is winning. Apple has pulled out all the stops to prevent the web from providing a safer, more private alternative to native apps, winning long reprieves thus far. And you should see what they did on the native side. phew.

This is extremely embarrassing for the EU, which has attempted to respond thoughtfully and reasonably to every provocation. But appeasement of Cupertino isn't working.

Apple didn't author the playbook it's using now; it has been developed and honed for 40 years by toxic industries trying feverishly to escape the consequences of their actions. Most famously, the tactics Apple is gleefully employing served Big Oil and Big Tobacco extremely well:

  1. Lobbying (legalised corruption).

    This works a shockingly large fraction of the time, and was sufficient to kill effective regulation of the smartphone ecosystem in the US during Biden's term, despite overwhelming evidence from the NTIA's exhaustive report.

    If first-party lobbying hits roadblocks — e.g., being absolutely outclassed by civil society groups that see the real threats and costs of your actions — there are alternatives, including...

  2. Astroturf.

    If "ACT" rings a bell, it might be for the role it played in the '90s in the (hollow) defence of Microsoft.

    These days, it's a bought-and-paid-for megaphone for Apple, and it is working to worm its way into EU policy circles too. For a flavour of the Klein bottle arguments it serves up regularly, imagine arguing that more choice than a single App Store would be bad because it would hurt small developers. 🤯

  3. Market false comparison points.

    Apple has a point about privacy and security when it comes to Android, e.g., but why is that our comparison point? Why is the App Store's historic failure to safeguard anyone from anything (by comparison to browsers) our collective counterfactual? It's nonsense. Hot tosh. But it's marketed with gusto and zeal in the hopes that nobody will notice. And the tech press, to their everlasting discredit, have more than played along.

    When that doesn't work...

  4. Perform petulance.

    If a regulator or government has the stones to stand up for its own democratic polity in a way that attacks your profit potential, go to the matresses. Let everyone know they'll be sorry. Throw as big a hissyfit as you can manage, then make the same false claims over and over.

    To make obvious falsehoods stick, shameless repetition, at volume, is a must.

  5. Market your "compliance."

    But under no circumstances, actually comply.

    The trick here is to performatively roll out processes and press briefings telling everyone how seriously you take all of this while actively gumming up the works. Apple have done this on the regular since the DMA's passage, and if that rhymes with various ongoing plots against right-to-repair after public claims to support it...well...

  6. Claim law hurts kittens, blame regulators

    Doesn't have to be kittens, obviously. Jobs, privacy, apple pie...anything that seems sympathetic will do. Just make fig-leaf claims to support your side and lean heavily into "won't someone think of the kids?"

    Assuming your bought-and-paid-for astroturfers and docile access-journalism reporters are still on speed dial, GOTO 2, safe in the assumption that regulators won't do much.

Honestly, it's a winning formula, assuming you have more money than Croesus and lawyers willing to make claims they know to be untrue. All they better if they're so well paid that they can't be bothered to check.

It only falls apart if the folks whose job it is to ferret out the truth, and the people whose role is to stand up for citizens against vested interests, do their jobs.

At the risk of stating the obvious, it does not matter what your relationship to Cupertino is. It's in your head. They threaten to call your boss? Or to stop talking to you? So what? They always do that. To everyone. All the time. Ask anyone. There's no upside, and you can never be on the inside. You were always being managed.

Cupertino's gonna threaten to stop running ads on your site? Print that.

Where are your stones? Why did you get into journalism? Or technology? Or technology journalism? What, would you say, 'ya do here?

The obvious stenography around competition issues is mind-numbing when the theory of the case is so bloody simple. Apple and Google want everyone in their app stores because that's how they maintain power, and through that power, profits. They have arranged things such that being in the App Store is the only way developers get access to critical APIs, and those APIs are the only way to make functional apps.

Apple's fighting real browsers and the DMA because they're a threat to that model. Browsers, and the PWAs they enable, are the open, tax-free alternative to app stores, and the Duopolists are extremely displeased that they exist, even in the degraded form they currently allow.

This is simple. Obvious. Incredibly transparent.

But almost nobody will connect the dots in print. And that's a scandal, too.

]]>
Comforting Myths 2025-09-23T00:00:00Z https://infrequently.org/2025/09/cupertinos-comforting-myths/

Update:
The charts below were inappropriately snapped, compressing the range of values. This has been corrected. Feature availability numbers also changed since drafting and have been updated.

In several recent posts, I've attempted to address how the structure of standards bodies, and their adjacent incubation venues, accelerates or suppresses the potential of the web as a platform. The pace of progress matters because platforms are competitions, and actors that prevent expansions of basic capabilities risk consigning the web to the dustbin.

Inside that framework, there is much to argue over regarding the relative merits of specific features and evolutionary directions. This is healthy and natural. We should openly discuss features and their risks, try to prevent bad consequences, and work to mend what breaks. This competitive process has made browsers incredibly safe and powerful for 30 years.

Until iOS, that is.

Imagine my surprise upon hearing that Apple isn't attempting to freeze the web in amber, preserving advantages for its proprietary platform, and that it instead offers to redesign proposals it disagrees with.

As I have occasionally documented, this has not been my experience. I have relatively broad exposure to the patterns of Apple's collaboration, having designed, advised on, or led teams that built dozens of features across disparate areas of the platform since the Blink fork.1

But perhaps this was the wrong slice from which to judge? I've been hearing of Apple's openness to collaboration on challenging APIs so often that either my priors are invalid, or something else is at work. To find out, I needed data.

A specific parry gets deployed whenever WebKit's sluggish feature pace comes up: “controversial” features “lack consensus” or “are not standards” or “have privacy and security problems” (unspecified). The corollary being that Apple engages in good-faith to address these developer needs in other ways, even in areas where they have overtly objected.

Apple's engine has indisputably trailed Blink and Gecko in all manner of features over the past decade. This would not be a major problem, except that Apple prevents other browsers from delivering better and more competitive web engines on iOS.

Normally, consequences for not adopting certain features arrive in the market. Browsers that fail to meet important needs, or drop the ball on quality lose share. This does not hold on iOS because no browser can ship a less-buggy or more capable engine than Apple's WebKit.

Because competitors are reduced to rebadging WebKit, Apple has created new responsibilities and expectations for itself.2 Everyone knows iOS is the only way to reach wealthy users, and no browser can afford to be shut out of that slice of the mobile market. Therefore, the quality and features of Apple's implementation matter greatly to the health and competitiveness of the web.

This put's Apple's actions squarely in the spotlight.

It's possible to size up Apple's appetite for problem-solving in several ways. We can look to understand how frequently Apple ships features ahead of, or concurrently with, other engines because near-simultaneous delivery is an indicator of co-design. We can also look for visible indications of willingness to engage on thorny designs, searching for counter-proposals and shipped alternatives along the way.

This chart tracks single-engine omissions over the past decade; a count of designs which two engines have implement but which a single holdout prevents from web-wide availability:

Features missing in one engine, '15-25. Lower is better.<br>Safari consistently trails every other engine, and APIs missing from it impact every iOS browser.
Features missing in one engine, '15-25. Lower is better.
Safari consistently trails every other engine, and APIs missing from it impact every iOS browser.

Thanks to the same Web Features data set, many views are possible. This data shows that there are currently 178 features in Chromium that are not available in Safari, and 34 features in Safari that are not yet in Chromium. (or 179 and 37 for mobile, respectively). But as I've noted previously, point-in-time evaluations may not tell us very much.

I was curious about delays in addition to omissions. How often do we see evidence of simultaneous shipping, indicating strong collaboration? Is that more or less likely than leading vendors feeling the need to go it alone, either because of a lack of collaborative outreach, or because other vendors do not engage when asked?

To get a sense, I downloaded all the available data (JSON file), removed features with no implementations, removed features introduced before 2015, filtered to Chrome, Safari, and Firefox, then aggregated by year. The resulting data set is here (JSON file).

The data can't determine causality, but can provide directional hints:

Safari rarely leads, but that does not mean other vendor's designs will stand the test of time. But if Apple engages in solving the same problems, we would expect to see Safari leading on alternatives3 or driving up the rates of simultaneously shipping features once consensus emerges. But these aren't markedly increased. Apple can, of course, afford to fund work into substitutes for “problematic” APIs, but it doesn't seem to.

Narratives about collaboration in tricky areas take more hits from Safari's higher incidence of catch-up launches. These indicate Apple shipping the same design that other vendors led with, but on a delay of two years or more from their first introduction. This is not redesign. If there were true objections to these APIs, we wouldn't expect to see them arrive at all, yet Apple has done more catching up over the past several years than it has shipped APIs with other vendors.

This fails to rebut intuitions developed from recent drops of Safari features (1, 2) composed primarily of APIs that Apple's engineers were not primary designers of.

But perhaps this data is misleading, or maybe I analysed it incorrectly. I have heard allusions to engagement regarding APIs that Apple has publicly rejected. Perhaps those are where Cupertino's standards engineers have invested their time?

Most of the hard cases concern APIs that Apple (and others) have rightly described as having potentially concerning privacy and security implications. Chromium engineers agreed those concerns have merit and worked to address them; we called it “Project Fugu” for a reason. In addition to meticulous design to mitigate risks, part of the care taken included continually requesting engagement from other vendors.

Consider the tricky cases of Web MIDI, Web USB, and Web Bluetooth.

Apple has supported MIDI in macOS for at least 20 years — likely much longer — and added support for MIDI on iOS with 2010's introduction of Core MIDI in iOS 4.2. By the time the first Web MIDI proposals broke cover in 2012, MIDI hardware and software were the backbone of digital music and a billion dollar business; Apple's own physical stores were stocking racks of MIDI devices for sale. Today, an overwhelming fraction of MIDI devices explicitly list their compatibility with iOS and macOS.

It was therefore a clear statement of Apple's intent to cap web capabilities when it objected to Web MIDI's development just before the Blink fork. The objections by Apple were by turns harsh, condescendingly ignorant and imbued with self-fulfilling stop-energy; patterns that would repeat post-fork.

After the fork and several years of open development (which Apple declined to participate in), Web MIDI shipped in Chromium in early 2015. Despite a decade to engage, Safari has not shipped Web MIDI, Apple has not provided a “standards position” for it4, and has not proposed an alternative. To the best of my knowledge, Apple have also not engaged in conversations about alternatives, despite being a member of the W3C's Audio Working Group which has published many Working Drafts of the API. That group has consistently included publication of Web MIDI as a goal since 2012.

Across 11 charters or re-charters since then, I can find no public objection within the group's mailing list from anyone with an @apple.com email address.5 Indeed, I can find no mentions of MIDI from anyone at Apple on the public list. Obviously, that is not the same thing as agreeing to publication as a Recommendation, but it also not indicative of any attempts at providing an alternative.

But perhaps alternatives emerged elsewhere, e.g., in an Incubation venue?

There's no counter-proposal listed in the WebKit explainers repository, but maybe it was developed elsewhere?

We can look for features available behind flags in Safari Technology Preview and read the Tech Preview release notes. To check them, I used curl to fetch each of the 127 JSON files that are, apparently, the format for Safari's release notes, pretty-printed them with jq, then grepped case-insensitively for mention of “audio” and “midi”. Every mention of “audio” was in relation to the Web Audio API, the Web Speech API, WebRTC, the <audio> element, or general media playback issues. There were zero (0) mentions of MIDI.

I also cannot locate any public feedback on Web MIDI from anyone I know to have an @apple.com email address in the issue tracker for the Web Audio Working Gropup or in WICG except for a single issue requesting that WICG designs look “less official.”

The now-closed Web MIDI Community Group, likewise, had zero (0) involvement by Apple employees on its mailing list or on the successor Audio Community Group mailing list. There were also no (0) proposals covering similar ground that I was able to discern on the Audio CG issue list.

Instead, Apple have issued a missive decrying Web MIDI as a privacy risk. As far as anyone can tell, this was done without substantive analysis or engagement with the evidence from nearly a decade of deploying it in Chromium-based browsers.

If Apple ever offered an alternative, or to collaborate on a redesign, or even an evidence-based case for opposing it,6 I cannot find them in the public record.7


Missing alt text

USB is a security sensitive API, and Web USB was designed with those concerns in mind. All browsers that ship Web USB today present “choosers” that force users to affirmatively select each device they provide access to, from which sites, and always show ambient usage indicators that let users revoke access. Further, sensitive device classes that are better covered by more specific APIs (e.g., the Filesystem Access API instead of USB Mass Storage) are restricted.

This is far from the Hacker News caricature of "letting any web page talk to your USB devices," allowing only point connections between individual devices and sites, with explicit controls always visible.

Web USB has been available to an overwhelming majority of the web's users for the better part of a decade. The sky has not fallen.
Web USB has been available to an overwhelming majority of the web's users for the better part of a decade. The sky has not fallen.

After two years of public development and a series of public Origin Trials lasting seven months (1, 2), the first version of the API shipped in Chrome 61, released September 2017.

I am unable to locate any substantive engagement from Apple about alternatives for the motivating use-cases outlined in the spec.

With many years of shipping experience, we can show that these needs have been successfully addressed by WebUSB; e.g. teaching programming in classrooms. More than a decade after it was first approached about them, it's unclear what Apple's alternative is. Hollowing out school budgets to buy Cupertino's high-end devices to run unsafe, privacy-invading native apps?

Apple have included Web USB on the list of APIs they “decline to implement” and quite belatedly issued a “standards position” opposing the design. But no counter-proposal was developed or linked from those threads, despite being asked directly if there might be more palatable alternatives.

I can locate no appetite from Apple's standards engineers to address these use-cases, know of no enquiries into data about our experiences shipping them, and can find no constructive counterproposals. Which raises the obvious question: if Apple does engage to develop counterproposals in tricky areas, how long are counterparties meant to wait? More than eight years?

Like Web USB, Web Bluetooth was designed from the ground-up with safety in mind and, as a result, has been incredibly safe and deployed at massive scale for eight years. It relies on the same chooser model in Chromium-based browsers.

Prior to getting user consent, this PWA can't talk to any devices.

Pairing is only possible when users explicitly choose a device.

Once connected, data can only be sent over the safer GATT subset of the larger Bluetooth protocol.

As with all Project Fugu device APIs, Web Bluetooth was designed to reduce ambient risks — no access to Bluetooth Classic, available only on secure sites, and only in <iframe>s with explicit delegation, etc. — and to give implementers flexibility about the UIs they present to maximise trust and minimise risk. This included intentionally designing flexibility for restricting access based on context; e.g., only from installed PWAs, if a vendor chooses that.

The parallels with Web USB continue on the standards track. I can locate no engagement from any @apple.com or @webkit.org email addresses on the public-web-bluetooth mailing list. In contrast, when design work began in 2014 and every browser vendor was invited to participate, Mozilla engaged. I can find no evidence of similar openness on the part of Apple, nor practical counter-proposals.

Over more than three years of design and gestation in public, including very public Origin Trials, Apple did not provide constructive feedback, develop counter-proposals, or offer to engage in any other way I can find.

This appears to be a pattern.

From a deep read of the “standards position” threads for designs Apple opposes, I cannot find evidence that Cupertino has ever offered a counter-proposal to any API it disfavours.

These threads do demonstrate a history of downplaying clearly phrased developer needs, rather than proactive engagement, and it seems the pattern is that parties must beg Apple to belatedly form an opinion. When there is push-back, often after years of radio silence, requesters (not Apple) also have to invent potential alternatives, which Apple may leave hanging without engagement for years.

Worse, there are performative expressions of disinterest which Apple's standards engineers know are in bad-faith. An implementer withholding engagement from a group, then claiming a lack of implementer engagement in that same venue as a reason not to support a design, is the sort of self-serving, disingenuous circularity worthy of disdain.

Perhaps both the general trends and these specific high-profile examples are aberrant. Perhaps Apple's modus operandi isn't to:

  • Ignore new incubations, even when explicitly asked to participate.
  • Fail to register concerns early and collaboratively, where open design processes could address them.
  • Force web developers and other implementers to request “positions” at the end of the design process because Apple's disengagement makes it challenging to understand Cupertino's level of support (or antipathy).

Mayhaps it's not simply the predicable result of paltry re-investments in the web by a firm that takes eye watering sums from it.

Committers to Blink, V8, WebKit, and JSC by affiliation. Green is Apple, blue is Google, tan is Igalia, and brown is Microsoft.
Committers to Blink, V8, WebKit, and JSC by affiliation. Green is Apple, blue is Google, tan is Igalia, and brown is Microsoft.

The true scale of the funding gap becomes clearer as more projects are included. Here we add WebRTC and Web Platform Tests, both of which are shared between browsers and take contributions from many engines.
The true scale of the funding gap becomes clearer as more projects are included. Here we add WebRTC and Web Platform Tests, both of which are shared between browsers and take contributions from many engines.

If so, I would welcome evidence to that effect. But the burden of proof no longer rests with me.8

It's hard to say why some folks are under the impression that Apple are generous co-designers, or believe Apple's evocative statements about hard-case features are grounded in analysis or evidence. We can only guess at motive.

The most generous case I can construct is that Apple's own privacy and security failures in native apps have scared it away, and that spreading FUD is cover for those sins. The more likely reality is that upper management fears PWAs and wants to keep them from threatening the App Store with safe alternatives that don't require paying Apple's vig.

Whatever the cause, the data does not support the idea that Apple visibly engages in constructive critique or counter-proposal in these areas.

Moreover, it shows that many of Apple's objections and delays were unprincipled. It should be every browser's right to control the features it enables, and Cupertino is entirely within those rights to avoid shipping features in Safari. But the huge number of recent “catch-up” features tells a story that aligns more with covering for embarrassing oversights, rather than holding a line on quality, privacy, or security.

On the upside, this suggests that if and when web developers press hard for capabilities that have been safe on other platforms, Cupertino will relent or be regulated into doing so. It scarcely has a choice while simultaneously skimming billions from the web and making arguments like these to regulators (PDF, p35):

Apple is arguing to regulators that the App Store faces plenty of competition, including from web apps. The very same web apps it continually hamstrings, deprives of features, and does not demonstrably engage to improve.
Apple is arguing to regulators that the App Store faces plenty of competition, including from web apps. The very same web apps it continually hamstrings, deprives of features, and does not demonstrably engage to improve.

The moment iPhone users around the world can install high-quality browsers, the conversational temperature about missing features and reliability will drop considerably. Until then, it remains important that Apple bear responsibility for the problems Apple is causing not only for Apple, but for us all.

FOOTNOTES

  1. My various roles since the Blink fork have included:

    In my role as TAG member, Fizz/Fugu TL, and API OWNER I've designed, reviewed, or provided input on dozens of web APIs. All of this work has been collaborative, but these positions have given me a nearly unique perch from to observe the ebb and flow of new designs from, particularly on the "spicy" end of the spectrum.

  2. Apple has many options for returning voluntarity to the market for iOS browsers.

    Most obviously, Apple can simply allow secure browsers to use their own engines. There is no debate that this is possible, that competitors generally do a better job regarding security than Apple, and that competitors would avail themselves of these freedoms if allowed.

    But Apple has not allowed them.

    Open Web Advocacy has exhaustively documented the land mines that Apple has strewn in front of competitors that have the temerity to attempt to bring their own engines to EU users. Apple's choice to geofence engine choice to the EU, indefensible implementation roadblocks, poison-pill distribution terms, and the continued prevarications and falsehoods offered in their defence, are choices that Apple is affirmatively and continually making.

    Less effectively, Apple could provide runtime flags for other browsers to enable features in the engine which Apple itself does not use in Safari. Paired with a commitment to implement features in this way on a short timeline after they are launched in other engines on other OSes, competing vendors could risk their own brands without Apple relenting on its single-implementer demands. This option has been available to Apple since the introduction of competing browsers in the App Store. As I have argued elsewhere, near-simultaneous introduction of features is the minimum developers should expect of a firm that skims something like $19BN/yr in profits from the web (a ~95% profit rate, versus current outlays on Safari and WebKit).

    Lastly, Apple could simply forbid browsers and web content on iOS. This policy would neatly resolve the entire problem. Removing Safari, along with every other iOS browser, is intellectually and competitively defensible as it removes the “special boy” nature of Safari and WebKit. This would also rid Apple of the ethical stain of continuing to string developers and competitors along within standards venues when it is demonstrably an enemy of those processes.

  3. Apple regularly goes it alone when it is convinced about a design. We have seen this in areas as diverse as touch events, notch CSS, web payments, "liquid glass" effects, and much else. It is not credible to assume that Apple will only ship APIs that have an official seal of an SDO given Cupertino's rich track record of launch-and-pray web APIs over the years.

  4. In fairness to Apple regarding a "standards position" for Web MIDI, the feature predates Apple's process. But this brings up the origin of the system.

    Why does this repository exist? Shouldn't it be rather obvious what other implementers think of that feature, assuming they are engaged in co-design?

    Yes, but that assumes engagement.

    Just after the Blink fork, a series of incidents took place in which Chromium engineers extrapolated from vaguely positive-sounding feedback in standards meetings when asked about other vendor's positions as part of the Blink Launch Process. This feedback was not a commitment from Apple (or anyone else) to implement, and various WebKit leaders objected to the charachterisations. As a way to avoid over-reading tea leaves in the absence of more fullsome co-design, the "standards position" process was erected in WebKit (and Gecko) so that Chromium developers could solicit "official" positions in the many instances where they were leading on design, in lieu of clearer (tho long invited) engagement.

    If this does not sound like it augurs well for assertions that Apple engages to help shape designs in a timely way...well, you might very well think that. I couldn't possibly comment.

  5. There may have been Formal Objections (as defined by the W3C process) in private communications, but Member Confidentiality at the W3C precludes me from saying either way. If Apple did object in this way, it will have to provide evidence of that objection for the public record, as I cannot.

  6. Apple's various objections to powerful features have never tried to square the obvious circle: why are camera and microphone access (via getUserMedia()) OK, but MIDI et. al are not? What evidence supports the notion that adding chooser-based UIs will lead to pervasive privacy issues that cannot be addressed through mechanisms like those Apple is happy to adopt for Geolocation? Why, despite their horrendous track records, are native apps the better alternative?

  7. Mozilla objected to Web MIDI on various grounds over the years, and after getting utterly roasted by its own users over failing to support the API, shipped support in Firefox 108 (Dec '22).

    The larger question of Mozilla's relationship to device APIs was a winding road. It eventually culminated (for me) in a long discussion at a TAG meeting in Stockholm with EKR of TLS and Mozilla fame.

    By 2016, Mozilla was licking its wounds from the failure of Firefox OS and retrenching around a less expansive vision of the future of the web. Long gone were the aspirations for "WebAPIs". Just a few short years earlier, Mozilla would have engaged (if not agreed) about work in this space, but an overwhelming tenor of conservativism and desktop-centricity radiated from Mozilla by the time of this overlapping IETF/W3C meeting.

    It didn't make the notes, but my personal recollection of how we left things late in the afternoon in Stockholm was EKR claiming that bandwidth for security reviews was the biggest blocker that and that it was fine if we (Chromium) went ahead with these sorts of designs to prove they wouldn't blow up the world. Only then would Mozilla perhaps consider versions of them.

    True to his word, Mozilla eventually shipped Web MIDI on EKR's watch. If past is prologue, we'll only need to wait another three to five years before Web Bluetooth et al. join them.

  8. My memory is famously faulty, and I have been engaged in a long-running battle with Apple's legal folks relating to the suppression of browser choice on iOS. All of that colours my vision, and so here I have tried to disabuse myself of less generous notions by consulting public evidence to support Apple's case.

    From what I was able to gather over many hours was overwhelmingly inculpatory. It is not possible from reading these threads and data points — rather relying on my own recollections — to sustain a belief that Apple have either provided timely constructive feedback on tricky APIs, or worked to solve the problems they address. But I am, in the end, heavily biased.

    If my conclusions or evidence are wrong, I would very much appreciate corrections; my inbox and DMs are open.

    If reliable evidence is provided, I will update this blog post to include it, and I encourage others to post on this topic in opposition to my conclusions. It should not be hard for Apple to make the case, assuming there is evidence to support it, that I've missed important facts. It would have both regulatory and persuasive valence regarding questions I have raised relating to Apple's footprint in the internet standards community.

]]>
Web Standards and the Fall of the House of Iamus 2025-09-21T00:00:00Z https://infrequently.org/2025/09/standards-and-the-fall-of-iamus/

Photo by Photo by Artan Sadiku
Photo by Photo by Artan Sadiku

Commentary about browsers and the features they support is sometimes pejorative towards pre-standardisation features. Given Apple's constriction of Mozilla's revenue stream and its own strategic under-funding of WebKit, this usually takes the form "Chromium just ships whatever it wants."

This is true, of course, but not in the way commenters intend; and not only because Blink's rigorous launch process frequently prevents unvalidated designs from shipping.

Except for iOS, where Apple attempts to hide its role in undermining the competitive foundation of web standards, every vendor always ships "whatever it wants." That is the point of voluntary standards, and competition is the wind in those sails.

Working Groups don't gate what browsers ship, nor do they define what's useful or worthy. There are no seers; no Delphic oracles that fleetingly glimpse a true logos. Working Groups do not invent the future, nor do they hand down revealed truths like prophets of the House of Iamus.

In practice, they are thoughtful historians of recent design expeditions, critiquing, tweaking, then spreading the good news of proposals that already work through Web Standards ratified years after features first ship, serving to licence designs liberally to increase their spread.

In the end, this is the role of standards and the Working Groups that develop them: to license patents that read on existing designs, reducing risks to implementers who adopt them.

Anyone who tries to convince you otherwise, or invites you to try your hand at invention within a chartered Working Group, does not understand what those groups are designed to do. Sadly, this includes some folks who spend a lot of time in them.

Complaints about certain engines leading reveals a bias to fighting the last-war; fear of a return to the late 90s, when “irrational exuberance” applied not just to internet stocks, but also browser engines.

Unfounded optimism about our ability to deal with externalities of shipping prior to consensus created a great deal of pain. Misreadings of how browsers evolve in practice over-taught lessons that still invoke fear. But we have enough miles on the tires to see that standards aren't what create responsible feature development; thoughtful launch processes are.1

As we will see, consensus is never a precondition for shipping, nor could it be. If it were, nothing would move forward, for it is the threat (and sometimes reality) of popular features shipping without a priori agreement that brings parties to the table. It's also the only reliable defence against the fifth column problem.

Before we get into the role of the charters that define Standards Development Organisation (SDO) Working Groups, we must define some terms:2

Proposal
Any design, in any sort of document (Specification, Explainer, or scrawl on the back of a napkin) offered inside an SDO for standardisation. Proposals are revised through collaborative processes which function best inside SDOs thanks to carve-outs in competition laws allowing competitors to collaborate in writing standards.
Proposals are implemented, and even shipped, well ahead of formal status as Web Standards in order to gain developer feedback, iterate on designs, and build confidence that they meet needs effectively.3
Specification
A document that describes the properties of an implementation, often in technical shorthand (e.g., Web IDL) and with reference to other specifications.
Specifications are by and for implementers, rather than working web developers, and are the documents to which the IPR commitments of Web Standards attach.
Explainer
An overview of a proposal. Explainers are not specifications, but encapsulate the key points of designs in terms of the problems they solve.4 The goal is to explain the why along with the what, in terms web developers can weigh.
In well-incubated designs, Explainers evolve in parallel to specifications. This forces API designers to think like web developers. Explainers serve as foci for considering alternatives and as well as “missing manuals” for early designs.
Web Standard
A specification that has been put forward by a chartered Working Group to the full membership, and granted official status after ratification. For example, by receiving the status as a Recommendation of the W3C by a vote of the Advisory Committee, or in the case of ECMA TC39, publication of an International Standard at ISO.
Web standards create clarity around IPR concerns which pre-standards specifications cannot. These commitments license the patents of parties participating in a Working Group under Royalty Free terms.
In practice, this also commits major firms to the defence of potential implementers, reducing risks across the ecosystem.
Standards Track
A specification document is "on the standards track" if its developers have signalled interest in formal, chartered Working Groups taking up their design for publication as a Web Standard, subject to clarification along the way. It is not necessary for any specific Working Group to agree for this description to hold.
Designs in this state often carry IPR grants, but provide less patent clarity ("contribution only") than official Web Standards.
Incubation
Any process outside a chartered Working Group in which web developers, browser vendors, and other interested parties meet to understand problems and design solutions. Drafts of incubated designs are often volatile. Eventually, if a problem is judged important by participants, one or more designs congeal into brief, high-level "Explainers".
Before features launch without flags in stable-channel browsers, developer and implementer feedback can lead to large changes in incubated designs. Designers and participants must actively solicit input and respond to substantive feedback constructively.

Proposals whose specifications receive formal status within SDOs are described as “adopted”, “recommended”, or “ratified”. This process binds the intellectual property of chartered Working Group participants to the licensing regime of the SDO by approval of the membership.

From this we can see that is nonsensical to discuss “proposals” and "standards" in the same breath. They are a binary pair. A specification has either one status or the other; it cannot be both.

No design that becomes a standard can start as anything but a proposal,5 and the liminal space of non-standard proposal is not somehow less-than or an aberration. Rather, it is an embryonic stage every design must pass through, and which every de jure Web Standard has taken at some point.

Armed with better language, we can develop an understanding of how features actually come to the web. Through this, we'll see why our optimism for standards has to come from our own powers of collaborative problem-solving, rather than waiting for designs to be handed down from Olympus.

Jake Archibald pointed out that linguistic precision is uncommon, and developers use the phrase “standard” or “web standard” loosely; a shorthand for features they can use without worry. That ground covers both official Web Standards and de facto standards. The example of WebP was raised, and was even more instructive than I recalled.6

A brief refresher: WebP was introduced by Google in 2010, shipping in Chromium in 2011. At that point, and for many years thereafter, WebP was not a standard. A high-level specification broke cover in 2012, but was not brought to an SDO. Patent rights and other IP questions were addressed only by Google's waiver and the Open Source licensing of libwebp.

Nearly all Chromium-based browsers adopted WebP by 2012. Microsoft Edge added support in 2018, Firefox joined the party in 2019, and Safari added support in 2020.

It was not until April 2021, a full decade after the format's introduction and after all major engines supported it, that WebP arrived at the IETF for standardisation. It finally earned RFC status in November 2024, four years after Apple unlocked wide compatibility, having been the last hold-out.

WebP gained tremendous adoption despite not being a standard for nearly 15 years. This has led some to suggest it was a Web Standard, as it had many of the properties expected of successful standards. Namely:

  • Broad adoption. All major engines gained support before standardisation began in earnest.
  • Stability. Despite adding opacity and animation later, every WebP image produced across the life of the format still renders correctly.
  • Interoperability. The same WebP images work across implementations.
  • IPR clarity. As the primary contributor to the format and reference implementation, Google's substantial legal and patent assets provided adopters with assurances related to the provenance of both the code and the codec fundamentals.

Many assume that Web Standards are the only path to achieving these attributes, but WebP shows us that this is mistaken. Web Standards deliver these properties most effectively and reliably, but WebP shows us that “being a standard” is not a requirement for interoperable implementation at scale, even in browsers.

Interoperability and adoption are, by inspection, independent of official status. Many formal Web Standards have spotty adoption or fail to achieve interoperability. The web also has a long history of de facto behaviours gaining official status long after interoperability and stability manifest in the market. This sits comfortably with the reality that every single web feature is non-standard for some fraction of its lifetime. At the limit, that fraction can be rather large.

Conflation of de facto standards, official Web Standards, wide deployment, and interoperability are likely to persist. However, those of us who work on browsers can and should be more precise, preferring “interoperability”, “widely deployed”, and “standards track” to “Web Standard” when those are more appropriate.

If we admit, as we must, that all features begin as non-standard proposals which proceed through official processes to become Web Standards, quality is never as simple as “standard good, non-standard bad.” A design can pass unaltered through every lifecycle stage without change to its technical shape or merits, and questions of wide implementation and interoperability are clearly extrinsic to the status of a design within an SDO.

We must locate definitions of relative goodness in more directly descriptive properties. I have given a few above, namely adoption, stability, interoperability, and IPR clarity. For a full account, we need to cover design-time attributes, including testability, specification rigour, wide review, and responsiveness to developer feedback . Taken together, they define what it means to solve important problems well on the standards track.

Developers and implementers weigh up these properties when judging APIs. Some designs, like WebP,7 can score highly on everything but adoption. WebP was not technically worse in 2015 than in 2019, or 2020, or 2024; only less widely adopted. Working web developers were free to exploit its benefits the whole time. Many did, and because it had been responsibly developed, interoperability followed, even before a proposal had been submitted to an internet standards body.

Likewise, we cannot judge other APIs shipped in a single engine to be low-quality on the basis of SDO status. Nor can we credit statements opposing them on that basis, particularly from parties that have not offered counterproposals. A single implementation may preclude features from becoming standards, but that says nothing about design fitness, or even prospects for eventual adoption.

This understanding draws our focus away from formal SDO status and toward more important properties of features. It also helps us weigh up the processes that gestate them:

  • Are they friendly and open to developers, or are they smoke-filled rooms where old-timers hold sway?
  • Do they encourage new work at the pace of emerging developer needs, or are they artificially constricted by a proliferation of veto players?
  • Do those processes generate good tests and strong specifications, or do a few powerful editors regularly cut corners?
  • Are features submitted for wide review to competent and responsive experts, or is Working Group "go fever" allowed to dominate?
  • Do processes put evidence of solving end-user and developer problems8 ahead of groupthink and reckons?
  • Do those processes allow high-quality features to ship ahead of full consensus, based on evidence?
  • Do they adequately protect implementers from IPR risks?

These questions do not line up clearly with corporate jerseys. They do not favour one company's designs over another's. Google, Microsoft, Apple, and Mozilla all have fielded designs that easily pass these tests, and others that flunked. That's the thing about quality; metrics don't care about status or affiliation. Quality doesn't have a “team”, it has units of measure.

Quality tests help us evaluate features in the market-based ecosystem of voluntary standards. The presumed working mode of Web SDOs is that many ideas will be tried (responsibly, hopefully) and the best ideas should gain traction. Eventually, better designs sway the market, leading to calls for standardisation.9 Competitors who objected, even strenuously, can then adopt liberally-licensed designs, provide alternatives, or lose share. This logic does not designate grandees or smoke-filled rooms as arbiters of quality. Instead, we rely on voluntary adoption, the testable qualities of shipped designs, and developers voting with their feet.

Responsible, quality-oriented feature development processes reject priesthoods, seers, and oracles, putting evidence-based engineering in their place. This focuses us on solving problems we can see, showing our work, inviting critique, and engaging constructively.

There were never curtains from behind which a feature's essential “standardness” could be revealed, and we should not invent them because the messier, egalitarian reality of feature evolution seems disquieting. Like “Best Picture”, formal SDO status isn't a reliable indicator of quality. Each must judge for themselves.

In the end, the wider internet engineering community always decides if problems are relevant and if designs are “correct.”

There are no blinding flashes of insight, only the push and pull of competing proposals to solve user and developer needs. When standards fail to meet needs, developers patch over the gaps at great cost, or route around standards-based platforms altogether. It is no coincidence that the phases of the Blink Launch Process and tools like Origin Trials present practical steps away from Working Group mysticism. Moving, however haltingly, towards egalitarian, pluralistic collaboration has delivered tremendous progress, despite recalcitrant participants.10

Along with other experienced colleagues at Google and Microsoft, I have had the privilege to teach the practice and theory of standards-track feature development to talented engineers working inside Chromium.

The role of chartered Working Groups looms large in those conversations, but it is far from central. This is because the primary job of a Working Group is to accelerate adoption of existing designs that have enough support to move forward and insulate them from IPR-based attacks. This is both their official job (per SDO process documents) and their primary value.

“I” dotting and “t” crossing is expected, and it can slightly alter designs, but chartered Working Groups never work well as venues for effectively canvassing developers for problems to solve. Nor are they good environments for testing ideas. If the goal is to solve new problems or design new solutions, chartered Working Groups are never the best forum for the simple reason that it is not what they are built to do.

Consider the most recent charters from a few lively W3C Working Groups:

Per the formal process governing charters, each group's convening document contains several sections that outline:

  • A problem statement, laid out at the group's (re)charter.
  • An explicit scope; i.e., technologies the group can consider in addressing that problem.
  • An a priori list of documents and deliverables the WG will produce.

Everything in these charters works to provide clarity and precision about what the group will deliver because that is how members (firms, in the W3C's case) weigh up the costs and benefits of joining relative to their patent portfolios.

Yes, it's all about patents. Standards Development Organisations are, practically speaking, IPR clearing houses. From this perspective, all the hoops one must jump through to join a chartered group make sense:

  • Q: Why do companies have to officially join each Working Group separately?

    A: Patents.

  • Q: Why does each participant need to be approved by a company's main W3C delegate?

    A: Patents.

  • Q: Why are there so many checkpoints in the process that appear to do little more than waste time?

    A: Patents.

  • Q: Why does the whole membership need to vote to progress a document to a formal standard?

    A: Patents.

Features named in Working Group charters are the only designs a group can standardise, and groups run into trouble with lawyers when they exceed charter scopes, because participating firms may not have done a search in their portfolio (or their competitor's) regarding designs outside the pre-determined scope. When this happens, all progress can halt for an indeterminate amount of time. SDO processes and documents, therefore, work to cabin the risk, moving early design out of Working Groups and into other venues.

All of this drives to a simple set of obvious conclusions: venues other than chartered Working Groups are necessary in order to do the sort of high-iteration-pace design work that leads to higher-quality designs.

It's a punchline that standards committees are bad at design, but this is by design. They do not have the cycles, mission, or tools needed to explore developer problems, let alone iterate towards compelling solutions. Their job, per the official process documents that convene them, is to ensure that certain quality attributes are tended to (e.g., tests and complete specifications) on the way towards licensing of the intellectual property embodied in specifications.

To this end, internet SDOs have many alternatives that are better suited to design work, and which create glide paths for the fittest proposals into chartered Working Groups to gain eventual status as Web Standards. Here are a few:11

It's always possible for designs to fail to solve important problems, or fail to solve them well, and better designs flow naturally from a higher pace of iteration and feedback. Incubation venues tend to work better for design because they lower some strictures Working Groups erect to create patent clarity. Instead of needing to pre-define their deliverables, these groups can fail fast and iterate. When they work, they can be submitted to Working Groups to get patent and IP issues finalised.

Incubation venues display very different working modes and membership than formal Working Groups. Instead of pre-decided, formal deliverables, groups can form around nebulous problem statements and define the issues long before landing on a single design. There is no stable set of delegates from member firms, largely dominated by implementers. Rather, interest groups may form organically, even from web developers interested in shared problems. Instead of predefined timelines and pre-ordained deliverables, these groups work at the speed of their need and understanding. And instead of design pivots representing potential failure, incubation venues can test multiple alternatives. Discarding entire designs is even healthy while the furious iteration of incubation plays out.

Lower barriers to entry for web developers, along with structures that encourage dynamic problem-solving coalitions, tend to create higher pace and more focused work. This, in turn, allows consideration of more alternatives and helps facilitate the iterations that hone design. Not every Incubation venue succeeds, and not every design they consider wins. But their outsized success vs. design-in-committee approaches has been apparent for more than a decade.

Only those mistakenly wedded to, or cynically selling, mystical notions of Working Groups as diviners of revealed truth try to rubbish the web's incubation venue's outstanding track record of delivery. Given the overwhelming success of incubation, including the demonstrated benefits of fail-fast iteration, we must then ask why there are still proponents of Working Group misuse and process opacity.

Splitting the needs of design from the strictures of standardisation isn't against the spirit of standardisation, nor is shipping ahead of a vote by the full membership of an SDO problematic. These approaches are written into the process documents of the internet's most effective bodies. Implementation experience is required for a proposal to become a ratified standard, and Working Groups are enjoined by process from undertaking the explorations incubation venues specialise in. Which is why those same SDOs build and encourage incubation groups.

This is an uncomfortable acknowledgement for some.

Folks I have talked to seem to imagine rooms filled with people who know everything and always make correct decisions. There are no such rooms. We're all just engineers trying to solve problems, from the heights of the TAG and the Internet Architecture Board to the scrappiest W3C Community Group or IETF Bar BOF. And sometimes we get it very wrong. But that should not immobilise us in fear; it should cause us to do what we always do when engineering: adopt processes and guardrails to improve quality and increase the rate of delivery.

Problems only get solved by people of good cheer working hard to spot them,12 trying out solutions, and working collaboratively to document the ones that work best. There are no oracles in standards work, only engineers. And that's more than enough.

FOOTNOTES

  1. While the Blink Launch Process raised the bar in some ways for every project, many parts of Blink's risk-based logic for feature leadership still lack analogues for tools like Origin Trials in other projects. Over the past decade, this has generally not been a major problem because other projects rarely lead (sadly).

  2. And, it would seem, long-time browser engineers and standards participants.

  3. The IETF famously phrases this principal as "rough consensus and running code", which indicates that that formal status as an official standard should be reserved for designs with experience in the market.

    A frustrating fraction of the web's erstwhile friends twist the arrow of causality, demanding formal standards status before they (or anyone else) dare to ship a proposal. This is, of course, entirely backwards, but sounds convincing to web developers who desire interoperability which comes from uniform implementation status which, historically, correlates with de jure status.

    This rhetorical bait-and-switch is deployed very frequently by fifth columnists.

  4. Most contemporary web standards explainers use a template derived from a document I authored for use by Google's Web Standards engineers, and which was adopted by the W3C's Technical Architecture Group during my service there.

    The current template has drifted from the original somewhat, but the initial intent is in tact: to favour the perspective of working developers over implementers when weighing up potential designs.

    This explicit goal is realised through many steps of the Blink Launch Process. It's that process which forces Blink engineers to request TAG design review for new features. This combines with the reality that most new web features begin life within Chromium and Blink's “coalition of the willing”. As a result, designers of important new features experience Blink's process guardrails as the most constricting influence on quality as they work towards shipping. And one of those guardrails is to explain features in terms that web developers can understand via Explainers.

    As a sociotechnical system, the Blink Launch Process, and its integrations with the TAG and various incubation bodies, are calculated to act in a Popperian mode, rejecting claims to design authority and oracular Working Group pronouncements. In their place, we explicitly delegate power to groups that have done a good job of representing wider constituencies. The goal is to place the key question "does this design solve an important problem well?" at the centre of feature development, and to trust users (web developers) as judge of the results.

    We reject both essentialist and inductive intuitions about where features “should” come from, and instead require designers walk a mile in the shoes of their users. This is easiest to do in incubation venues like the WICG, where the barriers to bringing web developers into the conversation are lowest.

    This also is why the Blink API OWNERS expect “Considered Alternatives” sections of explainers to be exhaustive, and that examples appear the languages developers use (HTML, CSS, and JS), not the languages implementers think in (WebIDL and C++). Domain experts may understand the system, by sympathy for the machine is not a substitute for web developers trying designs and rendering their own judgement. The Blink Launch Process guards against go fever while enabling feature designers to launch responsibly, even when forced to go first by the disinterest of other vendors in solving problems.

  5. Even if a proposal is, for some blighted reason, floated first within a chartered Working Group, it is still only that: a proposal without any standing as a standard. It is only upon publication of a formal standard document months or years later that a proposal, encoded in a specification, can be called a Web Standard and carry any of the attendant IP protections.

    That necessarily happens after the proposal has been offered, and generally happens months or years after all design work has completed and multiple interoperable implementations are fielded.

  6. Until prompted to consider it, I had not been aware that WebP had made the transition to a formal standard in any venue.

  7. It would, obviously, have been better for WebP to be floated at IETF many years prior, but adoption in every major engine occurred before that process began.

  8. In that order, always.

  9. In reality, new web features are a thinly traded market, and long experience (e.g., CSS's “prefixpocalypse") has taught the web community that uncontrolled experiments are disruptive and costly. This lesson has, in my view, been heavily over-sold.

    Responsible approaches (e.g., Origin Trials) now have a long track record of success. Those who argue that we must sort out differences in perspective exclusively by thinking hard about designs within the teleconferences of chartered Working Groups have not demonstrated that their approach is superior in any way that I can see. Simply trying ideas and gathering developer feedback has, despite tremendous resistance from Apple, generated a many important and high-quality features over the past decade using this model.

    Had we waited for Working Groups to recover from stupors induced by fifth columnist's disinterest, the quality of the resulting designs would not have improved substantially. Nor, in my estimation, would they be more likely to eventually achieve status as ratified and widely adopted Web Standards.

    There will always be a role for chartered Working Groups to improve on extant designs, but we have falsified the thesis that it is necessary for design to happen within them. The positive qualities of successful features are not down to venue, but the constituencies involved, and the evidence used to evolve them. That developer involvement and vendor flexibility towards feedback overlapped with Working Group-centered design efforts at certain moments in the past gives rise to much confusion. New entrants, thankfully, can skip old muddles and move straight to beginning their work in more entrepreneurial, high-iteration-rate venues built for incubation.

  10. Many big, old chartered Working Groups attract participants who are earnestly confused about the value of their own contributions or the correct scope of the group's activities. Luckily, it is not necessary to convince them that they should not have outsized power in these domains. Instead, we only need to route around by moving early design work to incubation venues that enable more productive and egalitarian practices.

    There is little that hard-done-by WG delegates can do when this happens, other than to muddy the waters about what the best order of operations is. They cannot prevent design work from decamping to more fertile ground because it is in full alignment with the roles assigned to Working Groups and their feeder venues by the formal processes that govern SDOs.

    Design work should never have been on their docket, and it is always illegitimate Working Groups to attempt to monopolise it. Their charters make this extremely clear.

  11. The WHATWG is unique among internet SDOs in disavowing any sort of incubation or early design process, which largely serves to make the W3C's various appendages (particularly the WICG) more useful and vibrant.

  12. Taking developers seriously and not being dismissive about the needs they express is the first step in this work. It is therefore extremely problematic when implementers gaslight developers or claim the needs they are taking time to express are somehow illegitimate.

    There are many cases where an SDO or Working Group can't, or shouldn't, work to solve certain problems. But we can engage those questions without being dismissive, and should show our work along the way. Vendors who make a habit of brushing off developers, therefore, bring disrepute to the whole goal of standards work.

]]>
Apple's Assault on Standards 2025-09-02T00:00:00Z https://infrequently.org/2025/09/apples-crimes-against-the-internet-community/

TL;DR: Market competition underlies the enterprise of standards. It creates the only functional test of designs and lets standards-based ecosystems route around single-vendor damage. Without competition, standards bodies have no purpose, and neither they, nor the ecosystems they support, can retain relevance. Apple has poisoned the well through a monopoly on influence, which it has parleyed into suppression of browser choice. This is an existential threat to the web, but also renders web and internet standards moot. Internet standards bodies should recognise the threat and respond.

Internet enthusiasts of the previous century sometimes expressed the power of code by declaring the sovereignty of cyberspace, or that "code is law."

As odd as these claims sound today, they hit a deep truth: end-users, and even governments, lack power to re-litigate choices embodied in software. Software vendors, therefore, have power. Backed by deeply embedded control chokepoints, and without a proportional response from other interests, this control is akin to state power.

Both fear and fervour about these properties developed against a backdrop of libertarian1 attitudes toward regulation and competition. Attenuating vendor power through interoperability was, among other values, a shared foundation of collaboration for internet pioneers.

The most fervent commitment of this strain was faith in markets to sort out information distribution problems through pricing signals,2 and that view became embedded deeply into the internet's governance mechanisms.3 If competition does not function, neither do standards.

The internet's most consequential designs took competitive markets as granted. Many participants believed hardware and software markets would (and should) continue to decouple; that it would be easier for end-users to bring their own software to devices they owned. It is odd from the perspective of 2025 to suggest that swapping browsers, e.g., should come at the cost of replacing hardware. This is why Apple have worked so hard to obscure that this is exactly the situation it has engineered, to the detriment of users, the web, and internet standards.

Internet standards bodies assumed the properties of open operating systems and low-cost software replacement to such an extent that their founding documents scarcely bother to mention them.4 Only later did statements of shared values see fit to make the subtext clear.

And it has worked. Internet standards have facilitated interoperability that has blunted lock-in, outsized pricing power, and other monopolistic abuses. This role is the entire point of standards at a societal level, and the primary reason that competition law carves out space for competitors to collaborate in developing them.5

But this is not purely an economic project. Standards attenuate the power of firms that might seek to arrogate code's privileges. Functional interoperability enables competition, reallocating power to users. Standardisation is therefore, at least partially a political project; one that aligns with the values of open societies:

We must ask whether ... we should not prepare for the worst leaders, and hope for the best. But this leads to a new approach to the problem of politics, for it forces us to replace the question: "Who should rule?" by the new question: "How can we so organize political institutions that bad or incompetent rulers can be prevented from doing too much damage?"

— Karl Popper,
"The Open Society and Its Enemies"

Without a counterweight, network effects allow successful tech firms to concentrate wealth and political influence. This power allows them to degrade potential competitive challenges, enabling rent extraction for services that would otherwise be commodities. This mechanism operates through (often legalised) corruption of judicial, regulatory, and electoral systems. When left to fester, it corrodes democracy itself.6

Apple has deftly used a false cloak of security and privacy to move the internet, and the web in particular, toward enclosure and irrelevance. Cupertino acts as a corrupted, and indeed incompetent, autocrat in our digital lives, even for folks who don't carry iPhones. It accomplishes this trick through abuse of a unique monopoly, allowing it to extract rents, including on the last remnants of open ecosystems it tolerates.

Worse, Apple's centralisation through the App Store entrenches the positions of peer big tech firms, harming the prospects of competitors in turn. Apple have been, over the course of many years, poisonous to internet standards and the moral commitments of that grand project.7

Despite near continuous horse-race coverage in the tech press, the consequences of this regression in civic/technical affairs is not well socialised.

Standardisation expands the reach of interoperable technology, pushing firms to innovate, rather than extracting rents on commodities.

Interoperability-in-being gives users choice, forcing competitors to differentiate on quality and not-yet-standard features. Standards expedite interoperability by lowering the costs for implementers and reducing tail risks, e.g. from patent trolls. Over time, a complete enough set of standards can attenuate the power of vendors to extract rents and prevent progress in important domains.

Interoperability is not the only mechanism that can reduce the power of dominant firms, but it is the most powerful. Free and Open Source software (FOSS) can provide a counterweight too, but OSS is not a full solution.8

Interoperability, and the economic surpluses that flow from it, are underpinned by voluntary adoption. This is enshrined in the "open stand" principles agreed to by no less than ISOC, IETF, IAB, IEEE, and the W3C.:

...

5. Voluntary Adoption

Standards are voluntarily adopted and success is determined by the market.

— The IAB, IEEE, IETF, ISOC, and W3C,
"The OpenStand Principles"

This final principle is the shortest, and many readers will understand it as a dodge; a way for Standards Development Organisations (SDOs) to avoid being seen to pick winners. But it is more than that.

Voluntary adoption is necessary for internet standards to function, and it creates a presumption of fair play.

Several implications bear mentioning.

First, the principle of voluntary adoption is necessary for effective standards development. Without a mechanism for determining which designs are better, we are unable to make consistent progress. That test never comes from within an SDO; it is always customer-defined. Writing a standard is not a test of quality, and without a functional market to test designs in, SDOs are irrelevant.

Second, this principle outlines a live-and-let-live doctrine, both within standards bodies and in the market. Participants may want their design to win, but are enjoined from procedural shenanigans to prevent competing designs from also being standardised.

Lastly, voluntary adoption marks customers (developers) and suppliers (browser vendors, etc.) as peers and creates norms of mutual respect within the walls of SDOs.

For all of these reasons, voluntary adoption must be defended, and actions taken to undermine it met with resistance and eventual sanction.

Regulators have had no difficulty in building market tests that demonstrate the power Apple holds in the lives of users.

Some of these tests have produced comical contortions as Apple has attempted to weasel out of its responsibilities. Consider the Trinitarian claim that Safari is simultaneously one product, and also three. Or that iPadOS, despite sharing nearly all code with iOS, marketed under that brand for years, supporting identical features, running the same third-party software, releasing on the same cadence, and running exclusively on the same hardware architecture is (somehow) an entirely different product.9

But for as clear and effective as regulator's tests have been in piercing these smoke screens, they do not capture the most important aspects of Apple's influence on the market for smartphone software: the monopoly on wealth.

Apple suppresses standards-based platforms and their disruptive potential through a mechanism every software developer understands: wealthy users carry iPhones, and they have all the influence.

Competition law does not explicitly recognise this distortion of the market, but the connection between the influence of the wealthy and the choices available to other end-users is indisputable.

Despite selling fewer than a quarter of smartphones globally for the past decade, Apple has built and anti-competitively maintained a supermonopoly among the wealthiest 10-20% of the world's population who, in turn, control effectively all wealth on the planet. This has handed it control over software distribution due to the social effects of wealth. Apple pairs legitimately superior product attributes (e.g., leading chip design) with anti-user and anticompetitive tactics — e.g., green vs. blue chat bubbles, suppression of browsers — to maintain this position.

Apple has put in place a set of interlocking restrictions to ensure the web cannot disrupt native apps within this user base. Those users, in turn, warp the behaviour of software developers thanks their spending power and positions within the software industry. Developers understand that if they cannot demo their wares to bosses and VCs (all of whom are wealthy) on their own devices, their software might as well not exist.

The long-stable propensity of users who make more than $100K USD/year to carry iPhones combines with Apple's suppression of browsers and PWA capabilities to ensure developers have no choice but to build native applications. These effects were visible in population-level statistics a decade ago and have been stable ever since:

Knowing whether someone owns an iPad in 2016 allows us to guess correctly whether the person is in the top or bottom income quartile 69 percent of the time. Across all years in our data, no individual brand is as predictive of being high-income as owning an Apple iPhone in 2016.

— Marianne Bertrand and Emir Kamenica,
"Coming Apart? Cultural Distances in the United States Over Time"

Developers are forced into the App Store by missing web capabilities, ensuring an advantage for Apple's proprietary ecosystem. This induces wealthy and influential users to default to the App Store for software, further damping the competitiveness of open platforms.

The monopoly on influence explains why Apple is wedded to legalistic, dissembling tactics in order to prevent the spread of web apps. Should the work of internet and web standards bodies ever become relevant, Cupertino understands the market for software will transform in ways it cannot control or tax.

Developers are forced into the App Store by a lack of functionality in browsers and web apps. By contrast, Apple has liberally meted out fingerprintable system capabilities to privacy-invading native apps. It has also allowed them to suppress pro-privacy interventions when it serves them.10

Apple professes that "privacy is a human right", but this as an attempt to turn the consequences of Apple's own largesse towards data abusers into a marketing asset. Wide-scale privacy erosion has depended, fundamentally, on Apple's own decisions, and it has not recanted.

It was Apple's choice to introduce less safe, less privacy-preserving native apps into iOS. It was also Apple's choice to deny competing browsers engines the freedom of voluntary feature adoption. This has ensured that important underlying capabilities could only ever be accessed through Apple's proprietary APIs, and only ever by those who are willing to agree to Cupertino's extractive terms.

The result has been API enclosure; appropriation of commodity capabilities that themselves are standards-based — e.g., rich graphics, USB, Bluetooth, NFC, file storage, etc. — by a proprietary ecosystem. Meanwhile, it has delayed and undermined the emergence of safe and privacy-preserving versions of those features on the web.

This, in turn, has created a winner-take-all dynamic inside the app store, harming privacy, security, and competition in the process.

This strategy relies on interlocking policies that harm competitors, and the sabotage of voluntary standards adoption lies at its heart. A Bill of Particulars for crimes against the internet community springs from a small set of undisputed facts.

Apple has:

  • Restricted competitors from shipping their own implementations of web and internet standards, depriving them access to influential users.

  • Forced all iOS browsers to use Apple's own defective, impoverished implementation.

  • Undermined user security and privacy through iOS browser engine monoculture.

  • Used contractual terms to dissuade competitors from supporting standards Apple disfavours.

  • Objected spuriously within standards bodies to prevent standardisation of features which Apple offered no counter-proposal.11

  • Engaged in marketing and product UI that discourages use of web-based alternatives.

  • Misled regulators and the public when presented with evidence of the harm from these actions.

Through these and other overt acts, Apple has worked to disempower users, depriving them of choice by preventing open platforms from challenging native apps.12

This isn't just a fierce market participant competing aggressively. Apple has done violence to the founding ethos of internet and web standards development. Instead of honourably withdrawing from those groups, Apple has maintained a charade of engagement, and gaslights other participants while actively sabotaging the principle of voluntary adoption that internet standards rely on.

Apple has never been forced to suppress its competitors, nor to create an anticompetitive landscape. Cupertino's senior management have intellectually consistent options that would allow it to pursue growth of the superior (we are told) native app ecosystem without threatening browser choice or the good functioning of internet standards.

Let's consider two: all safe browsers, and no browsers.

Apple could, of course, simply enable the same sort of level playing field for high-quality browsers that every competing general-purpose OS vendor has for the web's entire history. Apple itself facilitates this sort of ecosystem on macOS.13 Any plausible restrictions stemming from available system resources have long been overcome by progress in mobile hardware, particularly within Apple's ecosystem.14

The only reasonable restrictions on competing browser engines relate to security. As other vendors have generally had a better track records than Apple regarding sandboxing, incident response, patch gaps, and support for older OS versions, this should be no practical obstacle.

Lest Apple's defenders worry about the impact on Safari, recall that under true browser choice, Apple retains considerable market advantages, including (but not limited to) pre-installation, lower structural costs15, and continued differentiation through integration.16 Such bulwarks have allowed Safari to retain considerable share on macOS in spite of stiff competition and Safari's poor track record on security and standards conformance.

Alternatively, Apple could withdraw Safari while forbidding web content on iOS. This is a fully consistent position, and has been available to Apple from the moment of the App Store's release. The iPod did not include a browser, and many subsequent Apple OSes lack functional browsers. iOS and visionOS are uniquely deficient in this regard.17

Either way, the decision to undermine choice and standards rests entirely with Apple. It has always had intellectually honest solutions to the problems it has created. Apple cannot claim the situation is anyone else's fault, or that it has had no alternative.

Under voluntary implementation, every vendor is free to ship what they please, including Apple. It may be sad, or even damaging, when features go missing from important products, but that is not a calamity; just an input to be priced by the market — a reason for a specific browser to gain or lose share.

It's not enough to cite a lack of features, bungled implementations, peevish behaviour in working groups — or even rank dishonesty — as reason for censure. These are, to greater or lesser degrees, players playing the game. Certain tactics may be distasteful, but are squarely inside the "awful but lawful" zone. Standards venues should allow them, with sanctions for poor behaviour meted out in the social realm.

But by undercutting voluntary implementation, Apple has committed a more fundamental and more dangerous outrage.18

Some will see a parallel to the Paradox of Tolerance, and I do not believe this is mistaken. Standards bodies can, and should, admit many positions by their participants, but granting membership in good standing to those who are actively poison the basis for standards is madness. It ensures that standards, and the ecosystems that depend on them, wither and die.

By subverting the voluntary nature of open standards, Apple has defanged them as tools for users against the totalising power of technology firms in their digital lives. This high-modernist approach is antithetical to the foundational commitments of internet standards bodies and, over time, erode them.

Indeed, no other vendor has achieved what Apple has in suppressing the web through anticompetitive means. We must not imagine that Apple would stop at the Application layer given a chance. The same mechanism threatens voluntary feature adoption in networking and every other layer down the stack, too.

The web and internet communities should understand the threat and clock the cumulative harm done to internet standards and open technology ecosystems. It seems to me that this point has hardly been engaged, let alone won, within the walls of SDOs. But if it were, then what? What can be done?

The founding documents of internet SDOs do not include censure mechanisms for sabotage. The W3C's bylaws, for example, only relate membership in good standing with paying dues. Regardless, it is possible — and I believe urgent — to do more.

First, proposals can be raised to amend bylaws to include mechanisms for censure by votes of the membership for the kinds of outrages alleged here. These are likely to fail, and will surely be rejected at first, but the act of raising these questions has power. Bringing up these issues in plenary meetings can, at a minimum, elicit a response. That, on its own, is valuable to the community.

Next, leadership boards with moral authority can consider the question and issue guidance. The W3C's Advisory Board and Technical Architecture Group and the IETF's Internet Architecture Board have the ear of the membership, even on non-technical topics, should they choose to weigh in on the side of their own continued relevance.

Most importantly, individual delegates to Working Groups can recognise that Apple's forced monoculture is illegitimate and corrosive. They can resolve not to accept "Apple does not comment on future products" in response to questions about implementation timelines.

As long as Cupertino demands a monopoly, we must demand it take responsibility for the consequences.

Today, Apple alone chooses which features ship in WebKit, and which can be used by competing browsers. Even if it does not to enable them in Safari, it can provide more to others. It is simply illegitimate for Apple to claim that it cannot, or should not, allow other vendors to reach feature parity between their iOS browsers and other products that use their own engines.

WebKit purports to be Open Source, but in practice Apple has used it to undermine the "bring your own code" foundation of OSS. It is illogical for Apple to cite a disinterest in a feature in Safari as a reason for Apple not to be expected to implement those features in iOS's WebKit binary, making them available for other embedders to flag on.

The sham of WebKit as an Open Source project is incompatible with preventing other vendors from introducing features they would turn on for their iOS users if allowed. The destruction of voluntary adoption is not a shield against critique. Instead, it must heighten expectations. Apple agues it should be singularly entrusted with control over all web features on iOS, despite Safari and WebKit's trailing record on standards' conformance, security, and even privacy. This is nonsense, but so long as it is the law of the land in code, Apple should bear the costs.

Apple alone must be on the hook to implement any and every web platform feature shipped by any and every other engine. It does not need to enable them in Safari, but must make them available for use by others as they see fit.

So long as competing vendors are forced into the App Store and required to use Apple's engine, Cupertino owes much more when it comes to completeness and quality. So long as Cupertino compels use of WebKit, the demand should be echoed back: parity with browser features on other Operating Systems is the minimum bar.

Fundamentally, the web and internet community must stop accepting the premise that Apple should benefit from the protections and privileges of voluntary feature adoption while denying it to others.19

Lastly, and perhaps most controversially, delegates and organisations can use their positions to vote against Apple's personnel in elections to leadership positions within internet and web SDOs. It is inconsistent for Apple to hold positions of influence in organisations it works to sabotage, and fellow participants are under no obligation to pretend otherwise. Handing Apple formal or persuasive power within these groups is a mistake, and one that can be corrected without changes to bylaws.

In raising these questions, colleagues have invariably asked "why now? What changed?"

Beyond the threshold point that the damage is cumulative, and therefore it isn't necessary to identify specific instances to discuss the spreading rot, it's fair to ask why anyone should be agitated tomorrow when they might not have been yesterday.

Most of the factors involved have indeed changed very gradually, and humans are famously poor judges of slowly emergent threats. Apple's monopoly on influence, Cupertino's post-2009 WebKit priorities, the suppression of browser competitors, and the never-ending parade of showstopping bugs are all gradually emergent factors. Despite all of this long-running, unrefuted evidence, many continue to think of Apple an ally of the web for helpful acts now more than 15 years old.

But recent events must shake us awake. Apple's petulant attempt to duck regulations, destroy the web as a competitor for good, and frame regulators for the dirty deed was shocking. In recurring misrepresentations to regulators before and since, it has dissembled about its role in suppressing the web, and through its demand for secrecy in quasi-standards processes, has worked tirelessly to cover its tracks.

Taken individually, and in ignorance of iOS's coerced WebKit use by competing browsers, forced monoculture, habitual security failures, and strategic starvation of the Safari team, these shameful acts would simply indicate another monopolist behaving badly. It is only when considered alongside the wider set of facts that the anti-standards strategy and impact become clear.

Like most who have dedicated the greater proportion of their working lives to the cause of an open and interoperable web, the conclusions offered here shock me as well.

In the end, however, the question "do standards matter any more?" is intrusive, despite my aversion to reconsidering a question whose answer I thought obvious. But in light of the past decade's sidelining of the web, we must grapple with the consequences. To allow Apple to continue to abuse the foundation of standards without acknowledgement would be a failure of honesty towards my own intellectual commitments.

My personal affinity for the many talented and thoughtful people that Apple has sent to standards bodies over the years — including those I think of as friends — has, on reflection, been an emotional blind spot. But here we are. The realisation that they have been an unwitting fifth column against the web is nauseating for me, and I expect many will loudly reject the conclusion. I do not blame them.

For all the harm Apple has done to the web and to competition, I had hoped that it would relent before any of this became necessary. Like most web developers, I harboured hope that, true to Steve Job's promise in '07, Apple would let the web be a "sweet solution" for delivering safe, powerful applications. Piggybacked on that hope was a belief that a relevant mobile web would bolster the relevance of standards. But Cupertino has gone a different way, choosing profit over collaboration and the needs of users.

The web matters too much for the standards-based future it represents to fade without so much as a nod.

FOOTNOTES

  1. Or, if you like, "liberaltarian".

  2. A faith I do not share. Markets fail frequently, never mind that most goods are hardly substitutable, thinly traded, and lack reliable public prices. All of this means that the information capacity of markets a priori the rationalising effect of standards and market fairness regulation is heavily suspect.

    But even for those that take a market fundamentalist perspective, restrictions on trade such as Apple has imposed are offensive to the basic logic of the market's role in bettering society. Only those who would see markets subjugate all, forever, because of one-time power imbalances can be sanguine about what Apple has done.

    We do not have to grant pro-totalitarian arguments the assumption of good faith given what we know about where unchecked power leads.

  3. It is not original to note that there is an inherent contradiction in the idea of liber(al)tarian participants in standards bodies collaborating through non-market mechanisms. We do not have to ignore it, however, or even treat those holding both an affinity to open standards and libertarian ideals as hypocrites, in order to accept that the development of standards is often a creative act of critique for which there is no other functional venue (or, if you like, "market").

    Proposals for open internet standards often begin as personal drafts of individual authors, working in a community of like-minded developers experiencing similar challenges and working to design solutions. These proposals are situated in a context that is often opaque to those outside narrow circles, and the communities that form around them trade in reputation as much as any other currency.

    In addition to personal status, there is a distinguishable ideology at work:

    The open systems ideology that was developed in computing between the 1970s and the 1990s embodied several assumptions articulated in previous open system visions in diplomacy, economics, philosophy, and engineering. These assumptions included:

    • an economic commitment to global markets;
    • a moral support of international and multicultural ties;
    • a political opposition to centralized power — either in governments or in monopolies — that threatened individual autonomy;
    • a belief that technical professionals could achieve these economic, moral, and political aspirations through cooperation and standardization.

    — Andrew L. Russell,
    "Open Standards and the Digital Age"

    These principles stand as critique to older, less open ways of working. Technology that is more interoperable makes a larger space for critique through code and the market, and that is essential to the openness of any society that depends on software as much as modern, western nations do today.

  4. The history of Bell Labs, the antitrust battles of IBM, the birth of Unix, and the expectations of "common carrier" treatment for data transmission help to situate the thinking of participants in setting up the foundational standards bodies that we now take for granted. Major battles were fought to keep computing out of the hands of a few corporations with outsized power, and the consequences continue to reverberate.

    It is my personal view that it has been the good work of antitrust and anti-monopoly reformers — both within and outside government — that made it possible for other parties to believe in the abstraction of functional markets. Functioning markets are not, in fact, self-organising, and are enabled explicitly by law that helps to minimise noise in the channel (e.g., from fraud).

    That self-identified libertarians (along with the rest of society) have benefited dramatically from the gains enabled by these anti-monopoly regimes is not in doubt; the only mystery is how fervently some cling to the belief that regulation must be problematic as a way to skip past its content and ignore engaging on substance.

  5. Without an "all clear" signal for explicit standardisation, antitrust law in most advanced nations would explicitly forbid the sorts of coordination among industry peers that standards bodies facilitate. Authorities permit, and even encourage, standardisation in contrast to other forms of collaboration because the positive externality benefits of reduced friction to trade from interoperability is so compelling. For more on the history and structure of this now-international regime, see Russell (2014)

  6. One needs to look no further than tech CEOs lining up to hail the rise of explicitly fascist leaders and ply them with gifts, to understand the ways in which this corruption corrodes our hopes for open, tolerant societies.

  7. This is not an accusation to be levelled lightly, and not without overwhelming evidence. But that evidence has accrued, and so I feel compelled to speak.

    Apple are known both to be capricious and retributive, and its antipathy towards Khronos provides an excellent example of the decade-length grudge-holding for which Cupertino is infamous. I therefore write this post advisedly and with trepidation.

  8. More than twenty years into the experience of Open Source as a force in software, we can clearly discern that the licence of source code is not, in-and-of itself, a solution to the imbalances inherent in software's relationship to users, or even to other parties with the capacity to develop software.

    I believe that history demonstrates clearly that (F)OSS licensing and effective governance are only related by the intellectual and commercial commitments of those building software, and are therefore easily co-opted by wealthy and powerful firms.

    We have witnessed the limits of forking as a bulwark against bad behaviour, through both the persistent upward reach of firmware into "open" systems, and through capture in higher layers due to the carrying costs of complex systems. Licensing has also been little help in courts against large and determined enemies of OSS and the freedoms it attempts preserve. The limits of licensing, and the lack of IP defence pooling inherent in common licences, create risks that successful firms and projects hit regularly.

    At the limit, (F)OSS licensing is not a significantly disruptive force against wealthy and powerful technology firms, but rather a tool that is useful to peer-level adversaries. For OSS licences and defences to have purchase in practice, a significantly endowed group of technologist's interests must be at stake. The interests of informed and financially capable technologists and the firms that use their software bear only passing resemblance to the needs and interests of society more broadly.

    From this perspective, we can understand (F)OSS as complementary to standards development, but unable to fully replace standards-based interoperability as an attenuating force on the power of software in the lives of users. Standards retain a unique ability to build space for challengers within and between proprietary products, which (F)OSS do not. The power of SDOs to pool the patent interests of proprietary players has parallels in (F)OSS, but with more clarity and lower risk, further accelerating practical interoperability.

    Therefore, we must understand that it is a rhetorical and intellectual trap to consider (F)OSS a replacement for standards. Each hasten interoperability and attenuate power in different and complementary — but not substitutable — ways, and both mechanisms are healthier when the other succeeds.

  9. The EC, having borne witness to Apple's appalling behaviour within its borders, and more generally to boot, was having none of it.

    The full finding (PDF) is a masterclass in even-handed consideration, and as such it is not surprising that Apple's legal and marketing fictions failed.

  10. Helpfully for Apple, the conversation around privacy and technology has barely progressed past Apple's anti-Google and anti-Facebook kayfabes.

    Privacy advocates are regularly taken in by Apple's marketing, and the tech press remains in a largely stenographic mode. None of this is to say that Google or Facebook are good actors (they are not), only that Apple is also not on the level.

    We can begin an analysis by noting that Apple does not encourage users to move their computing to the web where browsers can attenuate the worst invasions of privacy. Browsers do not provide nearly as much information to trackers as even the most locked-down native app in Apple's ecosystem, and Apple know this.

    Next, Apple has not funded lobbying and regulatory outreach efforts to establish privacy laws worth a damn. Instead, it channels huge amounts into defeating right-to-repair legislation and defeating browser engine choice around the world, including going as far as funding astroturf groups to provide Cupertino's views in stereo.

    Moreover, Apple takes no responsibility for its historical role in the growth of tracking via the native API surfaces Apple created relative to the web. Nor does it demand audits of data use by App Store participants, or to even set policy about acceptable uses of data collected via App Store-vended applications.

    Apple does not even forbid pervasive "ad blocker blocking" by the worst actors in its ecosystem, even when interventions are trivial from a policy and technical perspective.

    That all of this aligns with Apple's preference for control over, and taxation of, developers cannot escape comment. It further forces us to entertain the idea that Apple's position as a defender of privacy is a cynical show, or that it is incompetent in assessing privacy risks, or that it is out of touch with the behaviour of snooping developers. Any of these would be enough to judge it an incompetent regulator, asleep at the switch.

    Depressingly, these are not even exclusive choices.

  11. It is prima facie evidence of bad faith for a vendor that skims as much cash from the open web as Apple does, and which insists it must maintain a monopoly on browser engines, to object to providing implementations of proposals that other vendors have shipped safely for years, without offering their own designs to address similar needs,

    This case is bolstered by Apple late-implementing designs which it spuriously objected to as regulatory pressure has grown.

    By failing to engage in solving problems, then citing its own objections as reason not to implement the resulting demonstrably safe features, Apple have cut the legs out from underneath any protection from critique.

  12. Platforms are in competition.

    It cannot escape notice that Apple's undermining of voluntary adoption most damages interoperable platforms that might challenge the App Store-based native app ecosystem which supports Apple's vertical integration agenda.

  13. Here we must mention other instances of operating systems imperilling browser choice. Microsoft, for instance, was credibly accused of making text less readable on Netscape Navigator.

    Google, for its part, introduced ChromeOS without any provision for competing browsers. Subsequently, it allowed Android versions of competing browser products using their own engines to register as the default system browser. This is an unsatisfying solution, as Play-based apps are a poor experience on low-end devices and competing browsers are prevented from integrating as deeply as Chrome does. This may not rise to the level of Apple or Microsoft's acts as ChromeOS remains a niche product, ranks lowest in influence, and features a capable browser — regardless, the precedent, combined with prior episodes of Microsoft and Apple turning away from their successful and capable browsers, remains troubling.

  14. No company in the world is more aware of the awesome power and capability of Apple's A-series chips than Apple itself.

    Apple executives know, for instance, that even the least expensive iPhone produced in the past five years is hundreds of times faster than the first iPhones which gave rise to resource-based restrictions on engine choice. They are either aware, or can trivially deduce, that there is no legitimate system health or resource-related reason to disallow competing browser engines.

  15. As I have discussed before, Apple economises on the development of WebKit and Safari not only by failing to fund development of web features in a timely way, drafting on the path-breaking work of others, but also through an (over)reliance on OS components in lieu of more easily defended abstractions. This goes much deeper than only needing to support a single family of operating systems; unlike competitors, Apple directly leans on OS systems where competitors rebuild large swaths of the runtime to enable the web where underlying OS APIs are more limited.

    The result, for Apple, is reduced headcount and a lower bill-of-materials when producing devices (read: higher profits), owing to higher code page sharing across applications.

    The consequence for users is an implementation monoculture and slower patch delivery, harming security. Apple's users remain vulnerable to attacks for longer and without recourse to competing alternatives with stronger track records. The downside of all high modernism is brittleness, borne of uniformity, and iOS is no exception. The lack of the ecological diversity that Apple demands undermines security and resilience. This is not a price society should pay for the convenience of a blue chat bubble.

  16. In testimony before various competition authorities around the world, and in peevish screeds in response to those arguments failing, Apple habitually attempts a rhetorical redirection that unmasks its anti-Popperian thirst for control.

    It appears to believe, on the strength of brand, that it can win on the ground of "who should rule?," rather than "how can institutions keep bad rulers from doing too much damage?" In so doing, Apple offers an authoritarian model of technology. Cupertino proposes that all sources of control attenuation are themselves totalising, and any compromise forced on the ruler therefore equivalent to a coup.

    This is nonsense, both practically and historically, and technologists that support democratic norms should reject this framing.

    History shows clearly that open systems in open societies enable civil society to take many effective positions that attenuate untrammelled power, both of the state and of other actors. We can further see that neither open nor closed systems are full bulwarks against government coercion. The prospects for improved societies must not be invested in autocratic technology firms for the simple reason that they are powerless to deliver it.

    Apple itself has been forced to compromise in areas such as right-to-repair without any of the apocalyptic prophecies of Cupertino's lobbyists coming to pass.

    Here Apple, and its extremely vocal band of apologists, will bring up overreaches by others, including disastrous and ill-advised anti-encryption pushes by national governments. The goal of this argument is to sully the idea of democratic control of technology. "How", it asks leadingly, "can anyone trust these people to do what's right for users?"

    How indeed! For the central point is that we do not have to. Recourse is available (in functioning democracies) through elected representatives and the responsiveness to citizen's concerns. We can even accept that democracies will get many of these issues wrong for a time without seceding any ground to Apple's authoritarian offer. It is intellectually consistent to reject both noxious positions taken by elected officials and offers of high modernist control by firms opposed to the current positions of a government.

    Apple's arguments are microns deep on the merits. They attempt to erase both the successes of civil society (rather than Apple) in curbing abuses along with Apple's own shameful track record of capitulation to overtly authoritarian regimes.20

    Apple's framing asserts that total control by a famously unaccountable firm is a Panglossian utopia. That any decision Cupertino makes now, has ever made, or has ever reversed, have unquestionably been the best of all possible options. That by paying a thousand dollars for a phone, we are but lucky to bask in the glow of such resplendent wisdom.

    It's all too much to take, at least for anyone with a working memory. As I hope I have demonstrated here, this posture fails on ethical as well as practical grounds.

    Apple, with all of its power and money, could be an ally to civil society in curbing abuses without claiming total and unaccountable control for itself in the process. It is incumbent on any thinking technologist to reject such land grabs, even when they come wrapped in causes we otherwise support. To do otherwise is to co-sign the doctrine of "who should rule?," a framing that is caustic to both our short-term technical and long-term societal interests.

  17. Paradoxically, it may be Apple's own malign behaviour regarding browser choice that might prevent a "no browsers" policy from being possible today.

    Even if we were to ignore Cupertino's incentive to maintain browsers on iOS owing to the shockingly large flows of money from Google for search placement (which Apple records as nearly pure profit), regulators may require it to pursue an "all browsers" alternative to fairly redress the harms from the previous 15 years of abuses. Should this come to pass, Cupertino would (again) have no party to blame but itself.

  18. The damage Apple has done to the cause of internet and web standardisation is analogous to the (US) concept of impeachable "high crimes and misdemeanours", rather than better enumerated, more pedestrian infractions. This category covers acts that are destructive to the foundational principles of the enterprise, regardless of narrow legality.

    Internet SDOs are not set up well to police or react to this class of offence, which may help explain why Apple's violence against the community has gone unremarked for so many years.

  19. Apple engineers should be questioned (politely, but insistently) about timelines for implementation of any feature that any other engine chooses to ship, and failure to provide that timeline should be viewed as ongoing evidence of malfeasance against the internet and web community for as long as Apple withholds the ability for others to bring their own engines and whatever features they choose.

    If this causes Apple to retreat from sponsoring work in these venues, that is regrettable, but must also be understood as a choice that is entirely within Apple's power to reverse.

  20. The fuller airdrop story creates a cross-pressured narrative in which it is possible to claim that Apple was acting to protect, rather than suppress, A4 protesters. But such a pose must also contend with Apple's own history of failure, first to build a truly private version of a system marketed in those terms, thereby encouraging users in sensitive situations to expose themselves to great risk. Second, Apple's unwillingness to patch post-disclosure.

    Owing to AirDrop's use of closed protocols that Apple does not make available to developers of competing applications, it also falls exclusively on Apple's shoulders that better solutions were not available to replace Apple's own botched implementation.

    The story is more complex than "Apple sold protesters up the river," but no publicly available version of events is a defence the argument that Apple is a bellicose marketing outfit with a situational and opportunistic relationship to user privacy and security.

    When the effect is to expose or debilitate users that rely on the very properties Apple so truculently asserts its competitors fail to deliver, it becomes impossible to defend Cupertino's insistence that it alone should be trusted.

]]>
Apple vs. Facebook is Kayfabe 2025-08-25T00:00:00Z https://infrequently.org/2025/08/apple-vs-fb-kayfabe/

Photo by Claudia Raya
Photo by Claudia Raya

Apple vs. Facebook is, and always was, kayfabe. In reality, Apple is Facebook's chauffeur; holding Zuck's coat while Facebook1 wantonly surveils iPhones owners.2

Facebook's gross profit over time.
Facebook's gross profit over time.

Facebook and Apple mugged convincingly for the cameras as "App Tracking Transparency" rolled out, talking up the impact to Facebook's business. But San Mateo's profits tell a very different story. Net income dipped between late 2021 and early 2023 thanks to accelerating capital expenditures, not reductions in revenue. Despite strenuous efforts to sell the move, it's hard to discern any impact from ATT whatsoever.

How can we be sure Apple's wise? Because Cupertino continues to allow Facebook's wide-scale abuse of In-App Browsers:

Open Web Advocacy - In-App Browser Primer

Apple has long facilitated and enriched mass surveillance through native apps, both directly from in-app activity, but much more insidiously through the In-App Browsers that lurk behind every link in Facebook, Instagram, Messenger, Threads, TikTok, Pinterest, etc.

I have written about them before, and they still stink to high heavens. Facebook couldn't ask for a better or more willing accomplice than Apple as it glides into the second decade of its browserjacking spree.

In-App Browsers are, for Facebook's purposes, ad-blocker blockers. Cheat codes for the enterprising panopticon proprietor. Much wringing of hands transpires every time one of these knockoff browsers is suspected of injecting script into web pages. The fear is that this will enable a level of tracking by apps that is not otherwise possible.

As scary (and real) as the threat is, it is also a misdirection.

To effect total surveillance, Facebook et al. don't need to inject scripts into the runtime, they only need your browser not to block their "ad tags" that are already embedded in every high-traffic page on the internet. Combined with the ability to watch every URL you navigate to in a WebView, this is more than enough to correlate in-app activity with web browsing without leaving overt fingerprints.

As long as users remain in the web purgatory of native apps, data collected from tracking endpoints remains immeasurably richer. It is, in effect, a loophole that only requires users be denied access to their browser of choice whenever it is convenient for native apps.

Real browsers matter because they are user agents; they represent the interests of users, rather than ad networks (Facebook, Google, ByteDance) and the OS vendors that are desperate to keep apps from decamping to portable, interoperable alternatives like PWAs. Users configure their browsers' privacy and security settings, knowing they will synchronize between devices. They can expand those protections with extensions that further ward against unwanted snooping.

By contrast, IABs from Facebook and ByteDance (etc.) do not feature many privacy-preserving settings or extensions, are fiendishly hard to disable, and do not sync between devices in the same app. They don't even synchronise preferences between multiple apps from the same company on the same device.3

Promising-looking options for users trying to disable Facebook's IAB. Now to find the right one...maybe we can just search? Promising-looking options for users trying to disable Facebook's IAB. Now to find the right one...maybe we can just search? Promising-looking options for users trying to disable Facebook's IAB. Now to find the right one...maybe we can just search?
Promising-looking options for users trying to disable Facebook's IAB. Now to find the right one...maybe we can just search?

Ok, let's search for 'web'...ah! 'Apps and websites', looks promising...well...no...maybe if we scroll. Hmmm. Nope. Ok, let's search for 'web'...ah! 'Apps and websites', looks promising...well...no...maybe if we scroll. Hmmm. Nope. Ok, let's search for 'web'...ah! 'Apps and websites', looks promising...well...no...maybe if we scroll. Hmmm. Nope.
Ok, let's search for 'web'...ah! 'Apps and websites', looks promising...well...no...maybe if we scroll. Hmmm. Nope.

Well, what if we search for 'browser' instead? Tapping on 'Browser' takes us to a tab that...doesn't let us turn it off? Hm. Let's go back to the start...might have seen something about 'Content Preferences'...hrm, nope, not that. Well, what if we search for 'browser' instead? Tapping on 'Browser' takes us to a tab that...doesn't let us turn it off? Hm. Let's go back to the start...might have seen something about 'Content Preferences'...hrm, nope, not that. Well, what if we search for 'browser' instead? Tapping on 'Browser' takes us to a tab that...doesn't let us turn it off? Hm. Let's go back to the start...might have seen something about 'Content Preferences'...hrm, nope, not that.
Well, what if we search for 'browser' instead? Tapping on 'Browser' takes us to a tab that...doesn't let us turn it off? Hm. Let's go back to the start...might have seen something about 'Content Preferences'...hrm, nope, not that.

Ok, it has to be in 'Settings and Privacy'...but where? We've tried 'Browser', and that's not it. It's not 'Content Preferences'...maybe 'Media'? Worth a shot... Ok, it has to be in 'Settings and Privacy'...but where? We've tried 'Browser', and that's not it. It's not 'Content Preferences'...maybe 'Media'? Worth a shot... Ok, it has to be in 'Settings and Privacy'...but where? We've tried 'Browser', and that's not it. It's not 'Content Preferences'...maybe 'Media'? Worth a shot...
Ok, it has to be in 'Settings and Privacy'...but where? We've tried 'Browser', and that's not it. It's not 'Content Preferences'...maybe 'Media'? Worth a shot...

I'm sorry...what? Are you actually kidding? The option is hidden two levels deep, not in the 'Browser' section, but in 'Media', and you have to scroll to get to it? I'm sorry...what? Are you actually kidding? The option is hidden two levels deep, not in the 'Browser' section, but in 'Media', and you have to scroll to get to it? I'm sorry...what? Are you actually kidding? The option is hidden two levels deep, not in the 'Browser' section, but in 'Media', and you have to scroll to get to it?
I'm sorry...what? Are you actually kidding? The option is hidden two levels deep, not in the 'Browser' section, but in 'Media', and you have to scroll to get to it?

Bewildering placement
Be honest: would you think to look in:
"Menu" > "Settings & privacy" >
"Settings" > [ scroll to "Preferences" ] >
"Media" > [ scroll to bottom ] >
"Open links in external browser"?

This goat rodeo is wilfully obtuse in a way that only an organisation dedicated through-and-through to A/B testing can accomplish. You might very well think that Facebook is working hard to trick users into a knockoff browser in the hopes they don't notice.

Even without assuming intentional obfuscation, it is shocking how laughably incomplete the privacy and security settings of Facebook's IAB remain, more than a dozen years after it was introduced:

Facebook's IAB fails to provide standard security, privacy, and accessibility settings. This single page is the entire set of available configuration options, and they are not sync'd.
Facebook's IAB fails to provide standard security, privacy, and accessibility settings. This single page is the entire set of available configuration options, and they are not sync'd.

An identical thicket of pain awaits anyone trying to disable the Facebook IAB on iOS. Facebook's UI is screen-for-screen the same across mobile OSes, and iOS users enjoy no advantage in finding the settings to disable FB's IAB.

Chrome and Edge's sync'd settings are vast by comparison, as are the offerings from every other responsible browser vendor:

Firefox for Android has such detailed settings that users have to scroll twice to see just the top-level entry points. The 'Privacy and Security' section alone offers as many options as all of FB's IAB. Firefox for Android has such detailed settings that users have to scroll twice to see just the top-level entry points. The 'Privacy and Security' section alone offers as many options as all of FB's IAB. Firefox for Android has such detailed settings that users have to scroll twice to see just the top-level entry points. The 'Privacy and Security' section alone offers as many options as all of FB's IAB.
Firefox for Android has such detailed settings that users have to scroll twice to see just the top-level entry points. The 'Privacy and Security' section alone offers as many options as all of FB's IAB.

Opera is similarly flexible and settings follow logged-in users through sync. Opera is similarly flexible and settings follow logged-in users through sync.
Opera is similarly flexible and settings follow logged-in users through sync.

The difficulty in disabling Facebook's IAB, failure to synchronize opt-out choices, and a crippled privacy features are calculated to enable maximum tracking when tapping links. Even when Apple's vaunted "App Tracking Transparency" is enabled:

App Tracking Transparency was enabled, yet Facebook's IAB continues to be invoked when tapping on links. The IAB was also disabled on the same account via another device within the hour, showing that Facebook disregards privacy settings when it suits them.

Here are the results of the EFF's "Cover Your Tracks" testing tool on (left-to-right) the iOS Facebook IAB, Firefox Focus, and the DuckDuckGo browser, all with default settings under iPadOS 17.7.10:

FB's IAB scores worst of all FB's IAB scores worst of all FB's IAB scores worst of all
Facebook's IAB reports 'some protection' with 'some gaps', leaking a unique fingerprint, while both Firefox Focus and DuckDuckGo do better, featuring 'strong protection', despite using exactly the same browser engine under the hood, as Apple (unfairly) requires.

Users that install browsers, configure them to preserve their privacy, then use Facebook app's after selecting "Ask App Not to Track" are reliably sold up the river by Apple, who seem content to facilitate this end-run around the rules.

Privacy settings? Gone. Extensions? Poof! And because Facebook has code on every top site, the tracking suddenly roars back to life, out of band of the channels that Apple wants you to believe are important (or at least that it has taken heat for in the past).

FB's tracking is so pervasive in modern web pages that it doesn't need to exfiltrate data from the IAB to track you. It just needs to keep you away from your real browser, where it might not be able to join up clicks and taps.

It isn't exactly clear if the IAB is the basis for recent reports of secretive "deterministic matching" efforts, but it's safe to assume that Facebook's bullheaded determination to steal clicks and deny users their choice in browsers isn't simply an oversight.

None of this is news to Apple. They have read the posts and have made timid interventions to avoid being blamed for the most obviously nasty versions of IAB tracking. But acting against the deep rot? No joy.

Not only has Apple not responded to advocacy on behalf of users from groups like OWA, it has failed to impose either common-sense, pro-privacy restrictions on IABs, or to support action by regulators. As we've seen above, it doesn't even require apps to avoid privacy-degrading IABs when ATT is enabled or provide a global opt-out.

And Apple is absolutely aware of these concerns because they have been raised publicly and in regulatory circles for years. Yet it does not act.

Why?

Facebook's dark patterns are directly facilitated by Apple and Google. It is their SDKs and policies that make this not only possible, but pervasive. So why do they deliver users unto perdition?

Denying users true browser choice helps keep the big app vendors in the store. Those whales understand that native APIs offer increased data collection, which they monetize.

Keeping big fish in the store, in turn, helps Apple and Google corral others into their API enclosure ghettos. If users know that to get their "main" apps, they must go to the store, then the store becomes the place to look for all software. Moreover, for competitors to have any hope of equivalent profits, they must enter the same Store-enabled race to the privacy bottom.

And that race to the bottom helps the duopolists, no matter how much they want you to believe otherwise.

Apple and Google are trying to maintain a distribution model chokehold over mobile software. Their duopoly allows them to tax developers outrageously for access to commodity APIs. Dependence on proprietary versions of bog-stock APIs, in turn, makes it hard for software vendors to consider building for other platforms with their limited engineering budgets.

This, not coincidentally, reduces the size of the (potentially) portable software catalogue, harming the prospects of new entrants that might challenge the duopoly. For users, a lack of apps in open ecosystems make it hard to escape for less predatory alternatives.

The entire point of the multi-layered exercise, in the end, is to subvert interoperability. And to do that, it's necessary to keep the anchor apps happy. Which is why Apple and Google let Facebook spy on you via through the web.

Apple isn't defending your privacy, it's retreating just far enough into the hedge that you won't notice the App Store dangling Facebook, Messenger, Instagram, and TikTok to take the blame for the APIs that Apple itself has recklessly provided. Compared to the web, iOS native apps have facilitated a universe of privacy invasion that was previously unthinkable. Apple did that, and continues to do that, and now it wants credit for forcing developers to "comply" (wink wink) with anti-tracking rules it can't be bothered to enforce.

The answer is blindingly obvious: forbid IABs, particularly under ATT, and/or force native apps to use the system-provided browser-overlay systems — SFSafariViewController and Android Custom Tabs — where user's choice of browser and customisations will be respected.

This isn't hard. In fact, it's one of the simplest interventions possible. And yet neither Apple nor Google are willing to pick a real fight with Facebook and do right by users.

And until they do, you can be certain the privacy preening is all for show.4

FOOTNOTES

  1. I will continue to refer to Facebook as "Facebook" and Twitter as "Twitter" despite their sweaty, grasping rebrands.

    It is an act of complicity to assist folks this guilty in turning the page on their transgressions. Using their new names helps oligarchs who retain standing through collective amnesia by disconnecting what is from what was, and they know it. Which is why they tried to rebrand in the first place.

    The US legal fiction of "corporate personhood" does not entitle that corporation to respect. As a corporation's feelings cannot be hurt, either because it has none, or because it is a sociopath (as the market demands). If a billionaire is so embarrassed of their monster that they long for its rebirth it, I say fine; go ahead. I'll happily call it something else, just so long as it dies first. Liquidate the assets, pay your bloody taxes, fire everyone, put the winnings into a holding LLC, then start afresh. Then I'll call the new thing whatever they please.

    Never deadname a trans person, but the pet amphisbaenas of billionares? Always and forever.

  2. At least until Apple can wrestle away Facebook's ad business for itself.

  3. A reliable measure of a tech firm's disrespect is making users frequently re-select privacy-enhancing choices. A good proxy for this is which settings are sync'd. Chrome's disregard for choices which Google dislikes is evident when logging into each new device:

    Every time a user visits settings on a new device, they're obliquely informed that the settings they laboriously configured on previous devices have been disregarded.
    Every time a user visits settings on a new device, they're obliquely informed that the settings they laboriously configured on previous devices have been disregarded.

    Chrome settings after verifying that sync has completed for the profile.
    Chrome settings after verifying that sync has completed for the profile.

    All of this has been carefully considered, and it is exceedingly likely that Figma mocks exist for versions that respect user choices. Those designs were not put into production. It's impossible to know from the outside exctly who made the call, but it's a reliable guess that the ads team won a Product Manager cage match.

    Facebook, for its part, is even worse, failing to join up browser choice settings on any surface or across apps. Hardly a surprise, but in a world where almost every other setting is synchronised, the difference is confirmation of anti-user intent.

  4. It should go without saying that the only truly effective solutions in this space will be legislative and regulatory. That neither Apple nor Google have put their aggressive lobbying teams to work to get effective privacy laws passed should also be a warning flag.

    Until and unless they're willing to put the same sorts of money behind drafting model legislation and donating to the coffers of electeds via proxies that they do to degrade right-to-repair and browser engine choice, it's all kayfabe.

]]>
Put Names and Dates On Documents 2025-08-13T00:00:00Z https://infrequently.org/2025/08/names-and-dates-on-docs-every-time/

Anyone who has worked closely with me, or followed on social media [, ], will have seen a post or comment to the effect of:

Names and dates on docs. Every time. Don't forget.

This is most often tacked onto design documents lacking inline attribution, and is phrased provocatively to make it sticky.

Why do I care enough about this to be prescriptive — bordering on pushy — when colleagues accuse me of being Socratic to a fault about everything else? Because not only is unattributed writing a reliable time-waster, the careers of authors hang in the balance.

Having to work to find the best person to discuss a topic with is annoying, but in large organisations, the probability of authors having work appropriated without credit goes up when they fail to claim ownership of their writing. It should go without saying that this is toxic, and that functional engineering cultures look harshly on it. But to ward off bad behaviour, it helps to model what's healthy.

The best reminder to cite work is for authors to name themselves. Doing this increases their stature, unsubtly encourages others to link and cite, and leaves a trail of evidence for authors to reference when building a case for promotion.

The importance of evidence to support claims of design work in technical fields cannot be overstated. Having served on hiring and promotion committees for many years, I can unequivocally say that this collateral is often pivotal. The difference between "[x] promote" and "[x] do not promote" often comes down to the names on documents. Reviewers will pay attention to both the authors list and the propensity of design doc authors to cite others.1

In response to unsubtle nudges, several recurring arguments are offered against explicit authorship notices.

This is fair, but does not outweigh the needs of future readers who may be working to trace a chain of events or ideas. Nor does it outweigh the needs of authors for credit regarding their work at a future date.

This crutch fails in any number of ways:

  • PDFs and printed copies do not include authorship data that is not in body text.
  • Some systems (e.g. Google Docs) do not make the history available to non-editors.
  • Documents may be copied or re-published in ways that disconnect the content from the original revision tracking system; e.g., in a systems transition as part of an acquisition.

Moreover, design is a complex and collaborative process. Ideas and concepts captured in documents are not always the contribution of the person writing down the conclusions of a whiteboarding session. A clear, consistent way to give credit helps everyone feel included and encourages future collaboration.

This is usually offered as a claim of superfluousness. If everyone knows everything and everyone working in a system, why does attribution matter?

Perhaps a team is small now, but will it always be? I am not in a position to tell, and my interlocutors also lack crystal balls. Given the downside risks, attribution is a pittance of an insurance premium.

This is easily countered with invitations in comments and drafts for contributors to add themselves to the authorship section. Generous and collaborative folks — the sorts of people we want to promote — reliably add their collaborators to documents proactively and set the expectation that others will do the same. Over time, practice becomes habit, which crystallises into culture.

A final concern I hear is that these blocks require a great deal of upkeep. Long-form revision logs might be onerous, but the minimum viable attribution style only needs three elements: names, emails, and dates. These should be provided on the very first page, ideally just below the title.

The primary date always refers first drafting, even if that is before publication. If deemed necessary, authors can optionally add a "Last Updated" field below the primary date, but this is optional. Documents authored in a single sitting by a lone author should avoid extra visual noise.

Revision logs are generally unnecessary, and my personal view is that they distract from content; if they're a requirement (e.g., in a heavily regulated industry), record them in an Appendix, but do not remove minimum viable attributions.

Here's a screenshot of an explainer I helped draft many years ago showing the basic form2:

Minimum Viable Attribution in a Markdown document.
Minimum Viable Attribution in a Markdown document.

If a document is still an early draft, it can be helpful to put some indication in the title — I use a prefix like "[ Draft ] ..." — and invite collaborators to add themselves to the authors list by including an entry there of the form "Your Name <[email protected]>". Once a document is circulated widely, remove these inline markers.

Thanks to Andy Luhrs for his feedback on drafts of this post.

FOOTNOTES

  1. If you write technical design docs, it is always a good sign if you cite prior work and parallel efforts, including competing designs. Omitting those references is something that both technical and promotion reviewers are alert to and are primed to think poorly of. No design is entirely new, and it is a sign of maturity to give others their due.

  2. Don't worry, all of these email addresses are now inactive.

    On the question of emails and spam in authoring public documents, views are split. I favour always using them, but understand if authors prefer other sorts of contact information; e.g., their personal website. Best not to be too fussy about this sort of thing, except to ensure that internal documents always contain email addresses.

]]>
How Do Committees Fail To Invent? 2025-08-11T00:00:00Z https://infrequently.org/2025/08/how-do-committees-fail-to-invent/

Mel Conway's seminal paper "How Do Committees Invent?" (PDF) is commonly paraphrased as Conway's Law:

Organizations which design systems are (broadly) constrained to produce designs which are copies of the communication structures of these organizations.

This is deep organisational insight that engineering leaders ignore at their peril, and everyone who delivers code for a living benefits from a (re)read of "The Mythical Man-Month", available at fine retailers everywhere.

Conway's Law is generally invoked to describe organisations working to solve well-defined problems, in which everyone is working towards a solution. But what if there are defectors? And what if they can prevent forward progress without paying any price? This problem is rarely analysed for the simple reason that such an organisation would be deranged.

But what if that happened regularly?

I was reminded of the possibility while chatting with a colleague joining a new (to them) Working Group at the W3C. The most cursed expressions of Conway's Law regularly occur in Standards Development Organisations (SDOs); specifically, when delegates refuse to communicate their true intentions, either through spurious objection or tactical silence.

This special case is the fifth column problem.

Expressed in Conwayist terms, the fifth column problem describes the way organisations mirror the miscommunication patterns of their participants when they fail to deliver designs of any sort. This pathology presents when the goal of the majority is antithetical to a small minority with a veto.

Reticence of certain SDO participants to consider important problems is endemic, in part, due to open membership. Unlike corporate environments where alignment is at least partially top-down, the ability of any firm to join an SDO practically invites subterfuge. This plays out calamitously when representatives of pivotal firms obfuscate their willingness to implement, or endlessly delay consideration of important designs.

Open membership alone would not lead to crisis, except that certain working groups adopt rules or norms that create an explosion of veto players.

These veto-happy environments combine with bad information to transmute opaque communication into arterial blockage. SDO torpidity, in turn, colours the perception of web developers against standards. And who's to say they're wrong? Bemoaning phlegmatic working groups is only so cathartic.

Eventually, code must ship, and if important features are missing from standards-based platforms, it becomes inevitable that developers will decamp to proprietary solutions.

Instances of the fifth column problem are frequently raised to me by engineers working in these groups. "How," they ask, "can large gatherings of engineers meet regularly, accomplish next-to-nothing, yet continue pat themselves on the back?"

The sad answer is "quite easily."

This dynamic has recurred with surprising regularity over the web's history,1 and preventing it from clogging the works is critical to the good order of the entire system.

The primary mechanism that produces consequential fifth column episodes is internal competition within tech giants.

Because the web is a platform, and because platforms are competitions, it's natural that companies that make both browsers and proprietary platforms favour their owned-and-operated offerings.

The rent extraction opportunities from controlling OS APIs are much more lucrative than an open ecosystem's direct competition. Decisions about closed platforms also do not have to consider other players as potential collaborators. Strategically speaking, managing a proprietary platform is playing on easy mode. When that isn't possible, the web can provide a powerful bridge to the temporarily embarrassed monopolist, but that is never their preference.

This competition is often hard to see because the decisive battles between proprietary systems and open platforms are fought behind the tallest, most heavily guarded walls of tech giants: budget planning.

Because budgets are secret, news of the web losing status is also a secret. It's a firing offence to leak budget details, or even share internally to those without a need to know, so line engineers may never be explicitly told that their mission has fundamentally changed. They can experience transitions away from the web as "Plan A" to "Plan Z" without their day-to-day changing much. The team just stops growing, and different types of features get priority.

OS vendors that walk away from the web after it has served their businesses can simply freeze browser teams, trimming ambitions around the edges without even telling the team that their status has been lowered. Capping and diverting funding away from new capabilities is easily explained; iteration on existing, non-threatening features is important, after all, and there are always benchmarks to optimise for.

Most fifth columnists are, therefore, unwitting and unknowing accomplices to corporate machinations well above their own pay grades. They cannot acknowledge that a vibrant web is not valuable to their firm because they will have never been told as much, and should they enquire, will hear only that the important work they are doing continues to be valued.

Until recently, the impact of fifth column tactics was obscured in a haze of legacy engines. Web developers take universality seriously, so improvements in capabilities have historically been experienced as the rate at which legacy UAs fall below a nominal relevance threshold.2 Thanks to pervasive auto-updates, the historical problem of dead-code-walking has largely been resolved. Major versions of important browsers may have entire cradle-to-grave lifecycles on the order of two or three months, rather than IE 6's half-decade half-life.

This has clarified the impact of (dis)investments by certain vendors and makes fifth columnists easier to spot. Web developers now benefit from, or are held back by, recent decisions by the companies (under)funding browser development. If sites remain unable to use features launched in leading-edge engines, it's only because of deficiencies in recent versions of competing engines. This is a much easier gap to close — in theory, anyway.

Web standards are voluntary, and the legal structures that create SDO safe-harbours (PDF) create the space in, and rules under, which SDOs must operate. SDOs may find their designs written into legislation after the fact, or as implementation guides, but there is a strong aversion to being told what to design by governments within the web community, particularly among implementers. Some new participants in standards arrive with the expectation that the de jure nature of a formal standard creates a requirement for implementation, but nothing could be further from fact. This sometimes leads to great frustration. Enshrining a design in a ratified standard does not obligate anyone to do anything, and many volumes of pure specifiction have been issued to little effect.

The voluntary nature of web standards is based on the autonomy of browsers, servers, and web developers to implement whatever they please under own brand.

Until Apple's flagrantly anticompetitive iOS policies, this right was inviolable because, when compromised, erosions of feature sovereignty undermine the premise of SDOs.

When products lose the ability to differentiate on features, quality, performance, safety, and standards conformance, the market logic underpinning voluntary standards becomes a dead letter. There's no reason to propose an improvement to the collective body of the web when another party can prevent you from winning share by supporting that feature.

The harms of implementation compellence are proportional to market influence. iOS's monopoly on the pockets of the wealthy (read: influential) has decisively undermined the logic of the open internet and the browser market. Not coincidentally, this has also desolated the prospect of a thriving mobile web.

The Mobile Web: MIA

It's no exaggeration to say that it is anti-web to constrain which standards vendors can implement within their browsers, and implementation coercion is antithetical to the good functioning of SDOs and the broader web ecosystem.3

In a perverse way, Apple's policy predations, strategic under-investment, and abusive incompetence clarify the basic terms of the web's power structure: SDOs are downstream of browser competition, and browser competition depends on open operating systems.

Why do vendors spend time and money to work in standards, only to give away their IP in the form of patent licences covering ratified documents?

When vendors enjoy product autonomy, they develop standards to increase interoperability at the behest of customers who dislike lock-in. Standards also lower vendor's legal risk through joint licensing, and increase marketability of their products. Behind this nondescript summary often lies a furious battle for market share between bitter competitors, and standards development is famous for playing a subtle role. Shapiro and Varian's classic paper "The Art of Standards Wars" (PDF) is a quarter-century old now, but its description of these sorts of battles is no less relevant today that it was in '99.

Like this classic, most discussions of standards battles highlight parties with differing visions but similar goals — should margin-sizing or border-sizing be the default? — rather than situations where one vendor has a proprietary agenda and wishes to grow or maintain a capability gap versus standards-based alternatives. Denying features to open ecosystems makes high-tax proprietary platforms more attractive, and gridlock in standards is an effective strategy for accomplishing it. These cases are under-discussed, in part, because they're hard to perceive over short time periods or by observing a single working group.

Parties that maintain a footprint in standards, but are unhappy to see standards-based platforms compete with their proprietary offerings, only need a few pages from the Simple Sabotage manual (PDF).

Often, they will send delegates to important groups and take visible leadership positions. Combined with juuuuuust enough constructive technical engagement, obstreperous parties scarcely have to say anything about their intent. Others will raise their own hopes, and cite (tepid) participation as evidence of good faith. The fifth columnist doesn't need to raise a finger, which is handy, as doing nothing is the goal.4

Working group composition also favours delays at the hands of fifth columnists. Encouraged by defectors, they regularly divert focus from important problems, instead spending huge amounts of time on trivialities because few customers (web developers) have the time, money, and energy to represent their own needs in committee. Without those voices, it's hard to keep things on track.

Worse, web developers generally lack an understanding of browser implementation details and don't intone the linguistic airs and shorthands of committeespeak, which vary from venue to venue. This hampers their ability to be taken seriously if they do attend. At the limit, dismissing pressing problems on technicalities can become something of a committee pastime.5

There's also a great deal of opportunity for fifth columnists to misrepresent the clearly stated needs of web developers, launching projects to solve adjacent (but unimportant) sub-issues, while failing to address the issues of the day. This is particularly problematic in big, old rooms.

A competent fifth columnist only needs to signal in-group membership to amplify the impact of their own disinterest in topics they would prefer to avoid. Ambiguous "concerns" and scary sounding caveats are raised, and oven-ready designs which do arrive are reframed as naive proposals by outsiders. Process-level critique, in lieu of discussing substance, is the final line of defence.

Deflecting important work is shockingly easy to pull off because organisations that wish to defeat progress can send delegates that can appeal to rooms full of C++/Rust engineers as peers. The tripwires in web API design are not obvious to the uninitiated, so it's easy to move entire areas of design off the agenda through critique of small, incongruent details.

The most depressing thing about this pattern is that these tactics work because other vendors allow it.

One problem facing new areas in standards is that chartered Working Groups are just that: chartered. They have to define what they will deliver years in advance, and anything not on the agenda is, by definition, not in scope. The window in many SDO processes for putting something new into the hopper is both short and biased towards small revisions of the existing features. Spinning up new Working Groups is a huge effort that requires political clout.

Technically, this is a feature of SDOs; they jointly licence the IP of members to reduce risks to implementers and customers of adopting standards-based products. Patent trolls have to consider the mutual defences of the whole group's membership and cannot easily pick off smaller prey. Most developers never give a second thought to patent portfolios and do not live in fear of being personally sued for infringement.

This is a sign web standards are a smashing success, but also makes it unlikely that working developers understand that standards processes are designed with bounded IP commitments in mind. Internet SDOs have been so successful at caging the beast that generations of developers have never considered the threat or how it intersects with their interests.

This creates a tension: over the long term, SDOs and the ecosystems can only succeed if they take on new problems that are adjacent to the current set, but the Working Groups they create are primed by their composition and history to avoid taking on substantial expansions of their scope. After all, a good v2 of a spec is one that fixes all the problems of v1 and introduces relatively few new ones.

To work around this restriction, functional SDOs create incubation venues. These take different guises, but the core features are the same. Unlike chartered Working Groups, incubation groups are simple to create; no charter votes or large, up-front IP commitments. They also feature low bars to participation, can be easily shut down, and do not produce documents for formal standardisation, although they can produce "Notes" or other specification documents that Working Groups can take up.

Instead, they tend to have substantial contributor-only grants of IP, ad-hoc meeting and internal debate mechanisms, and attract only those interested in working on solutions in a new problem space. In functioning SDOs, such "fail-fast" groups naturally become feeders for chartered Working Groups, iterating on problems and solutions at a rate which is not possible under the plodding bureaucracy of a chartered Working Groups's minuted and agenda-driven meeting cadence.

And that's why these sorts of groups are a first priority for sabotage by fifth columnists. The usual tactics deployed to subvert incubation include:

  • Aspersions of bad faith by those working in incubation venues, either on the grounds that the groups are "amateur", "not standards-track",6 or "do not have the right people."
  • Avoidance of engagement in incubation groups, robbing them of timely feedback while creating a self-fulfilling "lack of expertise" critique.
  • Citing a high fraction failed designs within these groups as an indicator that they are not useful, obscuring the reality that the entire point of incubation is to fail fast and iterate furiously.7
  • Accuse those who implement incubated proposals of "not following the process", "ignoring standards", or "shipping whatever they want"; twisting the goals of those doing design in the open, in good faith, under the SDO's IP umbrella.
  • Demanding formalities akin to chartered Working Groups to slow the pace of design progress in incubation venues that are too successful to ignore.

The fifth columnist also works behind the scenes to reduce the standing and reputation of incubation groups among the SDO's grandees, claiming that they represent a threat to the stability of the overall organisation. Because that constituency is largely divorced from the sausage-making, this sort of treachery works more often than it should, causing those who want to solve problems to burn time defending the existence of venues where real progress is being made.

The picture presented this far is of Working Groups meeting in The Upside Down. After all, it's only web developers who can provide a real test of a design, or even the legitimacy of a problem.

This problem becomes endemic in many groups, and entire SDOs can become captured by the internal dramas and preferences of implementers, effectively locking customers out. Without more dynamic, fail-fast forums that enable coalitions of the willing to design and ship around obstructionists, working groups can lay exclusive claim to important technologies and retreat into irrelevance without paying a reputational cost.

The alternative — hard forking specifications — is a nuclear option. The fallout can easily blow back into the camp of those launching a fork, and the effort involved is stupendous. Given the limited near-term upside and unclear results, few are brave or foolish enough to consider forking to route around a single intransigent party.

This feeds the eclipse of an SDO's relevance because legitimacy deficits become toxic to the host only slowly. Risk of obsolescence can creep unnoticed until it's too late. As long as the ancient forms and traditions are followed, a decade or more can pass before the fecklessness of an important group rises to the notice of anyone with the power to provoke change. All the while, external observers will wonder why they must resort to increasingly tall piles of workarounds and transpilers. Some may even come to view deadening stasis, incompatibility, and waste as the natural state of affairs, declining to invest any further hope for change in the work of SDOs. At this point, the fifth columnist has won.

One of the self-renewing arrows in the fifth column's arsenal is the tendency of large and old working groups to indulge in survivorship bias.

Logically, there's no reason why folks whose designs won a lottery in the last round of market jousting8 should be gatekeepers regarding the next tranche of features. Having proposed winning designs in the past is not, in itself, a reliable credential. And yet, many of these folks become embedded within working groups, sometimes for decades, holding sway by dint of years of service and interpersonal capital. Experience can be helpful, but only when it is directed to constructive engagement, and too many group chairs allow bad behavior, verging on incivility, at the hands of la vieille garde. This, of course, actively discourages new and important work, and instead clears the ground for yet more navel-gazing.

This sort of in-group/out-group formation is natural in some sense, and even folks who have loathed each other from across a succession of identically drab conference rooms for years can find a sort of camaraderie in it. But the social lives of habitual TPAC and TC39 attendees are no reason to accept unproductive monopolies on progress; particularly when the folks in those rooms become unwitting dupes of fifth columnists, defending the honour of the group against those assailing it for not doing enough.

The dysfunctions of this dynamic mirrors those of lightly moderated email lists: small rooms of people all trying to solve the same problem can be incredibly productive, no matter how open. Large rooms with high alignment of aims can make progress if leadership is evident (e.g., strong moderation). What is reliably toxic are large, open rooms with a mix of "old timers" who are never moderated and "newbies" who have no social standing. Without either a clear destination or any effective means of making decisions, these sorts of venues become vitriolic over even the slightest things. As applied to working groups, without incredibly strong chairing, interpersonal dynamics of long-standing groups can make a mockery of the responsibilities resting on the shoulders of charter. But it's unusual for anyone on the outside to get any the wiser. Who has time to decipher meeting minutes or decode in-group shorthands?

And so it is precisely because fifth columnists can hire old timers9 that they are able to pivot groups away from addressing pressing concerns to the majority of the ecosystem, particularly in the absence of functional incubation venues challenging sclerotic groups to move faster.

One useful lens for discussing the fifth column problem is the now-common Political Science analysis of systems through "veto points" or "veto players" (Tsebelis, '95; PDF):

Policy stability is different from both government stability and regime stability. In fact ... they are inversely related: policy stability causes government or regime instability. This analysis is based on the concept of the veto player in different institutional settings.

If we substitute "capability stability" for "policy stability" and "platform relevance" for "government/regime stability," the situation becomes clear:

Capability stability is different from platform relevance. In fact ... they are inversely related: capability stability causes platform irrelevance.

Any platform that cannot grow to incorporate new capabilities, or change to address pressing problems, eventually suffers irrelevance, then collapse. And the availability of veto points to players entirely hostile to the success of the platform is, therefore, an existential risk10 — both to the platform and to the SDOs that standardise it, not to mention the careers of developers invested in the platform.

This isn't hard to understand, so how does the enterprising fifth columnist cover their tracks? By claiming that they are not opposed to proposals, but that they "need work," without either offering to do that work, develop counter-proposals, or make any commitment to ship any version of proposals in their own products. This works too often because a pattern of practice must develop before participants can see that blockage is not a one-off rant by a passionate engineer.

Participants are given wide berths, both because of the presumption of voluntarity implementations,11 and the social attitudes of standards-inclined developers. Most SDO participants are community-minded and collaboration-oriented. It's troubling to imagine that someone would show up to such a gathering without an intent to work to solve problems, as that would amount to bad faith. But it has recurred frequently enough that we must accept it does happen. Having accepted its possibility, we must learn to spot the signs, remain on guard, and call it out as evidence accumulates.

The meta-goal is to ensure no action for developers, with delay to the point of irrelevance as a fallback position, so it is essential to the veto wielder that this delay be viewed as desirable in some other dimension. If this sounds similar to "but neighbourhood character!" arguments by NIMBYs offer, that's no accident. Without a valid argument to forestall efforts to solve pressing problems, the fifth column must appeal to latent, second-order values that are generally accepted by the assembled to pre-empt the first-order concern. This works a shocking fraction of the time.

It works all the better in committees with a strong internal identity. It's much easier to claim that external developers demanding solutions "just don't get it" when the group already views their role as self-styled bulwarks against bad ideas.

The final, most pernicious, building block of Working Group decay is the introduction of easy vetoes via consensus decision-making. When vetoes are available to anyone in a large group at many points, the set of proposals that can be offered without a presumption of failure shrinks to a tiny set, in line with the findings of Tsebelis.

This is not a new problem in standards development, but the language gets muddy. I perceive two distinct versions:

  • Strong Consensus refers to working modes in which the assent of every participant is affirmatively required to move proposals forward.
  • Weak Consensus are modes in which preferences are polled, but "the sense of the room" can carry a proposal forward over even strenuous objections by small minorities.

Every long-term functional SDO operates by some version of Weak Consensus. The IETF bandies this about so often that the phrase "rough consensus and running code" is synonymous with the organisation.

But not every group within these SDOs are chaired by folks willing to overrule objectors. In these situations, groups can revert to de facto strong consensus, which greatly multiplies the number of veto holders. Variations on this theme can be even less disciplined, with only an old guard having effective veto power, whilst newer participants may be more easily overturned.

Strong consensus is the camel's nose for long-term gridlock. Like unmoderated mailing lists, things can spiral without anyone quite knowing where the error was made. Small groups can start under strong consensus out of a sense of mutual respect, only to find it is nearly impossible to revoke a veto power once handed out. A sense of fair play may cause this right to be extended to each new participant, and as groups grow, affiliations change, and interests naturally diverge, it may belatedly dawn on those interested in progress that the very rooms where they once had so much luck driving things forward have become utterly dysfunctional. And under identical rules!

Having found the group no longer functions, delegates who have invested large portions of their careers to these spaces have a choice: they can acknowledge that it is not working and demand change, becoming incredibly unpopular amongst their closest peers in the process. Or they can keep their heads down and hope for the best, defending the honour of the group against attacks by "outsiders". Don't they know whose these people are?

Once it sets in, strong consensus modes are devilish to unpick, often requiring a changing of the guard, both among group chairs and influential veto-wielders. Groups can lose internal cohesion and technical expertise in the process, heaping disincentive to rock even the most unproductive boats.

The ways that web ecosystem SDOs and their participants can guard against embrittlement and fracture from the leeching effects of fifth columns are straightforward, if difficult to pull off socially:

  • Seek out and remove strong consensus processes.

    The timeless wisdom of weak consensus is generally prescribed by process documents governing SDOs, so the usual challenge is enforcement. The difficulty in shaking strong consensus practices is frequently compounded by the status of influential individuals from important working groups who prefer it. Regardless, the consequences of allowing strong consensus to fester in rooms big enough to justify chairing is dire, and it must be eliminated root and branch.

  • Aggressively encourage "counterproposal or GTFO" culture.

    Fifth columnists thrive in creating ambiguity for the prospects of meaningful proposals while paying no cost for "just asking questions." This should be actively discouraged, particularly among implementers, within the social compact of web SDOs. The price for imposing delay must be higher than having vague "concerns".

  • Require Working Groups to list incubators they accept proposals from. Require they prove it.

    Many groups that fifth columnists exploit demonstrate a relative imperviousness to new ideas through a combination of social norms and studious ignorance. To break this pattern, SDOs should require all re-charters include clear evidence of proposals coming from outside the group itself. Without such collateral, charters should be at risk.

  • Defend incubators from process attacks.

    Far from being sideshows, incubation venues are the lifeblood of vibrant SDOs. They must be encouraged, nurtured, and highlighted to the membership as essential to the success of the ecosystem and the organisation.

    In the same vein, process shenanigans to destabilise successful incubators must be fended off; including but not limited to making them harder to join or create, efforts to deny their work products a seat in working group chartering, or tactics that make their internal operations more veto-centric.

It takes a long time, but like the gravitational effects of a wandering planet out in the OORT cloud, the informational content of the fifth columnist's agenda eventually becomes legible by side effect. Because an anti-web agenda is easy to pass off under other cover, it requires a great number of observations to understand that this part of the committee does not want the platform to evolve. From that point forward, it becomes easier to understand the information being communicated as noise, rather than signal.

Once demonstrated, we must route around the damage, raising the cost of efforts to undermine the single most successful standards-based ecosystem of our lifetimes; one that I believe is worth defending from insider threats as well as external attack.

FOOTNOTES

  1. The most substantial periods of institutional decrepitude in web standards are highly correlated with veto players (vendors with more than ~10% total share) walking away from efforts to push the web forward.

    The most famous period of SDO decay is probably the W3C's troubled period after Microsoft disbanded the Internet Explorer team after IE 6.0's triumphant release in 2001. Even if folks from Microsoft continued to go to meetings, there was nobody left to implement new or different designs and no product to launch them in.

    Standards debate went from pitched battles over essential features of systems being actively developed to creative writing contests about futures it might be nice to have. Without the disciplining function of vendors shipping, working groups just become expensive and drab pantomimes.

    With Microsoft circa 2002 casting the IE team to the wind and pivoting hard to XAML and proprietary, Windows-centric technologies, along with the collapse of Netscape, the W3C was left rudderless, allowing it to drift into failed XHTML escapades that inspired revulsion among the remaining staffed engine projects.

    This all came to a head over proposed future directions at 2004's Web Applications and Compound Document Workshop. WHATWG was founded in the explosion's crater, and the rest is (contested) history.

    The seeds of the next failure epoch were planted at the launch of the iOS App Store in 2008, where it first became clear that other "browsers" would be allowed on Cupertino's best-selling devices, but not if they included their own engines. Unlike the big-bang of Microsoft walking away from browsers for 3+ years, Apple's undermining of the W3C, IETF, and ECMA become visible only gradually as the total global market share of mobile devices accelerated. Apple also "lost" its early lead in the smartphone market share as Android ate up the low end's explosive growth. The result was a two-track mobile universe, where Apple retained nearly all influence and profits, whilst most new smartphone users encountered the predations of Samsung, HTC, LG, Xiaomi, and a hundred other cut-price brands.

    Apple's internal debates about which platform for iOS was going to "win" may have been unsettled at the launch of the App Store13, but shortly thereafter the fate of Safari and the web on iOS was sealed when Push Notifications appeared for native apps but not web apps.

    Cupertino leveraged its monopoly on influence to destroy the web's chances, while Mozilla, Google, and others who should have spoken up remained silent. Whether that cowardice was borne of fear, hope, or ignorance hardly matters now. The price of silence is now plain, and the web so weakened that it may succumb entirely to the next threat; after all, it has no champions among the megacorps that have built their businesses on its back.

    First among equals, Apple remains at the vanguard of efforts to suppress the web, spending vast sums to mislead web developers, regulators, legislators, and civil society. That last group uncomfortably includes SDOs, and it's horrifying to see the gaslighting plan work while, in parallel, Cupertino sues for delay and offers easily disproven nonsense in rooms where knowing misrepresentation should carry sanction.

    All this to preclude a competent version of the web on iPhones, either from Apple or (horrors!) from anyone else. Query why.

  2. The market share at which any browser obtains "blocking share" is not well theorized, but is demonstrably below 5% for previously dominant players, and perhaps higher for browsers or brands that never achieved market plurality status.

    Browsers and engines which never gain share above about 10% are not considered "relevant" by most developers and can be born, live, and die entirely out of view of the mainstream. For other players, particularly across form-factors, the salience of any specific engine is more contextual. Contractual terms, tooling support, and even the personal experience of influential developers all play a role. This situation is not helped by major sites and CDNs — with the partial exception of Cloudflare — declining to share statistics on the mix of browsers their services see.

    Regardless, web-wide market share below 2% for any specific version of any engine is generally accepted as irrelevance; the point at which developers no longer put in even minimal effort to continue to support a browser except with "fallback" experiences.

  3. It's not an exaggeration to suggest that the W3C, IETF, and ECMA have been fundamentally undermined by Apple's coercion regarding browser engines on iOS, turning the entire organisation into a sort of Potempkin village with semi-independent burgs taking shape on the outskirts through Community Groups like the WICG, which Apple regularly tries to tear down through procedural attacks it hopes the wider community will not trace back to the source.

    When competitors cannot ship their best ideas, the venues where voluntary standards are codified lose both their role as patent-pooling accelerators for adoption, as well as their techno-social role as mediators and neutral ground.

    The corporeal form continues long after the ghost leaves the body, but once the vivvifying force of feature autonomy is removed, an SDO's roof only serves to collect skeletons, eventually compromising the origanisation itself. On these grounds, self-aware versions of the W3C, IETF, and ECMA would have long ago ejected Apple from membership, but self-awareness is not their strong suit. And as long as the meetings continue and new drafts are published, it hardly deserves mention that the SDO's role in facilitating truly disruptive change will never again be roused. After all, the membership documents companies sign do not require them to refrain from shivving their competition; only that everyone keep their voices down and the tone civil.

    What's truly sad is how few convening services or reading the liturgy from the pews seem disturbed that their prayers can never again be heard.

  4. This is about the point where folks will come crawling out of the walls to tell stories about IBM or Rambus or Oracle or any of the codec sharks that have played the heel in standards at one point or another. Don't bother; I've got a pretty full file of those stories, and I can't repeat them here anyway. But if you do manage to blog one of them in an entertaining way without getting sued, please drop a line.

  5. You know, in case you're wondering what CSS WG was doing from '04-'21. I wonder what changed?

  6. It's particularly disingenuous for fifth columnists to claim proposals they don't like are "not standards track" as they know full-well that the reason they aren't being advanced within chartered working groups is their own opposition.

    The circularity is gobsmacking, but often works. This reduces pressure fifth columnists from credulous web developers. Excusemaking is only possible because other vendors fail to call bullshit. Apple, e.g., would not be getting away with snuffing out the mobile web's chances were it not for a cosy set of fellow travellers at Mozilla and Google.

  7. Internet APIs and protocols do not spring fully-formed from the head of Zeus.

    Getting to good, or even good-enough, requires furious iteration, and that means testing and prodding at proposals. The only way to get miles under a proposal is to try things, and that's what incubation venues specialise in. It is not a sign of failure that many proposals change shape in response to feedback, or that certain evolutionary branches are abandoned altogether. Much the opposite, in fact.

    It is only by trying, testing, iterating, and yes, abandoning many designs that we arrive at productive progress in web specs. Anyone who tells you differently is carrying water for fifth columnists and should be put on notice. They may not personally intend to undermine the web's future, but that's what treating iteration in design as failure does by proxy.

  8. Most Web API design processes cannot claim any kinship to the scientific method, although we have tried mightily to open a larger space for testing of alternative hypotheses within the Chromium project over the past decade.

    Even so, much of the design work of APIs on the web platform is shaped by the specific and peculiar preferences of powerful individuals, many of whom are not and have never been working web developers.

  9. Hiring "known quantities" to do the wrangling within a Working Group you want to scupper is generally cheaper than doing constructive design work, so putting in-group old-timers on the payroll is a reliable way for fifth columnists to appear aligned with the goals of the majority while working against them in practice.

  10. One rhetorical mode for those working to constrain the web platform's capabilities is to attempt to conflate any additions with instability, and specifically, the threat that sites that work today will stop working tomorrow. This is misdirection, as stability for the ecosystem is not a function of standards debates, but rather the self-interested actions of each vendor in the market.

    When true browser competition is allowed, the largest disciplining force on vendor behaviour is incompatibility. Browsers that fail to load important web pages lose share to those that have better web compatibility. This is as close as you can get to an iron law of browser engineering, and every vendor knows that their own engine teams have spent gargantuan amounts of time and money to increase compatibility over the years.

    Put more succinctly, backwards compatibility on the web is not seriously at risk from capability expansions. Proposals that would imperil back compat12 are viewed as non-starters in all web standards venues, and major schisms have formed over proposed, incompatible divergence, with the compatibility-minded winning nearly every skirmish.

    No SDO representatitive from these teams is ignorant of these facts, and so attempts to argue against solving important problems by invoking the spectre of "too much change, too fast" or "breaking the web" are sleights of hand.

    They know that most web developers value stability and don't understand these background facts, creating space for a shell game in which the threat of too much change serves to obsure their own attempts at sabatoge through inaction. Because web standards are voluntary and market share matters tremendously to every vendor, nothing that actually breaks the web will be allowed to ship. So armed, you can now call out this bait-and-switch wherever it appears. Doing so is important, as the power to muddy these waters stems from the relative ignorance of web developers. Educating them to the real power dynamics at work is our best bullwark against the fifth column.

  11. There's no surer sign of the blindness many SDO participants exhibit toward the breakage of the voluntary implementation regime than that they extend deference on that basis to Apple.

    Cupertino's SDO delegates do not bear a heightened presumption that they will implement as soon as other ship. To the contrary, Apple have so thoroughly lowered expectations that nobody expects timely feedback on proposals, let alone implmementation commitments. Even when Apple is the last holdout. Cupertino sullies the brands they force to use WebKit's worst-of-the-worst implementation. It has been going on so long that it is now simply accepted as the status quo.

    This is entirely backwards, and Apple's representatitives should, instead, be expected to provide implementation timlines for features shipped by other vendors. God knows they can afford it.

    Until such time as Apple allows engine competition worldwide, it's only fair to expect (and demand) parity. Every Apple employee should feel the heat of shame every as they mutter "Apple does not comment on future product releases" while indefensibly harming the web.

  12. The browser and web infrastructure community have implemented large transitions away from some regretted technologies, but the care and discipline needed to do this without breaking the web is the stuff of legend. Big ones that others should write long histories of are the sunsetting of AppCache and the move away from unencrypted network connections.

    Both played out on the order of half a decade or more, took dozens of stakeholders to pull off, and were games of inches adding up to miles. New tools like The Reporting API, Deprecation Reports, and Reverse Origin Trials had to be invented to augment the "usual" tool bag of anonymised analytics trawls, developer outreach, new limits on unwanted behaviour, and nudging UI.

    In both cases (among many more small deprecations we have done over the years), the care taken ensured that only a small fraction of the ecosystem was impacted at any moment, lowering the temperature and allowing for an orderly transition to better technology.

  13. Your correspondent has heard different stories from folks who had reason to know about the period from '08-'10 when Apple pulled its foot off the gas with Safari.

    Given the extreme compartmentalisation of Apple's teams, the strategic import of any decision, and the usual opacity of tech firms around funding levels ("headcount") to even relatively senior managers, this is both frustrating and expected.

    The earliest dating puts the death of the web as "Plan A" before Steve Jobs's announcement of the iPhone at Macworld in June 2007. The evidence offered for this view was that a bake off for system apps and the home screen launcher had already been lost by WebKit. Others suggest it wasn't until the success of the App Store in '09 and '10 that Apple began to pull away from the web as a top-tier platform. Either way, it was all over by early 2011 at the very latest.

    WebKit would never again be asked to compete as a primary mobile app platform, and skeletal funding for Safari ensured it would never be allowed to break out of the OS's strategic straightjacket the way IE 4-6 had.

]]>
Links? Links! 2025-07-16T00:00:00Z https://infrequently.org/2025/07/links/

Frances has urged me for years to collect resources for folks getting into performance and platform-oriented web development. The effort has always seemed daunting, but the lack of such a list came up again at work, prompting me to take on the side-quest amidst a different performance yak-shave. If that sounds like procrastination, well, you might very well think that. I couldn't possibly comment.

The result is a new links and resources page page which you can find over in the navigation rail. It's part list-of-things-I-keep-sending-people, part background reading, and part blogroll.

The blogroll section also prompted me to create an OPML export , which you can download or send directly to your feed reader of choice.

The page now contains more than 250 pointers to people and work that I view as important to a culture that is intentional about building a web worth wanting. Hopefully maintenance won't be onerous from here on in. The process of tracking down links to blogs and feeds is a slog, no matter how good the tooling. Very often, this involved heading to people's sites and reading the view-source://

Having done this dozens of times on the sites of brilliant and talented web developers in a short period of time, a few things stood out.

First — and I cannot emphasise this enough — holy cow.

The things creative folks can do today with CSS, HTML, and SVG in good browsers is astonishing. If you want to be inspired about what's possible without dragging bloated legacy frameworks along, the work of Ana Tudor, Jhey, Julia Miocene, Bramus, Adam, and so many others can't help but raise your spirits. The CodePen community, in particular, is incredible, and I could (and have) spend hours just clicking through and dissecting clever uses of the platform from the site's "best of" links.

Second, 11ty and Astro have won the hearts of frontend's best minds.

It's not universal, but the overwhelming bulk of personal pages by the most talented frontenders are now built with SSGs that put them in total control. React, Next, and even Nuxt are absent from pages of the folks who really know what they're doing. This ought to be a strong signal to hiring managers looking to cut through the noise.

Next, when did RSS/Atom previews get so dang beautiful?

The art and effort put into XSLT styling like Elly Loel's is gobsmacking. I am verklempt that not only does my feed not look that good, my site doesn't look that polished.

Last, whimsy isn't dead.

Webrings, guestbooks, ASCII art in comments, and every other fun and silly flourish are out there, going strong, just below the surface of the JavaScript-Industrial Complex's thinkfluencer hype recycling.

And it's wonderful.

My overwhelming feeling after composing this collection is gratitude. So many wonderful people are doing great things, based on values that put users first. Sitting with their work gives me hope, and I hope their inspiration can spark something similar for you.

]]>
Conferences, Clarity, and Smokescreens 2025-06-27T00:00:00Z https://infrequently.org/2025/06/conferences-clarity-and-smokescreens/

Before saying anything else, I'd like to thank the organisers of JSNation for inviting me to speak in Amsterdam. I particularly appreciate the folks who were brave enough to disagree at the Q&A sessions afterwards. Engaged debate about problems we can see and evidence we can measure makes our work better.

The conference venue was lovely, and speakers were more than well looked after. Many of the JSNation talks were of exactly the sort I'd hope to see as our discipline belatedly confronts a lost decade, particularly Jeremias Menichelli's lighting talk. It masterfully outlined how many of the hacks we have become accustomed to are no longer needed, even in the worst contemporary engines. view-source:// on the demo site he made to see what I mean.

Vinicius Dallacqua's talk on LoAF was on-point, and the full JSNation line-up included knowledgeable and wise folks, including Jo, Charlie, Thomas, Barry, Nico, and Eva. There was also a strong set of accessibility talks from presenters I'm less familiar with, but whose topics were timely and went deeper than the surface. They even let me present a spicier topic than I think they might have been initially comfortable with.

All-in-all, JSNation was a lovely time, in good company, with a strong bent toward doing a great job for users. Recommended.

Day 21React Summit 2025 — could not have been more different. While I was in a parallel framework authors meeting for much of the morning,2 I did attend talks in the afternoon, studied the schedule, and went back through many more after the fact on the stream. Aside from Xuan Huang's talk on Lynx and Luca Mezzalira's talk on architecture, there was little in the program that challenged frameworkist dogma, and much that played to it.

This matters because conferences succeed by foregrounding the hot topics within a community. Agendas are curated to reflect the tides of debate in the zeitgeist, and can be read as a map of the narratives a community's leaders wish to see debated. My day-to-day consulting work, along with high-visibility industry data, shows that the React community is mired in a deep, measurable quality crisis. But attendees of React Summit who didn't already know wouldn't hear about it.

Near as I can tell, the schedule of React Summit mirrors the content of other recent and pending React conferences (1, 2, 3, 4, 5, 6) in that these are not engineering conferences; they are marketing events.

How can we tell the difference? The short answer is also a question: "who are we building for?"

The longer form requires distinguishing between occupations and professions.

In a 1912 commencement address, the great American jurist and antitrust reformer Louis Brandeis hoped that a different occupation — business management — would aspire to service:

The peculiar characteristics of a profession as distinguished from other occupations, I take to be these:

First. A profession is an occupation for which the necessary preliminary training is intellectual in character, involving knowledge and to some extent learning, as distinguished from mere skill.

Second. It is an occupation which is pursued largely for others and not merely for one's self.

Third. It is an occupation in which the amount of financial return is not the accepted measure of success.

In the same talk, Brandeis named engineering a discipline already worthy of a professional distinction. Most software development can't share the benefit of the nominative doubt, no matter how often "engineer" appears on CVs and business cards. If React Summit and Co. are anything to go by, frontend is mired in the same ethical tar that causes Wharton, Kellogg, and Stanford grads to experience midlife crises.3

It may seem slanderous to compare React conference agendas to MBA curricula, but if anything it's letting the lemon vendors off too easily. Conferences crystallise consensus about which problems matter, and React Summit succeeded in projecting a clear perspective — namely that it's time to party like it's 2013.

A patient waking from a decade-long coma would find the themes instantly legible. In no particular order: React is good because it is popular. There is no other way to evaluate framework choice, and that it's daft to try because "everyone knows React".4 Investments in React are simultaneously solid long-term choices, but also fragile machines in need of constant maintenance lest they wash away under the annual tax of breaking changes, toolchain instability, and changing solutions to problems React itself introduces. Form validation is not a solved problem, and in our glorious future, the transpilers compilers will save us.

Above all else, the consensus remains that SPAs are unquestionably a good idea, and that React makes sense because you need complex data and state management abstractions to make transitions between app sections seem fluid in an SPA. And if you're worried about the serially terrible performance of React on mobile, don't worry; for the low, low price of capitulating to App Store gatekeepers, React Native has you covered.5

At no point would our theoretical patient risk learning that rephrasing everything in JSX is now optional thanks to React 19 finally unblocking interoperability via Web Components.6 Nor would they become aware that new platform APIs like cross-document View Transitions and the Navigation API invalidate foundational premises of the architectures that React itself is justified on. They wouldn't even learn that React hasn't solved the one problem it was pitched to address.

Conspicuously missing from the various "State Of" talks was discussion of the pressing and pervasive UX quality issues that are rampant in the React ecosystem.

Per the 2024 Web Almanac, less than half of sites earn passing grades on mobile, where most users are.
Per the 2024 Web Almanac, less than half of sites earn passing grades on mobile, where most users are.

We don't need to get distracted looking inside these results. Treating them as black boxes is enough. And at that level we can see that, in aggregate, JS-centric stacks aren't positively correlated with delivering good user-experiences.

2024's switch from FID to INP caused React (particularly Next and Gatsby) sites which already had low pass-rates to drop more than sites constructed on many other stacks.
2024's switch from FID to INP caused React (particularly Next and Gatsby) sites which already had low pass-rates to drop more than sites constructed on many other stacks.

This implies that organisations adopting React do not contain the requisite variety needed to manage the new complexity that comes from React ecosystem tools, practices, and community habits. Whatever the source, it is clearly a package deal. The result are systems that are out of control and behave in dynamically unstable ways relative to business goals.

The evidence that React-based stacks frequently fail to deliver good experiences is everywhere. Weren't "fluid user experiences" the point of the JS/SPA/React boondoggle?7

We have witnessed high-cost, low-quality JS-stack rewrites of otherwise functional HTML-first sites ambush businesses with reduced revenue and higher costs for a decade. It is no less of a scandal for how pervasive it has become.

But good luck finding solutions to, or even acknowledgement of, that scandal on React conference agendas. The reality is that the more React spreads, the worse the results get despite the eye-watering sums spent on conversions away from functional "legacy" HTML-first approaches. Many at React Summit were happy to make these points to me in private, but not on the main stage. The JS-industrial-complex omertà is intense.

No speaker I heard connected the dots between this crisis and the moves of the React team in response to the emergence of comparative quality metrics. React Fiber (née "Concurrent"), React Server Components, the switch away from Create React App, and the React Compiler were discussed as logical next steps, rather than what they are: attempts to stay one step ahead of the law. Everyone in the room was expected to use their employer's money to adopt all of these technologies, rather than reflect on why all of this has been uniquely necessary in the land of the Over Reactors.8

The treadmill is real, but even at this late date, developers are expected to take promises of quality and productivity at face value, even as they wade through another swamp of configuration cruft, bugs, and upgrade toil.

React cannot fail, it can only be failed.

And then there was the privilege bubble. Speaker after speaker stressed development speed, including the ability to ship to mobile and desktop from the same React code. The implications for complexity-management, user-experience, and access were less of a focus.

The most egregious example of the day came from Evan Bacon in his talk about Expo, in which he presented Burger King's website as an example of a brand successfully shipping simultaneously to web and native from the same codebase. Here it is under WebPageTest.org's desktop setup:9


As you might expect, putting 75% of the 3.5MB JS payload (15MB unzipped) in the critical path does unpleasant things to the user experience, but none of the dizzying array of tools involved in constructing bk.com steered this team away from failure.10

The fact that Expo enables Burger King to ship a native app from the same codebase seems not to have prevented the overwhelming majority of users from visiting the site in browsers on their mobile devices, where weaker mobile CPUs struggle mightily:


The CrUX data is damning:


This sort of omnishambles is what folks mean when they say that "JavaScript broke the web and called it progress".

Is waiting 30 seconds for a loading spinner bad?<br>Asking for an industry.
Is waiting 30 seconds for a loading spinner bad?
Asking for an industry.

The other poster child for Expo is Bluesky, a site that also serves web and React Native from the same codebase. It's so bewilderingly laden with React-ish excesses that their engineering choices qualify as gifts-in-kind to Elon Musk and Mark Zuckerberg:


Why is Bluesky so slow? A huge, steaming pile of critical-path JS, same as Burger King:


Again, we don't need to look deeply into the black box to understand that there's something rotten about the compromises involved in choosing React Native + Expo + React Web. This combination clearly prevents teams from effectively managing performance or even adding offline resilience via Service Workers. Pinafore and Elk manage to get both right, providing great PWA experiences while being built on a comparative shoestring. It's possible to build a great social SPA experience, but maybe just not with React:

If we're going to get out of this mess, we need to stop conflating failure with success. The <em>entire point</em> of this tech was to deliver better user experiences, and so the essential job of management is to ask: does it?
If we're going to get out of this mess, we need to stop conflating failure with success. The entire point of this tech was to deliver better user experiences, and so the essential job of management is to ask: does it?

The unflattering comparisons are everywhere when you start looking. Tanner Linsley's talk on TanStack (not yet online) was, in essence, a victory lap. It promised high quality web software and better time-to-market, leaning on popularity contest results and unverifiable, untested claims about productivity to pitch the assembled. To say that this mode of argument is methodologically unsound is an understatement. Rejecting it is necessary if we're going to do engineering rather that marketing.

Popularity is not an accepted unit of engineering quality measurement.
Popularity is not an accepted unit of engineering quality measurement.

The TanStack website cites this social proof as an argument for why their software is great, but the proof of the pudding is in the eating:


The contrast grows stark as we push further outside the privilege bubble. Here are the same sites, using the same network configuration as before, but with the CPU emulation modelling a cheap Android instead:

An absolute rout. The main difference? The amount of JS each site sends, which is a direct reflection of values and philosophy.
An absolute rout. The main difference? The amount of JS each site sends, which is a direct reflection of values and philosophy.

Site Wire JS Decoded JS TBT (ms)
astro.build 11.1 kB 28.9 kB 23
hotwired.dev 1.8 kB 3.6 kB 0
11ty.dev 13.1 kB 42.2 kB 0
expo.dev 1,526.1 kB 5,037.6 kB 578
tanstack.com 1,143.8 kB 3,754.8 kB 366

Yes, these websites target developers on fast machines. So what? The choices they make speak to the values of their creators. And those values shine through the fog of marketing when we use objective quality measures. The same sorts of engineers who care to shave a few bytes of JS for users on fast machines will care about the lived UX quality of their approach all the way down the wealth curve. The opposite also holds.

It is my long experience that cultures that claim "it's fine" to pay for a lot of JS up-front to gain (unquantified) benefits in another dimension almost never check to see if either side of the trade comes up good.

Programming-as-pop-culture is oppositional to the rigour required of engineering. We need to collectively recalibrate when the folks talking loudest about "scale" and "high quality" and "delivery speed" — without metrics or measurement — continually plop out crappy experiences but are given huge megaphones anyway.

There were some bright spots at React Summit, though. A few brave souls tried to sneak perspective in through the side door, and I applaud their efforts:

If the Q&A sessions after my talk are any indication, Luca faced serious risk of being ignored as a heretic for putting this on a slide.
If the Q&A sessions after my talk are any indication, Luca faced serious risk of being ignored as a heretic for putting this on a slide.

If frontend aspires to be a profession11something we do for others, not just ourselves — then we need a culture that can learn to use statistical methods for measuring quality and reject the sorts of marketing that still dominates the React discourse.

And if that means we have to jettison React along the way, so be it.

FOOTNOTES

  1. For attendees, JSNation and React Summit were separate events, although one could buy passes that provided access to both. My impression is that many did. As they were in the same venue, this may have simplified some logistics for the organisers, and it was a good way to structure content for adjacent, but not strictly overlapping, communities of interest.

  2. Again, my thanks to the organisers for letting me sit in on this meeting. As with much of my work, my goal was to learn about what's top of mind to the folks solving problems for developers in order to prioritise work on the Web Platform.

    Without giving away confidences from a closed-door meeting, I'll just say that it was refreshing to hear framework authors tell us that they need better HTML elements and that JSX's default implementations are scaling exactly as poorly ecosystem-wide as theory and my own experience suggest. This is down to React's devil-may-care attitude to memory.

    It's not unusual to see heavy GC stalls on the client as a result of Facebook's always-wrong assertion that browsers are magical and that CPU costs don't matter. But memory is a tricksy issue, and it it's a limiting factor on the server too.

    Lots to chew on from those hours, and I thank the folks who participated for their candour, which was likely made easier since nobody from the React team deigned to join.

  3. Or worse, don't.

    Luckily, some who experience the tug of conscience punch out and write about it. Any post-McKinsey tell-all will do, but Anand Giridharadas is particularly good value for money in the genre.

  4. Circular logic is a constant in discussions with frameworkists. A few classics of the genre that got dusted off in conversations over the conference:

    • "The framework makes us more productive."

      Oh? And what's the objective evidence for that productivity gain?

      Surely, if it's large as frameworkists claim, economists would have noted the effects in aggregate statistics. But we have not seen that. Indeed, there's no credible evidence that we are seeing anything more than the bog-stock gains from learning in any technical field. The combinatorial complexity of JS frameworks may, in itself, reduce those gains; we don't know, so we can't make claims either way.

      Nobody's running real studies that compare proficient HTML&CSS or jQuery developers to React developers under objective criteria. In the place of research, personal progression is frequently cited as evidence for collective gains, which is obviously nonsensical.

      Indeed, it's just gossip.

    • "But we can hire for the framework."

      😮 sigh 😮‍💨

    • "The problem isn't React, it's the developers."

      Hearing this self-accusation offered at a React conference was truly surreal.

      In a room free of discussions about real engineering constraints, victim-blaming casts a shadow of next-level cluelessness. But frameworkists soldier on, no matter the damage it does to their argument. Volume and repetition seem key to pressing this line with a straight face.

  5. A frequently missed consequence of regulators scrutinising Apple's shocking (lack of) oversight of its app store has been Apple relaxing restrictions on iOS PWAs. Previously, PWA submissions were rejected often enough to warn businesses away. But that's over now. To reach app stores on Windows, iOS, and Android, you need is a cromulent website and PWABuilder.

    For most developers, the entire raison d'être for React Native is kaput; entirely overcome by events. Not that you'd hear about it at an assemblage of Over Reactors.

  6. Instead of describing React's exclusive ownership of subtrees of the DOM, along with the introduction of a proprietary, brittle, and hard-to-integrate parallel lifecycle as a totalising framework that demands bespoke integration effort, the marketing term "composability" was substituted to describe the feeling of giving everything over to JSX-flavoured angle brackets every time a utility is needed.

  7. It has been nearly a decade since the failure of React to reliably deliver better user experiences gave rise to the "Developer Experience" bait-and-switch.

  8. Mark Erikson's talk was ground-zero for this sort of obfuscation. At the time of writing, the recording isn't up yet, but I'll update this post with analysis when it is. I don't want to heavily critique from my fallible memory.

  9. WPT continues to default desktop tests to a configuration that throttles to 5Mbps up, 1Mbps down, with 28ms of RTT latency added to each packet. All tests in this post use a somewhat faster configuration (9Mbps up and down) but with 170ms RTT to better emulate usage from marginal network locations and the effects of full pipes.

  10. I read the bundles so you don't have to.

    So what's in the main, 2.7MB (12.5MB unzipped) bk.com bundle? What follows is a stream-of-consciousness rundown as I read the pretty-printed text top-to-bottom. At the time of writing, it appears to include:

    • A sea of JS objects allocated by the output of a truly cursed "CSS-in-JS" system. As a reminder, "CSS-in-JS" systems with so-called "runtimes" are the slowest possible way to provide styling to web UI. An ominous start.

    • React Native Reanimated (no, I'm not linking to it), which generates rAF-based animations on the web in The Year of Our Lord 2025, a full five years after Safari finally dragged its ass into the 2010s and implemented the Web Animation API.

      As a result, React Native Renaimated is Jank City.

      Jank Town. George Clinton and the Parliment Jankidellic. DJ Janky Jeff. Janky Jank and the Janky Bunch. Ole Jankypants. The Undefeated Heavyweight Champion of Jank.

      You get the idea; it drops frames.

    • Redefinitions of the built-in CSS colour names, because at no point traversing the towering inferno of build tools was it possible to know that this web-targeted artefact would be deployed to, you know, browsers.

    • But this makes some sense, because the build includes React Native Web, which is exactly what it sounds like: a set of React components that emulate the web. This allows RN project to provide a subset of the layout that browsers are natively capable of, but it does not include many of the batteries that the web includes for free.

      Which really tells you everything you need to know about how teams get into this sort of mess.

    • Huge amounts of code duplication via inline strings that include the text of functions right next to the functions themselves.

      Yes, you're reading that right: some part of this toolchain is doubling up the code in the bundle, presumably for the benefit of a native debugger. Bro, do you even sourcemap?

      At this point it feels like I'm repeating myself, but none of this is necessary on the web, and none of the (many, many) compiler passes saw fit to eliminate this waste in a web-targeted build artefact.

    • Another redefinition of the built-in CSS colour names and values. In browsers that support them natively. I feel like I'm taking crazy pills.

    • A full copy of React, which is almost 10x larger than it needs to be in order to support legacy browsers and React Native.

    • Tens Hundreds of thousands of lines of auto-generated schema validation structures and repeated, useless getter functions for data that will never be validated on the client. How did this ungodly cruft get into the bundle? One guess, and it rhymes with "schmopallo".

    • Of course, no bundle this disastrous would be complete without multiple copies of polyfills for widely supported JS features like Object.assign(), class private fields, generators, spread, async iterators, and much more.

    • Inline'd WASM code, appearing as a gigantic JS array. No, this is not a joke.

    • A copy of Lottie. Obviously.

    • What looks to be the entire AWS Amplify SDK. So much for tree-shaking.

    • A userland implementation of elliptic curve cryptography primitives that are natively supported in every modern browser via Web Crypto.

    • Inline'd SVGs, but not as strings. No, that would be too efficient. They're inlined as React components.

    • A copy of the app's Web App Manifest, inline, as a string. You cannot make this up.

    Given all of this high-cost, low-quality output, it might not surprise you to learn that the browser's coverage tool reports that more than 75% of functions are totally unused after loading and clicking around a bit.

  11. I'll be the first to point out that what Brandeis is appealing to is distinct from credentialing. As a state-school dropout, that difference matters to me very personally, and it has not been edifying to see credentialism (in the form of dubious boot camps) erode both the content and form of learning in "tech" over the past few years.

    The term "professional" continues to leave me uncomfortable for many of the affective connotations it now carries. But I do believe we should all aspire to do our work in a way that is compatible with Brandeis' description of a profession. To do otherwise is to endanger any hope of self-respect and even our social licence to operate.

]]>
Safari at WWDC '25: The Ghost of Christmas Past 2025-06-13T00:00:00Z https://infrequently.org/2025/06/the-ghost-of-christmas-past/ At Apple's annual developer marketing conference, the Safari team announced a sizeable set of features that will be available in a few months. Substantially all of them are already shipped in leading-edge browsers. Here's the list, prefixed by the year that these features shipped to stable in Chromium:

In many cases, these features were available to developers even earlier via the Origin Trials mechanism. WebGPU, e.g., ran trials for a year, allowing developers to try the in-development feature on live sites in Chrome and Edge as early as September 2021.

There are features that Apple appears to be leading on in this release, but it's not clear that they will become available in Safari before Chromium-based browsers launch them, given that the announcement is about a beta:

The announced support for CSS image crossorigin() and referrerpolicy() modifiers has an unclear relationship to other browsers, judging by the wpt.fyi tests.

On balance, this is a lot of catch-up with sparse sprinklings of leadership. This makes sense, because Safari is in usually in last place when it comes to feature completeness:

A graph of features missing from only one engine. Over the past decade, Safari and WebKit have consistently brought up the caboose.
A graph of features missing from only one engine. Over the past decade, Safari and WebKit have consistently brought up the caboose.

And that is important because Apple's incredibly shoddy work impacts every single browser on iOS.

You might recall that Apple was required by the EC to enable browser engine choice for EU citizens under the Digital Markets Act. Cupertino, per usual, was extremely chill about it, threatening to end PWAs entirely and offering APIs that are inadequate or broken.

And those are just the technical obstacles that Apple has put up. The proposed contractual terms (pdf) are so obviously onerous that no browser vendor could ever accept them, and are transparently disallowed under the DMA's plain language. But respecting the plain language of the law isn't Apple's bag.

All of this is to say that Apple is not going to allow better browsers on iOS without a fight, and it remains dramatically behind the best engines in performance, security, and features. Meanwhile, we now know that Apple is likely skimming something like $19BN per year in pure profit from it's $20+BN/yr of revenue from its deal with Google. That's a 90+% profit rate, which is only reduced by the paltry amount it re-invests into WebKit and Safari.

So to recap: Apple's Developer Relations folks want you to be grateful to Cupertino for unlocking access to features that Apple has been the singular obstacle to.

And they want to you ignore the fact that for the past decade it has hobbled the web while skimming obscene profits from the ecosystem.

Don't fall for it. Ignore the gaslighting. Apple could 10x the size of the WebKit team without causing the CFO to break a sweat, and there are plenty of great browser engineers on the market today. Suppressing the web is a choice — Apple's choice — and not one that we need to feel gratitude toward.

]]>
If Not React, Then What? 2024-11-29T00:00:00Z https://infrequently.org/2024/11/if-not-react-then-what/ Over the past decade, my work has centred on partnering with teams to build ambitious products for the web across both desktop and mobile. This has provided a ring-side seat to a sweeping variety of teams, products, and technology stacks across more than 100 engagements.

While I'd like to be spending most of this time working through improvements to web APIs, the majority of time spent with partners goes to remediating performance and accessibility issues caused by “modern” frontend frameworks and the culture surrounding them. Today, these issues are most pronounced in React-based stacks.

This is disquieting because React is legacy technology, but it continues to appear in greenfield applications.

Surprisingly, some continue to insist that React is “modern.” Perhaps we can square the circle if we understand “modern” to apply to React in the way it applies to art. Neither demonstrate contemporary design and construction. They are not built for current needs and do not meet contemporary performance standards, but pose as expensive objets harkening back to an earlier era's antiquated methods.

In the hope of steering the next team away from the rocks, I've found myself penning advocacy pieces and research into the state of play, as well as giving talks to alert managers and developers of the dangers of today's frontend orthodoxy.

In short, nobody should start a new project in the 2020s based on React. Full stop.1

Code that runs on the server can be fully costed. Performance and availability of server-side systems are under the control of the provisioning organisation, and latency can be actively managed.

Code that runs on the client, by contrast, is running on The Devil's Computer.2 Almost nothing about the latency, client resources, or even API availability are under the developer's control.

Client-side web development is perhaps best conceived of as influence-oriented programming. Once code has left the datacentre, all a web developer can do is send thoughts and prayers.

As a direct consequence, an unreasonably effective strategy is to send less code. Declarative forms generate more functional UI per byte sent. In practice, this means favouring HTML and CSS over JavaScript, as they degrade gracefully and feature higher compression ratios. These improvements in resilience and reductions in costs are beneficial in compounding ways over a site's lifetime.

Stacks based on React, Angular, and other legacy-oriented, desktop-focused JavaScript frameworks generally take the opposite bet. These ecosystems pay lip service the controls that are necessary to prevent horrific proliferations of unnecessary client-side cruft. The predictable consequence are NPM-amalgamated bundles full of redundancies like core-js, lodash, underscore, polyfills for browsers that no longer exist, userland ECC libraries, moment.js, and a hundred other horrors.

This culture is so out of hand that it seems 2024's React developers are constitutionally unable to build chatbots without including all of these 2010s holdovers, plus at least one chonky MathML or TeX library in the critical path to display an <input>. A tiny fraction of query responses need to display formulas — and yet.

Tech leads and managers need to break this spell. Ownership has to be created over decisions affecting the client. In practice, this means forbidding React in new work.

This question comes in two flavours that take some work to tease apart:

  • The narrow form:

    "Assuming we have a well-qualified need for client-side rendering, what specific technologies would you recommend instead of React?"

  • The broad form:

    "Our product stack has bet on React and the various mythologies that the cool kids talk about on React-centric podcasts. You're asking us to rethink the whole thing. Which silver bullet should we adopt instead?"

Teams that have grounded their product decisions appropriately can work through the narrow form by running objective bake offs. Building multiple small PoCs to determine each approach's scaling factors and limits can even be a great deal of fun.3 It's the rewarding side of real engineering; trying out new materials under well-understood constraints to improve outcomes.

Just the prep work to run bake offs tends to generate value.

In most teams, constraints on tech stack decisions have materially shifted since they were last examined. For some, identifying use-cases reveals a reality that's vastly different than managers and tech leads expect. Gathering data on these factors allows for first-pass cuts about stack choices, winnowing quickly to a smaller set of options to run bake offs for.4

But the teams we spend the most time with don't have good reasons to stick with client-side rendering in the first place.

Many folks asking "if not React, then what?" think they're asking in the narrow form but are grappling with the broader version. A shocking fraction of decent, well-meaning product managers and engineers haven't thought through the whys and wherefores of their architectures, opting instead to go with what's popular in a responsibility fire brigade.5

For some, provocations to abandon React induce an unmoored feeling; a suspicion that they might not understand the world any more.6

Teams in this position are working through the epistemology of their values and decisions.7 How can they know their technology choices are better than the alternatives? Why should they pick one stack over another?

Many need help deciding which end of the telescope to use when examining frontend challenges because frameworkism has become the dominant creed of our discourse.

Frameworkism insists that all problems will be solved if teams just framework hard enough. This is non-sequitur, if not entirely backwards. In practice, the only thing that makes web experiences good is caring about the user experience — specifically, the experience of folks at the margins. Technologies come and go, but what always makes the difference is giving a toss about the user.

In less vulgar terms, the struggle is to convince managers and tech leads that they need to start with user needs. Or as Public Digital puts it, "design for user needs, not organisational convenience"

The essential component of this mindset shift is replacing hopes based on promises with constraints based on research and evidence. This aligns with what it means to commit wanton acts of engineering, because engineering is the practice of designing solutions for users and society under known constraints.

The opposite of engineering is imagining that constraints do not exist, or do not apply to your product. The shorthand for this is “bullshit.”

Ousting an engrained practice of bullshitting does not come easily. Frameworkism preaches that the way to improve user experiences is to adopt more (or different) tooling from within the framework's ecosystem. This provides adherents with something to do that looks plausibly like engineering, but isn't.

It can even become a totalising commitment; solutions to user problems outside the framework's expanded cinematic universe are unavailable to the frameworkist. Non-idiomatic patterns that unlock wins for users are bugs to be squashed, not insights to be celebrated. Without data or evidence to counterbalance the bullshit artist's assertions, who's to say they're wrong? So frameworkists root out and devalue practices that generate objective criteria in decision-making. Orthodoxy unmoored from measurement predictably spins into absurdity. Eventually, heresy carries heavy sanctions.

And it's all nonsense.

Realists do not wallow in abstraction-induced hallucinations about user experiences; they measure them. Realism requires reckoning with the world as it is, not as we wish it to be. In that way, realism is the opposite of frameworkism.

The most effective tools for breaking the spell are techniques that give managers a user-centred view of system performance. This can take the form of RUM data, such as Core Web Vitals (check yours now!), or lab results from well-configured test-benches (e.g., WPT). Instrumenting critical user journeys and talking through business goals are quick follow-ups that enable teams to seize the momentum and formulate business cases for change.

RUM and bench data sources are essential antidotes to frameworkism because they provide data-driven baselines to argue from, creating a shared observable reality. Instead of accepting the next increment of framework investment on faith, teams armed with data can weigh up the costs of fad chasing versus likely returns.

Prohibiting the spread of React (and other frameworkist totems) by policy is both an incredible cost savings and a helpful way to reorient teams towards delivery for users. However, better results only arrive once frameworkism itself is eliminated from decision-making. It's no good to spend the windfall from avoiding one sort of mistake on errors within the same category.

A general answer to the broad form of the problem has several parts:

  • User focus

    Decision-makers must accept that they are directly accountable for the results of their engineering choices. Either systems work well for users,8 including those at the margins, or they don't. Systems that do not perform must be replaced. No sacred cows, only problems to be solved with the appropriate application of constraints.

  • Evidence

    The essential shared commitment between management and engineering is a dedication to realism. Better evidence must win.

  • Guardrails

    Policies must be implemented to ward off hallucinatory frameworkist assertions about how better experiences are delivered. Good examples of this include the UK Government Digital Service's requirement that services be built using progressive enhancement techniques.

    Organisations can tweak guidance as appropriate — e.g., creating an escalation path for exceptions — but the important thing is to set a baseline. Evidence boiled down into policy has power.

  • Bake Offs

    No new system should be deployed without a clear list of critical user journeys. Those journeys embody what we users do most frequently, and once those definitions are in hand, teams can do bake offs to test how well various systems deliver given the constraints of the expected marginal user.

All of this casts the product manager's role in stark relief. Instead of suggesting an endless set of experiments to run (poorly), they must define a product thesis and commit to defining success as improving services for users. This will be uncomfortable. It's also the job. Graciously accept the resignations of PMs who decide managing products is not in their wheelhouse.

To see how realism and frameworkism differ in practice, it's helpful to work a few examples. As background, recall that our rubric9 for choosing technologies is based on the number of incremental updates to primary data in a session. Some classes of app, like editors, feature long sessions and many incremental updates where a local data model can be helpful in supporting timely application of updates, but this is the exception.

Sites with short average sessions cannot afford much JS up-front.
Sites with short average sessions cannot afford much JS up-front.

It's only in these exceptional instances that SPA architectures should be considered.

Very few sites will meet the qualifications to be built as an SPA

And only when an SPA architecture is required should tools designed to support optimistic updates against a local data model — including "frontend frameworks" and "state management" tools — ever become part of a site's architecture.

The choice isn't between JavaScript frameworks, it's whether SPA-oriented tools should be entertained at all.

For most sites, the answer is clearly "no".

We can examine broad classes of site to understand why this is true:

Sites built to inform should almost always be built using semantic HTML with optional progressive enhancement as necessary.

Static site generation tools like Hugo, Astro, 11ty, and Jekyll work well for many of these cases. Sites that have content that changes more frequently should look to "classic" CMSes or tools like WordPress to generate HTML and CSS.

Blogs, marketing sites, company home pages, and public information sites should minimise client-side JavaScript to the greatest extent possible. They should never be built using frameworks that are designed to enable SPA architectures.10

Informational sites have short sessions and server-owned application data models; that is, the source of truth for what's displayed on the page is always the server's to manage and own. This means that there is no need for a client-side data model abstraction or client-side component definitions that might be updated from such a data model.

Note: many informational sites include productivity components as distinct sub-applications, which can be evaluated independently. For example, CMSes such as Wordpress are comprised of two distinct surfaces; post editors that are low-traffic but high-interactivity, and published pages, which are high-traffic, low-interactivity viewers. Progressive enhancement should be considered for both, but is an absolute must for reader views which do not feature long sessions.9:1

E-commerce sites should be built using server-generated semantic HTML and progressive enhancement.

A large and stable performance gap between Amazon and its React-based competitors demonstrates how poorly SPA architectures perform in e-commerce applications. More than 70% of Walmart's traffic is mobile, making their bet on Next.js particularly problematic for the business.
A large and stable performance gap between Amazon and its React-based competitors demonstrates how poorly SPA architectures perform in e-commerce applications. More than 70% of Walmart's traffic is mobile, making their bet on Next.js particularly problematic for the business.

Many tools are available to support this architecture. Teams building e-commerce experiences should prefer stacks that deliver no JavaScript by default, and buttress that with controls on client-side script to prevent regressions in material business metrics.

The general form of e-commerce sites has been stable for more than 20 years:

  • Landing pages with current offers and a search function for finding products.
  • Search results pages which allow for filtering and comparison of products.
  • Product-detail pages that host media about products, ratings, reviews, and recommendations for alternatives.
  • Cart management, checkout, and account management screens.

Across all of these page types, a pervasive login and cart status widget will be displayed. Sometimes this widget, and the site's logo, are the only consistent elements.

Long experience demonstrates very little shared data across these page types, highly variable session lengths, and a need for fresh content (e.g., prices) from the server. The best way to reduce latency in e-commerce sites is to optimise for lightweight, server-generated pages. Aggressive caching, image optimisation, and page-weight reduction strategies all help.

Media consumption sites vary considerably in session length and data update potential. Most should start as progressively-enhanced markup-based experiences, adding complexity over time as product changes warrant it.

Many interactive elements on media consumption sites can be modelled as distinct islands of interactivity (e.g., comment threads). Many of these components present independent data models and can therefore be constructed as progressively-enhanced Web Components within a larger (static) page.

This model breaks down when media playback must continue across media browsing (think "mini-player" UIs). A fundamental limitation of today's web platform is that it is not possible to preserve some elements from a page across top-level navigations. Sites that must support features like this should consider using SPA technologies while setting strict guardrails for the allowed size of client-side JS per page.

Another reason to consider client-side logic for a media consumption app is offline playback. Managing a local (Service Worker-backed) media cache requires application logic and a way to synchronise information with the server.

Lightweight SPA-oriented frameworks may be appropriate here, along with connection-state resilient data systems such as Zero or Y.js.

Social media apps feature significant variety in session lengths and media capabilities. Many present infinite-scroll interfaces and complex post editing affordances. These are natural dividing lines in a design that align well with session depth and client-vs-server data model locality.

Most social media experiences involve a small, fixed number of actions on top of a server-owned data model ("liking" posts, etc.) as well as distinct update phase for new media arriving at an interval. This model works well with a hybrid approach as is found in Hotwire and many HTMX applications.

Islands of deep interactivity may make sense in social media applications, and aggressive client-side caching (e.g., for draft posts) may aid in building engagement. It may be helpful to think of these as unique app sections with distinct needs from the main site's role in displaying content.

Offline support may be another reason to download a snapshot of user data to the client. This should be as part of an approach that builds resilience against flaky networks. Teams in this situation should consider a Service Worker-based, multi-page apps with "stream stitching". This allows sites to stick with HTML, while enabling offline-first logic and synchronisation. Because offline support is so invasive to an architecture, this requirement must be identified up-front.

Note: Many assume that SPA-enabling tools and frameworks are required to build compelling Progressive Web Apps that work well offline. This is not the case. PWAs can be built using stream-stitching architectures that apply the equivalent of server-side templating to data on the client, within a Service Worker.

With the advent of multi-page view transitions, MPA architecture PWAs can present fluid transitions between user states without heavyweight JavaScript bundles clogging up the main thread. It may take several more years for the framework community to digest the implications of these technologies, but they are available today and work exceedingly well, both as foundational architecture pieces and as progressive enhancements.

Document-centric productivity apps may be the hardest class to reason about, as collaborative editing, offline support, and lightweight "viewing" modes with full document fidelity are hard product requirements.

Triage-oriented experiences (e.g. email clients) are also prime candidates for the potential benefits of SPA-based technology. But as with all SPAs, the ability to deliver a better experience hinges both on session depth and up-front payload cost. It's easy to lose this race, as this blog has examined in the past.

Editors of all sorts are a natural fit for local data models and SPA-based architectures to support modifications to them. However, the endemic complexity of these systems ensures that performance will remain a constant struggle. As a result, teams building applications in this style should consider strong performance guardrails, identify critical user journeys up-front, and ensure that instrumentation is in place to ward off unpleasant performance surprises.

Editors frequently feature many updates to the same data (e.g., for every keystroke or mouse drag). Applying updates optimistically and only informing the server asynchronously of edits can deliver a superior experience across long editing sessions.

However, teams should be aware that editors may also perform double duty as viewers and that the weight of up-front bundles may not be reasonable for both cases. Worse, it can be hard to tease viewing sessions apart from heavy editing sessions at page load time.

Teams that succeed in these conditions build extreme discipline about the modularity, phasing, and order of delayed package loading based on user needs (e.g., only loading editor components users need when they require them). Teams that get stuck tend to fail to apply controls over which team members can approve changes to critical-path payloads.

Some types of apps are intrinsically interactive, focus on access to local device hardware, or center on manipulating media types that HTML doesn't handle intrinsically. Examples include 3D CAD systems, programming editors, game streaming services, web-based games, media-editing, and music-making systems. These constraints often make client-side JavaScript UIs a natural fit, but each should be evaluated critically:

  • What are the critical user journeys?
  • How long will average sessions be?
  • Do many updates to the same data take place in a session?
  • What metrics will we track to ensure that performance remains acceptable?
  • How will we place tight controls on critical-path script and other resources?

Success in these app classes is possible on the web, but extreme care is required.

A Word On Enterprise Software: Some of the worst performance disasters I've helped remediate are from a category we can think of, generously, as "enterprise line-of-business apps". Dashboards, worfklow systems, corporate chat apps, that sort of thing.

Teams building these excruciatingly slow apps often assert that "startup performance isn't important because people start our app in the morning and keep it open all day". At the limit, this can be true, but what this attempted deflection obscures is that performance is cultural. Teams that fail to define and measure critical user journeys (include loading) always fail to manage post-load interactivity too.

The old saying "how you do anything is how you do everything" is never more true than in software usability.

One consequence of cultures that fail to put the user first are products whose usability is so poor that attributes which didn't matter at the time of sale (like performance) become reasons to switch.

If you've ever had the distinct displeasure of using Concur or Workday, you'll understand what I mean. Challengers win business from them not by being wonderful, but simply by being usable. These incumbents are powerless to respond because their problems are now rooted deeply in the behaviours they rewarded through hiring and promotion along the way. The resulting management blindspot becomes a self-reinforcing norm that no single leader can shake.

This is why it's caustic to product success and brand value to allow a culture of disrespect towards users in favour of venerating developers (e.g., "DX"). The only antidote is to stamp it out wherever it arises by demanding user-focused realism in decision making.

To get unstuck, managers and tech leads that become wedded to frameworkism have to work through a series of easily falsified rationales offered by Over Reactors in service of their chosen ideology. Note, as you read, that none of these protests put the user experience front-and-centre. This admission by omission is a reliable property of the conversations these sketches are drawn from.

This chestnut should be answered with the question: "for how long?"

The dominant outcome of fling-stuff-together-with-NPM, feels-fine-on-my-$3K-laptop development is to get teams stuck in the mud much sooner than anyone expects.

From major accessibility defects to brand-risk levels of lousy performance, the consequence of this approach has been crossing my desk every week for a decade. The one thing I can tell you that all of these teams and products have in common is that they are not moving faster.

Brands you've heard of and websites you used this week have come in for help, which we've dutifully provided. The general prescription is "spend a few weeks/months unpicking this Gordian knot of JavaScript."

The time spent in remediation does fix the revenue and accessibility problems that JavaScript exuberance cause, but teams are dead in the water while they belatedly add ship gates and bundle size controls and processes to prevent further regression.

This necessary, painful, and expensive remediation generally comes at the worst time and with little support, owing to the JavaScript-industrial-complex's omerta. Managers trapped in these systems experience a sinking realisation that choices made in haste are not so easily revised. Complex, inscrutable tools introduced in the "move fast" phase are now systems that teams must dedicate time to learn, understand deeply, and affirmatively operate. All the while the pace of feature delivery is dramatically reduced.

This isn't what managers think they're signing up for when accepting "but we need to move fast!"

But let's take the assertion at face value and assume a team that won't get stuck in the ditch (🤞): the idea embedded in this statement is, roughly, that there isn't time to do it right (so React?), but there will be time to do it over.

This is in direct opposition to identifying product-market-fit. After all, the way to find who will want your product is to make it as widely available as possible, then to add UX flourishes.

Teams I've worked with are frequently astonished to find that removing barriers to use opens up new markets and leads to growth in parts of a world they had under-valued.

Now, if you're selling Veblen goods, by all means, prioritise anything but accessibility. But in literally every other category, the returns to quality can be best understood as clarity of product thesis. A low-quality experience — which is what is being proposed when React is offered as an expedient — is a drag on the core growth argument for your service. And if the goal is scale, rather than exclusivity, building for legacy desktop browsers that Microsoft won't even sell you at the cost of harming the experience for the majority of the world's users is a strategic error.

To a statistical certainty, you aren't making Facebook. Your problems likely look nothing like Facebook's early 2010s problems, and even if they did, following their lead is a terrible idea.

And these tools aren't even working for Facebook (or IG, or Threads). They just happen to be a monopoly in various social categories and can afford to light money on fire. If that doesn't describe your situation, it's best not to over index on narratives premised on Facebook's perceived success.

React developers are web developers. They have to operate in a world of CSS, HTML, JavaScript, and DOM. It's inescapable. This means that React is the most fungible layer in the stack. Moving between templating systems (which is what JSX is) is what web developers have done fluidly for more than 30 years. Even folks with deep expertise in, say, Rails and ERB, can easily knock out Django or Laravel or WordPress or 11ty sites. There are differences, sure, but every web developer is a polyglot.

React knowledge is also not particularly valuable. Any team familiar with React's...baroque...conventions can easily master Preact, Stencil, Svelte, Lit, FAST, Qwik, or any of a dozen faster, smaller, reactive client-side systems that demand less mental bookkeeping.

The tech industry has just seen many of the most talented, empathetic, and user-focused engineers I know laid off for no reason other than their management couldn't figure out that there would be some mean reversion post-pandemic. Which is to say, there's a fire sale on talent right now, and you can ask for whatever skills you damn well please and get good returns.

If you cannot attract folks who know web standards and fundamentals, reach out. I'll help you formulate recs, recruiting materials, hiring rubrics, and promotion guides to value these folks the way you should: unreasonably effective collaborators that will do incredible good for your products at a fraction of the cost of solving the next problem the React community is finally acknowledging that React caused.

But even if you decide you want to run interview loops to filter for React knowledge, that's not a good reason to use it! Anyone who can master the dark thicket of build tools, typescript foibles, and the million little ways that JSX's fork of HTML and JavaScript syntax trips folks up is absolutely good enough to work in a different system.

Heck, they're already working in an ever-shifting maze of faddish churn. The treadmill is real, which means that the question isn't "will these folks be able to hit the ground running?" (answer: no, they'll spend weeks learning your specific setup regardless), it's "what technologies will provide the highest ROI over the life of our team?"

Given the extremely high costs of React and other frameworkist prescriptions, the odds that this calculus will favour the current flavour of the week over the lifetime of even a single project are vanishingly small.

It makes me nauseous to hear managers denigrate talented engineers, and there seems to be a rash of it going around. The idea that folks who come out of bootcamps — folks who just paid to learn whatever was on the syllabus — aren't able or willing to pick up some alternative stack is bollocks.

Bootcamp grads might be junior, and they are generally steeped in varying strengths of frameworkism, but they're not stupid. They want to do a good job, and it's management's job to define what that is. Many new grads might know React, but they'll learn a dozen other tools along the way, and React is by far the most (unnecessarily) complex of the bunch. The idea that folks who have mastered the horrors of useMemo and friends can't take on board DOM lifecycle methods or the event loop or modern CSS is insulting. It's unfairly stigmatising and limits the organisation's potential.

In other words, definitionally atrocious management.

For more than a decade, the core premise of frameworkism has been that client-side resources are cheap (or are getting increasingly inexpensive) and that it is, therefore, reasonable to trade some end-user performance for developer convenience.

This has been an absolute debacle. Since at least 2012, the rise of mobile falsified this contention, and (as this blog has meticulously catalogued) we are only just starting to turn the corner.

Frameworkist assertion that "everyone has fast phones" is many things, but first and foremost it's an admission that the folks offering it don't know what they're talking about — and they hope you don't either.

No business trying to make it on the web can afford what they're selling, and you are under no obligation to offer your product as sacrifice to a false god.

This is, at best, a comforting fiction.

At worst, it's a knowing falsity that serves to omit the variability in React-based stacks because, you see, React isn't one thing. It's more of a lifestyle, complete with choices to make about React itself (function components or class components?) languages and compilers (typescript or nah?), package managers and dependency tools (npm? yarn? pnpm? turbo?), bundlers (webpack? esbuild? swc? rollup?), meta-tools (vite? turbopack? nx?), "state management" tools (redux? mobx? apollo? something that actually manages state?) and so on and so forth. And that's before we discuss plugins to support different CSS transpilation, among other optional side-quests frameworkists insist are necessary.

Across more than 100 consulting engagements, I've never seen two identical React setups, save smaller cases where the defaults of Create React App were unchanged. CRA itself changed dramatically over the years before finally being removed from the React docs as the best way to get started.

There's nothing standard about any of this. It's all change, all the time, and anyone who tells you differently is not to be trusted.

Hopefully, if you've made it this far, you'll forgive a digression into how the "React is industry standard" misdirection became so embedded.

Given the overwhelming evidence that this stuff isn't even working on the sites of the titular React poster children, how did we end up with React in so many nooks and crannies of contemporary frontend?

Pushy know-it-alls, that's how. Frameworkists have a way of hijacking every conversation with assertions like "virtual DOM is fast" without ever understanding anything about how browsers work, let alone the GC costs of their (extremely chatty) alternatives. This same ignorance allows them to confidently assert that React is "fine" when cheaper alternatives exist in every dimension.

These are not serious people. You do not have to entertain arguments offered without evidence. But you do have to oppose them and create data-driven structures that put users first. The long-term costs of these errors are enormous, as witnessed by the parade of teams needing our help to achieve minimally decent performance using stacks that were supposed to be "performant" (sic).

Which part, exactly? Be extremely specific. Which packages are so valuable, yet wedded entirely to React, that a team should not entertain alternatives? Do they really not work with Preact? How much money is exactly the right amount to burn to use these libraries? Because that's the debate.

Even if you get the benefits of "the ecosystem" at Time 0, why do you think that will continue to pay out at T+1? Or T+N?

Every library presents a separate, stochastic risk of abandonment. Even the most heavily used systems fall out of favour with the JavaScript-industrial-complex's in-crowd. This strands teams in the same position they'd have been in if they accepted ownership of more of the stack up-front, but with less experience and agency. Is that a good trade? Does your boss agree?

And how's that "CSS-in-JS" adventure working out? Still writing class components, or did you have a big forced (and partial) migration that's still creating headaches?

The truth is that every single package that is part of a repo's devDependencies is, or will be, fully owned by the consumer of the package. The only bulwark against uncomfortable surprises is to consider NPM dependencies a high-interest loan collateralized by future engineering capacity.

The best way to prevent these costs spiralling out of control is to fully examine and approve each and every dependency for UI tools and build systems. If your team is not comfortable agreeing to own, patch, and improve every single one of those systems, they should not be part of your stack.

Do you feel lucky, punk? Do you?

You'll have to be lucky to beat the odds.

Sites built with Next.js perform materially worse than those from HTML-first systems like 11ty, Astro, et al.

It simply does not scale, and the fact that it drags React behind it like a ball and chain is a double demerit. The chonktastic default payload of delay-loaded JS in any Next.js site will compete with ads and other business-critical deferred content for bandwidth, and that's before custom components and routes are added. Even when using React Server Components. Which is to say, Next.js is a fast way to lose a lot of money while getting locked in to a VC-backed startup's proprietary APIs.

Next.js starts bad and only gets worse from a shocking baseline. No wonder the only Next sites that seem to perform well are those that enjoy overwhelmingly wealthy user bases, hand-tuning assistance from Vercel, or both.

So, do you feel lucky?

React Native is a good way to make a slow app that requires constant hand-tuning and an excellent way to make a terrible website. It has also been abandoned by it's poster children.

Companies that want to deliver compelling mobile experiences into app stores from the same codebase as their website are better served investigating Trusted Web Activities and PWABuilder. If those don't work, Capacitor and Cordova can deliver similar benefits. These approaches make most native capabilities available, but centralise UI investment on the web side, providing visibility and control via a single execution path. This, in turn, reduces duplicate optimisation and accessibility headaches.

These are essential guides for frontend realism. I recommend interested tech leads, engineering managers, and product managers digest them all:

These pieces are from teams and leaders that have succeeded in outrageously effective ways by applying the realist tenants of looking around for themselves and measuring. I wish you the same success.

Thanks to Mu-An Chiou, Hasan Ali, Josh Collinsworth, Ben Delarre, Katie Sylor-Miller, and Mary for their feedback on drafts of this post.

FOOTNOTES

  1. Why not React? Dozens of reasons, but a shortlist must include:

    • React is legacy technology. It was built for a world where IE 6 still had measurable share, and it shows.
    • Virtual DOM was never fast.
      • React was forced to back away from misleading performance claims almost immediately.11
      • In addition to being unnecessary to achieve reactivity, React's diffing model and poor support for dataflow management conspire to regularly generate extra main-thread work in the critical path. The "solution" is to learn (and zealously apply) a set of extremely baroque, React-specific solutions to problems React itself causes.
      • The only (positive) contribution to performance that React's doubled-up work model can, in theory, provide is a structured lifecycle that helps programmers avoid reading back style and layout information at the moments when it's most expensive.
      • In practice, React does not prevent forced layouts and is not able to even warn about them. Unsurprisingly, every React app that crosses my desk is littered with layout thrashing bugs.
      • The only defensible performance claims Reactors make for their work-doubling system are phrased as a trade; e.g. "CPUs are fast enough now that we can afford to do work twice for developer convenience."
        • Except they aren't. CPUs stopped getting faster about the same time as Reactors began to perpetuate this myth. This did not stop them from pouring JS into the ecosystem as though the old trends had held, with predictably disasterous results Sales volumes of the high-end devices that continues to get faster stagnated over the past decade. Meanwhile, the low end exploded in volume whole remaining stubbornly fixed in performance.
        • It isn't even necessary to do all the work twice to get reactivity! Every other reactive component system from the past decade is significantly more efficient, weighs less on the wire, and preserves the advantages of reactivitiy without creating horrible "re-render debugging" hunts that take weeks away from getting things done.
    • React's thought leaders have been wrong about frontend's constraints for more than a decade.
    • The money you'll save can be measured in truck-loads.
      • Teams that correctly cabin complexity to the server side can avoid paying inflated salaries to begin with.
      • Teams that do build SPAs can more easily control the costs of those architectures by starting with a cheaper baseline and building a mature performance culture into their organisations from the start.
    • Not for nothing, but avoiding React will insulate your team from the assertion-heavy, data-light React discourse.

    Why pick a slow, backwards-looking framework whose architecture is compromised to serve legacy browsers when smaller, faster, better alternatives with all of the upsides (and none of the downsides) have been production-ready and successful for years?

  2. Frontend web development, like other types of client-side programming, is under-valued by "generalists" who do not respect just how freaking hard it is to deliver fluid, interactive experiences on devices you don't own and can't control. Web development turns this up to eleven, presenting a wicked effective compression format for UIs (HTML & CSS) but forces experiences to load at runtime across high-latency, narrowband connections. To low-end devices. With no control over which browser will execute the code.

    And yet, browsers and web developers frequently collude to deliver outstanding interactivity under these conditions. Often enough, that "generalists" don't give a second thought to the miracle of HTML-centric Wikipedia and MDN articles loading consistently quickly, as they gleefully clog those narrow pipes with JavaScript payloads so large that they can't possibly deliver similarly good experiences. All because they neither understand nor respect client-side constraints.

    It's enough to make thoughtful engineers tear their hair out.

  3. Tom Stoppard's classic quip that "it's not the voting that's democracy; it's the counting" chimes with the importance of impartial and objective criteria for judging the results of bake offs.

    I've witnessed more than my fair share of stacked-deck proof-of-concept pantomimes, often inside large organisations with tremendous resources and managers who say all the right things. But honesty demands more than lip service.

    Organisations looking for a complicated way to excuse pre-ordained outcomes should skip the charade. It will only make good people cynical and increase resistance. Teams that want to set bales of benajmins on fire because of frameworkism shouldn't be afraid to say what they want.

    They were going to get it anyway; warts and all.

  4. An example of easy cut lines for teams considering contemporary development might be browser support versus bundle size.

    In 2024, no new application will need to support IE or even legacy versions of Edge. They are not a measurable part of the ecosystem. This means that tools that took the design constraints imposed by IE as a given can be discarded from consideration. The extra client-side weight they require to service IE's quirks makes them uncompetitive from a bundle size perspective.

    This eliminates React, Angular, and Ember from consideration without a single line of code being written; a tremendous savings of time and effort.

    Another example is lock-in. Do systems support interoperability across tools and frameworks? Or will porting to a different system require a total rewrite? A decent proxy for this choice is Web Components support.

    Teams looking to avoid lock-in can remove frameworks from consideration that do not support Web Components as both an export and import format. This will still leave many contenders, and management can rest assured they will not leave the team high-and-dry.14

  5. The stories we hear when interviewing members of these teams have an unmistakable buck-passing flavour. Engineers will claim (without evidence) that React is a great13 choice for their blog/e-commerce/marketing-microsite because "it needs to be interactive" — by which they mean it has a Carousel and maybe a menu and some parallax scrolling. None of this is an argument for React per se, but it can sound plausible to managers who trust technical staff about technical matters.

    Others claim that "it's an SPA". But should it be a Single Page App? Most are unprepared to answer that question for the simple reason they haven't thought it through.9:2

    For their part, contemporary product managers seem to spend a great deal of time doing things that do not have any relationship to managing the essential qualities of their products.

    Most need help making sense of the RUM data already available to them. Few are in touch with device and network realities of their current and future (🤞) users. PMs that clearly articulate critical-user-journeys for their teams are like hen's teeth. And I can count on one hand teams that have run bake offs — without resorting to binary.

  6. It's no exaggeration to say that team leaders encountering evidence that their React (or Angular, etc.) technology choices are letting down users and the business go through some things.

    Following the herd is an adaptation to prevent their specific decisions from standing out — tall poppies and all that — and it's uncomfortable when those decisions receive belated scrutiny. But when the evidence is incontrovertible, needs must. This creates cognitive dissonance.

    Few are so entitled and callous that they wallow in denial. Most want to improve. They don't come to work every day to make a bad product; they just thought the herd knew more than they did. It's disorienting when that turns out not to be true. That's more than understandable.

    Leaders in this situation work through the stages of grief in ways that speak to their character.

    Strong teams own the reality and look for ways to learn more about their users and the constraints that should shape product choices. The goal isn't to justify another rewrite, but to find targets the team should work towards, breaking down the problem into actionable steps. This is hard and often unfamiliar work, but it is rewarding. Setting accurate goalposts helps teams take credit as they make progress remediating the current mess. These are all markers of teams on the way to improving their performance management maturity.

    Some get stuck in anger, bargaining, or depression. Sadly, these teams are taxing to help. Supporting engineers and PMs through emotional turmoil is a big part of a performance consultant's job. The stronger the team's attachment to React community narratives, the harder it can be to accept responsibility for defining team success in terms of user success. But that's the only way out of the deep hole they've dug.

    Consulting experts can only do so much. Tech leads and managers that continue to prioritise "Developer Experience" (without metrics, obviously) and "the ecosystem" (pray tell, which parts?) in lieu of user outcomes can remain beyond reach, no matter how much empathy and technical analysis is provided. Sometimes, you have to cut bait and hope time and the costs of ongoing failure create the necessary conditions for change.

  7. Most are substituting (perceived) popularity for the work of understanding users and their needs. Starting with user needs creates constraints to work backwards from.

    Instead of doing this work-back, many sub in short-term popularity contest winners. This goes hand-in-glove with a predictable failure to deeply understand business goals.

    It's common to hear stories of companies shocked to find the PHP/Python/etc. systems they are replacing with React will require multiples of currently allocated server resources for the same userbase. The impacts of inevitably worse client-side lag cost dearly, but only show up later. And all of these costs are on top of the salaries for the bloated teams frameworkists demand.

    One team shared that avoidance of React was tantamount to a trade secret. If their React-based competitors understood how expensive React stacks are, they'd lose their (considerable) margin advantage. Wild times.

  8. UIs that works well for all users aren't charity, they're hard-nosed business choices about market expansion and development cost.

    Don't be confused: every time a developer makes a claim without evidence that a site doesn't need to work well on a low-end device, understand it as a true threat to your product's success, if not your own career.

    The point of building a web experience is to maximize reach for the lowest development outlay, otherwise you'd build a bunch of native apps for every platform instead. Organisations that aren't spending bundles to build per-OS proprietary apps...well...aren't doing that. In this context, unbacked claims about why it's OK to exclude large swaths of the web market to introduce legacy desktop-era frameworks designed for browsers that don't exist any more work directly against strategy. Do not suffer them gladly.

    In most product categories, quality and reach are the product attributes web developers can impact most directly. It's wasteful, bordering insubbordinate, to suggest that not delivering those properties is an effective use of scarce funding.

  9. Should a site be built as a Single Page App?

    A good way to work this question is to ask "what's the point of an SPA?". The answer is that they can (in theory) reduce interaction latency, which implies many interactions per session. It's also an (implicit) claim about the costs of loading code up-front versus on-demand. This sets us up to create a rule of thumb.

    Should this site be built as a Single Page App? A decision tree. (hint: at best, maybe)

    Sites should only be built as SPAs, or with SPA-premised technologies if and only if:

    • They are known to have long sessions (more than ten minutes) on average
    • More than ten updates are applied to the same (primary) data

    This instantly disqualifies almost every e-commerce experience, for example, as sessions generally involve traversing pages with entirely different primary data rather than updating a subset of an existing UI. Most also feature average sessions that fail the length and depth tests. Other common categories (blogs, marketing sites, etc.) are even easier to disqualify. At most, these categories can stand a dose of progressive enhancement (but not too much!) owing to their shallow sessions.

    What's left? Productivity and social apps, mainly.

    Of course, there are many sites with bi-modal session types or sub-apps, all of which might involve different tradeoffs. For example, a blogging site is two distinct systems combined by a database/CMS. The first is a long-session, heavy interaction post-writing and editing interface for a small set of users. The other is a short-session interface for a much larger audience who mostly interact by loading a page and then scrolling. As the browser, not developer code, handles scrolling, we omit from interaction counts. For most sessions, this leaves us only a single data update (initial page load) to divide all costs by.

    If the denominator of our equation is always close to one, it's nearly impossible to justify extra weight in anticipation of updates that will likely never happen.12

    To formalise slightly, we can understand average latency as the sum of latencies in a session, divided by the number of interactions. For multi-page architectures, a session's average latency (Lavg) is simply a session's summed LCP's divided by the number of navigations in a session (N):

    L m avg = i = 1 N LCP ( i ) N

    SPAs need to add initial navigation latency to the latencies of all other session interactions (I). The total number of interactions in a session N is:

    N=1+I

    The general form is of SPA average latency is:

    L avg = latency ( navigation ) + i = 1 I latency ( i ) N

    We can handwave a bit and use INP for each individual update (via the Performance Timeline) as our measure of in-page update lag. This leaves some room for gamesmanship — the React ecosystem is famous for attempting to duck metrics accountability with scheduling shenanigans — so a real measurement system will need to substitute end-to-end action completion (including server latency) for INP, but this is a reasonable bootstrap.

    INP also helpfully omits scrolling unless the programmer does something problematic. This is correct for the purposes of metric construction as scrolling gestures are generally handled by the browser, not application code, and our metric should only measure what developers control. SPA average latency simplifies to:

    L s avg = LCP + i = 1 I INP ( i ) N

    As a metric for architecture, this is simplistic and fails to capture variance, which SPA defenders will argue matters greatly. How might we incorporate it?

    Variance (σ2) across a session is straightforward if we have logs of the latencies of all interactions and an understanding of latency distributions. Assuming latencies follows the Erlang distribution, we would have to work to assess variance, except that complete logs simplify this to the usual population variance formula. Standard deviation (σ) is then just the square root:

    σ 2 = ( x - μ ) 2 N

    Where μ is the mean (average) of the population X, the set of measured latencies in a session, with this value summed across all sessions.

    We can use these tools to compare architectures and their outcomes, particularly the effects of larger up-front payloads for SPA architecture for sites with shallow sessions. Suffice to say, the smaller the deonominator (i.e., the shorter the session), the worse average latency will be for JavaScript-oriented designs and the more sensitive variance will be to population-level effects of hardware and networks.

    A fuller exploration will have to wait for a separate post.

  10. Certain frameworkists will claim that their framework is fine for use in informational scenarios because their systems do "Server-Side Rendering" (a.k.a., "SSR").

    Parking for a moment discussion of the linguistic crime that "SSR" represents, we can reject these claims by substituting a test: does the tool in question send a copy of a library to support SPA navigations down the wire by default?

    This test is helpful, as it shows us that React-based tools like Next.js are wholly unsuitable for this class of site, while React-friendly tools like Astro are appropriate.

    We lack a name for this test today, and I hope readers will suggest one.

  11. React's initial claims of good performance because it used a virtual DOM were never true, and the React team was forced to retract them by 2015. But like many zombie ideas, there seems to have been no reduction in the rate of junior engineers regurgitating this long-falsified idea as a reason to continue to choose React.

    How did such a baldly incorrect claim come to be offered in the first place? The options are unappetising; either the React team knew their work-doubling machine was not fast but allowed others to think it was, or they didn't know but should have.15

    Neither suggest the sort of grounded technical leadership that developers or businesses should invest heavily in.

  12. It should go without saying, but sites that aren't SPAs shouldn't use tools that are premised entirely on optimistic updates to client-side data because sites that aren't SPAs shouldn't be paying the cost of creating a (separate, expensive) client-side data store separate from the DOM representation of HTML.

    Which is the long way of saying that if there's React or Angular in your blogware, 'ya done fucked up, son.

  13. When it's pointed out that React is, in fact, not great in these contexts, the excuses come fast and thick. It's generally less than 10 minutes before they're rehashing some variant of how some other site is fast (without traces to prove it, obvs), and it uses React, so React is fine.

    Thus begins an infinite regression of easily falsified premises.

    The folks dutifully shovelling this bullshit aren't consciously trying to invoke Brandolini's Law in their defence, but that's the net effect. It's exhausting and principally serves to convince the challenged party not that they should try to understand user needs and build to them, but instead that you're an asshole.

  14. Most managers pay lip service to the idea of preferring reversible decisions. Frustratingly, failure to put this into action is in complete alignment with social science research into the psychology of decision-making biases (open access PDF summary).

    The job of managers is to manage these biases. Working against them involves building processes and objective frames of reference to nullify their effects. It isn't particularly challenging, but it is work. Teams that do not build this discipline pay for it dearly, particularly on the front end, where we program the devil's computer.2:1

    But make no mistake: choosing React is a one-way door; an irreversible decision that is costly to relitigate. Teams that buy into React implicitly opt into leaky abstractions like timing quirks of React's unique (as in, nobody else has one because it's costly and slow) synthentic event system and non-portable concepts like portals. React-based products are stuck, and the paths out are challenging.

    This will seem comforting, but the long-run maintenance costs of being trapped in this decision are excruciatingly high. No wonder Over Reactors believe they should command a salary premium.

    Whatcha gonna do, switch?

  15. Where do I come down on this?

    My interactions with React team members over the years, combined with their confidently incorrect public statements about how browsers work, have convinced me that honest ignorance about their system's performance sat underneath misleading early claims.

    This was likely exascerbated by a competitive landscape in which their customers (web developers) were unable to judge the veracity of the assertions, and a deference to authority; surely Facebook wouldn't mislead folks?

    The need for an edge against Angular and other competitors also likely played a role. It's underappreciated how tenuous the position of frontend and client-side framework teams are within Big Tech companies. The Closure library and compiler that powered Google's most successful web apps (Gmail, Docs, Drive, Sheets, Maps, etc.) was not staffed for most of its history. It was literally a 20% project that the entire company depended on. For the React team to justify headcount within Facebook, public success was likely essential.

    Understood in context, I don't entirely excuse the React team for their early errors, but they are understandable. What's not forgivable are the material and willful omissions by Facebook's React team once the evidence of terrible performance began to accumulate. The React team took no responsibility, did not explain the constraints that Facebook applied to their JavaScript-based UIs to make them perform as well as they do — particularly on mobile — and benefited greatly from pervasive misconceptions that continue to cast React is a better light than hard evidence can support.

]]>
Platform Strategy and Its Discontents 2024-10-27T00:00:00Z https://infrequently.org/2024/10/platforms-are-competitions/

This post is an edited and expanded version of a now-mangled Mastodon thread.

Some in the JavaScript community imagine that I harbour an irrational dislike of their tools when, in fact, I want nothing more than to stop thinking about them. Live-and-let-live is excellent guidance, and if it weren't for React et. al.'s predictably ruinous outcomes, the public side of my work wouldn't involve educating about the problems JS-first development has caused.

But that's not what strategy demands, and strategy is my job.1

I've been holding my fire (and the confidences of consulting counterparties) for most of the last decade. Until this year, I only occasionally posted traces documenting the worsening rot. I fear this has only served to make things look better than they are.

Over the past decade, my work helping teams deliver competitive PWAs gave me a front-row seat to a disturbing trend. The rate of failure to deliver usable experiences on phones was increasing over time, despite the eye-watering cost of JS-based stacks teams were reaching for. Worse and costlier is a bad combo, and the opposite of what competing ecosystems did.

Native developers reset hard when moving from desktop to mobile, getting deeply in touch with the new constraints. Sure, developing a codebase multiple times is more expensive than the web's write-once-test-everywhere approach, but at least you got speed for the extra cost.

That's not what web developers did. Contemporary frontend practice pretended that legacy-oriented, desktop-focused tools would perform fine in this new context, without ever checking if they did. When that didn't work, the toxic-positivity crowd blamed the messenger.2

Frontend's tragically timed turn towards JavaScript means the damage isn't limited to the public sector or "bad" developers. Some of the strongest engineers I know find themselves mired in the same quicksand. Today's popular JS-based approaches are simply unsafe at any speed. The rot is now ecosystem-wide, and JS-first culture owns a share of the responsibility.

But why do I care?

I want the web to win.

What does that mean? Concretely, folks should be able to accomplish most of their daily tasks on the web. But capability isn't sufficient; for the web to win in practice, users need to turn to the browser for those tasks because it's easier, faster, and more secure.

A reasonable metric of success is time spent as a percentage of time on device.3

But why should we prefer one platform over another when, in theory, they can deliver equivalently good experiences?

As I see it, the web is the only generational software platform that has a reasonable shot at delivering a potent set of benefits to users:

  • Fresh
  • Frictionless
  • Safe by default
  • Portable and interoperable
  • Gatekeeper-free (no prior restraint on publication)4
  • Standards-based, and therefore...
  • User-mediated (extensions, browser settings, etc.)
  • Open Source compatible

No other successful platform provides all of these, and others that could are too small to matter.

Platforms like Android and Flutter deliver subsets of these properties but capitulate to capture by the host OS agenda, allowing their developers to be taxed through app stores and proprietary API lock-in. Most treat user mediation like a bug to be fixed.

The web's inherent properties have created an ecosystem that is unique in the history of software, both in scope and resilience.

So why does this result in intermittent antagonism towards today's JS community?

Because the web is losing, and instead of recognising that we're all in it together, then pitching in to right the ship, the Lemon Vendors have decided that predatory delay and "I've got mine, Jack"-ism is the best response.

What do I mean by "losing"?

Going back to the time spent metric, the web is cleaning up on the desktop. The web's JTBD percentage and fraction of time spent both continue to rise as we add new capabilities to the platform, displacing other ways of writing and delivering software, one fraction of a percent every year.5

The web is desktop's indispensable ecosystem. Who, a decade ago, thought the web would be such a threat to Adobe's native app business that it would need to respond with a $20BN acquisition attempt and a full-fledged version of Photoshop (real Photoshop) on the web?

Model advantages grind slowly but finely. They create space for new competitors to introduce the intrinsic advantages of their platform in previously stable categories. But only when specific criteria are met.

First and foremost, challengers need a cost-competitive channel. That is, users have to be able to acquire software that runs on this new platform without a lot of extra work. The web drops channel costs to nearly zero, assuming...

80/20 capability. Essential use-cases in the domain have to be (reliably) possible for the vast majority (90+%) of the TAM. Some nice-to-haves might not be there, but the model advantage makes up for it. Lastly...

It has to feel good. Performance can't suck for core tasks.6 It's fine for UI consistency with native apps to wander a bit.7 It's even fine for there to be a large peak performance delta. But the gap can't be such a gulf that it generally changes the interaction class of common tasks.

So if the web is meeting all these requirements on desktop – even running away with the lead – why am I saying "the web is losing"?

Because more than 75% of new devices that can run full browsers are phones. And the web is getting destroyed on mobile.

Utterly routed.

It's not going well
It's not going well

This is what I started warning about in 2019, and more recently on this blog. The terrifying data I had access to five years ago is now visible from space.

Public data <a href='https://vimeo.com/364402896'>shows what I warned about, citing Google-private data, in 2019.</a> In the US, time spent in browsers continues to stagnate while smartphone use grows, and the situation is even more dire outside the states. The result is a falling fraction of time spent. This is not a recipe for a healthy web.
Public data shows what I warned about, citing Google-private data, in 2019. In the US, time spent in browsers continues to stagnate while smartphone use grows, and the situation is even more dire outside the states. The result is a falling fraction of time spent. This is not a recipe for a healthy web.

If that graph looks rough-but-survivable, understand that it's only this high in the US (and other Western markets) because the web was already a success in those geographies when mobile exploded.

That history isn't shared in the most vibrant growth markets, meaning the web has fallen from "minuscule" to "nonexistent" as a part of mobile-first daily life globally.

This is the landscape. The web is extremely likely to get cast in amber and will, in less than a technology generation, become a weird legacy curio.

What happens then? The market for web developers will stop expanding, and the safe, open, interoperable, gatekeeper-free future for computing will be entirely foreclosed — or at least the difficulty will go from "slow build" to "cold-start problem"; several orders of magnitude harder (and therefore unlikely).

This failure has many causes, but they're all tractable. This is why I have worked so hard to close the capability gaps with Service Workers, PWAs, Notifications, Project Fugu, and structural solutions to the governance problems that held back progress. All of these projects have been motivated by the logic of platform competition, and the urgency that comes from understanding that that web doesn't have a natural constituency.8

If you've read any of my writing over this time, it will be unsurprising that this is why I eventually had to break silence and call out what Apple has done on iOS, and what Facebook and Android have done to more quietly undermine browser choice.

These gatekeepers are kneecapping the web in different, but overlapping and reinforcing ways. There's much more to say here, but I've tried to lay out the landscape over the past few years,. But even if we break open the necessary 80/20 capabilities and restart engine competition, today's web is unlikely to succeed on mobile.

Web developers and browsers have capped the web's mobile potential by ensuring it will feel terrible on the phones most folks have. A web that can win is a web that doesn't feel like sludge. And today it does.

This failure has many fathers. Browsers have not done nearly enough to intercede on users' behalf; hell, we don't even warn users that links they tap on might take them to sites that lock up the main thread for seconds at a time!

Things have gotten so bad that even the extremely weak pushback on developer excess that Google's Core Web Vitals effort provides is a slow-motion earthquake. INP, in particular, is forcing even the worst JS-first lemon vendors to retreat to the server — a tacit acknowledgement that their shit stinks.

So this is the strategic logic of why web performance matters in 2024; for the web to survive, it must start to grow on mobile. For that growth to start, we need the web to be a credible way to deliver these sorts of 80/20 capability-enabled mobile experiences with not-trash performance. That depends both on browsers that don't suck (we see you, Apple) and websites that don't consistently lock up phones and drain batteries.

Toolchains and communities that retreat into the numbing comfort of desktop success are a threat to that potential.

There's (much) more for browsers to do here, but developers that want the web to succeed can start without us. Responsible, web-ecology-friendly development is more than possible today, and the great news is that it tends to make companies more money, too!

The JS-industrial-complex culture that pooh-poohs responsibility is self-limiting and a harm to our collective potential.

Nobody has ever hired me to work on performance.

It's (still) on my plate because terrible performance is a limiting factor on the web's potential to heal and grow. Spending nearly half my time diagnosing and remediating easily preventable failure is not fun. The teams I sit with are not having fun either, and goodness knows there are APIs I'd much rather be working on instead.

My work today, and for the past 8 years, has only included performance because until it's fixed the whole web is at risk.

It's actually that dire, and the research I publish indicates that we are not on track to cap our JS emissions or mitigate them with CPUs fast enough to prevent ecosystem collapse.

Contra the framework apologists, pervasive, preventable failure to deliver usable mobile experiences is often because we're dragging around IE 9 compat and long toolchains premised on outdated priors like a ball and chain.9

Things look bad, and I'd be remiss if I didn't acknowledge that it could just be too late. Apple and Google and Facebook, with the help of a pliant and credulous JavaScript community, might have succeeded where 90s-era Microsoft failed — we just don't know it yet.

But it seems equally likely that the web's advantages are just dormant. When browser competition is finally unlocked, and when web pages aren't bloated with half a megabyte of JavaScript (on average), we can expect a revolution. But we need to prepare for that day and do everything we can to make it possible.

Failure and collapse aren't pre-ordained. We can do better. We can grow the web again. But to do that, the frontend community has to decide that user experience, the web's health, and their own career prospects are more important than whatever JS-based dogma VC-backed SaaS vendors are shilling this month.

FOOTNOTES

  1. My business card says "Product Manager" which is an uncomfortable fudge in the same way "Software Engineer" was an odd fit in my dozen+ years on the Chrome team.

    My job on both teams has been somewhat closer to "Platform Strategist for the Web". But nobody hires platform strategists, and when they do, it's to support proprietary platforms. The tactics, habits of mind, and ways of thinking about platform competition for open vs. closed platforms could not be more different. Indeed, I've seen many successful proprietary-platform folks try their hand at open systems and bounce hard off the different constraints, cultures, and "soft power" thinking they require.

    Doing strategy on behalf of a collectively-owned, open system is extremely unusual. Getting paid to do it is almost unheard of. And the job itself is strange; because facts about the ecosystem develop slowly, there isn't a great deal to re-derive from current events.

    Companies also don't ask strategists to design and implement solutions in the opportunity spaces they identify. But solving problems is the only way to deliver progress, so along with others who do roughly similar work, I have camped out in roles that allow arguments about the health of the web ecosystem to motivate the concrete engineering projects necessary to light the fuse of web growth.

    Indeed, nobody asked me to work on web performance, just as nobody asked me to develop PWAs, in the same way that nobody asked me to work on the capability gap between web and native. Each one falls out of the sort of strategy analysis I'm sharing in this post for the first time. These projects are examples of the sort of work I think anyone would do once they understood the stakes and marinated in the same data.

    Luckily, inside of browser teams, I've found that largely to be true. Platform work attracts long-term thinkers, and those folks are willing to give strategy analysis a listen. This, in turn, has allowed the formation of large collaborations (like Project Fugu and Project Sidecar to tackle the burning issues that pro-web strategy analysis yields.

    Strategy without action isn't worth a damn, and action without strategy can easily misdirect scarce resources. It's a strange and surprising thing to have found a series of teams (and bosses) willing to support an oddball like me that works both sides of the problem space without direction.

    So what is it that I do for a living? Whatever working to make the web a success for another generation demands.

  2. Just how bad is it?

    This table shows the mobile Core Web Vitals scores for every production site listed on the Next.js showcase web page as of Oct 2024. It includes every site that gets enough traffic to report mobile-specific data, and ignores sites which no longer use Next.js:10

    Mobile Core Web Vitals statistics for Next.js sites from Vercel's showcase, as well as the fraction of mobile traffic to each site. The last column indicates CWV stats (LCP, INP, and CLS) that consistently passed over the past 90 days.
    Tap column headers to sort.
    Site mobile LCP (ms) INP (ms) CLS 90d pass
    Sonos 70% 3874 205 0.09 1
    Nike 75% 3122 285 0.12 0
    OpenAI 62% 2164 387 0.00 2
    Claude 30% 7237 705 0.08 1
    Spotify 28% 3086 417 0.02 1
    Nerdwallet 55% 2306 244 0.00 2
    Netflix Jobs 42% 2145 147 0.02 3
    Zapier 10% 2408 294 0.01 1
    Solana 48% 1915 188 0.07 2
    Plex 49% 1501 86 0.00 3
    Wegmans 58% 2206 122 0.10 3
    Wayfair 57% 2663 272 0.00 1
    Under Armour 78% 3966 226 0.17 0
    Devolver 68% 2053 210 0.00 1
    Anthropic 30% 4866 275 0.00 1
    Runway 66% 1907 164 0.00 1
    Parachute 55% 2064 211 0.03 2
    The Washington Post 50% 1428 155 0.01 3
    LG 85% 4898 681 0.27 0
    Perplexity 44% 3017 558 0.09 1
    TikTok 64% 2873 434 0.00 1
    Leonardo.ai 60% 3548 736 0.00 1
    Hulu 26% 2490 211 0.01 1
    Notion 4% 6170 484 0.12 0
    Target 56% 2575 233 0.07 1
    HBO Max 50% 5735 263 0.05 1
    realtor.com 66% 2004 296 0.05 1
    AT&T 49% 4235 258 0.18 0
    Tencent News 98% 1380 78 0.12 2
    IGN 76% 1986 355 0.18 1
    Playstation Comp Ctr. 85% 5348 192 0.10 0
    Ticketmaster 55% 3878 429 0.01 1
    Doordash 38% 3559 477 0.14 0
    Audible (Marketing) 21% 2529 137 0.00 1
    Typeform 49% 1719 366 0.00 1
    United 46% 4566 488 0.22 0
    Hilton 53% 4291 401 0.33 0
    Nvidia NGC 3% 8398 635 0.00 0
    TED 28% 4101 628 0.07 1
    Auth0 41% 2215 292 0.00 2
    Hostgator 34% 2375 208 0.01 1
    TFL "Have your say" 65% 2867 145 0.22 1
    Vodafone 80% 5306 484 0.53 0
    Product Hunt 48% 2783 305 0.11 1
    Invision 23% 2555 187 0.02 1
    Western Union 90% 10060 432 0.11 0
    Today 77% 2365 211 0.04 2
    Lego Kids 64% 3567 324 0.02 1
    Staples 35% 3387 263 0.29 0
    British Council 37% 3415 199 0.11 1
    Vercel 11% 2307 247 0.01 2
    TrueCar 69% 2483 396 0.06 1
    Hyundai Artlab 63% 4151 162 0.22 1
    Porsche 59% 3543 329 0.22 0
    elastic 11% 2834 206 0.10 1
    Leafly 88% 1958 196 0.03 2
    GoPro 54% 3143 162 0.17 1
    World Population Review 65% 1492 243 0.10 1
    replit 26% 4803 532 0.02 1
    Redbull Jobs 53% 1914 201 0.05 2
    Marvel 68% 2272 172 0.02 3
    Nubank 78% 2386 690 0.00 2
    Weedmaps 66% 2960 343 0.15 0
    Frontier 82% 2706 160 0.22 1
    Deliveroo 60% 2427 381 0.10 2
    MUI 4% 1510 358 0.00 2
    FRIDAY DIGITAL 90% 1674 217 0.30 1
    RealSelf 75% 1990 271 0.04 2
    Expo 32% 3778 269 0.01 1
    Plotly 8% 2504 245 0.01 1
    Sumup 70% 2668 888 0.01 1
    Eurostar 56% 2606 885 0.44 0
    Eaze 78% 3247 331 0.09 0
    Ferrari 65% 5055 310 0.03 1
    FTD 61% 1873 295 0.08 1
    Gartic.io 77% 2538 394 0.02 1
    Framer 16% 8388 222 0.00 1
    Open Collective 49% 3944 331 0.00 1
    Õhtuleht 80% 1687 136 0.20 2
    MovieTickets 76% 3777 169 0.08 2
    BANG & OLUFSEN 56% 3641 335 0.08 1
    TV Publica 83% 3706 296 0.23 0
    styled-components 4% 1875 378 0.00 1
    MPR News 78% 1836 126 0.51 2
    Me Salva! 41% 2831 272 0.20 1
    Suburbia 91% 5365 419 0.31 0
    Salesforce LDS 3% 2641 230 0.04 1
    Virgin 65% 3396 244 0.12 0
    GiveIndia 71% 1995 107 0.00 3
    DICE 72% 2262 273 0.00 2
    Scale 33% 2258 294 0.00 2
    TheHHub 57% 3396 264 0.01 1
    A+E 61% 2336 106 0.00 3
    Hyper 33% 2818 131 0.00 1
    Carbon 12% 2565 560 0.02 1
    Sanity 10% 2861 222 0.00 1
    Elton John 70% 2518 126 0.00 2
    InStitchu 27% 3186 122 0.09 2
    Starbucks Reserve 76% 1347 87 0.00 3
    Verge Currency 67% 2549 223 0.04 1
    FontBase 11% 3120 170 0.02 2
    Colorbox 36% 1421 49 0.00 3
    NileFM 63% 2869 186 0.36 0
    Syntax 40% 2531 129 0.06 2
    Frog 24% 4551 138 0.05 2
    Inflect 55% 3435 289 0.01 1
    Swoosh by Nike 78% 2081 99 0.01 1
    Passing % 38% 28% 72% 8%

    Needless to say, these results are significantly worse than those of responsible JS-centric metaframeworks like Astro or HTML-first systems like Eleventy. Spot-checking their results gives me hope.

    Failure doesn't have to be our destiny, we only need to change the way we build and support efforts that can close the capability gap.

  3. This phrasing — fraction of time spent, rather than absolute time — has the benefit of not being thirsty. It's also tracked by various parties.

    The fraction of "Jobs To Be Done" happening on the web would be the natural leading metric, but it's challenging to track.

  4. Web developers aren't the only ones shooting their future prospects in the foot.

    It's bewildering to see today's tech press capitulate to the gatekeepers. The Register and The New Stack stand apart in using above-the-fold column inches to cover just how rancid and self-dealing Apple, Google, and Facebook's suppression of the mobile web has been.

    Most can't even summon their opinion bytes to highlight how broken the situation has become, or how the alternative would benefit everyone, even though it would directly benefit those outlets.

    If the web has a posse, it doesn't include The Verge or TechCrunch.

  5. There are rarely KOs in platform competitions, only slow, grinding changes that look "boring" to a tech press that would rather report on social media contratemps.

    Even the smartphone revolution, which featured never before seen device sales rates, took most of a decade to overtake desktop as the dominant computing form-factor.

  6. The web wasn't always competitive with native on desktop, either. It took Ajax (new capabilities), Moore's Law (RIP), and broadband to make it so. There had to be enough overhang in CPUs and network availability to make the performance hit of the web's languages and architecture largely immaterial.

    This is why I continue to track the mobile device and network situation. For the mobile web to take off, it'll need to overcome similar hurdles to usability on most devices.

    The only way that's likely to happen in the short-to-medium term is for developers to emit less JS per page. And that is not what's happening.

  7. One common objection to metaplatforms like the web is that the look and feel versus "native" UI is not consistent. The theory goes that consistent affordances create less friction for users as they navigate their digital world. This is a nice theory, but in practice the more important question in the fight between web and native look-and-feel is which one users encounter more often.

    The dominant experience is what others are judged against. On today's desktops, browsers that set the pace, leaving OSes with reduced influence. OS purists decry this as an abomination, but it just is what it is. "Fixing" it gains users very little.

    This is particularly true in a world where OSes change up large aspects of their design languages every half decade. Even without the web's influence, this leaves a trail of unreconciled experiences sitting side-by-side uncomfortably. Web apps aren't an unholy disaster, just more of the same.

  8. No matter how much they protest that they love it (sus) and couldn't have succeeded without it (true), the web is, at best, an also-ran in the internal logic of today's tech megacorps. They can do platform strategy, too, and know that that exerting direct control over access to APIs is worth its weight in Californium. As a result, the preferred alternative is always the proprietary (read: directly controlled and therefore taxable) platform of their OS frameworks. API access gatekeeping enables distribution gatekeeping, which becomes a business model chokepoint over time.

    Even Google, the sponsor of the projects I pursued to close platform gaps, couldn't summon the organisational fortitude to inconvenience the Play team for the web's benefit. For its part, Apple froze Safari funding and API expansion almost as soon as the App Store took off. Both benefit from a cozy duopoly that forces businesses into the logic of heavily mediated app platforms.

    The web isn't suffering on mobile because it isn't good enough in some theoretical sense, it's failing on mobile because the duopolists directly benefit from keeping it in a weakened state. And because the web is a collectively-owned, reach-based platform, neither party has to have their fingerprints on the murder weapon. At current course and speed, the web will die from a lack of competitive vigour, and nobody will be able to point to the single moment when it happened.

    That's by design.

  9. Real teams, doing actual work, aren't nearly as wedded to React vs., e.g., Preact or Lit or FAST or Stencil or Qwik or Svelte as the JS-industrial-complex wants us all to believe. And moving back to the server entirely is even easier.

    React is the last of the totalising frameworks. Everyone else is living in the interoperable, plug-and-play future (via Web Components). Escape doesn't hurt, but it can be scary. But teams that want to do better aren't wanting for off-ramps.

  10. It's hard to overstate just how fair this is to the over-Reactors. The (still) more common Create React App metaframework starter kit would suffer terribly by comparison, and Vercel has had years of browser, network, and CPU progress to water down the negative consequences of Next.js's blighted architecture.

    Next has also been pitched since 2018 as "The React Framework for the Mobile Web" and "The React Framework for SEO-Friendly Sites". Nobody else's petard is doing the hoisting; these are the sites that Vercel is chosing to highlight, built using their technology, which has consistently advertised performance-oriented features like code-splitting and SSG support.

]]>
Reckoning: Part 4 — The Way Out 2024-08-16T00:00:00Z https://infrequently.org/2024/08/the-way-out/

Other posts in the series:

Frontend took ill with a bad case of JavaScript fever at the worst possible moment. The predictable consequence is a web that feels terrible most of the time, resulting in low and falling use of the web on smartphones.1

Public data <a href='https://vimeo.com/364402896'>shows what I warned about, citing Google-private data, in 2019.</a> In the US, time spent in browsers continues to stagnate while smartphone use grows, and the situation is even more dire outside the states. The result is a falling fraction of time spent. This is not a recipe for a healthy web.
Public data shows what I warned about, citing Google-private data, in 2019. In the US, time spent in browsers continues to stagnate while smartphone use grows, and the situation is even more dire outside the states. The result is a falling fraction of time spent. This is not a recipe for a healthy web.

If nothing changes, eventually, the web will become a footnote in the history of computing; a curio along side mainframes and minicomputers — never truly gone but lingering with an ashen palour and a necrotic odour.

We don't have to wait to see how this drama plays out to understand the very real consequences of JavaScript excess on users.2

Everyone in a site's production chain has agency to prevent disaster. Procurement leads, in-house IT staff, and the managers, PMs, and engineers working for contractors and subcontractors building the SNAP sites we examined all had voices that were more powerful than the users they under-served. Any of them could have acted to steer the ship away from the rocks.

Unacceptable performance is the consequence of a chain of failures to put the user first. Breaking the chain usually requires just one insistent advocate. Disasters like BenefitsCal are not inevitable.


The same failures play out in the commercial teams I sit with day-to-day. Failure is not a foregone conclusion, yet there's an endless queue of sites stuck in the same ditch, looking for help to pull themselves out. JavaScript overindulgence is always an affirmative decision, no matter how hard industry "thought leaders" gaslight us.

Marketing that casts highly volatile, serially failed frontend frameworks as "standard" or required is horse hockey. Nobody can force an entire industry to flop so often it limits its future prospects.

These are choices.

Teams that succeed resolve to stand for the users first, then explore techniques that build confidence.

So, assuming we want to put users first, what approaches can even the odds? There's no silver bullet,3 but some techniques are unreasonably effective.

Engineering managers, product managers, and tech leads can make small changes to turn the the larger ship dramatically.

First, institute critical-user-journey analysis.

Force peers and customers to agree about what actions users will take, in order, on the site most often. Document those flows end-to-end, then automate testing for them end-to-end in something like WebPageTest.org's scripting system. Then define key metrics around these journeys. Build dashboards to track performance end-to-end.

Next, reform your frontend hiring processes.

Never, ever hire for JavaScript framework skills. Instead, interview and hire only for fundamentals like web standards, accessibility, modern CSS, semantic HTML, and Web Components. This is doubly important if your system uses a framework.

The push-back to this sort of change comes from many quarters, but I can assure you from deep experience that the folks you want to hire can learn anything, so the framework on top of the platform is the least valuable part of any skills conversation. There's also a glut of folks with those talents on the market, and they're vastly underpriced vs. their effectiveness, so "ability to hire" isn't a legitimate concern. Teams that can't find those candidates aren't trying.

Some teams are in such a sorry state regarding fundamentals that they can't even vet candidates on those grounds. If that's your group, don't hestitate to reach out.

In addition to attracting the most capable folks at bargain-basement prices, delivering better work more reliably, and spending less on JavaScript treadmill taxes, publicising these criteria sends signals that will attract more of the right talent over time. Being the place that "does it right" generates compound value. The best developers want to work in teams that prize their deeper knowledge. Demonstrate that respect in your hiring process.

Next, issue every product and engineering leader cheap phones and laptops.

Senior leaders should set the expectation those devices will be used regularly and for real work, including visibly during team meetings. If we do not live as our customers, blind spots metastasise.

Lastly, climb the Performance Management Maturity ladder, starting with latency budgets for every project, based on the previously agreed critical user journeys. They are foundational in building a culture that does not backslide.

Success or failure is in your hands, literally. Others in the equation may have authority, but you have power.

Begin to use that power to make noise. Refuse to go along with plans to build YAJSD (Yet Another JavaScript Disaster). Engineering leaders look to their senior engineers for trusted guidance about what technologies to adopt. When someone inevitably proposes the React rewrite, do not be silent. Do not let the bullshit arguments and nonsense justifications pass unchallenged. Make it clear to engineering leadership that this stuff is expensive and is absolutely not "standard".

Demand bakeoffs and testing on low-end devices.

The best thing about cheap devices is they're cheap! So inexpensive that you can likely afford a low-end phone out-of-pocket, even if the org doesn't issue one. Alternatively, WebPageTest.org can generate high-quality, low-noise simulations and side-by-side comparisons of the low-end situation.

Write these comparisons into testing plans early.

Advocate for metrics and measurements that represent users at the margins.

Teams that have climbed the Performance Management Maturity ladder intuitively look at the highest percentiles to understand system performance. Get comfortable doing the same, and build that muscle in your engineering practice.

Build the infrastructure you'll need to show, rather than tell. This can be dashboards or app-specific instrumentation. Whatever it is, just build it. Nobody in a high-performing engineering organisation will be ungrateful for additional visibility.

Lastly, take side-by-side traces and wave them like a red shirt.

Remember, none of the other people in this equation are working to undercut users, but they rely on you to guide their decisions. Be a class traitor; do the right thing and speak up for users on the margins who can't be in the room where decisions are made.

If your organisation is unfamiliar with the UK Government Digital Service's excellent Service Manual, get reading.

Once everyone has put their copy down, institute the UK's progressive enhancement standard and make it an enforceable requirement in procurement.4 The cheapest architecture errors to fix are the ones that weren't committed.

Next, build out critical-user-journey maps to help bidders and in-house developers understand system health. Insist on dashboards to monitor those flows.

Use tender processes to send clear signals that proposals which include SPA architectures or heavy JS frameworks (React, Angular, etc.) will face acceptance challenges.

Next, make space in your budget to hire senior technologists and give them oversight power with teeth.

The root cause of many failures is the continuation of too-big-to-fail contracting. The antidote is scrutiny from folks versed in systems, not only requirements. An effective way to build and maintain that skill is to stop writing omnibus contracts in the first place.

Instead, farm out smaller bits of work to smaller shops across shorter timelines. Do the integration work in-house. That will necessitate maintaining enough tech talent to own and operate these systems, building confidence over time.

Reforming procurement is always challenging; old habits run deep. But it's possible to start with the very next RFP.

Today's frontend community is in crisis.

If it doesn't look that way, it's only because the instinct to deny the truth is now fully ingrained. But the crisis is incontrovertable in the data. If the web had grown at the same pace as mobile computing, mobile web browsing would be more than a 1/3 larger than it is today. Many things are holding the web back — Apple springs to mind — but pervasive JavaScript-based performance disasters are doing their fair share.

All of the failures I documented in public sector sites are things I've seen dozens of times in industry. When an e-commerce company loses tens or hundreds of millions of dollars because the new CTO fired the old guard out to make way for a bussload of Reactors, it's just (extremely stupid) business. But the consequences of frontend's accursed turn towards all-JavaScript-all-the-time are not so readily contained. Public sector services that should have known better are falling for the same malarkey.

Frontend's culture has more to answer for than lost profits; we consistently fail users and the companies that pay us to serve them because we've let unscrupulous bastards sell snake oil without consequence.

Consider the alternative.

Canadian engineers graduating college are all given an iron ring. It's a symbol of professional responsibility to society. It also recognises that every discipline must earn its social license to operate. Lastly, it serves as a reminder of the consequences of shoddy work and corner-cutting.

https://en.wikipedia.org/wiki/Iron_Ring
photo by ryan_tir

I want to be a part of a frontend culture that accepts and promotes our responsibilities to others, rather than wallowing in self-centred "DX" puffery. In the hierarchy of priorities, users must come first.

What we do in the world matters, particularly our vocations, not because of how it affects us, but because our actions improve or degrade life for others. It's hard to imagine that culture while the JavaScript-industrial-complex has seized the commanding heights, but we should try.

And then we should act, one project at a time, to make that culture a reality.

Thanks to Marco Rogers, and Frances Berriman for their encouragement in making this piece a series and for their thoughtful feedback on drafts.

FOOTNOTES

  1. Users and businesses aren't choosing apps because they love downloading apps. They're choosing them because experiences built with these technologies work as advertised as least as often as they fail.

    The same cannot be said for contemporary web development.

  2. This series is a brief, narrow tour of the consequences of these excesses. Situating these case studies in the US, I hope, can dispel the notion that "the problem" is "over there".

    It never was and still isn't. Friends, neighbours, and family all suffer when we do as terrible a job as has now become normalized in the JavaScript-first frontend conversation.

  3. Silver bullets aren't possible at the technical level, but culturally, giving a toss is always the secret ingreedient.

  4. Exceptions to a blanket policy requiring a Progressive Enhancement approach to frontend development should be carved out narrowly and only for sub-sections of progressively enhanced primary interfaces.

    Specific examples of UIs that might need islands of non-progressively enhanced, JavaScript-based UI include:

    • Visualisations, including GIS systems, complex charting, and dashboards.
    • Editors (rich text, diagramming tools, image editors, IDEs, etc.).
    • Real-time collaboration systems.
    • Hardware interfaces to legacy systems.
    • Videoconferencing.

    In cases where an exception is granted, a process must be defined to characterise and manage latency.

]]>
Reckoning: Part 3 — Caprock 2024-08-14T00:00:00Z https://infrequently.org/2024/08/caprock/

Other posts in the series:

Last time, we looked at how JavaScript-based web development compounded serving errors on US public sector service sites, slowing down access to critical services. These defects are not without injury. The pain of accessing SNAP assistance services in California, Massachusetts, Maryland, Tennessee, New Jersey, and Indiana likely isn't dominated by the shocking performance of their frontends, but their glacial delivery isn't helping.

Waits are a price that developers ask users to pay and loading spinners only buy so much time.

These SNAP application sites create hurdles to access because the departments procuring them made or green-lit inappropriate architecture choices. In fairness, those choices may have seemed reasonable given JavaScript-based development's capture of the industry.

Betting on JavaScript-based, client-side rendered architectures leads to complex and expensive tools. Judging by the code delivered over the wire, neither CalSAWS nor Deloitte understand those technologies well enough to operate them proficiently.

From long experience and a review of the metrics (pdf) the CalSAWS Joint Management Authority reports, it is plain as day that the level of organisational and cultural sophistication required to deploy a complex JavaScript frontend is missing in Sacramento:

Dora Militaru | Performance culture through the looking-glass

It's safe to assume a version of the same story played out in Annapolis, Nashville, Boston, Trenton, and Indianapolis.

JavaScript-based UIs are fundamentally more challenging to own and operate because the limiting factors on their success are outside the data center and not under the control of procuring teams. The slow, flaky networks and low-end devices that users bring to the party define the envelope of success for client-side rendered UI.

This means that any system that puts JavaScript in the critical path starts at a disadvantage. Not only does JavaScript cost 3x more in processing power, byte-for-byte, than HTML and CSS, but it also removes the browser's ability to parallelise page loading. SPA-oriented stacks also preload much of the functionality needed for future interactions by default. Preventing over-inclusion of ancilliary code generally requires extra effort; work that is not signposted up-front or well-understood in industry.

This, in turn, places hard limits on scalability that arrive much sooner than with HTML-first progressive enhancement.

Consider today's 75-percentile mobile phone1, a device like the Galaxy A50 or the Nokia G100:

The Nokia G100. It isn't fast, but it also doesn't run up-to-date Android. But for ~$170 (new, unlocked, at launch), you get a better browser than iPhones costing 10x more.
The Nokia G100. It isn't fast, but it also doesn't run up-to-date Android. But for ~$170 (new, unlocked, at launch), you get a better browser than iPhones costing 10x more.

This isn't much of a an improvement on the Moto G4 I recommended for testing in 2016, and it's light years from the top-end of the market today.

1/10th the price, 1/9th the performance.
1/10th the price, 1/9th the performance.

A device like this presents hard limits on network, RAM, and CPU resources. Because JavaScript is more expensive than HTML and CSS to process, and because SPA architectures frontload script, these devices create a cap on the scalability of SPAs.2 Any feature that needs to be added once the site's bundle reaches the cap is in tension with every other feature in the site until exotic route-based code-splitting tech is deployed.

JavaScript bundles tend to grow without constraint in the development phase and can easily tip into territory that creates an unacceptable experience for users on slow devices.

Only bad choices remain once a project has reached this state. I have worked with dozens of teams surprised to have found themselves in this ditch. They all feel slightly ashamed because they've been led to believe they're the first; that the technology is working fine for other folks.3 Except it isn't.

I can almost recite the initial remediation steps in my sleep.4

The remaining options are bad for compounding reasons:

  • Digging into an experience that has been broken with JS inevitably raises a litany of issues that all need unplanned remediation.
  • Every new feature on the backlog would add to, rather than subtract from, bundle size. This creates pressure on management as feature work grinds to a halt.
  • Removing code from the bundle involves investigating and investing in new tools and deeply learning parts of the tower of JS complexity everyone assumed was "fine".

These problems don't generally arise in HTML-first, progressively-enhanced experiences because costs are lower at every step in the process:

  • HTML and CSS are byte-for-byte cheaper for building an equivalent interface.
  • "Routing" is handled directly by the browser and the web platform through forms, links, and REST. This removes the need to load code to rebuild it.
  • Component definitions can live primarily on the server side, reducing code sent to the client.
  • Many approaches to progressive enhancement (rather than "rehydration") use browser-native Web Components, eliminating both initial and incremental costs of larger, legacy-oriented frameworks.
  • Because there isn't an assumption that script is critical to the UX, users succeed more often when it isn't there.

This model reduces the initial pressure level and keeps the temperature down by limiting the complexity of each page to what's needed.

Teams remediating underperforming JavaScript-based sites often make big initial gains, but the difficulty ramps up once egregious issues are fixed. The residual challenges highlight the higher structural costs of SPA architectures, and must be wrestled down the hard (read: "expensive") way.

Initial successes also create cognitive dissonance within the team. Engineers and managers armed with experience and evidence will begin to compare themselves to competitors and, eventually, question the architecture they adopted. Teams that embark on this journey can (slowly) become masters of their own destinies.

From the plateau of enlightenment, it's simple to look back and acknowledge that for most sites, the pain, chaos, churn, and cost associated with SPA technology stacks are entirely unnecessary. From that vantage point, a team can finally, firmly set policy.

Organisations that purchase and operate technology all have a base carrying capacity. The cumulative experience, training, and OpEx budgets of teams set the level.

Traditional web development presents a model that many organisations have learned to manage. The incremental costs of additional HTML-based frontends are well understood, from authorisation to database capacity to the intricacies of web servers

SPA-oriented frameworks? Not so much.

In practice, the complex interplay of bundlers, client-side routing mechanisms, GraphQL API endpoints, and the need to rebuild monitoring and logging infrastructure creates wholly unowned areas of endemic complexity. This complexity is experienced as a shock to the operational side of the house.

Before, developers deploying new UIs would cabin the complexity and cost within the data center, enabling mature tools to provide visibility. SPAs and client-side rendered UIs invalidate all of these assumptions. A common result is that the operational complexity of SPA-based technologies creates new, additive points of poorly monitored system failure — failures like the ones we have explored in this series.

This is an industry-wide scandal. Promoters of these technologies have not levelled with their customers. Instead, they continue to flog each new iteration as "the future" despite the widespread failure of these models outside sophisticated organisations.

The pitch for SPA-oriented frameworks like React and Angular has always been contingent — we might deliver better experiences if long chains of interactions can be handled faster on the client.

It's time to acknowledge this isn't what is happening. For most organisations building JavaScript-first, the next dollar spent after launch day is likely go towards recovering basic functionality rather than adding new features.

That's no way to run a railroad.

Public and private sector teams I consult with regularly experience ambush-by-JavaScript.

This is the predictable result of buying into frontend framework orthodoxy. That creed hinges on the idea that toweringly complex and expensive stacks of client-side JavaScript are necessary to deliver better user experiences.

But this has always been a contingent claim, at least among folks introspective enough to avoid suggesting JS frameworks for every site. Indeed, most web experiences should explicitly avoid both client-side rendered SPA UI and the component systems built to support them. Nearly all sites would be better off opting for progressive enhancement instead.

Doing otherwise is to invite the complexity fox into the OpEx henhouse. Before you know it, you're fighting with "SSR" and "islands" and "hybrid rendering" and "ISR" to get back to the sorts of results a bit of PHP or Ruby and some CSS deliver for a tenth the price.

So how can teams evaluate the appropriateness of SPAs and SPA-inspired frameworks? By revisiting the arguments offered by the early proponents of these approaches.

The entirety of SPA orthodoxy springs from the logic of the Ajax revolution. As a witness to, and early participant in, that movement, I can conclusively assert that the buzz around GMail and Google Maps and many other "Ajax web apps" was their ability to reduce latency for subsequent interactions once an up-front cost had been paid.

The logic of this trade depends, then, on the length of sessions. As we have discussed before, it's not clear that even GMail clears the bar in all cases.

The utility of the next quanta of script is intensely dependent on session depth.

Sites with short average sessions cannot afford much JS up-front.
Sites with short average sessions cannot afford much JS up-front.

Very few sites match the criteria for SPA architectures.

Questions managers can use to sort wheat from chaff:

  • Does the average session feature more than 20 updates to, or queries against, the same subset of global data?
  • Does this UI require features that naturally create long-lived sessions (e.g., chat, videoconferencing, etc.), and are those the site's primary purpose?
  • Is there a reason the experience can't be progressively enhanced (e.g., audio mixing, document editing, photo manipulation, video editing, mapping, hardware interaction, or gaming)?

Answering these questions requires understanding critical user journeys. Flows that are most important to a site or project should be written down, and then re-examined through the lens of the marginal networks and devices of the user base.

The rare use-cases that are natural fits for today's JS-first dogma include:

  • Document editors of all sorts
  • Chat and videoconferencing apps
  • Maps, geospatial, and BI visualisations

Very few sites should lead with JS-based, framework-centric development.

Teams can be led astray when sites include mutliple apps under a single umberella. The canonical example is WordPress; a blog reading experience for millions and a blog editing UI for dozens. Treating these as independent experiences with their own constraints and tech stacks is more helpful than pretending that they're actually a "single app". This is also the insight behind the "islands architecture", and transfers well to other contexts, assuming the base costs of an experience remain low.

DevTools product managers use the phrase "pit of success" to describe how they hope teams experience their tools. The alternative is the (more common) "pit of ignorance".

The primary benefit of progressive enhancement over SPAs and SPA-begotten frameworks is that they leave teams with simpler problems, closer to the metal. Those challenges require attention and focus on the lived experience, which can be remediated cheaply once identified.

The alternative is considerably worse. In a previous post I claimed that:

SPAs are "YOLO" for web development.

This is because an over-reliance on JavaScript moves responsibility for everything into the main thread in the most expensive way.

Predictably, teams whose next PR adds to JS weight rather than HTML and CSS payloads will find themselves in the drink faster, and with tougher path out.

What's gobsmacking is that so many folks have seen these bets go sideways, yet continue to participate in the pantomime of JavaScript framework conformism. These tools aren't delivering except in the breach, but nobody will say so.

And if we were only lighting the bonuses of overpaid bosses on fire through under-delivery, that might be fine. But the JavaScript-industrial-complex is now hurting families in my community trying to access the help they're entitled to.

It's not OK.

Frontend is mired in a practical and ethical tar pit.

Not only are teams paying unexpectedly large premiums to keep the lights on, a decade of increasing JavaScript complexity hasn't even delivered the better user experiences we were promised.

We are not getting better UX for the escalating capital and operational costs. Instead, the results are getting worse for folks on the margins. JavaScript-driven frontend complexity hasn't just driven out the CSS and semantic-markup experts that used to deliver usable experiences for everyone, it is now a magnifier of inequality.

As previously noted, engineering is designing under constraint to develop products that serve users and society. The opposite of engineering is bullshitting, substituting fairy tales for inquiry and evidence.

For the frontend to earn and keep its stripes as an engineering discipline, frontenders need to internalise the envelope of what's possible on most devices.

Then we must take responsibility.

Next: The Way Out.

Thanks to Marco Rogers, and Frances Berriman for their encouragement in making this piece a series and for their thoughtful feedback on drafts.

FOOTNOTES

  1. 3/4 of all devices are faster than this phone, which means 1/4 of phones are slower. Teams doing serious performance work tend to focus on even higher percentiles (P90, P95, etc.).

    The Nokia G100 is by no means a hard target. Teams aspiring to excellence should look further down the price and age curves for representative compute power. Phones with 28nm-fabbed A53 cores are still out there in volume.

  2. One response to the regressive performance of the sites enumerated here is a version of "they're just holding it wrong; everybody knows you should use Server-Side Rendering (SSR) to paint content quickly".

    Ignoring the factual inaccuracies undergirding SPA apologetics5, the promised approaches ("SSR + hydration", "concurrent mode", etc.) have not worked.

    We can definitively see they haven't worked because the arrival of INP has shocked the body politic. INP has created a disturbance in the JS ecosystem because, for the first time, it sets a price on main-thread excesses backed by real-world data.

    Teams that adopt all these techniques are still are not achieving minimally good results. This is likely why "React Server Components" exists; it represents a last-ditch effort to smuggle some of the most costly aspects of the SPA-based tech stack back to the server where it always belonged.

    At the risk of tedious repetition, what these INP numbers mean is that these are bad experiences for real users. And these bad experiences can be laid directly at the feet of tools and architectures that promised better experiences.

    Putting the lie to SPA theatrics doesn't require inventing a new, more objective value system. The only petard needed to hoist the React ecosystem into the stratosphere is its own sales pitch, which it has miserably and consistently failed to achieve in practice.

  3. The JavaScript community's omertà regarding the consistent failure of frontend frameworks to deliver reasonable results at acceptable cost is likely to be remembered as one of the most shameful aspects of frontend's lost decade.

    Had the risks been prominently signposted, dozens of teams I've worked with personally could have avoided months of painful remediation, and hundreds more sites I've traced could have avoided material revenue losses.

    Too many engineering leaders have found their teams beached and unproductive for no reason other than the JavaScript community's dedication to a marketing-over-results ethos of toxic positivity.

    Shockingly, cheerleaders for this pattern of failure have not recanted, even when confronted with the consequences. They are not trustworthy. An ethical frontend practice will never arise from this pit of lies and half-truths. New leaders who reject these excesses are needed, and I look forward to supporting their efforts.

  4. The first steps in remediating JS-based performance disasters are always the same:

    • Audit server configurations, including:
      • Check caching headers and server compression configurations.
      • Enable HTTP/2 (if not already enabled).
      • Removing extraneous critical-path connections, e.g. by serving assets from the primary rather than a CDN host.
    • Audit the contents of the main bundle and remove unneeded or under-used dependencies.
    • Implement code-splitting and dynamic imports.
    • Set bundle size budgets and implement CI/CD enforcement.
    • Form a group of senior engineers to act as a "latency council".
      • Require the group meet regularly to review key performance metrics.
      • Charter them to approve all changes that will impact latency.
      • Have them institute an actionable "IOU" system for short-term latency regression.
      • Require their collective input when drafting or grading OKRs.
    • Beg, borrow, or steal buy low-end devices for all product managers and follow up to ensure they're using them regularly.

    There's always more to explore. SpeedCurve's Performance Guide and WebPageTest.org's free course make good next steps.

  5. Apologists for SPA architectures tend to trot out arguments with the form "nobody does X any more" or "everybody knows not to do Y" when facing concrete evidence that sites with active maintenance are doing exactly the things they have recently dissavowed, proving instead that not only are wells uncapped, but the oil slicks aren't even boomed.

    It has never been true that in-group disfavour fully contains the spread of once-popular techniques. For chrissake, just look at the CSS-in-JS delusion! This anti-pattern appears in a huge fraction of the traces I look at from new projects today, and that fraction has only gone up since FB hipsters (who were warned directly by browser engineers that it was a terrible idea) finally declared it a terrible idea.

    Almost none of today's regretted projects carry disclaimers. None of the frameworks that have led to consistent disasters have posted warnings about their appropriate use. Few boosters for these technologies even outline what they had to do to stop the bleeding (and there is always bleeding) after adopting these complex, expensive, and slow architectures.

    Instead, teams are supposed to have followed every twist and turn of inscrutable faddism, spending effort to upgrade huge codebases whenever the new hotness changes.

    Of course, when you point out that this is what the apologists are saying, no-true-Scotsmanship unfurls like clockwork.

    It's irresponsibility theatre.

    Consulting with more than a hundred teams over the past eight years has given me ring-side season tickets to the touring production of this play. The first few performances contained frission, some mystery...but now it's all played out. There's no paradox — the lies by omission are fully revealed, and the workmanlike retelling by each new understudy is as charmless as the last.

    All that's left is to pen scathing reviews in the hopes that the tour closes for good.

]]>
Reckoning: Part 2 — Object Lesson 2024-08-13T00:00:00Z https://infrequently.org/2024/08/object-lesson/

Other posts in the series:

BenefitsCal is the state of California's recently developed portal for families that need to access SNAP benefits (née "food stamps"):1


BenefitsCal loading on a low-end device over a 9Mbps link with 170ms RTT latency via WebPageTest.org

Code for America's getcalfresh.org performs substantially the same function, providing a cleaner, faster, and easier-to-use alternative to California county and state benefits applications systems:


getcalfresh.org loading under the same conditions

The like-for-like, same-state comparison getcalfresh.org provides is unique. Few public sector services in the US have competing interfaces, and only Code for America was motivated to build one for SNAP applications.

getcalfresh.org finishes in 1/3 the time, becoming interactive not long after the BenefitsCal loading screen appears.

WebPageTest.org timelines document a painful progression. Users can begin to interact with getcalfresh.org before the first (of three) BenefitsCal loading screens finish (scroll right to advance):


Google's Core Web Vitals data backs up test bench assessments. Real-world users are having a challenging time accessing the site:

BenefitsCal is a poor experience on real phones.
BenefitsCal is a poor experience on real phones.

But wait! There's more. It's even worse than it looks.

The multi-loading-screen structure of BenefitsCal fakes out Chrome's heuristic for when to record Largest Contentful Paint.

On low-end devices, BenefitsCal appears to <em>almost</em> load at the 22 second mark, only to put up a second loading screen shortly after. Because this takes so long, Chromium's heuristics for Largest Contentful Paint are thrown off, incorrectly reporting the end of the first loading screen as the complete page.
On low-end devices, BenefitsCal appears to almost load at the 22 second mark, only to put up a second loading screen shortly after. Because this takes so long, Chromium's heuristics for Largest Contentful Paint are thrown off, incorrectly reporting the end of the first loading screen as the complete page.

The real-world experience is significantly worse than public CWV metrics suggest.

Getcalfresh.org uses a simpler, progressively enhanced, HTML-first architecture to deliver the same account and benefits signup process, driving nearly half of all signups for California benefits (per CalSAWS).

The results are night-and-day:2

getcalfresh.org generates almost half of the new filings to the CalSAWS system. Its relative usability presumably contributes to that success.
getcalfresh.org generates almost half of the new filings to the CalSAWS system. Its relative usability presumably contributes to that success.

And this is after the state spent a million dollars on work to achieve "GCF parity".

No failure this complete has a single father. It's easy to enumerate contributing factors from the WebPageTest.org trace, and a summary of the site's composition and caching make for a bracing read:

Type First View Repeat View
Wire (KB) Disk D:W Wire Disk D:W Cache
JS 17,435 25,865 1.5 15,950 16,754 1.1 9%
Other 1,341 1,341 1.0 1,337 1,337 1.0 1%
CSS 908 908 1.0 844 844 1.0 7%
Font 883 883 N/A 832 832 N/A 0%
Image 176 176 N/A 161 161 N/A 9%
HTML 6 7 1.1 4 4 1.0 N/A
Total 20,263 29,438 1.45 18,680 19,099 1.02 7%

The first problem is that this site relies on 25 megabytes of JavaScript (uncompressed, 17.4 MB on over the wire) and loads all of it before presenting any content to users. This would be unusably slow for many, even if served well. Users on connections worse than the P75 baseline emulated here experience excruciating wait times. This much script also increases the likelihood of tab crashes on low-end devices.3

Very little of this code is actually used on the home page, and loading the home page is presumably the most common thing users of the site do:4

Red is bad. DevTools shows less than a quarter of the JavaScript downloaded is executed.
Red is bad. DevTools shows less than a quarter of the JavaScript downloaded is executed.

As bad as that is, the wait to interact at all is made substantially worse by inept server configuration. Industry-standard gzip compression generally results in 4:1-8:1 data savings for text resources (HTML, CSS, JavaScript, and "other") depending on file size and contents. That would reduce ~28 megabytes of text, currently served in 19MB of partially compressed resources, to between 3.5MB and 7MB.

But compression is not enabled for most assets, subjecting users to wait for 19MB of content. If CalSAWS built BenefitsCal using progressive enhancement, early HTML and CSS would become interactive while JavaScript filigrees loaded in the background. No such luck for BenefitsCal users on slow connections.

For as bad as cell and land-line internet service are in dense California metros, the vast rural population experiences large areas with even less coverage.
For as bad as cell and land-line internet service are in dense California metros, the vast rural population experiences large areas with even less coverage.

Thanks to the site's React-centric, JavaScript-dependent, client-side rendered, single-page-app (SPA) architecture, nothing is usable until nearly the entire payload is downloaded and run, defeating built-in browser mitigations for slow pages. Had progressive enhancement been employed, even egregious server misconfigurations would have had a muted effect by comparison.5

Gzip compression has been industry standard on the web for more than 15 years, and more aggressive algorithms are now available. All popular web servers support compression, and some enable it by default. It's so common that nearly every web performance testing tool checks for it, including Google's PageSpeed Insights.6

Gzip would have reduced the largest script from 2.1MB to a comparatively svelte 340K; a 6.3x compression ratio:

$ gzip -k main.2fcf663c.js 
$ ls -l
> total 2.5M
> ... 2.1M Aug  1 17:47 main.2fcf663c.js
> ... 340K Aug  1 17:47 main.2fcf663c.js.gz

Not only does the site require a gobsmacking amount of data on first visit, it taxes users nearly the same amount every time they return.

Because most of the site's payload is static, the fraction cached between first and repeat views should be near 100%. BenefitsCal achieves just 7%.

This isn't just perverse; it's so far out of the norm that I struggled to understand how CalSAWS managed to so thoroughly misconfigure a web server modern enough to support HTTP/2.

The answer? Overlooked turnkey caching options in CloudFront's dashboard.7

This oversight might have been understandable at launch. The mystery remains how it persisted for nearly three years (pdf). The slower the device and network, the more sluggish the site feels. Unlike backend slowness, the effects of ambush-by-JavaScript can remain obscured by the fast phones and computers used by managers and developers.

But even if CalSAWS staff never leave the privilege bubble, there has been plenty of feedback:

A Reddit user responding to product-level concerns with: <br><em>'And it's so f'n slow.'</em>
A Reddit user responding to product-level concerns with:
'And it's so f'n slow.'

Having placed a bet on client-side rendering, CalSAWS, Gainwell, and Deloitte staff needed to add additional testing and monitoring to assess the site as customers experience it. This obviously did not happen.

The most generous assumption is they were not prepared to manage the downsides of the complex and expensive JavaScript-based architecture they chose over progressive enhancement.89

Analogous sites from other states point the way. For instance, Wisconsin's ACCESS system:

Six seconds isn't setting records, but it's a fifth as long as it takes to access BenefitsCal.

There's a lot that could be improved about WI ACCESS's performance. Fonts are loaded too late, and some of the images are too large. They could benefit from modern formats like WebP or AVIF. JavaScript could be delay-loaded and served from the same origin to reduce connection setup costs. HTTP/2 would left-shift many of the early resources fetched in this trace.

WI ACCESS harks from simpler times.
WI ACCESS harks from simpler times.

But the experience isn't on fire, listing, and taking on water.

Despite numerous opportunities for improvement, WI ACCESS's appropriate architecture keeps the site usable for all.
Despite numerous opportunities for improvement, WI ACCESS's appropriate architecture keeps the site usable for all.

Because the site is built in a progressively enhanced way, simple fixes can cheaply and quickly improve on an acceptable baseline.

Even today's "slow" networks and phones are so fast that sites can commit almost every kind of classic error and still deliver usable experiences. Sites like WI ACCESS would have felt sluggish just 5 years ago but work fine today. It takes extra effort to screw up as badly as BenefitsCal has.

To get a sense of what's truly possible, we can compare a similar service from the world leader in accessible, humane digital public services: gov.uk, a.k.a., the UK Government Digital Service (GDS).

gov.uk's Universal Credit page finishes loading before BenefitsCal's first loading screen even starts.

California enjoys a larger GDP and a reputation for technology excellence, and yet the UK consistently outperforms the Golden State's public services.

There are layered reasons for the UK's success:

The BenefitsCal omnishambles should trigger a fundamental rethink. Instead, the prime contractors have just been awarded another $1.3BN over a long time horizon. CalSAWS is now locked in an exclusive arrangement with the very folks that broke the site with JavaScript. Any attempt at fixing it now looks set to reward easily-avoided failure.

Too-big-to-fail procurement isn't just flourishing in California; it's thriving across public benefits application projects nationwide. No matter how badly service delivery is bungled, the money keeps flowing.

CalSAWS is by no means alone.

For years, I have been documenting the inequality exacerbating effects of JS-based frontend development based on the parade of private-sector failures that cross my desk.11

Over the past decade, those failures have not elicited a change in tone or behaviour from advocates for frameworks, but that might be starting to change, at least for high-traffic commercial sites.

Core Web Vitals is creating pressure on slow sites that value search engine traffic. It's less clear what effect it will have on public-sector monopsonies. The spread of unsafe-at-any-scale JavaScript frameworks into government is worrying as it's hard to imagine what will dislodge them. There's no comparison shopping for food stamps.12

The Massachusetts Executive Office of Health and Human Services (EOHHS) seems to have fallen for JavaScript marketing hook, line, and sinker.

DTA Connect is the result, a site so slow that it frequently takes me multiple attempts to load it from a Core i7 laptop attached to a gigabit network.

From the sort of device a smartphone-dependent mom might use to access the site? It's lookin' slow.


Introducing the Can You Hold Your Breath Longer Than It Takes to Load DTA Connect? Challenge.

I took this trace multiple times, as WebPageTest.org kept timing out. It's highly unusual for a site to take this long to load. Even tools explicitly designed to emulate low-end devices and networks needed coaxing to cope.

The underlying problem is by now familiar:

You don't have to be a web performance expert to understand that <a href='https://www.webpagetest.org/breakdown.php?test=240804_AiDcFG_31Y&run=1&end=visual'>10.2MB of JS</a> is a tad much, particularly when it is served without compression.
You don't have to be a web performance expert to understand that 10.2MB of JS is a tad much, particularly when it is served without compression.

Vexingly, whatever hosting infrastructure Massachusetts uses for this project throttles serving to 750KB/s. This bandwidth restriction combines with server misconfigurations to ensure the site takes forever to load, even on fast machines.13

It's a small mercy that DTA Connect sets caching headers, allowing repeat visits to load in "only" several seconds. Because of the SPA architecture, nothing renders until all the JavaScript gathers its thoughts at the speed of the local CPU.

The slower the device, the longer it takes.14

Even when everything is cached, DTA Connect takes multiple seconds to load on a low-end device owing to the time it takes to run this much JavaScript (yellow and grey in the 'Browser main thread' row).
Even when everything is cached, DTA Connect takes multiple seconds to load on a low-end device owing to the time it takes to run this much JavaScript (yellow and grey in the 'Browser main thread' row).


A page this simple, served entirely from cache, should render in much less than a second on a device like this.15

The correlation between states procuring extremely complex, newfangled JavaScript web apps and fumbling basic web serving is surprisingly high.

Case in point, the residents of Maryland wait seconds on a slow connection for megabytes of uncompressed JavaScript, thanks to the Angular 9-based SPA architecture of myMDTHINK.16


Shockingly, Maryland's myMDTHINK loads its 5.2MB of critical-path JS without gzip.

American legislators like to means test public services. In that spirit, perhaps browsers should decline to load multiple megabytes of JavaScript from developers that feel so well-to-do they can afford to skip zipping content for the benefit of others.

Tennessee, a state with higher-than-average child poverty, is at least using JavaScript to degrade the accessibility of its public services in unique ways.

Instead of misconfiguring web servers, The Volunteer State uses Angular to synchronously fetch JSON files that define the eventual UI in an onload event handler.

The enormous purple line represents four full seconds of main thread unresponsiveness.
The enormous purple line represents four full seconds of main thread unresponsiveness.

It's little better on second visit, owing to the CPU-bound nature of the problem.
It's little better on second visit, owing to the CPU-bound nature of the problem.

Needless to say, this does not read to the trained eye as competent work.

New Jersey's MyNJHelps.gov (yes, that's the actual name) mixes the old-timey slowness of multiple 4.5MB background stock photos with a nu-skool render-blocking Angular SPA payload that's 2.2MB on the wire (15.7MB unzipped), leading to first load times north of 20 seconds.

Despite serving the oversized JavaScript payload relatively well, the script itself is so slow that repeat visits take nearly 13 seconds to display fully:


What Qualcomm giveth, Angular taketh away.

Despite almost perfect caching, repeat visits take more than 10 seconds to render thanks to a slow JavaScript payload.
Despite almost perfect caching, repeat visits take more than 10 seconds to render thanks to a slow JavaScript payload.

Debugging the pathologies of this specific page are beyond the scope of this post, but it is a mystery how New Jersey managed to deploy an application that triggers a debugger; statement on every page load with DevTools open whilst also serving a 1.8MB (13.8MB unzipped) vendor.js file with no minification of any sort.

One wonders if anyone involved in the deployment of this site are developers, and if not, how it exists.

Nearly half of the 15 seconds required to load Indiana's FSSA Benefits Portal is consumed by a mountain of main-thread time burned in its 4.2MB (16MB unzipped) Angular 8 SPA bundle.

Combined with a failure to set appropriate caching headers, both timelines look identically terrible:


Can you spot the difference?

First view.
First view.

Repeat visit.
Repeat visit.

Neither can I.

Entirely defeating browser caching in 2024 takes serious effort, but Indiana has pulled it off.

The good news is that not every digital US public benefits portal has been so thoroughly degraded by JavaScript frameworks. Code for America's 2023 Benefits Enrolment Field Guide study helpfully ran numbers on many benefits portals, and a spot check shows that those that looked fine last year are generally in decent shape today.

Still, considering just the states examined in this post, one in five US residents will hit underperforming services, should they need them.

None of these sites need to be user hostile. All of them would be significantly faster if states abandoned client-side rendering, along with the legacy JavaScript frameworks (React, Angular, etc.) built to enable the SPA model.

GetCalFresh, Wisconsin, and the UK demonstrate a better future is possible today. To deliver that better future and make it stick, organisations need to learn the limits of their carrying capacity for complexity. They also need to study how different architectures fail in order to select solutions that degrade more gracefully.

Next: Caprock: Development without constraints isn't engineering.

Thanks to Marco Rogers, and Frances Berriman for their encouragement in making this piece a series and for their thoughtful feedback on drafts.

If you work on a site discussed in this post, I offer (free) consulting to public sector services. Please get in touch.

FOOTNOTES

  1. The JavaScript required to render anything on BenefitsCal embodies nearly every anti-pattern popularised (sometimes inadvertently, but no less predictably) by JavaScript influencers over the past decade, along with the most common pathologies of NPM-based, React-flavoured frontend development.

    A perusal of the code reveals:

    • Multiple reactive systems, namely React, Vue, and RxJS.
    • "Client-side routing" metadata for the entire site bundled into the main script.
    • React components for all UI surfaces across the site, including:
    • A so-called "CSS-in-JS" library that does not support compilation to an external stylesheet. This is categorically the slowest and least efficient way to style web-based UIs. On its own, it would justify remediation work.
    • Unnecessary polyfills and transpilation overhead, including:
      • class syntax transpilation.
      • Generator function transpilation and polyfills independently added to dozens of files.
      • Iterator transpilation and polyfills.
      • Standard library polyfills, including obsolete userland implementations of ArrayBuffer, Object.assign() and repeated inlining of polyfills for many others, including a litany of outdated TypeScript-generated polyfills, bloating every file.
      • Obselete DOM polyfills, including a copy of Sizzle to provide emulation for document.querySelectorAll() and a sizable colourspace conversion system, along with userland easing functions for all animations supported natively by modern CSS.
    • No fewer than 2...wait...5...no, 6 large — seemingly different! — User-Agent parsing libraries that support browsers as weird and wonderful as WebOS, Obigo, and iCab. What a delightful and unexpected blast from the past! (pdf)
    • What appears to be an HTML parser and userland DOM implementation!?!
    • A full copy of Underscore.
    • A full copy of Lodash.
    • A full copy of core-js.
    • A userland elliptic-curve cryptography implementation. Part of an on-page chatbot, naturally.
    • A full copy of Moment.js. in addition to the custom date and time parsing functions already added via bundling of the (overlarge) react-date-picker library.
    • An unnecessary OAuth library.
    • An emulated version of the Node.js buffer class, entirely redundant on modern browsers.
    • The entire Amazon Chime SDK, which includes all the code needed to do videoconferencing. This is loaded in the critical path and alone adds multiple megabytes of JS spread across dozens of webpack-chunked files. No features of the home page appear to trigger videoconferencing.
    • A full copy of the AWS JavaScript SDK, weighing 2.6MB, served separately.
    • Obviously, nothing this broken would be complete without a Service Worker that only caches image files.

    This is, to use the technical term, whack.

    The users of BenefitsCal are folks on the margins — often working families — trying to feed, clothe, and find healthcare for kids they want to give a better life. I can think of few groups that would be more poorly served by such baffling product and engineering mismanagement.

  2. getcalfresh.org isn't perfect from a performance standpoint.

    The site would feel considerably snappier if the heavy chat widget it embeds were loaded on demand with the facade pattern and if the Google Tag Manager bundle were audited to cut cruft.

  3. Browser engineers sweat the low end because that's where users are17, and when we do a good job for them, it generally translates into better experiences for everyone else too. One of the most durable lessons of that focus has been that users having a bad time in one dimension are much more likely to be experiencing slowness in others.

    Slow networks correlate heavily with older devices that have less RAM, slower disks, and higher taxes from "potentially unwanted software" (PUS). These machines may experience malware fighting with invasive antivirus, slowing disk operations to a crawl. Others may suffer from background tasks for app and OS updates that feel fast on newer machines but which drag on for hours, stealing resources from the user's real work the whole time.

    Correlated badness also means that users in these situations benefit from any part of the system using fewer resources. Because browsers are dynamic systems, reduced RAM consumption can make the system faster, both through reduced CPU load from zram, as well as rebalancing in auto-tuning algorithms to optimise for speed rather than space.

    The pursuit of excellent experiences at the margins is deep teacher about the systems we program, and a frequently humbling experience. If you want to become a better programmer or product manager, I recommend focusing on those cases. You'll always learn something.

  4. It's not surprising to see low code coverage percentages on the first load of an SPA. What's shocking is that the developers of BenefitsCal confused it with a site that could benefit from this architecture.

    To recap: the bet that SPA-oriented JavaScript frameworks make is that it's possible to deliver better experiences for users when the latency of going to the server can be shortcut by client-side JavaScript.

    I cannot stress this enough: the premise of this entire wing of web development practice is that expensive, complex, hard-to-operate, and wicked-to-maintain JavaScript-based UIs lead to better user experiences.

    It is more than fair to ask: do they?

    In the case of BenefitsCal and DTA Connect, the answer is "no".

    The contingent claim of potentially improved UI requires dividing any additional up-front latency by the number of interactions, then subtracting the average improvement-per-interaction from that total. It's almost impossible to imagine any app with sessions long enough to make 30-second up-front waits worthwhile, never mind a benefits application form.

    These projects should never have allowed "frontend frameworks" within a mile of their git repos. That they both picked React (a system with a lurid history of congenital failure) is not surprising, but it is dispiriting.

    Previous posts here have noted that site structure and critical user journeys largely constrain which architectures make sense:

    Sites with shorter average sessions can afford less JS up-front.
    Sites with short average sessions cannot afford much JS up-front.

    These portals serve many functions: education, account management, benefits signup, and status checks. None of these functions exhibit the sorts of 50+ interaction sessions of a lived-in document editor (Word, Figma) or email client (Gmail, Outlook). They are not "toothbrush" services that folks go to every day, or which they use over long sessions.

    Even the sections that might benefit from additional client-side assistance (rich form validation, e.g.) cannot justify loading all of that code up-front for all users.

    The failure to recognise how inappropriate JavaScript-based SPA architectures are for most sites is an industry-wide scandal. In the case of these services, that scandal takes on whole new dimension of reckless irresponsibility.

  5. JavaScript-based SPAs yank the reins away from the browser while simultaneously frontloading code at the most expensive time.

    SPA architectures and the frameworks built to support them put total responsibility for all aspects of site performance squarely on the shoulders of the developer. Site owners who are even occasionally less than omniscient can quickly end up in trouble. It's no wonder many teams I work with are astonished at how quickly these tools lead to disastrous results.

    SPAs are "YOLO" for web development.

    Their advocates' assumption of developer perfection is reminiscent of C/C++'s approach to memory safety. The predictable consequences should be enough to disqualify them from use in most new work. The sooner these tools and architectures are banned from the public sector, the better.

  6. Confoundingly, while CalSAWS has not figured out how to enable basic caching and compression, it has rolled out firewall rules that prevent many systems like PageSpeed Insights from evaluating the page through IP blocks.

    The same rules also prevent access from IPs geolocated to be outside the US. Perhaps it's also a misconfiguration? Surely CalSAWS isn't trying to cut off access to services for users who are temporarialy visiting family in an emergency, right?

  7. There's a lot to say about BenefitsCal's CloudFront configuration debacle.

    First, and most obviously: WTF, Amazon?

    It's great that these options are single-configuration and easy to find when customers go looking for them, but they should not have to go looking for them. The default for egress-oriented projects should be to enable this and then alert on easily detected double-compression attempts.

    Second: WTF, Deloitte?

    What sort of C-team are you stringing CalSAWS along with? Y'all should be ashamed. And the taxpayers of California should be looking to claw back funds for obscenely poor service.

    Lastly: this is on you, CalSAWS.

    As the procurer and approver of delivered work items, the failure to maintain a minimum level of in-house technical skill necessary to call BS on vendors is inexcusable.

    New and more appropriate metrics for user success should be integrated into public reporting. That conversation could consume an entire blog post; the current reports are little more than vanity metrics. The state should also redirect money it is spending with vendors to enhance in-house skills in building and maintaining these systems directly.

    It's an embarrassment that this site is as broken as it was when I began tracing it three years ago. It's a scandal that good money is being tossed after bad. Do better.

  8. It's more likely that CalSAWS are inept procurers and that Gainwell + Deloitte are hopeless developers.

    The alternative requires accepting that one or all of these parties knew better and did not act, undermining the struggling kids and families of California in the process. I can't square that with the idea of going to work every day for years to build and deliver these services.

  9. In fairness, building great websites doesn't seem to be Deloitte's passion.

    Deloitte.com performs poorly for real-world users, a population that presumably includes a higher percentage of high-end devices than other sites traced in this post.
    Deloitte.com performs poorly for real-world users, a population that presumably includes a higher percentage of high-end devices than other sites traced in this post.

    But even Deloitte could have fixed the BenefitsCal mess had CalSAWS demanded better.

  10. It rankles a bit that what the UK's GDS has put into action for the last decade is only now being recognised in the US.

    If US-centric folks need to call these things "products" instead of "services" to make the approach legible, so be it! Better late than never.

  11. I generally have not not posted traces of the private sector sites I have spent much of the last decade assisting, preferring instead to work quietly to improve their outcomes.

    The exception to this rule is the public sector, where I feel deeply cross-pressured about the sort of blow-back that underpaid civil servants may face. However, sunlight is an effective disinfectant, particularly for services we all pay for. The tipping point in choosing to post these traces is that by doing so, we might spark change across the whole culture of frontend development.

  12. getcalfresh.org is the only direct competitor I know of to a state's public benefits access portal, and today it drives nearly half of all SNAP signups in California. Per BenefitsCal meeting notes (pdf), it is scheduled to be decommissioned next year.

    Unless BenefitsCal improves dramatically, the only usable system for SNAP signup in the most populous state will disappear when it goes.

  13. Capping the effective bandwidth of a server is certainly one way to build solidarity between users on fast and slow devices.

    It does not appear to have worked.

    The glacial behaviour of the site for all implies managers in EOHHS must surely have experienced DTA Connect's slowness for themselves and declined to do anything about it.

  14. The content and structure of DTA Connect's JavaScript are just as horrifying as BenefitsCal's1:1 and served just as poorly. Pretty-printed, the main bundle runs to 302,316 lines.

    I won't attempt nearly as exhaustive inventory of the #fail it contains, but suffice to say, it's a Create React App special. CRAppy, indeed.

    Many obsolete polyfills and libraries are bundled, including (but not limited to):

    • A full copy of core-js
    • Polyfills for features as widely supported as fetch()
    • Transpilation down to ES5, with polyfills to match
    • A full userland elliptic-curve cryptography library
    • A userland implementation of BigInt
    • A copy of zlib.js
    • A full copy of the Public Suffix List
    • A full list of mime types (thousands of lines).
    • What appears to be a relatively large rainbow table.

    Seasoned engineers reading this list may break out in hives, and that's an understandable response. None of this is necessary, and none of it is useful in a modern browser. Yet all of it is in the critical path.

    Some truly unbelievable bloat is the result of all localized strings for the entire site occurring in the bundle. In every supported language.

    Any text ever presented to the user is included in English, Spanish, Portuguese, Chinese, and Vietnamese, adding megabytes to the download.

    A careless disregard for users, engineering, and society permeates this artefact. Massachusetts owes citizens better.

  15. Some junior managers still believe in the myth of the "10x" engineer, but this isn't what folks mean when they talk about "productivity". Or at least I hope it isn't.

  16. Angular is now on version 18, meaning Maryland faces a huge upgrade lift whenever it next decides to substantially improve myMDTHINK.

  17. Browsers port to macOS for CEOs, hipster developers, and the tech press. Macs are extremely niche devices owned exclusively by the 1-2%. Its ~5% browsing share is inflated by the 30% not yet online, almost none of whom will be able to afford Macs.

    Wealth-related factors also multiply the visibility of high-end devices (like Macs) in summary statistics. These include better networks and faster hardware, both of which correlate with heavier browsing. Relatively high penetration in geographies with strong web use also helps. For example, Macs have 30% share of desktop-class sales in the US, vs 15% worldwide..

    The overwhelming predominance of smartphones vs. desktops seals the deal. In 2023, smartphones outsold desktops and laptops by more than 4:1. This means that smartphones outnumber laptops and desktops to an even greater degree worldwide than they do in the US.

    Browser makers keep Linux ports ticking over because that's where developers live (including many of their own). It's also critical for the CI/CD systems that power much of the industry.

    Those constituencies are vocal and wealthy, giving them outsized influence. But iOS and and macOS aren't real life; Android and Windows are, particularly their low-end, bloatware-filled expressions.

    Them's the breaks.

]]>
Reckoning: Part 1 — The Landscape 2024-08-12T00:00:00Z https://infrequently.org/2024/08/the-landscape/

Instead of an omnibus mega-post, this investigation into JavaScript-first frontend culture and how it broke US public services has been released in four parts. Other posts in the series:


When you live in the shadow of a slow-moving crisis, it's natural to tell people about it. At volume. Doubly so when engineers can cheaply and easily address the root causes with minor tweaks. As things worsen, it's also hard not to build empathy for Cassandra.

In late 2011, I moved to London, where the Chrome team was beginning to build Google's first "real" browser for Android.1 The system default Android Browser had, up until that point, been based on the system WebView, locking its rate of progress to the glacial pace of device replacement.2

In a world where the Nexus 4's 2GB of RAM and 32-bit, 4-core CPU were the high-end, the memory savings the Android Browser achieved by reusing WebView code mattered immensely.3 Those limits presented enormous challenges for Chromium's safer (but memory-hungry) multi-process sandboxing. Android wasn't just spicy Linux; it was an entirely new ballgame.

Even then, it was clear the iPhone wasn't a fluke. Mobile was clearly on track to be the dominant form-factor, and we needed to adapt. Fast.4

Browsers made that turn, and by 2014, we had made enough progress to consider how the web could participate in mobile's app-based model. This work culminated in 2015's introduction of PWAs and Push Notifications.

Disturbing patterns emerged as we worked with folks building on this new platform. A surprisingly high fraction of them brought slow, desktop-oriented JavaScript frameworks with them to the mobile web. These modern, mobile-first projects neither needed nor could afford the extra bloat frameworks included to paper over the problems of legacy desktop browsers. Web developers needed to adapt the way browser developers had, but consistently failed to hit the mark.

By 2016, frontend practice had fully lapsed into wish-thinking. Alarms were pulled, claxons sounded, but nothing changed.

Chrome DevSummit 2016: Progressive Performance

It could not have come at a worse time.

By then, explosive growth at the low end was baked into the cake. Billions of feature-phone users had begun to trade up. Different brands endlessly reproduced 2016's mid-tier Androids under a dizzying array of names. The only constants were the middling specs and ever-cheaper prices. Specs that would set punters back $300 in 2016, sold for only $100 a few years later, opening up the internet to hundreds of millions along the way. The battle between the web and apps as the dominant platform was well and truly on.

<em>Tap for a larger version.</em><br>Geekbench 5 single-core scores for 'fastest iPhone', 'fastest Android', 'budget', and 'low-end' segments.<br><br>Nearly all growth in smartphone sales volume since the mid '10s occured in the 'budget' and 'low-end' categories.
Tap for a larger version.
Geekbench 5 single-core scores for 'fastest iPhone', 'fastest Android', 'budget', and 'low-end' segments.

Nearly all growth in smartphone sales volume since the mid '10s occured in the 'budget' and 'low-end' categories.

But the low-end revolution barely registered in web development circles. Frontenders poured JavaScript into the mobile web at the same rate as desktop, destroying any hope of a good experience for folks on a budget.

Median JavaScript bytes for Mobile and Desktop sites. <br>As this blog has <a href='/series/performance-inequality/'>covered at length,</a> median device specs were largely stagnant between 2014 and 2022. Meanwhile, web developers made sure the 'i' in 'iPhone' stood for 'inequality.'
Median JavaScript bytes for Mobile and Desktop sites.
As this blog has covered at length, median device specs were largely stagnant between 2014 and 2022. Meanwhile, web developers made sure the 'i' in 'iPhone' stood for 'inequality.'

Prices at the high end accelerated, yet average selling prices remained stuck between $300 and $350. The only way the emergence of the $1K phone didn't bump the average up was the explosive growth at the low end. To keep the average selling price at $325, three $100 low-end phones needed to sell for each $1K iPhone; which is exactly what happened.

And yet, the march of JavaScript-first, framework-centric dogma continued, no matter how incompatible it was with the new reality. Predictably, tools sold on the promise they would deliver "app-like experiences" did anything but.5

Billions of cheap phones that always have up-to-date browsers found their CPUs and networks clogged with bloated scripts designed to work around platform warts they don't have.

In 2019, Code for America published the first national-level survey of online access to benefits programs, which are built and operated by each state. The follow-up 2023 study provides important new data on the spread of digital access to benefits services.

One valuable artefact from CFA's 2019 research is a post by Dustin Palmer, documenting the missed opportunity among many online benefits portals to design for the coming mobile-first reality that was already the status quo in the rest of the world.

Worldwide mobile browsing surpassed desktop browsing sometime in 2016.
Worldwide mobile browsing surpassed desktop browsing sometime in 2016.

US browsing exhibited the same trend, slightly delayed, owing to comparatively high desktop and laptop ownership vs emerging markets.
US browsing exhibited the same trend, slightly delayed, owing to comparatively high desktop and laptop ownership vs emerging markets.

Moving these systems online only reduces administrative burdens in a contingent sense; if portals fail to work well on phones, smartphone-dependent folks are predictably excluded:

28% of US adults in households with less than $30K/yr income are smartphone-dependent, falling to only 19% for families making 30-70K/yr.
28% of US adults in households with less than $30K/yr income are smartphone-dependent, falling to only 19% for families making 30-70K/yr.

But poor design isn't the only potential administrative burden for smartphone-dependent users.6

The networks and devices folks use to access public support aren't latest-generation or top-of-the-line. They're squarely in the tail of the device price, age, and network performance distributions. Those are the overlapping conditions where the consistently falsified assumptions of frontend's lost decade have played out disastrously.

California is a rich mix of urban and hard-to-reach rural areas. Some of the poorest residents are in the least connected areas, ensuring they will struggle to use bloated sites.
California is a rich mix of urban and hard-to-reach rural areas. Some of the poorest residents are in the least connected areas, ensuring they will struggle to use bloated sites.

It would be tragic if public sector services adopted the JavaScript-heavy stacks that frontend influencers have popularised. Framework-based, "full-stack" development is now the default in Silicon Valley, but should obviously be avoided in universal services. Unwieldy and expensive stacks that have caused agony in the commercial context could never be introduced to the public sector with any hope of success.

Right?

Next: Object Lesson: a look at California's digital benefits services.

Thanks to Marco Rogers, and Frances Berriman for their encouragement in making this piece a series and for their thoughtful feedback on drafts.

FOOTNOTES

  1. A "real browser", as the Chrome team understood the term circa 2012, included:

    • Chromium's memory-hungry multi-process architecture which dramatically improved security and stability
    • Winning JavaScript performance using our own V8 engine
    • The Chromium network stack, including support for SPDY and experiments like WebRTC
    • Updates that were not locked to OS versions
  2. Of course, the Chrome team had wanted to build a proper mobile browser sooner, but Android was a paranoid fiefdom separate from Google's engineering culture and systems. And the Android team were intensely suspicious of the web, verging into outright hostility at times.

    But internal Google teams kept hitting the limits of what the Android Browser could do, including Search. And when Search says "jump", the only workable response is "how high?"

    WebKit-based though it was (as was Chrome), OS-locked features presented a familiar problem, one the Chrome team had solved with auto-update and Chrome Frame. A deal was eventually struck, and when Chrome for Android was delivered, the system WebView also became a Chromium-based, multi-process, sandboxed, auto-updating system. For most, that was job done.

    This made a certain sort of sense. From the perspective of Google's upper management, Android's role was to put a search box in front of everyone. If letting Andy et al. play around with an unproven Java-based app model was the price, OK. If that didn't work, the web would still be there. If it did, then Google could go from accepting someone else's platform to having one it owned outright.7 Win/win.

    Anyone trying to suggest a more web-friendly path for Android got shut down hard. The Android team always had legitimate system health concerns that they could use as cudgels, and they weilded them with abandon.

    The launch of PWAs in 2015 was an outcome Android saw coming a mile away and worked hard to prevent. But that's a story for another day.

  3. Android devices were already being spec'd with more RAM than contemporary iPhones, thanks to Dalvik's chonkyness. This, in turn, forced many OEMs to cut corners in other areas, including slower CPUs.

    This effect has been a silent looming factor in the past decade's divergence in CPU performance between top-end Android and iPhones. Not only did Android OEMs have to pay a distinct profit margin to Qualcomm for their chips, but they also had to dip into the Bill Of Materials (BOM) budget to afford more memory to keep things working well, leaving less for the CPU.

    Conversely, Apple's relative skimpiness on memory and burning desire to keep BOM costs low for parts it doesn't manufacture are reasons to oppose browser engine choice. If real browsers were allowed, end users might expect phones with decent specs. Apple keeps that in check, in part, by maximising code page reuse across browsers and apps that are forced to use the system WebView.

    That might dig into margins ever so slightly, and we can't have that, can we?

  4. It took browsers that were originally architected in a desktop-only world many years to digest the radically different hardware that mobile evolved. Not only were CPU speeds and memory budgets cut dramatically — nevermind the need to port to ARM, including JS engine JITs that were heavily optimised for x86 — but networks suddenly became intermittent and variable-latency.

    There were also upsides. Where GPUs had been rare on the desktop, every phone had a GPU. Mobile CPUs were slow enough that what had felt like a leisurely walk away from CPU-based rendering on desktop became an absolute necessity on phones. Similar stories played out across input devices, sensors, and storage.

    It's no exaggeration to say that the transition to mobile force-evolved browsers in a compressed time frame. If only websites had made the same transition.

  5. Let's take a minute to unpack what the JavaScript framework claims of "app-like experiences" were meant to convey.

    These were code words for more responsive UI, building on the Ajax momentum of the mid-naughties. Many boosters claimed this explicitly and built popular tools to support these specific architectures.

    As we wander through the burning wreckage of public services that adopted these technologies, remember one thing: they were supposed to make UIs better.

  6. When confronted with nearly unusable results from tools sold on the idea that they make sites easier, better, and faster to use, many technologists offer the variants of "but at least it's online!" and "it's fast enough for most people". The most insipid version implies causality, constructing a strawman but-for defense; "but these sites might not have even been built without these frameworks."9

    These points can be both true and immaterial at the same time. It isn't necessary for poor performance to entirely exclude folks at the margins for it to be a significant disincentive to accessing services.

    We know this because it has been proven and continually reconfirmed in commercial and lab settings.

  7. The web is unattractive to every Big Tech company in a hurry, even the ones that owe their existence to it.

    The web's joint custody arrangement rankles. The standards process is inscrutable and frustrating to PMs and engineering managers who have only ever had to build technology inside one company's walls. Playing on hard mode is unappealing to high-achievers who are used to running up the score.

    And then there's the technical prejudice. The web's languages offend "serious" computer scientists. In the bullshit hierarchy of programming language snobbery, everyone looks down on JavaScript, HTML, and CSS (in that order).

    The web's overwhelmingly successful languages present a paradox: for the comfort of the snob, they must simultaneously be unserious toys beneath the elevated palettes of "generalists" and also Gordian Knots too hard for anyone to possibly wield effectively. This dual posture justifies treating frontend as a less-than discipline, and browsers as anything but a serious application platform.

    This isn't universal, but it is common, particularly in Google's C++/Java-pilled upper ranks.8 Endless budgetary space for projects like the Android Framework, Dart, and Flutter were the result.

  8. Someday I'll write up the tale of how Google so thoroughly devalued frontend work that it couldn't even retain the unbelievably good web folks it hired in the mid-'00s. Their inevitable departures after years of being condescended to went hand-in-hand with an inability to hire replacements.

    Suffice to say, by the mid '10s, things were bad. So bad an exec finally noticed. This created a bit of space to fix it. A team of volunteers answered the call, and for more than a year we met to rework recruiting processes and collateral, interview loop structures, interview questions, and promotion ladder criteria.

    The hope was that folks who work in the cramped confines of someone else's computer could finally be recognised for their achievements. And for a few years, Google's frontends got markedly better.

    I'm told the mean has reasserted itself. Prejudice is an insidious thing.

  9. The but-for defense for underperforming frontend frameworks requires us to ignore both the 20 years of web development practice that preceeded these tools and the higher OpEx and CapEx costs associated with React-based stacks.

    Managers sometimes offer a hireability argument, suggesting they need to adopt these univerally more expensive and harder to operate tools because they need to be able to hire. This was always nonsense, but never more so than in 2024. Some of the best, most talented frontenders I know are looking for work and would leap at the chance to do good things in an organisation that puts user experience first.

    Others sometimes offer the idea that it would be too hard to retrain their teams. Often, these are engineering groups comprised of folks who recently retrained from other stacks to the new React hotness or who graduated boot camps armed only with these tools. The idea that either cohort cannot learn anything else is as inane as it is self-limiting.

    Frontenders can learn any framework and are constantly retraining just to stay on the treadmill. The idea that there are savings to be had in "following the herd" into Next.js or similar JS-first development cul-de-sacs has to meet an evidentiary burden that I have rarely seen teams clear.

    Managers who want to avoid these messes have options.

    First, they can crib Kellan's tests for new technologies. Extra points for digesting Glyph's thoughts on "innovation tokens."

    Next, they should identify the critical user journeys in their products. Technology choices are always situated in product constraints, but until the critical user journeys are enunciated, the selection of any specific architecture is likely to be wrong.

    Lastly, they should always run bakeoffs. Once critical user journeys are outlined and agreed, bakeoffs can provide teams with essential data about how different technology options will perform under those conditions. For frontend technologies, that means evaluating them under representative market conditions.

    And yes, there's almost always time to do several small prototypes. It's a damn sight cheaper than the months (or years) of painful remediation work. I'm sick to death of having to hand-hold teams whose products are suffocating under unusably large piles of cruft, slowly nursing their code-bases back to something like health as their management belatedely learns the value of knowing their systems deeply.

    Managers that do honest, user-focused bakeoffs for their frontend choices can avoid adding their teams to the dozens I've consulted with who adopted extremely popular, fundamentally inappropriate technologies that have had disasterous effects on their businesses and team velocity. Discarding popular stacks from consideration through evidence isn't a career risk; it's literally the reason to hire engineers and engineering leaders in the first place.

]]>
Misfire 2024-07-30T00:00:00Z https://infrequently.org/2024/07/misfire/ The W3C Technical Architecture Group1 is out with a blog post and an updated Finding regarding Google's recent announcement that it will not be imminently removing third-party cookies.

The current TAG members are competent technologists who have a long history of nuanced advice that looks past the shouting to get at the technical bedrock of complex situations. The TAG also plays a uniquely helpful role in boiling down the guidance it issues into actionable principles that developers can easily follow.

All of which makes these pronouncements seem like weak tea. To grok why, we need to walk through the threat model, look at the technology options, and try to understand the limits of technical interventions.

But before that, I should stipulate my personal position on third-party cookies: they aren't great!

They should be removed from browsers when replacements are good and ready, and Google's climbdown isn't helpful. That said, we have seen nothing of the hinted-at alternatives, so the jury's out on what the impact will be in practice.2

So why am I dissapointed in the TAG, given that my position is essentially what they wrote? Because it failed to acknowledge the limited and contingent upside of removing third-party cookies, or the thorny issues we're left with after they're gone.

So, what do third-party cookies do? And how do they relate to the privacy theat model?

Like a lot of web technology, third-party cookies have both positive and negative uses. Owing to a historcal lack of platform-level identity APIs, they form the backbone of nearly every large Single Sign-On (SSO) system. Thankfully, replacements have been developed and are being iterated on.

Unfortunately, some browsers have unilaterally removed them without developing such replacements, disrupting sign-in flows across the web, harming users and pushing businesses toward native mobile apps. That's bad, as native apps face no limits on communicating with third parties and are generally worse for tracking. They're not even subject to pro-user interventions like browser extensions. The TAG should have called out this aspect of the current debate in its Finding, encouraging vendors to adopt APIs that will make the transition smoother.

The creepy uses of third-party cookies relate to advertising. Third-party cookies provide ad networks and data brokers the ability to silently reidentify users as they browse the web. Some build "shadow profiles", and most target ads based on sites users visit. This targeting is at the core of the debate around third-party cookies.

Adtech companies like to claim targeting based on these dossiers allows them to put ads in front of users most likely to buy, reducing wasted ad spending. The industry even has a shorthand: "right people, right time, right place."

Despite the bold claims and a consensus that "targeting works," there's reason to believe pervasive surveillence doesn't deliver, and even when it does, isn't more effective.

Assuming the social utility of targeted ads is low — likely much lower than adtech firms claim — shouldn't we support the TAG's finding? Sadly, no. The TAG missed a critical opportunity to call for legislative fixes to the technically unfixable problems it failed to enumerate.

Privacy isn't just about collection, it's about correlation across time. Adtech can and will migrate to the server-side, meaning publishers will become active participants in tracking, funneling data back to ad networks directly from their own logs. Targeting pipelines will still work, with the largest adtech vendors consolidating market share in the process.

This is why "give us your email address for 30% discount" popups and account signup forms are suddenly everywhere. Email addresses are stable, long-lived reidentifiers. Overt mechanisms like this are already replacing third-party cookies. Make no mistake: post-removal, tracking will continue for as long as reidentification has perceived positive economic value. The only way to change that equation is legislation; anything else is a band-aid.

Pulling tracking out of the shadows is good, but a limited and contingent good. Users have a terrible time recognising and mitigating risk on the multi-month time-scales where privacy invasions play out. There's virtually no way to control or predict where collected data will end up in most jurisdictions, and long-term collection gets cheaper by the day.

Once correlates are established, or "consent" is given to process data in ways that facilitate unmasking, re-identification becomes trivial. It only takes giving a phone number to one delivery company, or an email address to one e-commerce site to suddenly light up a shadow profile, linking a vast amount of previously un-attributed browsing to a user. Clearing caches can reset things for a little while, but any tracking vendor that can observe a large proportion of browsing will eventually be able to join things back up.

Removal of third-party cookies can temporarily disrupt this reidentification while collection funnels are rebuilt to use "first party" data, but that's not going to improve the situation over the long haul. The problem isn't just what's being collected now, it's the ocean of dormant data that was previously slurped up.3 The only way to avoid pervasive collection and reidentification over the long term is to change the economics of correlation.

The TAG surely understands the only way to make that happen is for more jurisdictions to pass privacy laws worth a damn. It should say so.

The goal of tracking is to pick users out of crowds, or at least bucket them into small unique clusters. As I explained on Mastodon, this boils down to bits of entropy, and those bits are everywhere. From screen resolution and pixel density, to the intrinsic properties of the networks, to extensions, to language and accessibility settings that folks rely on to make browsing liveable. Every attribute that is even subtly different can be a building block for silent reidentification; A.K.A., "fingerprinting."4

In jurisdictions where laws allow collected data to remain the property of the collector, the risks posed by data-at-rest is only slightly attenuated by narrowing the funnel through which collection takes place.

It's possible to imagine computing that isn't fingerprintable, but that isn't what anyone is selling. For complex reasons, even the most cautious use of commodity computers is likely to be uniquely identifiable with enough time. This means that the question to answer isn't "do we think tracking is bad?", it's "given that we can't technically eliminate it, how can we rebuild privacy?". The TAG's new Finding doesn't wrestle with that question, doing the community a disservice in the process.

The most third-party cookie removal can deliver is temporary disruption. That disruption will affect distasteful collectors, costing them money in the short run. Many think of this as a win, I suspect because they fail to think through the longer-term consequences. The predictable effect will be a recalibration and entrenchment of surveillence methods. It will not put the panopticon out of business; only laws can do that.

For a preview of what this will look like, think back on Apple's "App Tracking Transparency" kayfabe, which did not visibly dent Facebook's long-term profits.

So this is not a solution to privacy, it's fire-and-movement tactics against corporate enemies. Because of the deep technical challenges in defeating fingerprinting4:1, even the most outspoken vendors have given up, introducing "nutrition labels" to shift responsibility for privacy onto consumers.

If the best vertically-integrated native ecosystems can do is to shift blame, the TAG should call out posturing about ineffective changes and push for real solutions. Vendors should loudly lobby for stronger laws that can truly change the game and the TAG should join those calls. The TAG should also advocate for the web, rather than playing into technically ungrounded fearmongering by folks trying to lock users into proprietary native apps whilst simultaneously depriving users of more private browsers.

The most generous take I can muster is that the TAG's work is half-done. Calling on vendors to drop third-party cookies has the virtue of being technical and actionable, properties I believe all TAG writing should embody. But having looked deeply at the situation, the TAG should have also called on browser vendors to support further reform along several axes — particularly vendors that also make native OSes.

First, if the TAG is serious about preventing tracking and improving the web ecosystem, it should call on all OS vendors to prohibit the use of "in-app browsers" when displaying third-party content within native apps.

It is not sufficient to prevent JavaScript injection because the largest native apps can simply convince the sites to include their scripts directly. For browser-based tracking attenuation to be effective, these side-doors must be closed. Firms grandstanding about browser privacy features without ensuring users can reliably enjoy the protections of their browser need to do better. The TAG is uniquely positioned to call for this erosion of privacy and the web ecosystem to end.

Next, the TAG should have outlined the limits of technical approaches to attenuating data collection. It should also call on browser vendors to adopt scale-based interventions (rather than absolutism) in mitigating high-entropy API use.5 The TAG should go first in moving past debates that don't acknowledge impossibilities in removing all reidentification, and encourage vendors to do the same. There's no solution to the privacy puzzle that can be solved by the purchase of a new phone, and the TAG should be clarion about what will end our privacy nightmare: privacy laws worth a damn.

Lastly, the TAG should highlight discrepancies between privacy marketing and the failure of vendors to push for strong privacy laws and enforcement. Because the threat model of privacy intrusion renders solely techincal interventions ineffective on long timeframes, this is the rare case in which the TAG should push past providing technical advice.

The TAG's role is to explain complex things with rigor and signpost credible ways forward. It has not done that yet regarding third-party cookies, but it's not too late.

FOOTNOTES

  1. Praise, as well as concern, in this post is specific to today's TAG's, not the output of the group while I served. I surely got a lot of things wrong, and the current TAG is providing a lot of value. My hope here is that it can extend this good work by expanding its new Finding.

  2. Also, James Roswell can go suck eggs.

  3. It's neither here nor there, but the TAG also failed in these posts to encourage users and developers to move their use of digital technology into real browsers and out of native apps which invasively track and fingerprint users to a degree web adtech vendors only fantasize about.

    A balanced finding would call on Apple to stop stonewalling the technologies needed to bring users to safer waters, including PWA installation prompts.

  4. As part of the drafting of the 2015 finding on Unsanctioned Web Tracking, the then-TAG (myself included) spent a great deal of time working through the details of potential fingerprinting vectors. What we came to realise was that only the Tor Browser had done the work to credibly analyse fingerprinting vectors and produce a coherent threat model. To the best of my knowledge, that remains true today.

    Other vendors continue to publish gussied-up marketing documents and stroppy blog posts that purport to cover the same ground, but consistently fail to do so. It's truly objectionable that those same vendors also prevent users from choosing disciplined privacy-focused browsers.

    To understand the difference, we can do a small thought experiment, enumerating what would be necessary to sand off currently-identifiable attributes of individual users. Because only 31 or 32 bits are needed to uniquely identify anybody (often less), we want a high safety factor. This means bundling users into very large crowds by removing distinct observable properties. To sand off variations between users, a truly private browser might:

    • Run the entire browser in a VM in order to:
      • Cap the number of CPU cores, frequency, and centralise on a single instruction set (e.g., emulating ARM when running on x86). Will likely result in a 2-5x slowdown.
      • Ensure (high) fixed latency for all disk access.
      • Set a uniform (low) cap on total memory.
    • Disable hardware acceleration for all graphics and media.
    • Disable JIT. Will slow JavaScript by 3-10x.
    • Only allow a fixed set of fonts, screen sizes, pixel densities, gamuts, and refresh rates; no more resizing browsers with a mouse. The web will pixelated and drab and animations will feel choppy.
    • Remove most accessibility settings.
    • Remove the ability to install extensions.
    • Eliminate direct typing and touch-based interactions, as those can leak timing information that's unique.
    • Run all traffic through Tor or a similarly high-latency VPN egress nodes.
    • Disable all reidentifying APIs (no more web-based video conferencing!)

    Only the Tor project is shipping a browser anything like this today, and it's how you can tell that most of what passes for "privacy" features in other browsers are anti-annoyance and anti-creep-factor interventions; they matter, but won't end the digital panopticon.

  5. It's not a problem that sign-in flows need third-party cookies today, but it is a problem that they're used for pervasive tracking.

    Likewise, the privacy problems inherent in email collection or camera access or filesystem folders aren't absolute, they're related to scale of use. There are important use-cases that demand these features, and computers aren't going to stop supporting them. This means the debate is only whether or not users can use the web to meet those needs. Folks who push an absolutist line are, in effect, working against the web's success. This is anti-user, as the alternatives are generally much more invasive native apps.

    Privacy problems arise at scale and across time. Browsers should be doing more to discourage high-quality reidentifiaction across cache clearing and in ways that escalate with risk. The first site you grant camera access isn't the issue; it's the 10th. Similarly, speed bumps should be put in place for use of reidentifying APIs on sites across cache clearing where possible.

    The TAG can be instrumental is calling for this sort of change in approach.

]]>
Why Browsers Get Built 2024-03-10T00:00:00Z https://infrequently.org/2024/03/why-browsers-get-built/ There are only two-and-a-half reasons to build a browser, and they couldn't be more different in intent and outcome, even when they look superficially similar. Learning to tell the difference is helpful for browser project managers and engineers, but also working web developers who struggle to develop theories of change for affecting browser teams.

Like Platform Adjacency Theory and The Core Web Platform Loop, this post started1 as a set of framing devices that I've been sketching on whiteboards for the best part of a decade. These lenses aren't perfect, but they provide starting points for thinking about the complex dynamics of browsers, OSes, "native" platforms, and the standards-based web platform.

The reasons to build browsers are most easily distinguished by the OSes they support and the size and composition of their teams ("platform" vs. "product"). Even so, there are subtleties that throw casual observers for a loop. In industrial-scale engineering projects like browser construction, headcount is destiny, but it isn't the whole story.

This is simultaneously the simplest and most vexing reason to build a browser.

Under this logic, browsers are strategically important to a broader business, and investments in platforms are investments in their future competitiveness compared with other platforms, not just other browsers. But none of those investments come good until the project has massive scale.

This strategy is exemplified by the ambitions of Netscape, crisply captured by Andreesen's quip that the goal was to render Windows "a poorly debugged set of device drivers".

The idea is that the web is where the action is, and that the browser winning more user Jobs To Be Done follows from increasing the web platform's capability. This developer-enabling flywheel aims to liberate computing from any single OS, supporting a services model.

A Web As Platform play depends on credibly keeping up with expansions in underlying OS features. The goal is to deliver safe portable, interoperable, and effective versions of important capabilities at a fast enough clip to maintain faith in the web as a viable ongoing investment.

In some sense it's a confidence-management exercise. A Web As Platform endgame requires the platform increases expressive capacity year over year. It must do as many new things each year as new devices can, even if the introduction of those features is delayed for the web by several years; the price of standards.

Platform-play browsers aim to grow and empower the web ecosystem, rather than contain it or treat it as a dying legacy. Examples of this strategic orientation include Netscape, Mozilla (before it lost the plot), Chrome, and Chromium-based Edge (on a good day).

  • Ship on many OSes and not just those owned by the sponsor
  • Large platform teams (>200 people and/or >40% of the browser team)
  • Visible, consistent investments in API leadership and capability expansion
  • Balanced benchmark focus
  • Large standards engagement footprint

There are two primary tactical modes of this strategic posture, both serving the same goal: to make an operating system look good by enabling a corpus of web content to run well on it while maintaining a competitive distance between the preferred (i.e., native, OS-specific) platform and the hopefully weaker web platform.

The two sub-variants differ in ambition owing to the market positions of their OS sponsors.

OSes treat browsers they sponsor as bridges or moats, but never the main object.
Photo by Paul Arky

OSes deploy browsers as a bridge for users into their environment when they're underdogs or fear disruption.

Of course, it would be better from the OS vendor's perspective if everyone simply wrote software specifically tailored to their proprietary platform, maximising OS feature differentiation. But smart vendors also know that's not possible when an OS isn't dominant.

OS challengers, therefore, strike a bargain. For the price of developing a browser, they gain the web's corpus of essential apps and services, serving to "de-risk" the purchase of a niche device by offering broad compatibility with existing software through the web. If they do a good job, a conflicted short-term investment can yield enough browser share to enable a future turn towards moat tactics (see below). Examples include Internet Explorer 3-6 as well as Safari on Mac OS X and the first iPhone.2

Conversely, incumbents fearing disruption may lower their API drawbridges and allow the web's power to expand far enough that the incumbent can gain share, even if it's not for their favoured platform; the classic example here being Internet Explorer in the late 90s. Once Microsoft knew it had Netscape well and truly beat, it simply disbanded the IE team, leaving the slowly rusting husk of IE to decay. And it would have worked, too, if it weren't for those pesky Googlers pushing IE6 beyond what was "possible"!

Without meaningful regulation, powerful incumbents can use anticompetitive tactics to suppress the web's potential to disrupt the OS and tilt the field towards the incumbent's proprietary software ecosystem.

This strategy works by maintaining sufficient browser share to influence developer choices while never allowing the browser team to deliver features sufficient to disrupt OS-specific alternatives.

In practice, moats are arbitrage on the unwillingness of web developers to understand or play the game, e.g. by loudly demanding timely features or recommending better browsers to users. Incumbents know that web developers are easily convinced to avoid "non standard" features and are happy to invent excuses for them. It's cheap to add a few features here and there to show you're "really trying," despite underfunding browser teams so much they can never do more than a glorified PR for the OS. This was the strategy behind IE 7-11 and EdgeHTML. Even relatively low share browsers can serve as effective moats if they can't be supplanted by competitive forces.

Apple has perfected the strategy, preventing competitors from even potentially offering disruptive features. This adds powerfully to the usual moat-digger's weaponisation of consensus processes. Engineering stop-energy in standards and quasi-standards bodies is nice, but it is so much more work than simply denying anyone the ability to ship the features that might threaten the proprietary agenda.

Bridge and moat tactics appear very different, but the common thread is control with an intent to suppress web platform expansion. In both cases, the OS will task the browser team to heavily prioritise integrations with the latest OS and hardware features at the expense of more broadly useful capabilities — e.g. shipping "notch" CSS and "force touch" events while neglecting Push.

Browser teams tasked to build bridges can grow quickly and have remit that looks similar to a browser with a platform agenda. Still, the overwhelming focus starts (and stays) on existing content, seldom providing time or space to deliver powerful new features to the Web. A few brave folks bucked this trend, using the fog of war to smuggle out powerful web platform improvements under a more limited bridge remit; particularly the IE 4-6 crew.

Teams tasked with defending (rather than digging) a moat will simply be starved by their OS overlords. Examples include IE 7+ and Safari from 2010 onward. It's the simplest way to keep web developers from getting uppity without leaving fingerprints. The "soft bigotry of low expectations", to quote a catastrophic American president.

  • Shipped only to the sponsor's OSes
  • Browser versions tied to OS versions
  • Small platform teams (<100 people and/or <30% of the browser team)
  • Skeleton standards footprint
  • Extreme focus on benchmarks of existing content
  • Consistent developer gaslighting regarding new capabilities
  • Anti-competitive tactics against competitors to maintain market share
  • Inconsistent feature leadership, largely focused on highlighting new OS and hardware features
  • Lagging quality

This is the "half-reason"; it's not so much a strategic posture as it is environment-surfing.

Over the years, many browsers that provide little more than searchboxes atop someone else's engine have come and gone. They lack staying power because their teams lack the skills, attitudes, and management priorities necessary to avoid being quickly supplanted by a fast-following competitor pursuing one of the other agendas.

These browsers also tend to be short-lived because they do not build platform engineering capacity. Without agency in most of their codebase, they either get washed away in unmanaged security debt, swamped by rebasing challenges (i.e., a failure to "work upstream"). They also lack the ability to staunch bleeding when their underlying engine fails to implement table-stakes features, which leads to lost market share.

Historical examples have included UC Browser, and more recently, the current crop of "secure enterprise browsers" (Chromium + keyloggers). Perhaps more controversially, I'd include Brave and Arc in this list, but their engineering chops make me think they could cross the chasm and choose to someday become platform-led browsers. They certainly have leaders who understand the difference.

  • Shipped to many OSes
  • Tiny platform teams (<20 people or <10% of the browser team)
  • Little benchmark interest or focus
  • No platform feature leadership
  • No standards footprint
  • Platform feature availability lags the underlying engine (e.g., UI and permissions not hooked up)
  • Platform potentially languishes multiple releases behind "upstream"

This model isn't perfect, but it has helped me tremendously in reliably predicting the next moves of various browser players, particularly regarding standards posture and platform feature pace.3

The implications are only sometimes actionable, but they can help us navigate. Should we hope that a vendor in a late-stage browser-as-moat crouch will suddenly turn things around? Well, that depends on the priorities and fortunes of the OS, which dictate the strategy of their browser team.

Similarly, a Web As Platform strategy will maximise a browser's reach and its developers' potential, albeit at the occasional expense of end-user features.

The most important takeaway for developers may be what this model implies about browser choice. Products with an OS-first agenda are always playing second fiddle to a larger goal that does not put web developers first, second, or even third. Coming to grips with this reality lets us more accurately recommend browsers to users that align with our collective interests in a vibrant, growing Web.

FOOTNOTES

  1. I hadn't planned to write this now, but an unruly footnote in an upcoming post, along with Frances' constant advice to break things up, made me realise that I already had 90% of it of ready.

  2. IE and iOS Safari demonstrate the capacity of OS vendors to burn bridges, stranding web developers in an unattractive bind through of disinvestment in browsers they control which retain blocking market share.

    As a result, it is not fear of engine monoculture that keeps me up at night, but rather the difficulty of displacing entrenched browsers from vendors with a proprietary agenda.

  3. Modern-day Mozilla presents a puzzle within this model.

    In theory, Mozilla's aims and interests align with growing the web as a platform; expanding its power to enable a larger market for browsers, and through it, a larger market for Firefox.

    In practice, that's not what's happening. Despite investing almost everything it makes back into browser development, Mozilla has also begun to slow-walk platform improvements. It walked away from PWAs and has continued to spread FUD about device APIs and other features that would indicate an appetite for an expansive vision of the platform.

    In a sense, it's playing the OS Agenda, but without an OS to profit from or a proprietary platform to benefit with delay and deflection. This is vexing, but perhaps expected within an organisation that has entered a revenue-crunch crouch. Another way to square the circle is to note that the Mozilla Manifesto doesn't actually speak about the web at all. If the web is just another fungible application running atop the internet (which the manifesto does centre), then it's fine for the web to be frozen in time, or even shrink.

    Still, Mozilla leadership should be thinking hard about the point of maintaining an engine. Is it to hold the coats of proprietary-favouring OS vendors? Or to make the web a true competitor?

]]>
Home Screen Advantage 2024-02-25T00:00:00Z https://infrequently.org/2024/02/home-screen-advantage/

Update: OWA is out with an open letter appealing to Apple to do better. If you care about the future of the web, I encourage you to sign it, particularly if you live in the EU or build products for the common market.

After weeks of confusion and chaos, Apple's plan to kneecap the web has crept into view, menacing a PWApocalypse as the March 6th deadline approaches for compliance with the EU's Digital Markets Act (DMA).

The view from Cupertino.
The view from Cupertino.

The DMA requires Apple to open the iPhone to competing app stores, and its lopsided proposal for "enabling" them is getting most of the press. But Apple knows it has native stores right where it wants them. Cupertino's noxious requirements will take years to litigate. Meanwhile, potential competitors are only that.

But Cupertino can't delay the DMA's other mandate: real browsers, downloaded from Apple's own app store. Since it can't bar them outright, it's trying to raise costs on competitors and lower their potential to disrupt Apple's cozy monopoly. How? By geofencing browser choice and kneecapping web apps, all while gaslighting users about who is breaking their web apps.

The immediate impact of iOS 17.4 in the EU will be broken apps and lost data, affecting schools, governments, startups, gamers, and anyone else with the temerity to look outside the one true app store for even a second. None of this is required by the DMA, as demonstrated by continuing presence of PWAs and the important features they enable on Windows and Android, both of which are in the same regulatory boat.

The data loss will be catastrophic for many, as will the removal of foundational features. Here's what the landscape looks like today vs. what Apple is threatening:

PWA Capability Windows Android iOS 17.3 iOS 17.4
App-like UI
Settings Integration
Reliable Storage
Push Notifications
Icon Badging
Share-to PWA
App Shortcuts
Device APIs

Apple's support for powerful web apps wasn't stellar, but this step in the wrong direction will just so happen to render PWAs useless to worldwide businesses looking to reach EU users.

Apple's interpretation of the DMA appears to be that features not available on March 6th don't need to be shared with competitors, and it doesn't want to share web apps. The solution almost writes itself: sabotage PWAs ahead of the deadline and give affected users, businesses, and competitors minimal time to react.

Cupertino's not just trying to vandalise PWAs and critical re-engagement features for Safari; it's working to prevent any browser from ever offering them on iOS. If Apple succeeds in the next two weeks, it will cement a future in which the mobile web will never be permitted to grow beyond marketing pages for native apps.

By hook or by crook, Apple's going to maintain its home screen advantage.

The business goal is obvious: force firms back into the app store Apple taxes and out of the only ecosystem it can't — at least not directly. Apple's justifications range from unfalsifiable smokescreens to blatant lies, but to know it you have to have a background in browser engineering and the DMA's legalese. The rest of this post will provide that context. Apologies in advance for the length.

If you'd like to stop reading here, take with you the knowledge that Cupertino's attempt to scuttle PWAs under cover of chaos is exactly what it appears to be: a shocking attempt to keep the web from ever emerging as a true threat to the App Store and blame regulators for Apple's own malicious choices.

And they just might get away with it if we don't all get involved ASAP.

Two weeks ago, Apple sprung its EU Digital Markets Act (DMA) compliance plans on the world as a fait accomplis.

The last-minute unveil after months of radio silence were calculated to give competitors minimal time to react to the complex terms, conditions, and APIs. This tactic tries to set Apple's proposal as a negotiating baseline, forcing competitors to burn time and money arguing down plainly unacceptable terms before they can enter the market.

For native app store hopefuls, this means years of expensive disputes before they can begin to access an artificially curtailed market. This was all wrapped in a peevish, belligerant presentation, which the good folks over at The Platform Law Blog have covered in depth.

Much of the analysis has focused on the raw deal Apple is offering native app store competitors, missing the forest for the trees: the threat Apple can't delay by years comes from within.

Deep in the sub-basement of Apple's tower of tomfoolery are APIs and policies that purport to enable browser engine choice. If you haven't been working on browsers for 15 years, the terms might seem reasonable, but to these eyes they're anything but. OWA has a lengthy dissection of the tricks Apple's trying to pull.

Apple's message of hope and optimism for a better web.
Apple's message of hope and optimism for a better web.

The proposals are maximally onerous, but you don't have to take my word for it; here's Mozilla:

We are ... extremely disappointed with Apple’s proposed plan to restrict the newly-announced BrowserEngineKit to EU-specific apps. The effect of this would be to force an independent browser like Firefox to build and maintain two separate browser implementations — a burden Apple themselves will not have to bear.

Apple’s proposals fail to give consumers viable choices by making it as painful as possible for others to provide competitive alternatives to Safari.

This is another example of Apple creating barriers to prevent true browser competition on iOS.

Mozilla spokesperson

The strategy is to raise costs and lower the value of porting browsers to iOS. Other browser vendors have cited exactly these concerns when asked about plans to bring their best products to iOS. Apple's play is to engineer an unusable alternative then cite the lack of adoption to other regulators as proof that mandating real engine choice is unwise.

Instead of facilitating worldwide browser choice in good faith, Apple's working to geofence progress; classic "divide and conquer" stuff, justified with serially falsified security excuses. Odious, brazen, and likely in violation of the DMA, but to the extent that it will now turn into a legal dispute, that's a feature (not a bug) from Apple's perspective.

When you're the monopolist, delay is winning.

All of this would be stock FruitCo doing anti-competitive FruitCo things, but they went further, attempting to silently shiv PWAs and blame regulators for it. And they did it in the dead of the night, silently disabling important features as close to the DMA compliance deadline as possible.

It's challenging, verging on impossible, to read this as anything but extrordinary bad faith, but Apple's tactics require context to understand.

The DMA came into force in 2022, putting everyone (including Apple) on notice that their biggest platforms and products would probably be "designated", and after designation, they would have six months to "comply". The first set of designation decisions went out last Sept, obligating Android, Windows, iOS, Chrome, and Safari to comply no later than March 6th, 2024.

Apple tried everything to <a href='https://www.theregister.com/2023/11/02/apple_safari_browser/'>shrink the scope of enforcement</a> and <a href='https://www.theverge.com/2024/1/8/23961923/apple-app-store-appeal-european-union-digital-markets-act-core-platform-service-gatekeeper'>delay compliance,</a> but in the end had the same two-years of notice and six-months warning from designation as everyone else.
Apple tried everything to shrink the scope of enforcement and delay compliance, but in the end had the same two-years of notice and six-months warning from designation as everyone else.

A maximally aggressive legal interpretation might try to exploit ambiguity in what it means to comply and when responsibilities actually attach.

Does compliance mean providing open and fair access starting from when iOS and Safari were designated, or does compliance obligation only attach six months later? The DMA's text is not ironclad here:

10: The gatekeeper shall comply with the obligations laid down in Articles 5, 6 and 7 within 6 months after a core platform service has been listed in the designation decision pursuant to paragraph 9 of this Article.

DMA Article 3, Clause 10

Firms looking to comply maliciously might try to remove troublesome features just before a compliance deadline, then argue they don't need to share them with competitors becuse they weren't available before the deadline set in. Apple looks set to argue, contra everyone else subject to the DMA, that the moment from which features must be made interoperable is the end of the fair-warning period, not the date of designation.

This appears to be Apple's play, and it stinks to high heavens.

Apple's change isn't merely cosmetic. In addition to immediate data loss, FruitCo's change will destroy:

  • App-like UI:

    Web apps are no longer going to look or work like apps in the task manager, systems settings, or any other surface. Homescreen web apps will be demoted to tabs in the default browser.

  • Reliable storage:

    PWAs were the only exemption to Apple's (frankly silly) seven day storage eviction policy, meaning the last safe harbour for anyone trying to build a serious, offline-first experience just had the rug pulled out from under them.

  • Push Notifications:

    Remember how Apple gaslit web developers over Web Push for the best part of a decade? And remember how, when they finally got around to it, did a comically inept job? Recall fretting and about how shite web Push Notifications look and work for iOS users? Well, rest easy, because they're going away too.

  • App Icon Badging:

    A kissing cousin of Push, Icon Badging allows PWAs to ambiently notify users of new messages, something iOS native apps have been able to do for nearly 15 years.

Removal of one would be a crisis. Together? Apple's engineering the PWApocalypse.

You can't build credible mobile experiences without these features. A social network without notifications? A notetaking app that randomly loses data? Businesses will get the message worldwide: if you want to be on the homescreen and deliver services that aren't foundationally compromised, the only game in town is Apple's app store.

Apple understands even the most aggressive legal theories about DMA timing wouldn't support kneecapping PWAs after March 6th. Even if you believe (as I do) their obligations attached back in September, there's at least an argument to be tested. Cupertino's white-shoe litigators would be laughed out of court and Apple would get fined ridiculous amounts for non-compliance if it denied these features to other browsers after the fair-warning period. To preserve the argument for litigation, it was necessary to do the dirty deed ahead of the last plausible deadline.

The first indication something was amiss was a conspicuous lack of APIs for PWA support in the BrowserEngineKit documentation, released Feb 1st alongside Apple's peevish, deeply misleading note that attempted to whitewash malicious compliance in a thin coat of security theatre.

Two days later, after developers inside the EU got their hands on the iOS 17.4 Beta, word leaked out that PWAs were broken. Nothing about the change was documented in iOS Beta or Safari release notes. Developers filed plaintive bugs and some directly pinged Apple employees, but Cupertino remained shtum. This created panic and confusion as the windows closed for DMA compliance and the inevitable iOS 17.4 final release ahead of March 6th.

iOS 17.4 beta: Progressive Web Apps (PWAs) are entirely disabled in the EU

Two more betas followed, but no documentation or acknowledgement of the "bug." Changes to the broken PWA behavior were introduced, but Apple failed to acknowledge the issue or confirm that it was intentional and therefore likely to persist. After two weeks of growing panic from web developers, Apple finally copped to crippling the only open, tax-free competitor to the app store.

Apple's Feb 15th statement is a masterclass in deflection and deceit. To understand why requires a deep understanding of browsers internals and how Apple's closed PWA — sorry, "home screen web app" — system for iOS works.

TL;DR? Apple's cover story is horseshit, stem to stern. Cupertino ought to be ashamed and web developers are excused for glowing incandescent with rage over being used as pawns; first ignored, then gaslit, and finally betrayed.

I really, really hate to do this, but Brandolini's Law dictates that to refute Apple's bullshit, I'm going to need to go through their gibberish excuses line-by-line to explain and translate.

Q: Why don’t users in the EU have access to Home Screen web apps?

Translation: "Why did you break functionality that has been a foundational part of iOS since 2007, but only in the EU?"

To comply with the Digital Markets Act, Apple has done an enormous amount of engineering work to add new functionality and capabilities for developers and users in the European Union — including more than 600 new APIs and a wide range of developer tools.

Translation: "We're so very tired, you see. All of this litigating to avoid compliance tuckered us right out. Plus, those big meanies at the EU made us do work. It's all very unfair."

It goes without saying, but Apple's burden to add APIs it should have long ago provided for competing native app stores has no bearing whatsoever on its obligation to provide fair access to APIs that browser competitors need. Apple also had the same two years warning as everyone else. It knew this was coming, and 11th hour special pleading has big "the dog ate my homework" energy.

The iOS system has traditionally provided support for Home Screen web apps by building directly on WebKit and its security architecture. That integration means Home Screen web apps are managed to align with the security and privacy model for native apps on iOS, including isolation of storage and enforcement of system prompts to access privacy impacting capabilities on a per-site basis.

Finally! A recitation of facts.

Yes, iOS has historically forced a uniquely underpowered model on PWAs, but iOS is not unique in providing system settings integration or providing durable storage or managing PWA permissions. Many OSes and browsers have created the sort of integration infrastructure that Apple describes. These systems leave the question of how PWAs are actually run (and where their storage lives) to the browser that installs them, and the sky has yet to fall. Apple is trying to gussy up preferences and present them as hard requirements without justification.

Apple is insinuating that it can't provide API surface areas to allow the sorts of integrations that others already have. Why? Because it might involve writing a lot of code.

Bless their hearts.

Without this type of isolation and enforcement, malicious web apps could read data from other web apps and recapture their permissions to gain access to a user’s camera, microphone or location without a user’s consent.

Keeping one website from abusing permissions or improperly accessing data from another website is what browsers do. It's Job #1.

Correctly separating principals is the very defintion of a "secure" browser. Every vendor (save Apple) treats subversion of the Same Origin Policy as a showstopping bug to be fixed ASAP. Unbelieveable amounts of engineering go to ensuring browsers overlay stronger sandboxing and more restrictive permissions on top of the universally weaker OS security primitives — iOS very much included.

Browser makers have become masters of origin separation because they run totally untrusted code from all over the internet. Security is paramount because browsers have to be paranoid. They can't just posture about how store reviews will keep users safe; they have to do the work.

Good browsers separate web apps better than bad ones. It's rich that Apple of all vendors is directly misleading this way. Its decade+ of under-investment in WebKit ensured Safari was less prepared for Spectre and Meltdown and Solar Winds than alternative engines. Competing browsers had invested hundreds of engineer years into more advanced Site Isolation. To this day, Apple's underfunding and coerced engine monoculture put all iOS users at risk.

With that as background, we can start to unpack Apple's garbled claims.

Cupertino is saying that it does not want to create APIs for syncing permission state through the thin shims every PWA-supporting OS uses to make websites first class. It doesn't want to add APIs for attributing storage use, clearing state, toggling notifications, and other common management tasks. This is a preference, but it is not responsive to Apple's DMA obligations.

If those APIs existed, Apple would still have permission management questions to work through, which its misdirections also allude to. But these aren't a problem in practice. Every browser offering PWA support would happily sign up to terms that required accurate synchronization of permission state between OS surfaces and web origins, in exactly the same way they treat cross-origin subversion as a fatal bug to be hot-fixed.

Apple's excusemaking is a mirror of Cupertino's years of scaremongering about alternate browser engine security, only to take up my proposal more-or-less wholesale when the rubber hit the road.

Nothing about this is monumental to build or challenging to manage; FruitCo's just hoping you don't know better. And why would you? The set of people who understand these details generously number in the low dozens.

Browsers also could install web apps on the system without a user’s awareness and consent.

Apple know this is a lie.

They retain full control over the system APIs that are called to add icons to the homescreen, install apps, and much else. They can shim in interstitial UI if they feel like doing so. If iOS left this to Safari and did not include these sorts of precautions, those are choices Apple has made and has been given two years notice to fix.

Cupertino seems to be saying "bad things might happen if we continued to do a shit job" and one can't help but agree. However, that's no way out of the DMA's obligations.

Addressing the complex security and privacy concerns associated with web apps using alternative browser engines would require building an entirely new integration architecture that does not currently exist in iOS and was not practical to undertake given the other demands of the DMA and the very low user adoption of Home Screen web apps.

[CITATION NEEDED]

Note the lack of data? Obviously this sort of unsubstantiated bluster fails Hitchen's Razor, but that's not the full story.

Apple is counting on the opacity of its own web suppression to keep commenters from understanding the game that's afoot. Through an enervating combination of strategic underinvestment and coerced monoculture, Apple created (and still maintains) a huge gap in discoverability and friction for installing web apps vs. their native competition. Stacking the deck for native has taken many forms:

This campaign of suppression has been wildly effective. If users don't know they can install PWAs, it's because Safari never tells them, and until this time last year, neither could any other browser. Developers also struggled to justify building them because Apple's repression extended to neglect of critical features, opening and maininting a substantial capability gap.

If PWAs use on iOS is low, that's a consequence of Apple's own actions. On every other OS where I've seen the data, not only are PWAs a success, they are growing rapidly. Perhaps that's why Apple feels a need to mislead by omission and fail to provide data to back their claim.

And so, to comply with the DMA’s requirements, we had to remove the Home Screen web apps feature in the EU.

Bullshit.

Apple's embedded argument expands to:

  • We don't want to comply with the plain-letter language of the law.

  • To avoid that, we've come up with a legal theory of compliance that's favourable to us.

  • To comply with that (dubious) theory, and to avoid doing any of the work we don't want to do, we've been forced to bump off the one competitor we can't tax.

Neat, tidy, and comprised entirely of bovine excrement.

EU users will be able to continue accessing websites directly from their Home Screen through a bookmark with minimal impact to their functionality. We expect this change to affect a small number of users. Still, we regret any impact this change — that was made as part of the work to comply with the DMA — may have on developers of Home Screen web apps and our users.

Translation: "Because fuck you, that's why"

The DMA doesn't require Apple to torpedo PWAs.

Windows and Android will continue supporting them just fine. Apple apparently hopes it can convince users to blame regulators for its own choices. Cupertino's counting on the element of surprise plus the press's poorly developed understanding of the situation to keep blowback from snowballing into effective oppostion.

There's no possible way to justify a "Core Technology Fee" tax on an open, interoperable, standardsized platform that competitors would provide secure implementations of for free. What Apple's attempting isn't just some hand-wavey removal of a "low use" feature ([CITATION NEEDED]), it's sabotage of the only credible alternative to its app store monopoly.

A slide from Apple's presentation in Apple v. Epic, attempting to make the claim Epic could have just made a PWA if they didn't like the App Store terms because circa '20 Safari was <em>so</em> capable. <br><br><a href='/2021/04/progress-delayed/'>LOL.</a>
A slide from Apple's presentation in Apple v. Epic, attempting to make the claim Epic could have just made a PWA if they didn't like the App Store terms because circa '20 Safari was so capable.

LOL.

Businesses will get the message: from now on, the only reliable way to get your service under the thumb, or in the notification tray, of the most valuable users in the world is to capitulate to Apple's extortionate App Store taxes.

If the last 15 years are anything to judge by, developers will take longer to understand what's going on, but this is an attempt to pull a "Thoughts on Flash" for the web. Apple's suppression of the web has taken many forms over the past decade, but the common thread has been inaction and anti-competitive scuppering of more capable engines. With one of those pillars crumbling, the knives glint a bit more brightly. This is Apple once and for all trying to relegate web development skills to the dustbin of the desktop.

Not only will Apple render web apps unreliable for Safari users, FruitCo is setting up an argument to prevent competitors from ever delivering features that challenge the app store in future. And it doesn't care who it hurts along the way.

This is exactly what it looks like: a single-fingered salute to the web and web developers. The removal of features that allowed the iPhone to exist at all. The end of Steve Jobs' promise that you'd be able to make great apps out of HTML, CSS, and JS.

For the past few years Apple has gamely sent $1,600/hr lawyers and astroturf lobbyists to argue it didn't need to be regulated. That Apple was really on the developer's side. That even if it overstepped occasionally, it was all in the best interest of users.

Tell that to the millions of EU PWA users about to lose data. Tell that to the public services built on open technology. Tell it to the businesses that will fold, having sweated to deliver compelling experiences using the shite tools Apple provides web developers. Apple's rug pull is anti-user, anti-developer, and anti-competition.

Now we see the whole effort in harsh relief. A web Apple can't sandbag and degrade is one it can't abide. FruitCo's fear and loathing of an open platform it can't tax is palpable. The lies told to cover for avarice are ridiculous — literally, "worthy of ridicule".

It's ok to withhold the benefit of the doubt from Safari and Apple. It's ok to be livid. These lies aren't little or white; they're directly aimed at our future. They're designed to influence the way software will be developed and delivered for decades to come.

If you're as peeved about this as I am, go join OWA in the fight and help them create the sort of pressure in the next 10 days that might actually stop a monopolist with money on their mind.

Thanks to Stuart Langride, Bruce Lawson, and Roderick Gadellaa for their feedback on drafts of this post.

]]>
The Performance Inequality Gap, 2024 2024-01-31T00:00:00Z https://infrequently.org/2024/01/performance-inequality-gap-2024/ The global device and network situation continues to evolve, and this series is an effort to provide an an up-to-date understanding for working web developers. So what's changed since last year? And how much HTML, CSS, and (particularly) JavaScript can a new project afford?

In a departure from previous years, two sets of baseline numbers are presented for first-load under five seconds on 75th (P75) percentile devices and networks1; one set for JavaScript-heavy content, and another for markup-centric stacks.

Budget @ P75 Markup-based JS-based
Total Markup JS Total Markup JS
3 seconds 1.4MiB 1.3MiB 75KiB 730KiB 365KiB 365KiB
5 seconds 2.5MiB 2.4MiB 100KiB 1.3MiB 650KiB 650KiB

This was data was available via last year's update, but was somewhat buried. Going forward, I'll produce both as top-line guidance. The usual caveats apply:

  • Performance is a deep and nuanced domain, and much can go wrong beyond content size and composition.
  • How sites manage resources after-load can have a big impact on perceived performance.
  • Your audience may justify more stringent, or more relaxed, limits.

Global baselines matter because many teams have low performance management maturity, and today's popular frameworks – including some that market performance as a feature – fail to ward against catastrophic results.

Until and unless teams have better data about their audience, the global baseline budget should be enforced.

This isn't charity; it's how products stay functional, accessible, and reliable in a market awash in bullshit. Limits help teams steer away from complexity and towards tools that generate simpler output that's easier to manage and repair.

Since at least 2015, building JavaScript-first websites has been a predictably terrible idea, yet most of the sites I trace on a daily basis remain mired in script.2 For these sites, we have to factor in the heavy cost of running JavaScript on the client when describing how much content we can afford.

HTML, CSS, images, and fonts can all be parsed and run at near wire speeds on low-end hardware, but JavaScript is at least three times more expensive, byte-for-byte.

Most sites, even those that aspire to be "lived in", are generally experienced through short sessions, which means they can't justify much in the way of up-front code. First impressions always matter.

Most sorts of sites have shallow sessions, making up-front script costs hard to justify.
Most sorts of sites have shallow sessions, making up-front script costs hard to justify.

Targeting the slower of our two representative devices, and opening only two connections over a P75 network, we can afford ~1.3MiB of compressed content to get interactive in five seconds. A page fitting this budget can afford:

  • 650KiB of HTML, CSS, images, and fonts
  • 650KiB of JavaScript

If we set the target a more reasonable three seconds, the budget shrinks to ~730KiB, with no more than 365KiB of compressed JavaScript.

Similarly, if we keep the five second target but open five TLS connections, the budget falls to ~1MiB. Sites trying to load in three seconds but which open five connections can afford only ~460KiB total, leaving only ~230KiB for script.

Sites largely comprised of HTML and CSS can afford a lot more, although CSS complexity and poorly-loaded fonts can still slow things down. Conservatively, to load in five seconds over two connections, we should try to keep content under 2.5MiB, including:

  • 2.4MiB of HTML, CSS, images, and fonts, and
  • 100KiB of JavaScript.

To hit the three second first-load target, we should aim for a max 1.4MiB transfer, made up of:

  • 1.325MiB of HTML, CSS, etc., and
  • 75KiB of JavaScript.

These are generous targets. The blog you're reading loads in ~1.2 seconds over a single connection on the target device and network profile. It consumes 120KiB of critical path resources to become interactive, only 8KiB of which is script.

As in years past, you can use the interactive estimator to understand how connections and devices impact budgets. This the tool has been updated to let you select from JavaScript-heavy and JavaScript-light content composition and defaults to the updated network and device baseline (see below).

<em>Tap to try the interactive version.</em>
Tap to try the interactive version.

It's straightforward to understand the number of critical path network connections and to eyeball the content composition from DevTools or WebPageTest. Armed with that information, it's possible to use this estimator to quickly understand what sort of first-load experience users at the margins can expect. Give it a try!

These recommendations are not context-free, and folks can reasonably disagree.

Indeed, many critiques are possible. The five second target first load)1:1 is arbitrary. A sample population comprised of all internet users may be inappropriate for some services (in both directions). A methodology of "informed reckons" leaves much to be desired. The methodological critiques write themselves.

The rest of this post works to present the thinking behind the estimates, both to spark more informed points of departure and to contextualise the low-key freakout taking place as INP begins to put a price on JavaScript externalities.

Another aim of this series is to build empathy. Developers are clearly out of touch with market ground-truth. Building an understanding of the differences in the experiences of the wealthy vs. working-class users can make the privilege bubble's one-way mirror perceptible from the inside.3

The "i" in iPhone stands for "inequality".

Premium devices are largely absent in markets with billions of users thanks to the chasm of global wealth inequality. India's iOS share has surged to an all-time high of 7% on the back of last-generation and refurbished devices. That's a market of 1.43 billion people where Apple doesn't even crack the top five in terms of shipments.

The Latin American (LATAM) region, home to more than 600 million people and nearly 200 million smartphones, shows a similar market composition:

In LATAM, iPhones make up less than 6% of total device shipments.
In LATAM, iPhones make up less than 6% of total device shipments.

Everywhere wealth is unequally distributed, the haves read about it in Apple News over 5G while the have-nots struggle to get reliable 4G coverage for their Androids. In country after country (PDF) the embedded inequality of our societies sorts ownership of devices by price. This, in turn, sorts by brand.

This matters because the properties of devices defines what we can deliver. In the U.S., the term "smartphone dependence" has been coined to describe folks without other ways to access the increasing fraction of essential services only available through the internet. Unsurprisingly, those who can't afford other internet-connected devices, or a fixed broadband subscription, are also likely to buy less expensive smartphones:

Missing alt text

As smartphone ownership and use grow, the frontends we deliver remain mediated by the properties of those devices. The inequality between the high-end and low-end is only growing, even in wealthy countries. What we choose to do in response defines what it means to practice UX engineering ethically.

Extending the SoC performance-by-price series with 2023's data, the picture remains ugly:

<em>Tap for a larger version.</em><br>Geekbench 5 single-core scores for 'fastest iPhone', 'fastest Android', 'budget', and 'low-end' segments.
Tap for a larger version.
Geekbench 5 single-core scores for 'fastest iPhone', 'fastest Android', 'budget', and 'low-end' segments.

Not only have fruity phones extended their single-core CPU performance lead over contemporary high-end Androids to a four year advantage, the performance-per-dollar curve remains unfavourable to Android buyers.

At the time of publication, the cheapest iPhone 15 Pro (the only device with the A17 Pro chip) is $999 MSRP, while the S23 (using the Snapdrago 8 gen 2) can be had for $860 from Samsung. This nets out to 2.32 points per dollar for the iPhone, but only 1.6 points per dollar for the S23.

Meanwhile, a $175 (new, unlocked) Samsung A24 scores a more reasonable 3.1 points per dollar on single-core performance, but is more than 4.25× slower than the leading contemporary iPhone.

The delta between the fastest iPhones and moderately price new devices rose from 1,522 points last year to 1,774 today.

Put another way, the performance gap between wealthy users and budget shoppers grew more this year (252 points) than the gains from improved chips delivered at the low end (174 points). Inequality is growing faster than the bottom-end can improve. This is particularly depressing because single-core performance tends to determine the responsiveness of web app workloads.

A less pronounced version of the same story continues to play out in multi-core performance:

<em>Tap for a larger version.</em><br>Round and round we go: Android ecosystem SoCs are improving, but the Performance Inequality Gap continues to grow. Even the fastest Androids are 18 months (or more) behind equivalently priced iOS-ecosystem devices.
Tap for a larger version.
Round and round we go: Android ecosystem SoCs are improving, but the Performance Inequality Gap continues to grow. Even the fastest Androids are 18 months (or more) behind equivalently priced iOS-ecosystem devices.

Recent advances in high-end Android multi-core performance have closed the previous three-year gap to 18 months. Meanwhile, budget segment devices have finally started to see improvement (as this series predicted), thanks to hand-me-down architecture and process node improvements. That's where the good news ends.

The multi-core performance gap between i-devices and budget Androids grew considerably, with the score delta rising from 4,318 points last year to 4,936 points in 2023.

Looking forward, we can expect high-end Androids to at least stop falling further behind owing to a new focus on performance by Qualcomm's Snapdragon 8 gen 3 and MediaTek's Dimensity 9300 offerings. This change is long, long overdue and will take years to filter down into positive outcomes for the rest of the ecosystem. Until that happens, the gap in experience for the wealthy versus the rest will not close.

iPhone owners experience a different world than high-end Android buyers, and live galaxies apart from the bulk of the market. No matter how you slice it, the performance inequality gap is growing for CPU-bound workloads like JavaScript-heavy web apps.

As ever, 2023 re-confirmed an essential product truth: when experiences are slow, users engage less. Doing a good job in an uneven network environment requires thinking about connection availability and engineering for resilience. It's always better to avoid testing the radio gods than spend weeks or months appeasing them after the damage is done.

5G network deployment continues apace, but as with the arrival of 4G, it is happening unevenly and in ways and places that exacerbate (rather than lessen) performance inequality.4

Data on mobile network evolution is sketchy,5 and the largest error bars in this series' analysis continue to reside in this section. Regardless, we can look industry summaries like the GSMA's report on "The Mobile Economy 2023" (PDF) for a directional understanding that we can triangulate with other data points to develop a strong intuition.

For instance, GSMA predicts that 5G will only comprise half of connections by 2030. Meanwhile, McKinsey predicts that high-quality 5G (networks that use 6GHz bands) will only cover a quarter of the world's population by 2030. Regulatory roadblocks are still being cleared.

As we said in 2021, "4G is a miracle, 5G is a mirage."

This doesn't mean that 4G is one thing, or that it's deployed evenly, or even that the available spectrum will remain stable within a single generation of radio technology. For example, India's network environment has continued to evolve since the Reliance Jio revolution that drove 4G into the mainstream and pushed the price of a mobile megabyte down by ~90% on every subcontinental carrier.

Speedtest.net's recent data shows dramatic gains, for example, and analysts credit this to improved infrastructure density, expanded spectrum, and back-haul improvements related to the 5G rollout — 4G users are getting better experiences than they did last year because of 5G's role in reducing contention.

India's speed test medians are moving quickly, but variance is orders-of-magnitude wide, with 5G penetration below 25% in the most populous areas.
India's speed test medians are moving quickly, but variance is orders-of-magnitude wide, with 5G penetration below 25% in the most populous areas.

These gains are easy to miss looking only at headline "4G vs. 5G" coverage. Improvements arrive unevenly, with the "big" story unfolding slowly. These effects reward us for looking at P75+, not just means or medians, and intentionally updating priors on a regular basis.

Events can turn our intuitions on their heads, too. Japan is famously well connected. I've personally experienced rock-solid 4G through entire Tokyo subway journeys, more than 40m underground and with no hiccups. And yet, the network environment has been largely unchanged by the introduction of 5G. Having provisioned more than adequately in the 4G era, new technology isn't having the same impact from pent-up demand. But despite consistent performance, the quality of service for all users is distributed in a much more egalitarian way:

Japan's network environment isn't the fastest, but is much more evenly distributed.
Japan's network environment isn't the fastest, but is much more evenly distributed.

Fleet device composition has big effects, owing to differences in signal-processing compute availability and spectrum compatibility. At a population level, these influences play out slowly as devices age out, but still have impressively positive impacts:

Device impact on network performance is visible in Opensignal's iPhone dataset.
Device impact on network performance is visible in Opensignal's iPhone dataset.

As inequality grows, averages and "generation" tags can become illusory and misleading. Our own experiences are no guide; we've got to keep our hands in the data to understand the texture of the world.

So, with all of that as prelude, what can we say about where the mobile network baseline should be set? In a departure from years prior, I'm going to use a unified network estimate (see below). You'll have to read on for what it is! But it won't be based on the sort of numbers that folks explicitly running speed tests see; those aren't real life.

The market forces this series previewed in 2017 have played out in roughly a straight line: smartphone penetration in emerging markets is approaching saturation, ensuring a growing fraction of purchases are made by upgrade shoppers. Those who upgrade see more value in their phones and save to buy better second and third devices. Combined with the emergence and growth of the "ultra premium" segment, average selling prices (ASPs) have risen.

2022 and 2023 have established an inflection point in the regard, with worldwide average selling prices jumping to more than $430, up from $300-$350 for much of the decade prior. Some price appreciation has been due to transient impacts of the U.S./China trade wars, but most of it appears driven by iOS ASPs which peaked above $1,000 for the first time in 2023. Android ASPs, meanwhile, continued a gradual rise to nearly $300, up from $250 five years ago.

Missing alt text

A weak market for handsets in 2023, plus stable sales for iOS, had an notable impact on prices. IDC expects global average prices to fall back below $400 by 2027 as Android volumes increase from an unusually soft 2023.

Counterpoint data shows declining sales in both 2022 and 2023.
Counterpoint data shows declining sales in both 2022 and 2023.

Shipment growth in late 2023 and beyond is coming from emerging markets like the Middle East and Africa. Samsung's A-series mid-tier is doing particularly well.
Shipment growth in late 2023 and beyond is coming from emerging markets like the Middle East and Africa. Samsung's A-series mid-tier is doing particularly well.

Despite falling sales, distribution of Android versus iOS sales remains largely unchanged:

Android sales reliably constitute 80-85% of worldwide volume.
Android sales reliably constitute 80-85% of worldwide volume.

Even in rich nations like Australia and the <a href='https://www.statista.com/statistics/262179/market-share-held-by-mobile-operating-systems-in-the-united-kingdom/'>the U.K.</a>, iPhones account for less than half of sales. Predictably, they are over-represented in analytics and logs owing to wealth-related factors including superior network access and performance hysteresis.
Even in rich nations like Australia and the the U.K., iPhones account for less than half of sales. Predictably, they are over-represented in analytics and logs owing to wealth-related factors including superior network access and performance hysteresis.

Smartphone replacement rates have remained roughly in line with previous years, although we should expect higher device longevity in future years. Survey reports and market analysts continue to estimate average replacement at 3-4 years, depending on segment. Premium devices last longer, and a higher fraction of devices may be older in wealthy geographies. Combined with discretionary spending pressure and inflationary impacts on household budgets, consumer intent to spend on electronics has taken a hit, which will be felt in device lifetime extension until conditions improve. Increasing demand for refurbished devices also adds to observable device aging.

The data paints a substantially similar picture to previous years: the web is experienced on devices that are slower and older than those carried by affluent developers and corporate directors whose purchasing decisions are not impacted by transitory inflation.

To serve users effectively, we must do extra work to live as our customers do.

Re-using last year's P75 device calculus, our estimate is based on a device sold new, unlocked for the mid-2020 to mid-2021 global ASP of ~$350-375.

Representative examples from that time period include the Samsung Galaxy A51 and the Pixel 4a. Neither model featured 5G,6 and we cannot expect 5G to play a significant role in worldwide baselines for at least the next several years.4:1

The A51 featured eight slow cores (4x2.3 GHz Cortex-A73 and 4x1.7 GHz Cortex-A53) on a 10nm process:

Geekbench 6 scores for the Galaxy A51 versus today's leading device.
Geekbench 6 scores for the Galaxy A51 versus today's leading device.

The Pixel 4a's slow, eight-core big.LITTLE configuration was fabricated on an 8nm process:

Google spent more on the SoC for the Pixel 4a and enjoyed a later launch date, boosting performance relative to the A51.
Google spent more on the SoC for the Pixel 4a and enjoyed a later launch date, boosting performance relative to the A51.

Pixels have never sold well, and Google's focus on strong >SoC performance per dollar was sadly not replicated across the Android ecosystem, forcing us to use the A51 as our stand-in.

Devices within the envelope of our attention are 15-25% as fast as those carried by programmers and their bosses — even in wealthy markets.

The Galaxy may be slightly faster than last year's recommendation of the Galaxy A50 for testing, but the picture is muddy:

Geekbench 5 shows almost no improvement between the A50 and the A51.
Geekbench 5 shows almost no improvement between the A50 and the A51.

Geekbench 6 shows the same story within the margin of error. The low-end is stagnant, and still <a href='https://www.statista.com/statistics/934471/smartphone-shipments-by-price-category-worldwide/' target='_new'>30% of worldwide volume</a>.
Geekbench 6 shows the same story within the margin of error. The low-end is stagnant, and still 30% of worldwide volume.

If you're building a test lab today, refurbished A51s can be had for ~$150. Even better, the newer Nokia G100 can be had for as little as $100, and it's faithful to the sluggish original in nearly every respect.7

If your test bench is based on last year's recommended A50 or Nokia G11, I do not recommend upgrading in 2024. The absolute gains are so slight that the difference will be hard to feel, and bench stability has a value all its own. Looking forward, we can also predict that our bench performance will be stable until 2025.

Claims about how "performant" modern frontend tools are have to be evaluated in this slow, stagnant context.

It's a bit easier to understand the Desktop situation because the Edge telemetry I have access to provides statistically significant insight into 85+% of the market.

The TL;DR for desktop performance is that Edge telemetry puts ~45% of devices in a "low-end" bucket, meaning they have <= 4 cores or <= 4GB of RAM.

Device Tier Fleet % Definition
Low-end 45% Either:
<= 4 cores, or
<= 4GB RAM
Medium 48% HDD (not SSD), or
4-16 GB RAM, or
4-8 cores
High 7% SSD +
> 8 cores +
> 16GB RAM

20% of users are on HDDs (not SSDs) and nearly all of those users also have low (and slow) cores.

You might be tempted to dismiss this data because it doesn't include Macs, which are faster than the PC cohort. Recall, however, that the snapshot also excludes ChromeOS.

ChromeOS share has veered wildly in recent years, representing 50%-200% of Mac shipments in a given per quarter. In '21 and '22, ChromeOS shipments regularly doubled Mac sales. Despite post-pandemic mean reversion, according to IDC ChromeOS devices outsold Macs ~5.7M to ~4.7M in 2023 Q2. The trend reversed in Q3, with Macs almost doubling ChromeOS sales, but slow ChromeOS devices aren't going away and, from a population perspective, more than offset Macs today. Analysts also predict growth in the low end of the market as educational institutions begin to refresh their past purchases.

Desktop-attached networks continue to improve, notably in the U.S. Regulatory intervention and subsidies have done much to spur enhancements in access to U.S. fixed broadband, although disparities in access remain and the gains may not persist.

This suggests that it's time to also bump our baseline for desktop tests beyond the 5Mbps/1Mbps/28ms configuration that WebPageTest.org's "Cable" profile has defaulted to for desktop tests.

How far should we bump it? Publicly available data is unclear, and I've come to find out that Edge's telemetry lacks good network observation statistics (doh!).

But the comedy of omissions doesn't end there: Windows telemetry doesn't capture a proxy for network quality, I no longer have access to Chrome's data, the population-level telemetry available from CrUX is unhelpful, and telcos li...er...sorry, "market their products in accordance with local laws and advertising standards."

All of this makes it difficult to construct an estimate.

One option is to use a population-level assessment of medians from something like the Speedtest.net data and then construct a histogram from median speeds. This is both time-consuming and error-prone, as population-level data varies widely across the world. Emerging markets with high mobile internet use and dense populations can feature poor fixed-line broadband penetration compared with Western markets.

Another option is to mathematically hand-wave using the best evidence we can get. This might allow us to reconstruct probable P75 and P90 values if we know something about the historical distribution of connections. From there, we can gut-check using other spot data. To do this, we need to assume some data set is representative, a fraught decision all its own.8 Biting the bullet, we could start from the Speedtest.net global survey data, which currently fails to provide anything but medians (P50):

Speedtest.net's global median values are unhelpful on their own, both because they represent users who are testing for speed (and not organic throughput) and because they don't give us a fuller understanding of the distribution.
Speedtest.net's global median values are unhelpful on their own, both because they represent users who are testing for speed (and not organic throughput) and because they don't give us a fuller understanding of the distribution.

After many attempted Stupid Math Tricks with poorly fitting curves (bandwidth seems to be a funky cousin of log-normal), I've decided to wing it and beg for help: instead of trying to be clever, I'm leaning on Cloudflare Radar's P25/P50/P75 distributions for populous, openly-connected countries with >= ~50M internet users. It's cheeky, but a weighted average of the P75 of download speeds (3/4ths of all connections are faster) should get us in the ballpark. We can then use the usual 5:1 downlink:uplink ratio to come up with an uplink estimate. We can also derive a weighted average for the P75 RTT from Cloudflare's data. Because Cloudflare doesn't distinguish mobile from desktop connections, this may be an overly conservative estimate, but it's still be more permissive than what we had been pegged to in years past:

National P75 Downlink and RTT
Country P75 Downlink (Mbps) P75 RTT (ms)
India 4 114
USA 11 58
Indonesia 5 81
Brazil 8 71
Nigeria 3 201
Pakistan 3 166
Bangladesh 5 114
Japan 17 42
Mexico 7 75
Egypt 4 100
Germany 16 36
Turkey 7 74
Philippines 7 72
Vietnam 7 72
United Kingdom 16 37
South Korea 24 26
Population Weighted Avg. 7.2 94

We, therefore, update our P75 link estimate 7.2Mbps down, 1.4Mbps up, and 94ms RTT.

This is a mild crime against statistics, not least of all because it averages unlike quantities and fails to sift mobile from desktop, but all the other methods available at time of writing are just as bad. Regardless, this new baseline is half again as much link capacity as last year, showing measurable improvement in networks worldwide.

If you or your company are able to generate a credible worldwide latency estimate in the higher percentiles for next year's update, please get in touch.

The forces that shape the PC population have been largely fixed for many years. Since 2010, volumes have been on a slow downward glide path, shrinking from ~350MM per year in a decade ago to ~260MM in 2018. The pandemic buying spree of 2021 pushed volumes above 300MM per year for the first time in eight years, with the vast majority of those devices being sold at low-end price points — think ~$300 Chromebooks rather than M1 MacBooks.

Lest we assume low-end means "short-lived", recent announcements regarding software support for these devices will considerably extend their impact. This low-end cohort will filter through the device population for years to come, pulling our performance budgets down, even as renewed process improvement is unlocking improved power efficiency and performance at the high end of the first-sale market. This won't be as pronounced as the diffusion of $100 smartphones has been in emerging markets, but the longer life-span of desktops is already a factor in our model.

Per our methodology from last year which uses the 5-8 year replacement cycle for a PC, we update our target date to late 2017 or early 2018, but leave the average-selling-price fixed between $600-700. Eventually we'll need to factor in the past couple of years of gyrations in inflation and supply chains into account when making an estimate, but not this year.

So what did $650, give or take, buy in late 2017 or early 2018?

One option was a naf looking tower from Dell, optimistically pitched at gamers, with a CPU that scores poorly versus a modern phone, but blessedly includes 8GB of RAM.

In laptops (the larger segment), ~$650 bought the Lenovo Yoga 720 (12"), with a 2-core (4-thread) Core i3-7100U and 4GB of RAM. Versions with more RAM and a faster chip were available, but cost considerably more than our budget. This was not a fast box. Here's a device with that CPU compared to a modern phone; not pretty:

The phones of wealthy developers absolutely smoke the baseline PC.
The phones of wealthy developers absolutely smoke the baseline PC.

It's considerably faster than some devices still being sold to schools, though.

What does this mean for our target devices? There's wild variation in performance per dollar below $600 which will only increase as inflation-affected cohorts grow to represent a larger fraction of the fleet. Intel's move off of 14nm (finally!) also means that gains are starting to arrive at the low end, but in an uneven way. General advice is therefore hard to issue. That said, we can triangulate based on what we know about the market:

My recommendation, then, to someone setting up a new lab today is not to spend more than $350 on new a test device. Consider laptops with chips like the N4120, N4500, or the N5105. Test devices should also have no more than 8GB of RAM, and preferably 4GB. The 2021 HP 14 is a fine proxy. The updated ~$375 version will do in a pinch, but try to spend less if you can. Test devices should preferably score no higher than 1,000 in single-core Geekbench 6 tests; a line the HP 14's N4120 easily ducks, clocking in at just over 350.

There's a lot of good news embedded in this year's update. Devices and networks have finally started to get faster (as predicted), pulling budgets upwards.

At the same time, the community remains in denial about the disastrous consequences of an over-reliance on JavaScript. This paints a picture of path dependence — frontend isn't moving on from approaches that hurt users, even as the costs shift back onto teams that have been degrading life for users at the margins.

We can anticipate continued improvement in devices, while network gains will level out as the uneven deployment of 5G stumbles forward. Regardless, the gap between the digital haves and have-nots continues to grow. Those least able to afford fast devices are suffering regressive taxation from developers high on DX fumes.

It's no mystery why folks in the privilege bubble are not building with empathy or humility when nobody calls them to account. What's mysterious is that anybody pays them to do it.

The PM and EM disciplines have utterly failed, neglecting to put business constraints on the enthusiasms of developers. This burden is falling, instead, on users and their browsers. Browsers have had to step in as the experience guardians of last resort, indicating a market-wide botching of the job in technology management ranks and an industry-scale principal-agent issue amongst developers.

Instead of cabining the FP crowd's proclivities for the benefit of the business, managers meekly repeat bullshit like "you can't hire for fundamentals" while bussing in loads of React bootcampers. It is not too much to ask that managers run bake-offs and hire for skills in platform fundamentals that serve businesses better over time. The alternative is continued failure, even for fellow privilege bubble dwellers.

Case in point: this post was partially drafted on airplane wifi, and I can assure you that wealthy folks also experience RTTs north of 500ms and channel capacity in the single-digit-Mbps.

Even the wealthiest users step into the wider world sometimes. Are these EMs and PMs really happy to lose that business?

<em>Tap for a larger version.</em><br>Wealthy users are going to experience networks with properties that are even worse than the 'bad' networks offered to the Next Billion Users. At an altitude of 40k feet and a ground speed for 580 MPH somewhere over Alberta, CA, your correspondent's bandwidth is scarce, lopsided, and laggy.
Tap for a larger version.
Wealthy users are going to experience networks with properties that are even worse than the 'bad' networks offered to the Next Billion Users. At an altitude of 40k feet and a ground speed for 580 MPH somewhere over Alberta, CA, your correspondent's bandwidth is scarce, lopsided, and laggy.

Of course, any trend that can't continue won't, and INP's impact is already being felt. The great JavaScript merry-go-round may grind to a stop, but the momentum of consistently bad choices is formidable. Like passengers on a cruise ship ramming a boardwalk at flank speed, JavaScript regret is dawning far too late. As the good ship Scripting shudders and lists on the remains of the ferris wheel, it's not exactly clear how to get off, but the choices that led us here are becoming visible, if only through their negative consequences.

We got to a place where performance has been a constant problem in large part because a tribe of programmers convinced themselves that it wasn't and wouldn't be. The circa '13 narrative asserted that:

  • CPUs would keep getting faster (just like they always had).
  • Networks would get better, or at least not get worse.
  • Organisations had all learned the lessons of Google and Facebook's adventures in Ajax.

It was all bullshit, and many of us spotted it a mile away.

The problem is now visible and demands a solution, but the answers will be largely social, not technical. User-centered values must contest the airtime previouly taken by failed trickle-down DX mantras. Only when the dominant story changes will better architectures and tools win.

How deep was the branch? And how many cycles will the fault cost us? If CPUs and networks continue to improve at the rate of the past two years, and INP finally forces a reckoning, the answer might be as little as a decade. I fear we will not be so lucky; an entire generation has been trained to ignore reality, to prize tribalism rather than engineering rigor, and to devalue fundamentals. Those folks may not find the next couple of years to their liking.

Frontend's hangover from the JavaScript party is gonna suck.

FOOTNOTES

  1. The five second first-load target is arbitrary, and has always been higher than I would prefer. Five seconds on a modern computer is an eternity, but in 2016 I was talked down from my preferred three-second target by Googlers that despaired that "nobody" could hit that mark on the devices and networks of that era.

    This series continues to report budgets with that target, but keen readers will see that I'm also providing three-second numbers. The interactive estimation tool was also updated this year to provides the ability to configure the budget target.

    If you've got thoughts about how this should be set in future, or how it could be handled better, plesae get in touch.

  2. Frontend developers are cursed to program The Devil's Computer. Web apps execute on slow devices we don't spec or provision, on runtimes we can barely reason about, lashed to disks and OSes taxed by malware and equally invasive security software, over networks with the variability of carrier pigeons.

    It's vexing, then, that contemporary web development practice has decided that the way to deliver great experiences is to lean into client CPUs and mobile networks, the most unreliable, unscalable properties of any stack.

    And yet, here we are in 2024, with Reactors somehow still anointed to decree how and where code should run, despite a decade of failure to predict the obvious, or even adapt to the world as it has been. The mobile web overtook desktop eight years ago, and the best time to call bullshit on JS-first development was when we could first see the trends clearly.

    The second best time is now.

  3. Engineering is design under constraint, with the goal to develop products that serve users and society.

    The opposite of engineering is bullshit; substituting fairy tales for inquiry and evidence.

    For the frontend to earn and keep its stripes as an engineering discipline, frontenders need to internalise the envelope of what's possible on most devices.

  4. For at least a decade to come, 5G will continue to deliver unevenly depending on factors including building materials, tower buildout, supported frequencies, device density, radio processing power, and weather. Yes, weather (PDF).

    Even with all of those caveats, 5G networks aren't the limiting factor in wealthy geographies; devices are. It will take years for the deployed base to be fully replaced with 5G-capable handsets, and we should expect the diffusion to be "lumpy", with wealthy markets seeing 5G device saturation at nearly all price points well in advance of less affluent countries where capital availability for 5G network roll-outs will dominate.

  5. Ookla! Opensignal! Cloudflare! Akamai! I beseech thee, hear my plea and take pity, oh mighty data collectors.

    Whilst you report medians and averages (sometimes interchangeably, though I cannot speculate why), you've stopped publishing useable histogram information about the global situation, making the reports nearly useless for anything but telco marketing. Opensignal has stopped reporting meaningful 4G data at all, endangering any attempt at making sense.

    Please, I beg of you, publish P50, P75, P90, and P95 results for each of your market reports! And about the global situation! Or reach out directly and share what you can in confidence so I can generate better guidance for web developers.

  6. Both the benchmark A51 and Pixel 4a devices were eventually sold in 5G variants (A51 5G, Pixel 4a 5G), but at a price of $500 brand-new, unlocked at launch, making them more than 40% above the price of the base models and well above our 2020-2021 ASP of $350-$375.

  7. Samsung's lineup is not uniform worldwide, with many devices being region-specific. The closest modern (Western) Samsung device to the A51 is the Samsung A23 5G, which scores in the range of the Pixel 4a. As a result of the high CPU score and 5G modem, it's hard to recommend it — or any other current Samsung model — as a lab replacement. Try the Nokia G100 instead.

  8. The idea that any of the publicly available data sets is globally representative should set off alarms.

    The obvious problems include (but are not limited to):

    • geographic differences in service availability and/or deployed infrastructure,
    • differences in market penetration of observation platforms (e.g., was a system properly localised? Equally advertised?), and
    • mandated legal gaps in coverage.

    Of all the hand-waving we're doing to construct an estimate, this is the biggest leap and one of the hardest to triangulate against.

]]>
Why Are Tech Reporters Sleeping On The Biggest App Store Story? 2024-01-01T00:00:00Z https://infrequently.org/2024/01/the-web-is-the-app-store/ The tech news is chockablock1 with antitrust rumblings and slow-motion happenings. Eagle-eyed press coverage, regulatory reports, and legal discovery have comprehensively documented the shady dealings of Apple and Google's app stores. Pressure for change has built to an unsustainable level. Something's gotta give.

This is the backdrop to the biggest app store story nobody is writing about: on pain of steep fines, gatekeepers are opening up to competing browsers. This, in turn, will enable competitors to replace app stores with directories of Progressive Web Apps. Capable browsers that expose web app installation and powerful features to developers can kickstart app portability, breaking open the mobile duopoly.

But you'd never know it reading Wired or The Verge.

With shockingly few exceptions, coverage of app store regulation assumes the answer to crummy, extractive native app stores is other native app stores. This unexamined framing shapes hundreds of pieces covering regulatory events, including by web-friendly authors. The tech press almost universally fails to mention the web as a substitute for native apps and fail to inform readers of its potential to disrupt app stores.

As Cory Doctorow observed:

"An app is just a web-page wrapped in enough IP to make it a crime to defend yourself against corporate predation."

The implication is clear: browsers unchained can do to mobile what the web did to desktop, where more than 70% of daily "jobs to be done" happen on the web.

Replacing mobile app stores will look different than the web's path to desktop centrality, but the enablers are waiting in the wings. It has gone largely unreported that Progressive Web Apps (PWAs) have been held back by Apple and Google denying competing browsers access to essential APIs.2

Thankfully, regulators haven't been waiting on the press to explain the situation. Recent interventions into mobile ecosystems include requirements to repair browser choice, and the analysis backing those regulations takes into account the web's role as a potential competitor (e.g., Japan's JFTC (pdf)).

Regulators seem to understand that:

  • App stores protect proprietary ecosystems through preferential discovery and capabilities.
  • Stores then extract rents from developers dependent on commodity capabilities duopolists provide only through proprietary APIs.
  • App portability threatens the proprietary agenda of app stores.
  • The web can interrupt this model by bringing portability to apps and over-the-top discovery through search. This has yet to happen because...
  • The duopolists, in different ways, have kneecapped competing browsers along with their own, keeping the web from contesting the role of app stores.

Apple and Google saw what the web did to desktop, and they've laid roadblocks to the competitive forces that would let history repeat on smartphones.

The web's potential to disrupt mobile is evident to regulators, advocates, and developers. So why does the tech news fail to explain the situation?

Consider just one of the many antitrust events of recent months. It was covered by The Verge, Mac Rumors, Apple Insider, and more.

None of the linked articles note browser competition's potential to upend app stores. Browsers unshackled have the potential to free businesses from build-it-twice proprietary ecosystems, end rapacious app store taxes, pave the way for new OS entrants — all without the valid security concerns side-loading introduces.

Lest you think this an isolated incident, this article on the impact of the EU's DMA lacks any hint of the web's potential to unseat app stores. You can repeat this trick with any DMA story from the past year. Or spot-check coverage of the NTIA's February report.

Reporters are "covering" these stories in the lightest sense of the word. Barrels of virtual ink has been spilt documenting unfair app store terms, conditions, and competition. And yet.

In an industry obsessed with "disruption," why is this David vs. Goliath story going untold? Some theories, in no particular order.

First, Mozilla isn't advocating for a web that can challenge native apps, and none of the other major browser vendors are telling the story either. Apple and Google have no interest in seeing their lucrative proprietary platforms supplanted, and Microsoft (your narrator's employer) famously lacks sustained mobile focus.

Next, it's hard to overlook that tech reporters live like wealthy people, iPhones and all. From that vantage point, it's often news that the web is significantly more capable on other OSes (never mind that they spend much of every day working in a desktop browser). It's hard to report on the potential of something you can't see for yourself.

Also, this might all be Greek. Reporters and editors aren't software engineers, so the potential of browser competition can remain understandably opaque. Stories that include mention of "alternative app stores" generally fail to mention that these stores may not be as safe, or that OS restrictions on features won't disappear just because of a different distribution mechanism, or that the security track records of the existing duopolist app stores are sketchy at best. Under these conditions, it's asking a lot to expect details-based discussion of alternatives, given the many technical wrinkles. Hopefully, someone can walk them through it.

Further, market contestability theory has only recently become a big part of the tech news beat. Regulators have been writing reports to convey their understanding of the market, and to shape effective legislation that will unchain the web, but smart folks unversed in both antitrust and browser minutiae might need help to pick up what regulators are putting down.

Lastly, it hasn't happened yet. Yes, Progressive Web Apps have been around for a few years, but they haven't had an impact on the iPhones that reporters and their circles almost universally carry. It's much easier to get folks to cover stories that directly affect them, and this is one that, so far, largely hasn't.

The seeds of web-based app store dislocation have already been sown, but the chicken-and-egg question at the heart of platform competition looms.

On the technology side, Apple has been enormously successful at denying essential capabilities to the web through a strategy of compelled monoculture combined with strategic foot-dragging.

Missing alt text

As an example, the eight-year delay in implementing Push Notifications for the web3 kept many businesses from giving the web a second thought. If they couldn't re-engage users at the same rates as native apps, the web might as well not exist on phones. This logic has played out on a loop over the last decade, category-by-category, with gatekeepers preventing competing browsers from bringing capabilities to web apps that would let them supplant app stores2:1 while simultaneously keeping them from being discovered through existing stores.

Proper browser choice could upend this situation, finally allowing the web to provide "table stakes" features in a compelling way. For the first time, developers could bring the modern web's full power to wealthy mobile users, enabling the "write once, test everywhere" vision, and cut out the app store middleman — all without sacrificing essential app features or undermining security.

Sunsetting the 30% tax requires a compelling alternative, and Apple's simultaneous underfunding of Safari and compelled adoption of its underpowered engine have interlocked to keep the web out of the game. No wonder Apple is massively funding lobbyists, lawyers, and astroturf groups to keep engine diversity at bay while belatedly battening the hatches.

On the business side, managers think about "mobile" as a category. Rather than digging into the texture of iOS, Android, and the differing web features available on each, businesses tend to bulk accept or reject the app store model. One sub-segment of "mobile" growing the ability to route around highway robbery Ts & Cs is tantalising, but not enough to change the game; the web, like other metaplatforms, is only a disruptive force when pervasive and capable.4

A prohibition on store discovery for web apps has buttressed Apple's denial of essential features to browsers:

Even if developers overcome the ridiculous hurdles that Apple's shoddy browser engine throws up, they're still <a href='https://developer.apple.com/app-store/review/guidelines/#2.4'>prevented by Apple policy</a> from making interoperable web apps discoverable where users look for them.
Even if developers overcome the ridiculous hurdles that Apple's shoddy browser engine throws up, they're still prevented by Apple policy from making interoperable web apps discoverable where users look for them.

Google's answer to web apps in Play is a dog's breakfast, but it does at least exist for developers willing to put in the effort, or for teams savvy enough to reach for PWA Builder.

Recent developments also point to a competitive future for capable web apps.

First, browser engine choice should become a reality on iOS in the EU in 2024, thanks to the plain language of the DMA. Apple will, of course, attempt to delay the entry of competing browsers through as-yet-unknown strategies, but the clock is ticking. Once browsers can enable capable web apps with easier distribution, the logic of the app store loses a bit of its lustre.

Work is also underway to give competing browsers a chance to facilitate PWAs that can install other PWAs. Web App Stores would then become a real possibility through browsers that support them, and we should expect that regulatory and legislative interventions will facilitate this in the near future. Removed from the need to police security (browsers have that covered) and handle distribution (websites update themselves), PWA app stores like store.app can become honest-to-goodness app management surfaces that can safely facilitate discovery and sync.

PWA app stores like Appscope and store.app exist, but they're hobbled by gatekeepers that have denied competing browsers access to APIs that could turn PWA directories into real contenders.

It's no surprise that Apple and Google have kept private the APIs needed to make this better future possible. They built the necessary infrastructure for the web to disrupt native, then kept it to themselves. This potential has remained locked away within organisations politically hamstrung by native app store agendas. But all of that is about to change.

This begs the question: where's the coverage? This is the most exciting moment in more than 15 years for the web vs. native story, but the tech press is whiffing it.

2024 will be packed to the gills with app store and browser news, from implementation of the DMA, to the UK's renewed push into mobile browsers and cloud gaming, to new legislation arriving in many jurisdictions, to the first attempts at shipping iOS ports of Blink and Gecko browsers. Each event is a chance to inform the public about the already-raging battle for the future of the phone.

It's still possible to reframe these events and provide better context. We need a fuller discussion about what it will mean for mobile OSes to have competing native app stores when the underlying OSes are foundationally insecure. There are also existing examples of ecosystems with this sort of choice (e.g., China), and more needs to be written about the implications for users and developers. Instead of nirvana, the insecure status quo of today's mobile OSes, combined with (even more) absentee app store purveyors, turns side-loading into an alternative form of lock-in, with a kicker of added insecurity for users. With such a foundation, the tech-buying public could understand why a browser's superior sandboxing, web search's better discovery, and frictionless links are better than dodgy curation side-deals and "beware of dog" sign security.

The more that folks understand the stakes, the more likely tech will genuinely change for the better. And isn't that what public interest journalism is for?

Thanks to Charlie, Stuart Langride, and Frances Berriman for feedback on drafts of this post.

FOOTNOTES

  1. Antitrust is now a significant tech beat, and recent events frequently include browser choice angles because regulators keep writing regulations that will enhance it. This beat is only getting more intense, giving the tech press ample column inches to explain the status quo more deeply and and educate around the most important issues.

    In just the last two months:

    All but one of the 19 links above are from just the last 60 days, a period which includes a holiday break in the US and Europe. With the EU's DMA coming into force in March and the CMA back on the job, browser antitrust enforcement is only accelerating. It sure would be great if reporters could occasionally connect these dots.

  2. The stories of how Apple and Google have kept browsers from becoming real app stores differ greatly in their details, but the effects have been nearly identical: only their browsers could offer installation of web apps, and those browsers have done shockingly little to support web developers who want to depend on the browser as the platform.

    The ways that Apple has undermined browser-based stores is relatively well known: no equivalent to PWA install or "Smart Banners" for the web, no way for sites to suppress promotion of native apps, no ability for competing browsers to trigger homescreen installation until just this year, etc. etc. The decade-long build of Apple's many and varied attacks on the web as a platform is a story that's both tired and under-told.

    Google's malfeasance has gotten substantially less airtime, even among web developers – nevermind the tech press.

    The story picks up in 2017, two years after the release of PWAs and Push Notifications in Chrome. At the time, the PWA install flow was something of a poorly practised parlour trick: installation used an unreliable homescreen shortcut API that failed on many devices with OEM-customised launchers. The shortcut API also came laden with baggage that prevented effective uninstall and cross-device sync.

    To improve this situation, "WebAPKs" were developed. This new method of installation allows for deep integration with the OS, similar to the Application Identity Proxy feature that Windows lets browsers to provide for PWAs, with one notable exception: on Android, only Chrome gets to use the WebAPK system.

    Without getting into the weeds, suffice to say many non-Chrome browsers requested access. Only Google could meaningfully provide this essential capability across the Android ecosystem. So important were WebAPKs that Samsung gave up begging and reverse engineered it for their browser on Samsung devices. This only worked on Samsung phones where Suwon's engineers could count on device services and system keys not available elsewhere. That hasn't helped other browsers, and it certainly isn't an answer to an ecosystem-level challenge.

    Without WebAPK API access, competing browsers can't innovate on PWA install UI and can't meaningfully offer PWA app stores. Instead, the ecosystem has been left to limp along at the excruciating pace of Chrome's PWA UI development.

    Sure, Chrome's PWA support has been a damn sight better than Safari's, but that's just damning with faith praise. Both Apple and Google have done their part to quietly engineer a decade of unchallenged native app dominance. Neither can be trusted as exclusive stewards of web competitiveness. Breaking the lock on the doors holding back real PWA installation competition will be a litmus test for the effectiveness of regulation now in-flight.

  3. Push Notifications were, without exaggeration, the single most requested mobile Safari feature in the eight years between Chromium browsers shipping and Apple's 2023 capitulation.

    It's unedifying to recount all of the ways Apple prevented competing iOS browsers from implementing Push while publicly gaslighting developers who requested this business-critical feature. Over and over and over again. It's also unhelpful to fixate on the runarounds that Apple privately gave companies with enough clout to somehow find an Apple rep to harangue directly. So, let's call it water under the bridge. Apple shipped, so we're good, right?

    Right?

    I regret to inform you, dear reader, that it is not, in fact, "good".

    Despite most of a decade to study up on the problem space, and nearly 15 years of of experience with Push, Apple's implementation is anything but complete.

    The first few releases exposed APIs that hinted at important functionality that was broken or missing. Features as core as closing notifications, or updating text when new data comes in. The implementation of Push that Apple shipped could not allow a chat app to show only the latest message, or a summary. Instead, Apple's broken system leaves a stream of notifications in the tray for every message.

    Many important features didn't work. Some still don't.. And the pathetic set of customisations provided for notifications are a sick, sad joke.

    Web developers have once again been left to dig through the wreckage to understand just how badly Apple's cough "minimalist" cough implementation is compromised. And boy howdy, is it bad.

    Apple's implementation might have passed surface-level tests (gotta drive up that score!), but it's unusable for serious products. It's possible to draw many possible conclusions from this terrible showing, but even the relative charity of Hanlon's Razor is damning.

    Nothing about this would be worse than any other under-funded, trailing-edge browser over the past three decades (which is to say, a bloody huge problem), except for Apple's well-funded, aggressive, belligerent ongoing protest to every regulatory attempt to allow true browser choice for iPhone owners.

    In the year 2024, you can have any iOS browser you like. You can even set them as default. They might even have APIs that look like they'll solve important product needs, but as long as they're forced to rely on Apple's shit-show implementation, the web can't ever be a competitive platform.

    When Apple gets to define the web's potential, the winner will always be native, and through it, Apple's bottom line.

  4. The muting effect of Apple's abuse of monopoly over wealthy users to kneecap the web's capabilities is aided by the self-censorship of web developers. The values of the web are a mirror world to native, where developers are feted for adopting bleeding-edge APIs. On the web, features aren't "available" until 90+% of all users have access to them. Because iOS is at least 20% of the pie), web developers don't go near features Apple fails to support. Which is a lot.

    caniuse.com's "Browser Score" is one way to understand the scale of the gap in features that Apple has forced on all iOS browsers.
    The Web Platform Tests dashboard highlights 'Browser Specific Failures', which only measure failures in tests for features the browser claims to support. Not only are iOS browsers held back by Apple's shockingly poor feature support, but the features that _are_ available are broken so often that many businesses feel no option but to retreat to native APIs that Apple doesn't break on a whim, forcing the logic of the app store on them if they want to reach valuable users.

    Apple's pocket veto over the web is no accident, and its abuse of that power is no bug.

    Native app stores can only take an outsized cut if the web remains weak and developers stay dependent on proprietary APIs to access commodity capabilities. A prohibition on capable engines prevents feature parity, suppressing competition. A feature-poor, unreliable open web is essential to prevent the dam from breaking.

    Why, then, have competing browser makers played along? Why aren't Google, Mozilla, Microsoft, and Opera on the ramparts, waving the flag of engine choice? Why do they silently lend their brands to Apple's campaign against the web? Why don't they rename their iOS browsers to "Chrome Lite" or "Firefox Lite" until genuine choice is possible? Why don't they ask users to write their representatives or sign petitions for effective browser choice? It's not like they shrink from it for other worthy causes.

    I'm shocked by not surprised by the tardiness of browser bosses to seize the initiative. Instead of standing up to unfair terms, they've rolled over time and time again. It makes a perverse sort of sense.

    More than 30 years have passed since we last saw effective tech regulation. The careers of those at the top have been forged under the unforgiving terms of late-stage, might-makes-right capitalism, rather than the logic of open markets and standards. Today's bosses didn't rise by sticking their necks above the parapets to argue virtue and principle. At best, they kept the open web dream alive by quietly nurturing the potential of open technology, hoping the situation would change.

    Now it has, and yet they cower.

    Organisations that value conflict aversion and "the web's lane is desktop" thinking get as much of it as they care to afford.

  5. Recall that Apple won an upset victory in March after litigating the meaning of the word "may" and arguing that the CMA wasn't wrong to find after multiple years of investigations that Apple were (to paraphrase) inveterate shitheels, but rather that the CMA waited too long (six months) to bring an action which might have had teeth.

    Yes, you're reading that right; Apple's actual argument to the Competition Appeal Tribunal amounted to a mashup of rugged, free-market fundamentalist " but mah regulatory certainty!", performative fainting into strategically placed couches, and feigned ignorance about issues it knows it'll have to address in other jurisdictions.

    Thankfully, the Court of Appeals was not to be taken for fools. Given the harsh (in British) language of the reversal, we can hope a chastened Competition Appeal Tribunal will roll over less readily in future.

  6. If you're getting the sense that legalistic hair-splitting is what Apple spends its billion-dollar-per-year legal budget on because it has neither the facts nor real benefits to society on its side, wait 'till you hear about some of the stuff it filed with Japan's Fair Trade Commission!

    A clear strategy is being deployed. Apple:

    • First claims there's no there there (pdf). When that fails...
    • Claims competitors that it has expressly ham-strung are credible substitutes. When that fails...
    • Claims security would suffer if reasonable competition were allowed. Rending of garments is performed while prophets of doom recycle the script that the sky will fall if competing browsers are allowed (which would, in turn, expand the web's capabilities). Many treatments of this script fill the inboxes of regulators worldwide. When those bodies investigate, e.g. the history of iOS's forced-web-monoculture insecurity, and inevitably reject these farcical arguments, Apple...
    • Uses any and every procedural hurdle to prevent intervention in the market it has broken.

    The modern administrative state indulges firms with "as much due process as money can buy", and Apple knows it, viciously contesting microscopic points. When bluster fails, huffingly implemented, legalistic, hair-splitting "fixes" are deployed on the slowest possible time scale. This strategy buys years of delay, and it's everywhere: browser and mail app defaults, payment alternatives, engine choice, and right-to-repair. Even charging cable standardisation took years longer than it should have thanks to stall tactics. This maximalist, joined-up legal and lobbying strategy works to exhaust regulators and bamboozle legislators. Delay favours the monopolist.

    A firm that can transform the economy of an entire nation just by paying a bit of the tax it owes won't even notice a line item for lawyers to argue the most outlandish things at every opportunity. Apple (correctly) calculates that regulators are gun-shy about punishing them for delay tactics, so engagement with process is a is a win by default. Compelling $1600/hr white-shoe associates to make ludicrous, unsupportable claims is a de facto win when delay brings in billions. Regulators are too politically cowed and legally ham-strung to do more, and Apple plays process like a fiddle.

]]>
Safari 16.4 Is An Admission 2023-02-22T00:00:00Z https://infrequently.org/2023/02/safari-16-4-is-an-admission/ If you're a web developer not living under a rock, you probably saw last week's big Safari 16.4 reveal. There's much to cheer, but we need to talk about why this mega-release is happening now, and what it means for the future.

But first, the list!

Apple's summary combines dozens of minor fixes with several big-ticket items. Here's an overview of the most notable features, prefixed with the year they shipped in Chromium:

  • : Web Push for iOS (but only for installed PWAs)
  • : PWA Badging API (for unread counts) and id support (making updates smoother)
  • : PWA installation for third-party browsers (but not to parity with "Smart Banners")
  • A bevy of Web Components features, many of which Apple had held up in standards bodies for years1, including:
  • Myriad small CSS improvements and animation fixes, but also:
    • : CSS Typed OM for faster styling from JavaScript
    • : CSS Custom Properties can now be animated
  • : <iframe> lazy loading
  • : Clear-Site-Data for Service Worker use at scale
  • : Web Codecs for video (but not audio)
  • : WASM SIMD for better ML and games
  • : Compression Streams
  • : Reporting API (for learning about crashes and metrics reporting)
  • : Screen Orientation & Screen Wake Lock APIs (critical for games)
  • : Offscreen Canvas (but only 2D, which isn't what folks really need)
  • Critical usability and quality fixes for WebRTC

A number of improvements look promising, but remain exclusive to macOS and iPadOS:

  • Fullscreen API fixes
  • AVIF and AV1 support

The lack of iOS support for Fullscreen API on <canvas> elements continues to harm game makers; likewise, the lack of AVIF and AV1 holds back media and streaming businesses.

Regardless, Safari 16.4 is astonishingly dense with delayed features, inadvertantly emphasising just how far behind WebKit has remained for many years and how effective the Blink Launch Process has been in allowing Chromium to ship responsibly while consensus was witheld in standards by Apple.

The requirements of that process accelerated Apple's catch-up implementations by mandating proof of developer enthusiasm for features, extensive test suites, and accurate specifications. This collateral put the catch-up process on rails for Apple.

The intentional, responsible leadership of Blink was no accident, but to see it rewarded so definitively is gratifying.

The size of the release was expected in some corners, owing to the torrent of WebKit blog posts over the last few weeks:

This is a lot, particularly considering that Apple has upped the pace of new releases to once every eight weeks (or thereabouts) over the past year and a half.

Leading browsers moved to 6-week update cadence by 2011 at the latest, routinely delivering fixes at a quick clip. It took another decade for Apple to finally adopt modern browser engineering and deployment practices.

Starting in September 2021, Safari moved to an eight-week cadence. This is a sea change all its own.

Before Safari 15, Apple only delivered two substantial releases per year, a pattern that had been stable since 2016:

  • New features were teased at WWDC in the early summer
  • They landed in the Fall alongside a new iOS version
  • A second set of small features trickled out the next Spring

For a decade, two releases per year meant that progress on WebKit bugs was a roulette that developers lost by default.

In even leaner years (2012-2015), a single Fall release was all we could expect. This excruciating cadence affected Safari along with every other iOS browser forced to put its badge on Apple's sub-par product.

Contrast Apple's manufactured scarcity around bug fix information with the open bug tracking and reliable candecne of delivery from leading browsers. Cupertino manages the actual work of Safari engineers through an Apple-internal system ("Radar"), making public bug reports a sort of parallel track. Once an issue is imported to a private Radar bug it's more likely to get developer attention, but this also obscures progress from view.

This lack of transparency is by design.

It provides Apple deniability while simultaneously setting low expectations, which are easier to meet. Developers facing showstopping bugs end up in a bind. Without competitive recourse, they can't even recommend a different browser because every iOS browser is forced to use WebKit, meaning every iOS browser is at least as broken as Safari.

Given the dire state of WebKit, and the challenges contributors face helping to plug the gaps, these heartbreaks have induced a learned helplessness in much of the web community. So little improved, for so long, that some assumed it never would.

But here we are, with six releases a year and WebKit accelerating the pace at which it's closing the (large) gap.

Many big-ticket items are missing from this release — iOS fullscreen API for <canvas>, Paint Worklets, true PWA installation APIs for competing browsers, Offscreen Canvas for WebGL, Device APIs (if only for installed web apps), etc. — but the pace is now blistering.

This is the power of just the threat of competition.

Apple's laywers have offered claims in court and in regulatory filings defending App Store rapaciousness because, in their telling, iOS browsers provide an alternative. If developers don't like the generous offer to take only 30% of revenue, there's always Cupertino's highly capable browser to fall back on.

The only problem is that regulators ask follow-up questions like "is it?" and "what do developers think?"

Which they did.

TL;DR: it wasn't, and developers had lots to say.

This is, as they say, a bad look.

And so Apple hedged, slowly at first, but ever faster as 2021 bled into 2022 and the momentum of additional staffing began to pay dividends.

Apple had the resources needed to build a world-beating browser for more than a decade. The choice to ship a slower, less secure, less capable engine was precisely that: a choice.

Starting in 2021, Apple made a different choice, opening up dozens of Safari team positions. From 2023 perspective of pervasive tech layoffs, this might look like the same exuberant hiring Apple's competitors recently engaged in, but recall Cupertino had maintained extreme discipline about Safari staffing for nearly two decades. Feast or famine, Safari wouldn't grow, and Apple wouldn't put significant new resourcing into WebKit, no matter how far it fell behind.

The decision to hire aggressively, including some "big gets" in standards-land, indicates more is afoot, and the reason isn't that Tim lost his cool. No, this is a strategy shift. New problems needed new (old) solutions.

Apple undoubtedly hopes that a less egregiously incompetent Safari will blunt the intensity of arguments for iOS engine choice. Combined with (previously winning) security scaremongering, reduced developer pressure might allow Cupertino to wriggle out of needing to compete worldwide, allowing it to ring-fence progress to markets too small to justify browser development resources (e.g., just the EU).

Increased investment also does double duty in the uncertain near future. In scenarios where Safari is exposed to real competition, a more capable engine provides fewer reasons for web developers to recommend other browsers. It takes time to board up the windows before a storm, and if competition is truly coming, this burst of energy looks like a belated attempt to batten the hatches.

It's critical to Apple that narrative discipline with both developers and regulators is maintained. Dilatory attempts at catch-up only work if developers tell each other that these changes are an inevitable outcome of Apple's long-standing commitment to the web (remember the first iPhone!?!). An easily distracted tech press will help spread the idea that this was always part of the plan; nobody is making Cupertino do anything it doesn't want to do, nevermind the frantic regulatory filings and legal briefings.

But what if developers see behind the veil? What if they begin to reflect and internalise Apple's abandonment of web apps after iOS 1.0 as an exercise of market power that held the web back for more than a decade?

That might lead developers to demand competition. Apple might not be able to ring-fence browser choice to a few geographies. The web might threaten Cupertino's ability to extract rents in precisely the way Apple represented in court that it already does.

Rumours of engine ports are afoot. The plain language of the EU's DMA is set to allow true browser choice on iOS. But the regulatory landscape is not at all settled. Apple might still prevent progress from spreading. It might yet sue its way to curtailing the potential size and scope of the market that will allow for the web to actually compete, and if it succeeds in that, no amount of fast catch-up in the next few quarters will pose a true threat to native.

Consider the omissions:

  • PWA installation prompting
  • Fullscreen for <canvas>
  • Real Offscreen Canvas
  • Improved codecs
  • Web Transport
  • WebGPU
  • Device APIs

Depending on the class of app, any of these can be a deal-breaker, and if Apple isn't facing ongoing, effective competition it can just reassign headcount to other, "more critical" projects when the threat blows over. It wouldn't be the first time.

So, this isn't done. Not by a long shot.

Safari 16.4 is an admission that competition is effective and that Apple is spooked, but it isn't an answer. Only genuine browser choice will ensure the taps stay open.

FOOTNOTES

  1. Apple's standards engineers have a long and inglorious history of stalling tactics in standards bodies to delay progress on important APIs, like Declarative Shadow DOM (DSD).

    The idea behind DSD was not new, and the intensity of developer demand had only increased since Dimitri's 2015 sketch. A 2017 attempt to revive it was shot down in 2018 by Apple engineers without evidence or data.

    Throughout this period, Apple would engage sparsely in conversations, sometimes only weighing in at biannual face-to-face meetings. It was gobsmacking to watch them argue that features were unnecessary directly to the developers in the room who were personally telling them otherwise. This was disheartening because a key goal of any proposal was to gain support from iOS. In a world where nobody else could ship-and-let-live, and where Mozilla could not muster an opinion (it did not ship Web Components until late 2018), any whiff of disinterest from Apple was sufficient to kill progress.

    The phrase "stop-energy" is often misused, but the dampening effect of Apple on the progress of Web Components after 2015-2016's burst of V1 design energy was palpable. After that, the only Web Components features that launched in leading-edge browsers were those that an engineer and PM were willing to accept could only reach part of the developer base.

    I cannot stress enough how effectively this slowed progress on Web Components. The pantomime of regular face-to-face meetings continued, but Apple just stopped shipping. What had been a grudging willingness to engage on new features became a stalemate.

    But needs must.

    In early 2020, after months of background conversations and research, Mason Freed posted a new set of design alternatives, which included extensive performance research. The conclusion was overwhelming: not only was Declarative Shadow DOM now in heavy demand by the community, but it would also make websites much faster.

    The proposal looked shockingly like those sketched in years past. In a world where <template> existed and Shadow DOM V1 had shipped, the design space for Declarative Shadow DOM alternatives was not large; we just needed to pick one.

    An updated proposal was presented to the Web Components Community Group in March 2020; Apple objected on spurious grounds, offering no constructive counter.2

    Residual questions revolved around security implications of changing parser behaviour, but these were also straightforward. The first draft of Mason's Explainer even calls out why the proposal is less invasive than a whole new element.

    Recall that Web Components and the <template> element themselves were large parser behaviour changes; the semantics for <template> even required changes to the long-settled grammar of XML (long story, don't ask). A drumbeat of (and proposals for) new elements and attributes post-HTML5 also represent identical security risks, and yet we barrel forward with them. These have notably included <picture>, <portal> (proposed), <fencedframe> (proposed), <dialog>, <selectmenu> (proposed), and <img srcset>.

    The addition of <template shadowroot="open"> would, indeed, change parser behaviour, but not in ways that were unknowably large or unprecedented. Chromium's usage data, along with the HTTP Archive crawl HAR file corpus, provided ample evidence about the prevalence of patterns that might cause issues. None were detected.

    And yet, at TPAC 2020, Apple's representatives continued to press the line that large security issues remained. This was all considered at length. Google's security teams audited the colossal volume of user-generated content Google hosts for problems and did not find significant concerns. And yet, Apple continued to apply stop-energy.

    The feature eventually shipped with heavy developer backing as part of Chromium 90 in April 2021 but without consensus. Apple persistently repeated objections that had already been answered with patient explication and evidence.

    Cupertino is now implementing this same design, and Safari will support DSD soon.

    This has not been the worst case of Apple's deflection and delay — looking at you, Push Notifications — but serves as an exemplar. Problem solvers have been forced into a series of high-stakes gambits to solve developer problems by Apple (and, to a lesser extent, Mozilla) over Cupertino's dozen years of engine disinvestment.

    Even in Chromium, DSD was delayed by several quarters. Because of the Apple Browser Ban, cross-OS availability was postponed by two years. The fact that Apple will ship DSD without changes and without counterproposals across the long arc of obstruction implies claims of caution were, at best, overstated.

    The only folks to bring data to the party were Googlers and web developers. No new thing was learned through groundless objection. No new understanding was derived from the delay. Apple did no research about the supposed risks. It has yet to argue why it's safe now, but wasn't then.

    So let's call it what it was: concern trolling.

    Uncritical acceptance of the high-quality design it had long delayed is an admission, of sorts. It demonstrates Apple's ennui about developer and user needs (until pressed), paired with great skill at deflection.

    The playbook is simple:

    • Use opaque standards processes to make it look like occasional attendance at a F2F meeting is the same thing as good-faith co-engineering.
    • "Just ask questions" when overstretched or uninterested in the problem.
    • Spread FUD about the security or privacy of a meticulously-vetted design.
    • When all else fails, say you will formally object and then claim that others are "shipping whatever they want" and "not following standards" when they carefully launch a specced and tested design you were long consulted about, but withheld good faith engagement to improve.

    The last step works because only insiders can distinguish between legitimate critiques and standards process jockeying. Hanging the first-mover risk around the neck of those working to solve problems is nearly cost-free when you can also prevent designs from moving forward in standards, paired with a market veto (thaks to anti-competitive shenanigans).

    Play this dynamic out over dozens of features across a decade, and you'll better understand why Chromium participants get exercised about the responsibility theatre various Apple engineers put on to avoid engaging substantively, while simultaneously blocking all forward movement. Understood in context, it decodes as delay and deflection; a way to avoid using standards bodies to help actually solve problems.

    Cupertino has paid no price for deploying these smoke screens, thanks to the Apple Browser Ban and a lack of curiosity in the press. Without those shields, Apple engineers would have had to offer convincing arguments from data for why their positions were correct. Instead, they have whatabouted for over three years, only to suddenly implement proposals they recently opposed when the piercing gaze of regulators finally fell on WebKit.3

  2. The presence or absence of a counterproposal when objecting to a design is a primary indicator of seriousness within a standards discussion. All parties will have been able to examine proposals before any meeting, and in groups that operate by consensus, blocking objections are understood to be used sparingly by serious parties.

    It's normal for disagreements to surface over proposed designs, but engaged and collaborative counter-parties will offer soft concerns – "we won't block on this, but we think it could be improved..." – or through the offer to bring a counterproposal. The benefits of a concrete counter are large. It demonstrates good faith in working to solve the problem and signals a willingness to ship the offered design. Threats to veto, or never implement a specific proposal, are just not done in the genteel world of web standards.

    Over the past decade, making veto threats while offering neither data nor a counterproposal have become a hallmark of Apple's web standards footprint. It's a bad look, but it continues because nobody in those rooms wants to risk pissing off Cupertino. Your narrator considered a direct accounting of just the consequences of these tactics a potentially career-ending move; that's how serious the stakes are.

    The true power of a monopoly in standards is silence — the ability to get away with things others blanch at because they fear you'll hold an even larger group of hostages next time.

  3. Apple has rolled out the same playbook in dozens of areas over the last decade, and we can learn a few things from this experience.

    First, Apple corporate does not care about the web, no matter how much the individuals that work on WebKit (deeply) care. Cupertino's artificial bandwidth constraints on WebKit engineering ensured that it implements only when pressured.

    That means that external pressure must be maintained. Cupertino must fear losing their market share for doing a lousy job. That's a feeling that hasn't been felt near the intersection of I-280 and CA Route 85 in a few years. For the web to deliver for users, gatekeepers must sleep poorly.

    Lastly, Apple had the capacity and resources to deliver a richer web for a decade but simply declined. This was a choice — a question of will, not of design correctness or security or privacy.

    Safari 16.4 is evidence, an admission that better was possible, and the delaying tactics were a sort of gaslighting. Apple disrespects the legitimate needs of web developers when allowed, so it must not be.

    Lack of competition was the primary reason Apple feared no consequence for failing to deliver. Apple's protectionism towards Safari's participation-prize under-achievement hasn't withstood even the faintest whiff of future challengers, which should be an enduring lesson: no vendor must ever be allowed to deny true and effective browser competition.

]]>
The Market for Lemons 2023-02-04T00:00:00Z https://infrequently.org/2023/02/the-market-for-lemons/ For most of the past decade, I have spent a considerable fraction of my professional life consulting with teams building on the web.

It is not going well.

Not only are new services being built to a self-defeatingly low UX and performance standard, existing experiences are pervasively re-developed on unspeakably slow, JS-taxed stacks. At a business level, this is a disaster, raising the question: "why are new teams buying into stacks that have failed so often before?"

In other words, "why is this market so inefficient?"

George Akerlof's most famous paper introduced economists to the idea that information asymmetries distort markets and reduce the quality of goods because sellers with more information can pass off low-quality products as more valuable than informed buyers appraise them to be. (PDF, summary)

Customers that can't assess the quality of products pay too much for poor quality goods, creating a disincentive for high-quality products to emerge while working against their success when they do. For many years, this effect has dominated the frontend technology market. Partisans for slow, complex frameworks have successfully marketed lemons as the hot new thing, despite the pervasive failures in their wake, crowding out higher-quality options in the process.1

These technologies were initially pitched on the back of "better user experiences", but have utterly failed to deliver on that promise outside of the high-management-maturity organisations in which they were born.2 Transplanted into the wider web, these new stacks have proven to be expensive duds.

The complexity merchants knew their environments weren't typical, but sold their highly specialised tools to folks shopping for general purpose solutions anyway. They understood most sites lack latency budgeting, dedicated performance teams, hawkish management reviews, ship gates to prevent regressions, and end-to-end measurements of critical user journeys. They grasped that massive investment in controlling complexity is the only way to scale JS-driven frontends, but warned none of their customers.

They also knew that their choices were hard to replicate. Few can afford to build and maintain 3+ versions of a site ("desktop", "mobile", and "lite"), and vanishingly few web experiences feature long sessions and login-gated content.3

Armed with this knowledge, they kept the caveats to themselves.

This information asymmetry persists; the worst actors still haven't levelled with their communities about what it takes to operate complex JS stacks at scale. They did not signpost the delicate balance of engineering constraints that allowed their products to adopt this new, slow, and complicated tech. Why? For the same reason used car dealers don't talk up average monthly repair costs.

The market for lemons depends on customers having less information than those selling shoddy products. Some who hyped these stacks early on were earnestly ignorant, which is forgivable when recognition of error leads to changes in behaviour. But that's not what the most popular frameworks of the last decade did.

As time passed, and the results continued to underwhelm, an initial lack of clarity was revealed to be intentional omission. These omissions have been material to both users and developers. Extensive evidence of these failures was provided directly to their marketeers, often by me. At some point (certainly by 2017) the omissions veered into intentional prevarication.

Faced with the dawning realisation that this tech mostly made things worse, not better, the JS-industrial-complex pulled an Exxon.

They could have copped to an honest error, admitted that these technologies require vast infrastructure to operate; that they are unscalable in the hands of all but the most sophisticated teams. They did the opposite, doubling down, breathlessly announcing vapourware year after year to forestall critical thinking about fundamental design flaws. They also worked behind the scenes to marginalise those who pointed out the disturbing results and extraordinary costs.

Credit where it's due, the complexity merchants have been incredibly effective in one regard: top-shelf marketing discipline.

Over the last ten years, they have worked overtime to make frontend an evidence-free zone. The hucksters knew that discussions about performance trade-offs would not end with teams investing more in their technology, so boosterism and misdirection were aggressively substituted for evidence and debate. Like a curtain of Halon descending to put out the fire of engineering dialogue, they blanketed the discourse with toxic positivity. Those who dared speak up were branded "negative" and "haters", no matter how much data they lugged in tow.

It was, of course, bullshit.

Astonishingly, gobsmackingly effective bullshit, but nonsense nonetheless. There was a point to it, though. Playing for time allowed the bullshitters to punt introspection of the always-wrong assumptions they'd built their entire technical ediface on:

In time, these misapprehensions would become cursed articles of faith.

All of this was falsified by 2016, but nobody wanted to turn on the house lights while the JS party was in full swing. Not the developers being showered with shiny tools and boffo praise for replacing "legacy" HTML and CSS that performed fine. Not the scoundrels peddling foul JavaScript elixirs and potions. Not the managers that craved a check to cut and a rewrite to take credit for in lieu of critical thinking about user needs and market research.

Consider the narrative Crazy Ivans that led to this point.

By 2013 the trashfuture was here, just not evenly distributed yet. Undeterred, the complexity merchants spent a decade selling <a href='/2022/12/performance-baseline-2023/'>inequality-exascerbating technology</a> as a cure-all tonic.
By 2013 the trashfuture was here, just not evenly distributed yet. Undeterred, the complexity merchants spent a decade selling inequality-exascerbating technology as a cure-all tonic.

It's challenging to summarise a vast discourse over the span of a decade, particularly one as dense with jargon and acronyms as that which led to today's status quo of overpriced failure. These are not quotes, but vignettes of distinct epochs in our tortured journey:

  • "Progressive Enhancement has failed! Multiple pages are slow and clunky!

    SPAs are a better user experience, and managing state is a big problem on the client side. You'll need a tool to help structure that complexity when rendering on the client side, and our framework works at scale"

    illustrative example  ]

  • "Instead of waiting on the JavaScript that will absolutely deliver a superior SPA experience...someday...why not render on the server as well, so that there's something for the user to look at while they wait for our awesome and totally scalable JavaScript to collect its thoughts?"

    an intro to "isomorphic javascript", a.k.a. "Server-Side Rendering", a.k.a. "SSR"  ]

  • "SPAs are a better experience, but everyone knows you'll need to do all the work twice because SSR makes that better experience minimally usable. But even with SSR, you might be sending so much JS that things feel bad. So give us credit for a promise of vapourware for delay-loading parts of your JS."

    impressive stage management  ]

  • "SPAs are a better experience. SSR is vital because SPAs take a long time to start up, and you aren't using our vapourware to split your code effectively. As a result, the main thread is often locked up, which could be bad?"

    Anyway, this is totally your fault and not the predictable result of us failing to advise you about the controls and budgets we found necessary to scale JS in our environment. Regardless, we see that you lock up main threads for seconds when using our slow system, so in a few years we'll create a parallel scheduler that will break up the work transparently"

    2017's beautiful overview of a fated errand and 2018's breathless re-brand  ]

  • "The scheduler isn't ready, but thanks for your patience; here's a new way to spell your component that introduces new timing issues but doesn't address the fact that our system is incredibly slow, built for browsers you no longer support, and that CPUs are not getting faster"

    representative pitch  ]

  • "Now that you're 'SSR'ing your SPA and have re-spelt all of your components, and given that the scheduler hasn't fixed things and CPUs haven't gotten faster, why not skip SPAs and settle for progressive enhancement of sections of a document?"

    "islands", "server components", etc.  ]

It's the Steamed Hams of technology pitches.

Like Chalmers, teams and managers often acquiesce to the contradictions embedded in the stacked rationalisations. Together, the community invented dozens of reasons to look the other way, from the theoretically plausible to the fully imaginary.

But even as the complexity merchant's well-intentioned victims meekly recite the koans of trickle-down UX — it can work this time, if only we try it hard enough! — the evidence mounts that "modern" web development is, in the main, an expensive failure.

The baroque and insular terminology of the in-group is a clue. It's functional purpose (outside of signalling) is to obscure furious plate spinning. The tech isn't working, but admitting as much would shrink the market for lemons.

You'd be forgiven for thinking the verbiage was designed to obfuscate. Little comfort, then, that folks selling new approaches must now wade through waist-deep jargon excrement to argue for the next increment of complexity.

The most recent turn is as predictable as it is bilious. Today's most successful complexity merchants have never backed down, never apologised, and never come clean about what they knew about the level of expense involved in keeping SPA-oriented technologies in check. But they expect you'll follow them down the next dark alley anyway:

An admission against interest.
An admission against interest.

And why not? The industry has been down to clown for so long it's hard to get in the door if you aren't wearing a red nose.

The substitution of heroic developer narratives for user success happened imperceptibly. Admitting it was a mistake would embarrass the good and the great alike. Once the lemon sellers embed the data-light idea that improved "Developer Experience" ("DX") leads to better user outcomes, improving "DX" became and end unto itself. Many who knew better felt forced to play along.

The long lead time for falsifying trickle-down UX was a feature, not a bug; they don't need you to succeed, only to keep buying.

As marketing goes, the "DX" bait-and-switch is brilliant, but the tech isn't delivering for anyone but developers.4 The highest goal of the complexity merchants is to put brands on showcase microsites and to make acqui-hiring failing startups easier. Performance and success of the resulting products is merely a nice-to-have.

You'd think there would be data, that we would be awash in case studies and blog posts attributing product success to adoption of SPAs and heavy frameworks in an incontrovertable way.

And yet, after more than a decade of JS hot air, the framework-centric pitch is still phrased in speculative terms because there's no there there. The complexity merchants can't cop to the fact that management competence and lower complexity — not baroque technology — are determinative of product and end-user success.

The simmering, widespread failure of SPA-premised approaches has belatedly forced the JS colporteurs to adapt their pitches. In each iteration, they must accept a smaller rhetorical lane to explain why this stack is still the future.

The excuses are running out.

At long last, the journey has culminated with the rollout of Core Web Vitals. It finally provides an objective quality measurement that prospective customers can use to assess frontend architectures.

It's no coincidence the final turn away from the SPA justification has happened just as buyers can see a linkage between the stacks they've bought and the monetary outcomes they already value; namely SEO. The objective buyer, circa 2023, will understand heavy JS stacks as a regrettable legacy, one that teams who have hollowed out their HTML and CSS skill bases will pay for dearly in years to come.

No doubt, many folks who know their JS-first stacks are slow will do as Akerlof predicts, and obfuscate for as long as possible. The market for lemons is, indeed, mostly a resale market, and the excesses of our lost decade will not be flushed from the ecosystem quickly. Beware tools pitching "100 on Lighthouse" without checking the real-world Core Web Vitals results.

A subtle aspect of Akerlof's theory is that markets in which lemons dominate eventually shrink. I've warned for years that the mobile web is under threat from within, and the depressing data I've cited about users moving to apps and away from terrible web experiences aligns with that theory.

When websites feel like categorically worse experiences to the folks who write the checks, why should anyone expect them to spend a lot on them? And when websites stop being where accurate information and useful services are, will anyone still believe there's a future in web development?

The lost decade we've suffered at the hands of lemon purveyors isn't just a local product travesty; it's also an ecosystem-level risk. Forget AI putting web developers out of jobs; JS-heavy web stacks have been shrinking the future market for your services for years.

As Stigliz memorably quipped:

Adam Smith's invisible hand — the idea that free markets lead to efficiency as if guided by unseen forces — is invisible, at least in part, because it is not there.

But dreams die hard.

I'm already hearing laments from folks who have been responsible citizens of framework-landia lo these many years. Oppressed as they were by the lemon vendors, they worry about babies being throw out with the bathwater, and I empathise. But for the sake of users, and for the new opportunities for the web that will open up when experiences finally improve, I say "chuck those tubs".

Chuck 'em hard, and post the photos of the unrepentant bastards that tried to palm off this nonsense behind the cash register.

Anti JavaScript JavaScript Club

We lost a decade to smooth talkers and hollow marketeering; folks who failed the most basic test of intellectual honesty: signposting known unknowns. Instead of engaging honestly with the emerging evidence, they sold lemons and shrunk the market for better solutions. Furiously playing catch-up to stay one step ahead of market rejection, frontend's anguished, belated return to quality has been hindered at every step by those who would stand to lose if their false premises and hollow promises were to be fully re-evaluated.

Toxic mimicry and recalcitrant ignorance must not be rewarded.

Vendor's random walk through frontend choices may eventually lead them to be right twice a day, but that's not a reason to keep following their lead. No, we need to move our attention back to the folks that have been right all along. The people who never gave up on semantic markup, CSS, and progressive enhancement for most sites. The people who, when slinging JS, have treated it as special occasion food. The tools and communities whose culture puts the user ahead of the developer and hold evidence of doing better for users in the highest regard.1:1

It's not healing, and it won't be enough to nurse the web back to health, but tossing the Vercels and the Facebooks out of polite conversation is, at least, a start.

Deepest thanks to Bruce Lawson, Heydon Pickering, Frances Berriman, and Taylor Hunt for their thoughtful feedback on drafts of this post.

FOOTNOTES

  1. You wouldn't know it from today's frontend discourse, but the modern era has never been without high-quality alternatives to React, Angular, Ember, and other legacy desktop-era frameworks.

    In a bazaar dominated by lemon vendors, many tools and communities have been respectful of today's mostly-mobile users at the expense of their own marketability. These are today's honest brokers and they deserve your attention far more than whatever solution to a problem created by React that the React community is on about this week.

    This has included JS frameworks with an emphasis on speed and low overhead vs. cadillac comfort of first-class IE8 support:

    It's possible to make slow sites with any of these tools, but the ethos of these communities is that what's good for users is essential while developer luxuries are nice-to-have — even as they compete furiously for developer attention. This uncompromising focus on real quality is what has been muffled under the blanket complexity merchants have thrown over today's frontend discourse.

    Similarly, the SPA orthodoxy that precipitated the market for frontend lemons has been challenged both by the continued success of "legacy" tools like WordPress, as well as a new crop of HTML-first systems that provide JS-friendly authoring but output that's largely HTML and CSS:

    The essential thing about tools that succeed more often than not is starting with simple output. The difficulty in managing what you've explicitly added based on incremental need, vs. what you've been bequeathed by an inscrutable Rube Goldberg-esque metaframework, is an order of magnitude in cost and usability. Teams that adopt tools with simpler default output start with simpler problems that tend to have better-understood solutions.

  2. Organisations that manage their systems (not the other way around) can succeed with any set of tools. They might pick some elegant ones and some awkward ones, but the sine qua non of their success isn't what they pick up, it's how they hold it.

    Recall that Facebook became a multi-billion dollar, globe-striding colossus using PHP and C++.

    The differences between FB and your applications are likely legion. This is why it's fundamentally lazy and wrong for TLs and PMs to accept any sort of argument along the lines of "X scales, FB uses it".

    Pigs can fly; it's only matter of how much force you apply — but if you aren't willing to fund creation of a large enough trebuchet, it's unlikley that porcine UI will take wing in your organisation.

  3. I hinted last year at and under-developed model for how we can evolve our discussion around web performance to take account of the larger factors that distinguish different kinds of sites.

    While it doesn't account for many corner-cases, and is insufficient on its own to describe multi-modal experiences like WordPress (a content-producing editor for a small fraction of important users vs. shallow content-consumption reader experience for most), I wind up thinking about the total latency incurred in a user's session divided by the number of interactions. This raises a follow-on question: what's an interaction? Elsewhere, I've defined it as "turns through the interaction loop", but can be more easily described as "taps or clicks that involve your code doing work". This helpfully excludes scrolling, but includes navigations.

    ANYWAY, all of this nets out a session-depth weighted intuition about when and where heavyweight frameworks make sense to load up-front:

    Sites with shorter average sessions can afford less JS up-front.
    Sites with shorter average sessions can afford less JS up-front.

    Social media sites that gate content behind a login (and can use the login process to pre-load bundles), and which have tons of data about session depth — not to mention ML-based per-user bundling, staffed performance teams, ship gates to prevent regressions, and the funding to build and maintain at least 3 different versions of the site — can afford to make fundamentally different choices about how much to load up-front and for which users.

    The rest of us, trying to serve all users from a single codebase, need to prefer conservative choices that align with our management capacity to keep complexity in check.

  4. The "DX" fixation hasn't even worked for developers, if we're being honest. Teams I work with suffer eye-watering build times, shockingly poor code velocity, mysterious performance cliffs, and some poor sod stuck in a broom closet that nobody bothers, lest the webs stop packing.

    And yet, these same teams are happy to tell me they couldn't live without the new ball-and-chain.

    One group, after weeks of debugging a particularly gnarly set of issues brought on by their preposterously inefficient "CSS-in-JS" solution, combined with React's penchant for terrible data flow management, actually said to me that they were so glad they'd moved everything to hooks because it was "so much cleaner" and that "CSS-in-JS" was great because "now they could reason about it"; nevermind the weeks they'd just lost to the combination of dirtier callstacks and harder to reason about runtime implications of heisenbug styling.

    Nothing about the lived experience of web development has meaningfully improved, except perhaps for TypeScript adding structure to large codebases. And yet, here we are. Celebrating failure as success while parroting narratives about developer productivity that have no data to back them up.

    Sunk-cost fallacy rules all we survey.

]]>
The Performance Inequality Gap, 2023 2022-12-19T00:00:00Z https://infrequently.org/2022/12/performance-baseline-2023/

TL;DR: To serve users at the 75th percentile (P75) of devices and networks, we can now afford ~150KiB of HTML/CSS/fonts and ~300-350KiB of JavaScript (gzipped). This is a slight improvement on last year's budgets, thanks to device and network improvements. Meanwhile, sites continue to send more script than is reasonable for 80+% of the world's users, widening the gap between the haves and the have-nots. This is an ethical crisis for frontend.

Last month, I had the honour of joining what seemed like the entire web performance community at performance.now() in Amsterdam.

The talks are up on YouTube behind a paywall, but my slides are mirrored here1:

performance.now(): The Global Baseline

The talk, like this post, is an update on network and CPU realities this series has documented since 2017. More importantly, it is also a look at what the latest data means for our collective performance budgets.

In the interest of brevity, here's what we should be aiming to send over the wire per page in 2023 to reach interactivity in less than 5 seconds on first load:23

  • ~150KiB of HTML, CSS, images, and render-blocking font resources
  • No more than ~300-350KiB of JavaScript

This implies a heavy JS payload, which most new sites suffer from for reasons both bad and beyond the scope of this post. With a more classic content profile — mostly HTML and CSSwe can afford much more in terms of total data, because JavaScript is still the costliest way to do things and CPUs at the global P75 are not fast.

These estimates also assume some serving discipline, including:

These targets are anchored to global estimates for networks and devices at the 75th percentile4.

More on how those estimates are constructed in a moment, but suffice to say, it's messy. Where the data is weak, we should always prefer conservative estimates.

Based on trends and historical precedent, there's little reason for optimism that things are better than they seem. Indeed, misplaced optimism about disk, network, and CPU resources is the background music to frontend's lost decade.


Interaction-to-Next Paint measures page responsivness, and shows a large gap between desktop and mobile today
Per the 2022 Web Almanac, which pulls data from real-world devices via the CrUX dataset, today's web offers poor performance for the majority of users who are on mobile devices.

It is not an exaggeration to say that modern frontend is so enamoured of post-scarcity fairy tales that it is mortgaging the web's future for another night drinking at the JavaScript party.

We're burning our inheritance and polluting the ecosystem on shockingly thin, perniciously marketed claims of "speed" and "agility" and "better UX" that have not panned out at all. Instead, each additional layer of JavaScript cruft has dragged us further from living within the limits of what we can truly afford.

No amount of framework vendor happy talk can hide the reality that we are sending an escalating and unaffordable amount of JavaScript.
No amount of framework vendor happy talk can hide the reality that we are sending an escalating and unaffordable amount of JavaScript.

This isn't working for users or for businesses that hire developers hopped up Facebook's latest JavaScript hopium. A correction is due.

In years past, I haven't paid as much attention to the situation on desktops. But researching this year's update has turned up sobering facts that should colour our collective understanding.

From Edge's telemetry, we see that nearly half of devices fall into our "low-end" designation, which means that they have:

  • HDDs (not SSDs)
  • 2-4 CPU cores
  • 4GB RAM or less

Add to this the fact that desktop devices have a lifespan between five and eight years, on average. This means the P75 device was sold in 2016.

As this series has emphasised in years past, Average Selling Price (ASP) is performance destiny. To understand our P75 device, we must imagine what the ASP device was at the P75 age.5 That is, what was the average device in 2016? It sure wasn't a $2,000 M1 MacBook Pro, that's for sure.

No, it was a $600-$700 device. Think (best-case) 2-core, 4-thread married to slow, spinning rust.

Desktop-attached networks are hugely variable worldwide, including in the U.S., where the shocking effects of digital red-lining continue this day. And that's on top of globally uncompetitive service, thanks to shockingly lax regulation and legalised corruption.

As a result, we are sticking to our conservative estimates for bandwidth in line with WebPageTest's throttled Cable profile of 5Mbps bandwidth and ~25ms RTT.

Speeds will be much slower than advertised in many areas, particularly for rural users.

We've been tracking the mobile device landscape more carefully over the years and, as with desktop, ASPs today are tomorrow's performance destiny. Thankfully, device turnover is faster, with the average handset surviving only three to four years.

Without beating around the bush, our ASP 2019 device was an Android that cost between $300-$350, new and unlocked. It featured poor single and multi-core performance, and the high-end experience has continued to pull away from it since:

<em>Tap for a larger version.</em><br>Updated Geekbench five single-core scores for each mobile price point. TL;DR: your iPhone isn't real life.
Tap for a larger version.
Updated Geekbench five single-core scores for each mobile price point. TL;DR: your iPhone isn't real life.

<em>Tap for a larger version.</em><br>Android ecosystem <abbr title='system-on-chip'>SoC</abbr>s fare slightly better on multi-core performance, but the Performance Inequality Gap is growing there, too.
Tap for a larger version.
Android ecosystem SoCs fare slightly better on multi-core performance, but the Performance Inequality Gap is growing there, too.

As you can see, the gap is widening, in part because the high end has risen dramatically in price.

The best analogue you can buy for a representative P75 device today are ~$200 Androids from the last year or two, such as the Samsung Galaxy A50 and the Nokia G11.

These devices feature:

  • Eight slow, big.LITTLE ARM cores (A75+A55, or A73+A53) built on last-generation processes with very little cache
  • 4GiB of RAM
  • 4G radios

These are depressingly similar specs to devices I recommended for testing in 2017. Qualcomm has some 'splainin to do.

5G is still in its early diffusion phase, and the inclusion of a 5G radio is hugely randomising for device specs at today's mid-market price-point. It'll take a couple of years for that to settle.

Trustworthy mobile network data is challenging to acquire. Geographic differences create huge effects that we can see as variability in various global indexes. This variance forces us towards the bottom of the range when estimating our baseline, as mobile networks are highly contextual.

Triangulating from both speedtest.net and OpenSignal data (which has declined markedly in usefuleness), we're also going to maintain our global network baseline from last year:

  • 9Mbps bandwidth
  • 170ms RTT

This is a higher bandwidth estimate than might be reasonable, but also a higher RTT to cover the effects of high network behaviour variance. I'm cautiously optimistic that we'll be able to bump one or both of these numbers in a positive direction next year. But they stay put for now.

You don't have to take my word for it. If your product behavior or your own team's data or market research suggests different tradeoffs, then it's only right to set your own per-product baseline.

For example, let's say you send more HTML and less JavaScript, or your serving game is on lock and all critical assets load over a single H/2 link. How should your estimates change?

Per usual, I've also updated the rinky-dink live model that you can use to select different combinations of device, network, and content type.

<em>Tap to try the interactive version.</em>
Tap to try the interactive version.

Essential public services are now delivered primarily through digital channels in many countries. This means what the frontend community celebrates and promotes has a stochastic effect on the provision of those services — which leads to an uncomfortable conversation because, taken as a whole, it isn't working.

Pervasively poor results are part of why responsible public sector organisations are forced to develop HTML-first, progressive enhancement guidance in stark opposition to the "frontend consensus".

This is an indictment: modern frontend's fascination with towering piles of JavasScript complexity is not delivering better experiences for most users.

For a genuinely raw example, consider California, the state where I live. In early November, it was brought to my attention that CA.gov "felt slow", so I gave it a look. It was bad on my local development box, so I put it under the WebPageTest microscope. The results were, to be blunt, a travesty.

How did this happen? Well, per the new usual, overly optimistic assumptions about the state of the world accreted until folks at the margins were excluded.

In the case of CA.gov, it was an official Twitter embed that, for some cursed reason, had been built using React, Next.js, and the full parade of modern horrors. Removing the embed, along with code optimistically built in a pre-JS-bloat era that blocked rendering until all resources were loaded, resulted in a dramatic improvement:


Thanks to some quick work by the CA.gov team, the experience of the site radically improved between early November and mid-December, giving Californians easier access to critical information.

This is not an isolated incident. These sorts of disasters have been arriving on my desk with shocking frequency for years.

Nor is this improvement a success story, but rather a cautionary tale about the assumptions and preferences of those who live inside the privilege bubble. When they are allowed to set the agenda, folks who are less well-off get hurt.

It wasn't the embed engineer getting paid hundreds of thousands of dollars a year to sling JavaScript who was marginalised by this gross misapplication of overly complex technology. No, it was Californians who could least afford fast devices and networks who were excluded. Likewise, it hasn't been those same well-to-do folks who have remediate the resulting disasters. They don't even clean up their own messes.

Frontend's failure to deliver in today's mostly-mobile, mostly-Android world is shocking, if only for the durability of the myths that sustain the indefensible. We can't keep doing this.

As they say, any trend that can't continue won't.

FOOTNOTES

  1. Apologies for the lack of speaker notes in this deck. If there's sufficient demand, I can go back through and add key points. Let me know if that would help you or your team over on Mastodon.

  2. Since at least 2017, I've grown increasingly frustrated at the way we collectively think about the tradeoffs in frontend metrics. Per this year's post on a unified theory of web performance, it's entirely possible to model nearly every interaction in terms of a full page load (and vice versa).

    What does this tell us? Well, briefly, it tells us that the interaction loop for each interaction is only part of the story. Recall the loop's phases:

    1. Interactive (ready to handle input)
    2. Receiving input
    3. Acknowledging input, beginning work
    4. Updating status
    5. Work ends, output displayed
    6. GOTO 1

    Now imagine we collect all the interactions a user performs in a session (ignoring scrolling, which is nearly always handled by the browser unless you screw up), and then we divide the total set of costs incurred by the number of turns through the loop.

    Since our goal is to ensure users complete each turn through the loop with low latency and low variance, we can see the colourable claim for SPA architectures take shape: by trading off some initial latency, we can reduce total latency and variance. But this also gives rise to the critique: OK, but does it work?

    The answer, shockingly, seems to be "no" — at least not as practised by most sites adopting this technology over the past decade.

    The web performance community should eventually come to a more session-depth-weighted understanding of metrics and goals. Still, until we pull into that station, per-page-load metrics are useful. They model the better style of app construction and represent the most actionable advice for developers.

  3. The target that this series has used consistently has been reaching a consistently interactive ("TTI") state in less than 5 seconds on the chosen device and network baseline.

    This isn't an ideal target.

    First, even with today's the P75 network and device, we can aim higher (lower?) and get compelling experiences loaded and main-thread clean in much less than 5 seconds.

    Second, this target was set in covnersation back in 2016 in preparation for a Google I/O talk, based on what was then possible. At the time, this was still not ambitious enough, but the impact of an additional connection shrunk the set of origins that could accomplish the feat significantly.

    Lastly, P75 is not where mature teams and developers spend their effort. Instead, they're looking up the percentiles and focusing on P90+, and so for mature teams looking to really make their experiences sing, I'd happily recommend that you target 5 second TTI at P90 instead. It's possible, and on a good stack with a good team and strong management, a goal you can be proud to hit.

  4. Looking at the P75 networks and devices may strike mature teams and managers as a sandbagged goal and, honestly, I struggle with this.

    On the one hand, yes, we should be looking into the higher percentiles. But weaker goals aren't within reach for most teams today. If we moved the ecosystem to a place where it could reliably hit these limits and hold them in place for a few years, the web would stand a significantly higher chance of remaining relevant.

    On the other hand, these difficulties stack. Additive error means that targeting the combination P75 network and P75 device likely puts you north of P90 in the experiential distribution, but it's hard to know.

  5. Data-minded folks will be keenly aware that simply extrapolating from average selling price over time can lead to some very bad conclusions. For example, what if device volumes fluctuate significantly? What if, in more recent years, ASPs fluctuate significantly? Or what if divergence in underlying data makes comparison across years otherwise unreliable.

    These are classic questions in data analysis, and thankfully the PC market has been relatively stable in volumes, prices, and segmentation, even through the pandemic.

    As covered later in this post, mobile is showing signs of heavy divergence in properties by segment, with the high-end pulling away in both capability and price. This is happening even as global ASPs remain relatively fixed, due to the increases in low-end volume over the past decade. Both desktop and mobile are staying within a narrow Average Selling Price band, but in both markets (though for different reasons), the P75 is not where folks looking only at today's new devices might expect it to be.

    In this way, we can think of the Performance Inequality Gap as being an expression of Alberto Cairo's visual data lessons: things may look descriptively similar at the level of movement of averages between desktop and mobile, but the underlying data tells a very different story.

]]>
Apple Is Not Defending Browser Engine Choice 2022-06-23T00:00:00Z https://infrequently.org/2022/06/apple-is-not-defending-browser-engine-choice/ Gentle reader, I made a terrible mistake. Yes, that's right: I read the comments on a MacRumors article. At my age, one knows better. And yet.

As penance for this error, and for being short with Miguel, I must deconstruct the ways Apple has undermined browser engine diversity. Contrary to claims of Apple partisans, iOS engine restrictions are not preventing a "takeover" by Chromium — at least that's not the primary effect. Apple uses its power over browsers to strip-mine and sabotage the web, hurting all engine projects and draining the web of future potential.

As we will see, both the present and future of browser engine choice are squarely within Cupertino's control.

A refresher on Apple's iOS browser policies:

  • From iOS 2.0 in '08 to iOS 14 in late '20, Apple would not allow any browser but Safari to be the default.
  • For 14 years and counting, Apple has prevented competing browsers from bringing their own engines, forcing vendors to build skins over Apple's WebKit binary, which has historically been slower, less secure, and lacking in features.
  • Apple will not even allow competing browsers to provide different runtime flags to WebKit. Instead, Fruit Co. publishes a paltry set of options that carry an unmistakable odour of first-party app requirements.
  • Apple continues to self-preference through exclusive API access for Safari; e.g., the ability to install PWAs to the home screen, implement media codecs, and much else.

Defenders of Apple's monopoly offer hard-to-test claims, but many boil down to the idea that Apple's product is inferior by necessity. This line is frankly insulting to the good people that work on WebKit. They're excellent engineers; some of the best, pound for pound, but there aren't enough of them. And that's a choice.

Nobody frames it precisely this way; instead they'll say, if WebKit weren't mandated, Chromium would take over, or Google would dominate the web if not for the WebKit restriction. That potential future requires mechanisms of action — something to cause Safari users to switch. What are those mechanisms? And why are some commenters so sure the end is nigh for WebKit?

Recall the status quo: websites can already ask iOS users to download alternative browsers. Thanks to (belated) questioning by Congress, they can even be set as the user's default, ensuring a higher probability to generate search traffic and derive associated revenue. None of that hinges on browser engine choice; it's just marketing. At the level of commerce, Apple's capitulation on default browser choice is a big deal, but it falls short of true differentiation.

So, websites can already put up banners asking users to get different browsers, If WebKit is doomed, its failure must lie in other stars; e.g., that Safari's WebKit is inferior to Gecko and Blink.

But the quality and completeness of WebKit is entirely within Apple's control.

Past swings away from OS default browsers happened because alternatives offered new features, better performance, improved security, and good site compatibility. These are properties intrinsic to the engine, not just the badge on the bonnet. Marketing and distribution have a role, but in recent browser battles, better engines have powered market shifts.

To truly differentiate and win, competitors must be able to bring their own engines. The leads of OS incumbents are not insurmountable because browsers are commodities with relatively low switching costs. Better products tend to win, if allowed, and Apple knows it.

Destkop <abbr>OS</abbr>es have long created a vibrant market for browser choice, enabling competitors not tied to OS defaults to flourish over the years.
Destkop OSes have long created a vibrant market for browser choice, enabling competitors not tied to OS defaults to flourish over the years.

Apple's prohibition on iOS browser engine competition has drained the potential of browser choice to deliver improvements. Without the ability to differentiate on features, security, performance, privacy, and compatibility, what's to sell? A slightly different UI? That's meaningful, but identically feeble web features cap the potential of every iOS browser. Nobody can pull ahead, and no product can offer future-looking capabilities that might make the web a more attractive platform.

This is working as intended:

<a href='https://developer.apple.com/app-store/review/guidelines/#2.5.6'>Apple's policies</a> explicitly prevent meaningful competition between browsers on iOS. In 2022, you can have any default you like, as long as it's as buggy as Safari.
Apple's policies explicitly prevent meaningful competition between browsers on iOS. In 2022, you can have any default you like, as long as it's as buggy as Safari.

On OSes with proper browser competition, sites can recommend browsers with engines that cost less to support or unlock crucial capabilities. Major sites asking users to switch can be incredibly effective in aggregate. Standards support is sometimes offered as a solution, but it's best to think of it as a trailing indicator.1 Critical capabilities often arrive in just one engine to start with, and developers that need these features may have incentive to prompt users to switch.

Developers are reluctanct to do this, however; turning away users isn't a winning growth strategy, and prompting visitors to switch is passé.

Still, in extremis, missing features and the parade of showstopping bugs render some services impossible to deliver. In these cases, suggesting an alternative beats losing users entirely.

But what if there's no better alternative? This is the situation that Apple has engineered on iOS. Cui bono? — who benefits?

All iOS browsers present as Safari to developers. There's no point in recommending a better browser because none is available. The combined mass of all iOS browsing pegged to the trailing edge means that folks must support WebKit or decamp for Apple's App Store, where it hands out capabilities like candy, but at a shocking price.

iOS's mandated inadequacy has convinced some that when engine choice is possible, users will stampede of away from Safari. This would, in turn, cause developers to skimp on testing for Apple's engine, making it inevitable that browsers based on WebKit and other minority engines could not compete. Or so the theory goes.

But is it predestined?

Perhaps some users will switch, but browser market changes take a great deal of time, and Apple enjoys numerous defences.

To the extent that Apple wants to win developers and avoid losing users, it has plenty of time.

It took over five years for Chrome to earn majority share on Windows with a superior product, and there's no reason to think iOS browser share will move faster. Then there's the countervailing evidence from macOS, where Safari manages to do just fine.

Regulatory mandates about engine choice will also take more than a year to come into force, giving Apple plenty of time to respond and improve the competitiveness of its engine. And that's the lower bound.

Apple's pattern of malaicious compliance will likely postpone true choice even futher. As Apple fights tooth-and-nail to prevent alternative browser engines, it will try to create ambiguity about vendor's ability to ship their best products worldwide, potentially delaying high-cost investment in ports with uncertain market reach.

Cupertino may also try to create arduous processes that force vendors to individually challenge the lack of each API, one geography at a time. In the best case, time will still be lost to this sort of brinksmanship. This is time that Apple can use to improve WebKit and Safari to be properly competitive.

Why would developers recommend alternatives if Safari adds features, improves security, prioritises performance, and fumigates for showstopping bugs? Remember: developers don't want to prompt users to switch; they only do it under duress. The features and quality of Safari are squarely in Apple's control.

So, given that Apple has plenty of time to catch up, is it a rational business decision to invest enough to compete?

Browsers are both big business and industrial-scale engineering projects. Hundreds of folks are needed to implement and maintain a competitive browser with specialisations in nearly every area of computing. World-class experts in graphics, networking, cryptography, databases, language design, VM implementation, security, usability (particularly usable security), power management, compilers, fonts, high-performance layout, codecs, real-time media, audio and video pipelines, and per-OS specialisation are required. And then you need infrastructure; lots of it.

How much does all of this cost? A reasonable floor comes from Mozilla's annual reports. The latest consolidated financials (PDF) are from 2020 and show that, without marketing expenses, Mozilla spends between $380 and $430 million US per year on software development. Salaries are the largest category of these costs (~$180-210 million), and Mozilla economises by hiring remote employees paid local market rates, without large bonuses or stock-based compensation.

From this data, we can assume a baseline cost to build and maintain a competitive, cross-platform browser at $450 million per year.

Browser vendors fund their industrial-scale software engineering projects through integrations. Search engines pay browser makers for default placement within their products. They, in turn, make a lot of money because browsers send them transactional and commercial intent searches as part of the query stream.

Advertisers bid huge sums to place ads against keywords in these categories. This market, in turn, funds all the R&D and operational costs of search engines, including "traffic acquisition costs" like browser search default deals.2

How much money are we talking about? Mozilla's $450 million in annual revenue comes from approximately 8% of the desktop market and negligible mobile share. Browsers are big, big business.

Despite being largely open source, browsers and their engines are not loss leaders.

Safari, in particular, is wildly profitable. The New York Times reported in late 2020 that Google now pays Apple between $8-12 billion per year to remain Safari's default search engine, up from $1 billion in 2014. Other estimates put the current payments in the $15 billion range. What does this almighty torrent of cash buy Google? Searches, preferably of the commercial intent sort.

Mobile accounts for two-thirds of web traffic (or thereabouts), making outsized iOS adoption among wealthy users particularly salient to publishers and advertisers. Google's payments to Apple are largely driven by the iPhone rather than its niche desktop products where effective browser competition has reduced the influence of Apple's defaults.

Against considerable competition, Safari was used by 52% of visitors to <abbr>US</abbr> Government websites from macOS devices from <time datetime='2022-03-06'>March 6<sup>th</sup></time> to <time datetime='2022-04-04'>April 4<sup>th</sup>, 2022</time>
Against considerable competition, Safari was used by 52% of visitors to US Government websites from macOS devices from to


The influence of a dozen years of suppressed browser choice is evident on iOS, where Safari is used 90% of the time. Apple's policies caused Mozilla to delay producing an iOS browser for seven years, and its de minimus iOS share (versus 3.6% on macOS) is a predictable result.
The influence of a dozen years of suppressed browser choice is evident on iOS, where Safari is used 90% of the time. Apple's policies caused Mozilla to delay producing an iOS browser for seven years, and its de minimus iOS share (versus 3.6% on macOS) is a predictable result.


iOS represents 75% of all visits to <abbr>US</abbr> Government websites from Apple <abbr>OS</abbr>es
iOS represents 75% of all visits to US Government websites from Apple OSes

Even with Apple's somewhat higher salaries per engineer, the skeleton staffing of WebKit, combined with the easier task of supporting fewer platforms, suggests that Apple is unlikely to spend considerably more than Mozilla does on browser development. In 2014, Apple would have enjoyed a profit margin of 50% if it had spent half a billion on browser engineering. Today, that margin would be 94-97%, depending on which figure you believe for Google's payments.

In absolute terms, that's more profit than Apple makes selling Macs.

Compare Cupertino's 3-6% search revenue reinvestment in the web with Mozilla's near 100% commitment, then recall that Mozilla has consistently delivered a superior engine to more platforms. I don't know what's more embarrassing: that some folks argue with a straight face that Apple is trying hard to build a good browser, or that it is consistently overmatched in performance, security, and compatibility by a plucky non-profit foundation that makes just ~5% of Apple's web revenue.

Steve Jobs launched Safari for Windows in the same WWDC keynote that unveiled the iPhone.

WWDC 2007 - One More Thing: Safari for Windows

Commenters often fixate on the iPhone's original web-based pitch, but don't give Apple stick for reducing engine diversity by abandoning Windows three versions later.

Today, Apple doesn't compete outside its home turf, and when it has agency, it prevents others from doing so. These are not the actions of a firm that is consciously attempting to promote engine diversity. If Apple is an ally in that cause, it is only by accident.

Theories that postulate a takeover by Chromium dismiss Apple's power over a situation it created and recommits to annually through its budgeting process.

This is not a question of resources. Recall that Apple spends $85 billion per year on stock buybacks3, $15 billion on dividends, enjoys free cash flow larger than the annual budgets of 47 nations, and retain tens of billions of dollars of cash on hand.4 And that's to say nothing of Apple's $100+ billion in non-business-related long-term investments.

Even if Safari were a loss leader, Apple would be able to avoid producing a slower, stifled, less secure, famously buggy engine without breaking the bank.

Apple needs fewer staff to deliver equivalent features because Safari supports fewer OSes. The necessary investments are also R&D expenses that receive heavy tax advantages. Apple enjoys enviable discounts to produce a credible browser, but refuses to do so.

Unlike Microsoft's late and underpowered efforts with IE 7-11, Safari enjoys tolerable web compatibility, more than 90% share on a popular OS, and an unheard-of war chest with which to finance a defence. The postulated apocalypse seems far away and entirely within Apple's power to forestall.

One way to understand the voluntary nature of Safari's poor competitiveness is to put Cupertino's recent burst of effort in context.

When regulators and legislators began asking questions in 2019, a response was required. Following Congress' query about default browser choice, Apple quietly allowed it through iOS 14 (however ham-fistedly) the following year. This underscores Apple's gatekeeper status and the tiny scale of investment required to enable large changes.

In the past six months, the Safari team has gone on a veritable hiring spree. This month's WWDC announcements showcased returns on that investment. By spending more in response to regulatory pressure, Apple has eviscerated notions that it could not have delivered a safer, more capable, and competitive browser many years earlier.

Safari's incremental headcount allocation has been large compared to the previous size of the Safari team, but in terms of Apple's P&L, it's loose change. Predictably, hiring talent to catch up has come at no appreciable loss to profitability.

The competitive potential of any browser hinges on headcount, and Apple is not limited in its ability to hire engineering talent. Recent efforts demonstrate that Apple has been able to build a better browser all along and, year after year, chose not to.

For over a dozen years, setting any browser other than Safari as the iOS default was impossible. This spotted Safari a massive market share head-start. Meanwhile, restrictions on engine choice continue to hamstring competitors, removing arguments for why users should switch. But don't take my word for it; here's the recent "UK CMA Final Report on Mobile Ecosystems" summarising submissions by Mozilla and others (pages 154-155):

5.48 The WebKit restriction also means that browser vendors that want to use Blink or Gecko on other operating systems have to build their browser on two different browser engines. Several browser vendors submitted that needing to code their browser for both WebKit and the browser engine they use on Android results in higher costs and features being deployed more slowly.

5.49 Two browser vendors submitted that they do not offer a mobile browser for iOS due to the lack of differentiation and the extra costs, while Mozilla told us that the WebKit restriction delayed its entrance into iOS by around seven years

That's seven years of marketing, feature iteration, and brand loyalty that Mozilla sacrificed on the principle that if they could not bring their core differentiator, there was no point.

It would have been better if Mozilla had made a ruckus, rather than hoping the world would notice its stoic virtue, but thankfully the T-rex has roused from its slumber.

Given the hard times the Mozilla Foundation has found itself in, it seems worth trying to quantify the costs.

To start, Mozilla must fund a separate team to re-develop features atop a less-capable runtime. Every feature that interacts with web content must be rebuilt in an ad-hoc way using inferior tools. Everything from form autofill to password management to content blocking requires extra resources to build for iOS. Not only does this tax development of the iOS product, it makes coordinated feature launches more costly across all ports.

Most substantially, iOS policies against default browser choice — combined with "in-app-browser" and search entry point shenanigans — have delayed and devalued browser choice.

Until late 2020, users needed to explicitly tap the Firefox icon on the home screen to get back to their browser. Naïvely tapping links would, instead, load content in Safari. This split experience causes a sort of pervasive forgetfulness, making the web less useful.

Continuous partial amnesia about browser-managed information is bad for users, but it hurts browser makers too. On OSes with functional competition, convincing a user to download a new browser has a chance of converting nearly all of their browsing to that product. iOS (along with Android and Facebook's mobile apps) undermine this by constantly splitting browsing, ignoring the user's default. When users don't end up in their browser, searches occur through it less often, affecting revenue. Web developers also experience this as a reduction in visible share of browsing from competing products, reducing incentives to support alternative engines.

A foregetful web also hurts publishers. Ad bid rates are suppressed, and users struggle to access pay-walled content when browsing is split. The conspicuious lack of re-engagement features like Push Notifications are the rotten cherry on top, forcing sites to push users to the App Store where Apple doesn't randomly log users out, or deprive publishers of key features.

Users, browser makers, web developers, and web businesses all lose. The hat-trick of value destruction.

The pantomime of browser choice on iOS has created an anaemic, amnesiac web. Tapping links is more slogging than surfing when autofill fails, passwords are lost, and login state is forgotten. Browsers become less valuable as the web stops being a reliable way to complete tasks.

Can we quantify these losses?

Estimating lost business from user frustration and ad rate depression is challenging. But we can extrapolate what a dozen years of choice might have meant for Mozilla from what we know about how Apple monetises the web.

For the purposes of argument, let's assume Mozilla would be paid for web traffic at the same rate as Apple; $8-15 billion per year for ~75% share of traffic from Apple OSes.

If the traffic numbers to US government websites are reasonable proxies for the iOS/macOS traffic mix (big "if"s), then equal share for Firefox on iOS to macOS would be worth $215-400 million per year.5 Put differently; there's reason to think that Mozilla would not have suffered layoffs if Apple were an ally of engine choice.

Apple's policies have made the web a less compelling ecosystem, its anti-competitive behaviours drive up competitor's costs, and it simultaneously starves them of revenue by undermining browser choice.

If Apple is a friend of engine diversity, who needs enemies?

There is a narrow, fetid sense in which Apple's influence is nominally pro-diversity. Having anchored a significant fraction of web traffic at the trailing edge, businesses that do not decamp for the App Store may feel obliged to support WebKit.

This is a malignant form of diversity, not unlike other lagging engines through the years that harmed users and web-based businesses by externalizing costs. But on OSes with true browser choice, alternatives were meaningful.

Consider the loathed memory of IE 6, a browser that overstayed its welcome by nearly a decade. For as bad as it was, folks could recommend alternatives. Plugins also allowed us to transparently upgrade the platform.

Before the rise of open-source engines, the end of one browser lineage may have been a deep loss to ecosystem diversity, but in the past 15 years, the primary way new engines emerge has been through forking and remixing.

But the fact of an engine being different does not make that difference valuable, and WebKit's differences are incremental. Sure, Blink now has a faster layout engine, better security, more features, and fewer bugs, but like WebKit, it is also derived from KHTML. Both engines are forks and owe many present-day traits to their ancestors.

The history of browsers includes many forks and remixes. It's naïve to think that will end if iOS becomes hospitable to browser competition. After all, it has been competition that spurred engine improvements and forks.
The history of browsers includes many forks and remixes. It's naïve to think that will end if iOS becomes hospitable to browser competition. After all, it has been competition that spurred engine improvements and forks.

Today's KHTML descendants are not the end of the story. Future forks are possible. New codebases can be built from parts. Indeed, there's already valuable cross-pollination in code between Gecko, WebKit, and Chromium. Unlike the '90s and early 2000s, diversity can arrive in valuable increments through forking and recombination.

What's necessary for leading edge diversity, however, is funding.

By simultaneously taking a massive pot of cash for browser-building off the table, returning the least it can to engine development, and preventing others from filling the gap, Apple has foundationally imperilled the web ecosystem by destroying the utility of a diverse population of browsers and engines.

Apple has agency. It is not a victim, and it is not defending engine diversity.

A better, brighter future for the web is possible, and thanks to belated movement by regulators, increasingly likely. The good folks over at Open Web Advocacy are leading the way, clearly explaining to anyone who will listen both what's at stake and what it will take to improve the situation.

Investigations are now underway worldwide, so if you think Apple shouldn't be afraid of a bit of competition if it will help the web thrive, consider getting involved. And if you're in the UK or do business there, consider helping the CMA help the web before July 22nd, 2022. The future isn't written yet, and we can change it for the better.

FOOTNOTES

  1. Many commenters come to debates about compatibility and standards compliance with a mistaken view of how standards are made. As a result, they perceive vendors with better standards conformance (rather than content compatibility) to occupy a sort of moral high ground. They do not. Instead, it usually represents a broken standards-setting process.

    This can happen for several reasons. Sometimes standards bodies shutter, and the state of the art moves forward without them. This presents some risk for vendors that forge ahead without the cover of an SDO's protective IP umbrella, but that risk is often temporary and measured. SDOs aren't hard to come by; if new features are valuable, they can be standardised in a new venue. Alternatively, vendors can renovate the old one if others are interested in the work.

    More often, working groups move at the speed of their most obstinate participants, uncomfortably prolonging technical debates already settled in the market and preventing definitive documentation of the winning design. In other cases, a vendor may play games with intellectual property claims to delay standardisation or lure competitors into a patent minefield (as Apple did with Touch Events).

    At the leading edge, vendors need space to try new ideas without the need for the a priori consensus represented by a standard. However, compatibility concerns expressed by developers take on a different tinge over time.

    When the specific API details and capabilities of ageing features do not converge, a continual tax is placed on folks trying to build sites using features from that set. When developers stress the need for compatibility, it is often in this respect.

    Disingenuous actors sometimes try to misrepresent this interest and claim that all features must become standards before they are introduced in any engine. This interpretation runs against the long practice of internet standards development and almost always hides an ulterior motive.

    The role of standards is to consolidate gains introduced at the leading edge through responsible competition. Vendors that fail to participate constructively in this process earn scorn. They bring ignominy upon their houses by failing to bring implementations in line with the rough (documented and tested) consensus or by playing the heel in SDOs to forestall progress they find inconvenient.

    Vendors like Apple.

  2. In the financial reports of internet businesses, you will see the costs to acquire business through channels reported as "Traffic Acquisition Costs" or "TACM". Many startups report their revenue "excluding TAC" or "ex-TAC". These are all ways of saying, "we paid for lead generation", and search engines are no different.

  3. This is money Apple believes it cannot figure out a way to invest in its products. That's literally what share buybacks indicate. They're an admission that a company is not smart enough to invest the money in something productive. Buybacks are attractive to managers because they create artificial scarcity for shares to drive up realised employee compensation — their own included. Employees who are cheesed to realise that their projects are perennially short-staffed are encouraged not to make a stink through RSU appreciation. Everyone gets a cut, RSU-rich managers most of all.

  4. Different analysts use different ways of describing Apple's "cash on hand". Some analysts lump in all marketable securities, current and non-current, which consistently pushes the number north of $150 billion. Others report only the literal cash value on the books ($34 billion as of

    The picture is also clouded by changes in the way Apple manages its cash horde. Over the past two years, Apple has begun to draw from this almighty pile of dollars and spend more to inflate its stock price through share buybacks and dividends. This may cast Apple as more cash-poor than it is. A better understanding of the actual situation is derived from free cash flow. Perhaps Apple will continue to draw down from its tall cash mountain to inflate its stock price via buybacks, but that's not a material change in the amount Apple can potentially spend on improving its products.

  5. Since this post first ran, several commenters have noted a point I considered while writing, but omitted in order to avoid heaping scorn on a victim; namely that Mozilla's management has been asleep at the switch regarding the business of its business.

    Historically, when public records were available for both Opera and Mozilla, it was easy to understand how poorly Mozilla negotiated with search partners. Under successive leaders, Mozilla negotiated deals that led to payments less than as half as much per point of share. There's no reason to think MoCo's negotiating skills have improved dramatically in recent years. Apple, therefore, is likely to capture much more revenue per search than Mozilla ears from a Firefox user.

    But even if Mozilla only made 1/3 of Apple's haul for equivalent use, the combined taxes of iOS feature re-development and loss of revenue would be material to the Mozilla Foundation's bottom line.

    Obviously, to get that share, Mozilla would need to prioritise mobile, which it has not done. This is a deep own-goal and a point of continued sadness for me.

    A noble house reduced to rubble is a tragedy no matter who demolishes the final wall. Management incompetence is in evidence, and Mozilla's Directors are clearly not fit for purpose.

    But none of that detracts from what others have done to the Foundation and the web, and it would be wrong to claim Mozilla should have been perfect in ways its enemies and competitors were not.

]]>
A Management Maturity Model for Performance 2022-05-09T00:00:00Z https://infrequently.org/2022/05/performance-management-maturity/ Since 2015 I have been lucky to collaborate with more than a hundred teams building PWAs and consult on some of the world's largest sites. Engineers and managers on these teams universally want to deliver great experiences and have many questions about how to approach common challenges. Thankfully, much of what once needed hand-debugging by browser engineers has become automated and self-serve thanks to those collaborations.

Despite advances in browser tooling, automated evaluation, lab tools, guidance, and runtimes, teams I've worked with consistently struggle to deliver minimally acceptable performance with today's popular frameworks. This is not a technical problem per se — it's a management issue, and one that teams can conquer with the right frame of mind and support.

It may seem a silly question, but what is performance, exactly?

This is a complex topic, but to borrow from a recent post, web performance expands access to information and services by reducing latency and variance across interactions in a session, with a particular focus on the tail of the distribution (P75+). Performance isn't a binary and there are no silver bullets.

Only teams that master their systems can make intentional trade-offs. Organisations that serve their tools will tread water no matter how advanced their technology, while groups that understand and intentionally manage their systems can succeed on any stack.1

The value of performance is deeply understood within a specific community and in teams that have achieved high maturity. But outside those contexts it can be challenging to communicate. One helpful lens is to view the difference between good and bad performance as a gap between expectations and reality.

For executives that value:

Performance is rarely the single determinant of product success, but it can be the margin of victory. Improving latency and reducing variance allows teams to test other product hypotheses with less noise. A senior product leader recently framed a big performance win as creating space that allows us to be fallible in other areas.

Like accessibility, security, UI coherence, privacy, and testability, performance is an aggregate result. Any single component of a system can regress latency or create variance, which means that like other cross-cutting product properties, performance must be managed as a commons. The approaches that work over time are horizontal, culturally-based, and require continual investment to sustain.

Teams I've consulted with are too often wrenched between celebration over launching "the big rewrite" and the morning-after realisation that the new stack is tanking business metrics.

Now saddled with the excesses of npm, webpack, React, and a thousand promises of "great performance" that were never critically evaluated, it's easy for managers to lose hope. These organisations sometimes spiral into recrimination and mistrust. Where hopes once flourished, the horrors of a Bundle Buddy readout looms. Who owns this code? Why is it there? How did it get away from the team so quickly?

Many "big rewrite" projects begin with the promise of better performance. Prototypes "seem fast", but nobody's actually benchmarking them on low-end hardware. Things go fine for a while, but when sibling teams are brought in to integrate late in the process, attention to the cumulative experience may suffer. Before anyone knows it, the whole thing is as slow as molasses, but "there's no going back"... and so the lemon launches with predictably sour results.

In the midst of these crises, thoughtful organisations begin to develop a performance management discipline. This, in turn, helps to create a culture grounded in high expectations. Healthy performance cultures bake the scientific method into their processes and approaches; they understand that modern systems are incredibly complex and that nobody knows everything — and so we learn together and investigate the unknown to develop an actionable understanding.

Products that maintain a healthy performance culture elevate management of latency, variance, and other performance attributes to OKRs because they understand how those factors affect the business.

Performance management isn't widely understood to be part of what it means to operate a high-functioning team. This is a communcation challenge with upper management, but also a potential differentiator or even a strategic advantage. Teams that develop these advantages progress through a hierarchy of management practice phases. In drafting this post, I was pointed to similar work developed independently by others3; that experienced consultants have observed similar trends helps give me confidence in this assessment:

Hear no evil, see no evil, speak no evil.
Photo by von Vix

Level 0 teams do not know they have a problem. They may be passively collecting some data (e.g., through one of the dozens of analytics tools they've inevitably integrated over the years), but nobody looks at it. It isn't anyone's job description to do so.

Folks at this level of awareness might also simply assume that "it's the web, of course it's slow" and reach for native apps as a panacea (they aren't). The site "works" on their laptops and phones. What's the problem?

Managers in Level 0 teams are unaware that performance can be a serious product problem; they instead assume the technology they acquired on the back of big promises will be fine. This blindspot usually extends up to the C-suite. They do not have latency priorities and they uncritically accept assertions that a tool or architecture is "performant" or "blazing fast". They lack the technical depth to validate assertions, and move from one framework to another without enunciating which outcomes are good and which are unacceptable. Faith-based product management, if you will.

Level 0 PMs fail to build processes or cultivate trusted advisors to assess the performance impacts of decisions. These organisations often greenlight rewrites because we can hire easily for X, and we aren't on it yet. These are vapid narratives, but Level 0 managers don't have the situational awareness, experience, or confidence to push back appropriately.

These organisations may perform incidental data collection (from business analytics tools, e.g.) but are inconsistently reviewing performance metrics or considering them when formulating KPIs and OKRs.

Shit's on fire, yo.
Photo by Jay Heike

At Level 1, managers will have been made aware that the performance of the service is unacceptable.4

Service quality has degraded so much that even fellow travelers in the tech privilege bubble4:1 have noticed. Folks with powerful laptops, new iPhones, and low-latency networks are noticing, which is a very bad sign. When an executive enquires about why something is slow, a response is required.

This is the start of a painful remediation journey that can lead to heightened performance management maturity. But first, the fire must be extinguished.

Level 1 managers will not have a strong theory about what's amiss, and an investigation will commence. This inevitably uncovers a wealth of potential metrics and data points to worry about; a few of those will be selected and tracked throughout the remediation process. But were those the right ones? Will tracking them from now on keep things from going bad? The first firefight instills gnawing uncertainty about what it even means to "be fast". On teams without good leadership or a bias towards scientific inquiry, it can be easy for Level 1 investigations to get preoccupied with one factor while ignoring others. This sort of anchoring effect can be overcome by pulling in external talent, but this is often counter-intuitive and sometimes even threatening to green teams.

Competent managers will begin to look for more general "industry standard" baseline metrics to report against their data. The industry's default metrics are moving to a better place, but Level 1 managers are unequipped to understand them deeply. Teams at Level 1 (and 2) may blindly chase metrics because they have neither a strong, shared model of their users, nor an understanding of their own systems that would allow them to focus more tightly on what matters to the eventual user experience. They aren't thinking about the marginal user yet, so even when they do make progress on directionally aligned metrics, nasty surprises can reoccur.

Low levels of performance management maturity are synonymous with low mastery of systems and an undeveloped understanding of user needs. This leaves teams unable to quickly track down culprits when good scores on select metrics fail to consistently deliver great experiences.

Level 1 teams are in transition, and managers of those teams are in the most fraught part of their journey. Some begin an unproductive blame game, accusing tech leads of incompetence, or worse. Wise PMs will perceive performance remediation work as akin to a service outage and apply the principles of observability culture, including "blameless postmortems".

It's never just one thing that's amiss on a site that prompts Level 1 awareness. Effective managers can use the collective learning process of remediation to improve a team's understanding of its systems. Discoveries will be made about the patterns and practices that lead to slowness. Sharing and celebrating these discoveries is a crucial positive attribute.

Strong Level 1 managers will begin to create dashboards and request reports about factors that have previously caused problems in the product. Level 1 teams tend not to staff or plan for continual attention to these details, and the systems often become untrustworthy.

Teams can get stuck at Level 1, treating each turn through a Development ➡️ Remediation ➡️ Celebration loop as "the last time". This is pernicious for several reasons. Upper management will celebrate the first doused fire but will begin to ask questions about the fourth and fifth blazes. Are their services just remarkably flammable? Is there something wrong with their team? Losing an organisation's confidence is a poor recipe for maximising personal or group potential.

Next, firefighting harms teams, and doubly so when management is unwilling to adopt incident response framing. Besides potential acrimony, each incident drains the team's ability to deliver solutions. Noticeably bad performance is an expression of an existing feature working below spec, and remediation is inherently in conflict with new feature development. Level 1 incidents are de facto roadmap delays.

Lastly, teams stuck in a Level 1 loop risk losing top talent. Many managers imagine this is fine because they're optimising for something else, e.g. the legibility of their stack to boot camp grads. A lack of respect for the ways that institutional knowledge accelerates development is all too common.

It's difficult for managers who do not perceive the opportunities that lie beyond firefighting to comprehend how much stress they're placing on teams through constant remediation. Fluctuating between Levels 1 and 0 ensures a team never achieves consistent velocity, and top performers hate failing to deliver.

The extent to which managers care about this — and other aspects of the commons, such as a11y and security — is a reasonable proxy for their leadership skills. Line managers can prevent regression back to Level 0 by bolstering learning and inquiry within their key personnel, including junior developers who show a flair for performance investigation.

Think globally, then reset.
The global baseline isn't what folks in the privilege bubble assume.

Thoughtful managers become uncomfortable as repeated Level 1 incidents cut into schedules, hurt morale, and create questions about system architecture. They sense their previous beliefs about what's "reasonable" need to be re-calibrated... but against what baseline?

It's challenging for teams climbing the maturity ladder to sift through the many available browser and tool-vendor data points to understand which ones to measure and manage. Selected metrics are what influence future investments, and identifying the right ones allows teams to avoid firefighting and prevent blindspots.

A diagram of the W3C Navigation Timing timline events
Browsers provide a lot of data about site performance. Harnessing it requires a deep understanding of the product and its users.

Teams looking to grow past Level 1 develop (or uncover they already had) Real User Monitoring ("RUM data") infrastructure in previous cycles. They will begin to report to management against these aggregates.

Against the need for quicker feedback and a fog of metrics, managers who achieve Level 2 maturity look for objective, industry-standard reference points that correlate with business success. Thankfully, the web performance community has been busy developing increasingly representative and trustworthy measurements. Still, Level 2 teams will not yet have learned to live with the dissatisfaction that lab measurements cannot always predict a system's field behavior. Part of mastery is accepting that the system is complex and must be investigated, rather than fully modeled. Teams at Level 2 are just beginning to learn this lesson.

Strong Level 2 managers acknowledge that they don't know what they don't know. They calibrate their progress against studies published by peers and respected firms doing work in this area. These data points reflect a global baseline that may (or may not) be appropriate for the product in question, but they're significantly better than nothing.

Managers who bring teams to Level 2 spread lessons from remediation incidents, create a sense of shared ownership over performance, and try to describe performance work in terms of business value. They work with their tech leads and business partners to adopt industry-standard metrics and set expectations based on them.

Level 2 teams buy or build services that help them turn incidental data collection into continual reporting against those standard metrics. These reports tend to focus on averages and may not be sliced to focus on specific segments (e.g., mobile vs. desktop) and geographic attributes. Level 2 (and 3) teams may begin drowning in data, with too many data points being collected and sliced. Without careful shepherding to uncover the most meaningful metrics to the business, this can engender boredom and frustration, leading to reduced focus on important RUM data sources.

Strong Level 2 managers will become unsatisfied with how global rules of thumb and metrics fail to map directly into their product's experience and may begin to look for better, more situated data that describe more of the user journeys they care about. The canniest Level 2 managers worry that their teams lack confidence that their work won't regress these metrics.

Teams that achieve Level 2 competence can regress to Level 1 under product pressure (removing space to watch and manage metrics), team turnover, or assertions that "the new architecture" is somehow "too different" to measure.

Level 3 teams are starting to fly the plane instead of being passengers on an uncomfortable journey
Photo by Launde Morel

The unease of strong Level 2 management regarding metric appropriateness can lead to Level 3 awareness and exploration. At this stage, managers and TLs become convinced that the global numbers they're watching "aren't the full picture" — and they're right!

At Level 3, teams begin to document important user journeys within their products and track the influence of performance across the full conversion funnel. This leads to introducing metrics that aren't industry-standard, but are more sensitive and better represent business outcomes. The considerable cost to develop and validate this understanding seems like a drop in the bucket compared to flying blind, so Level 3 teams do it, in part, to eliminate the discomfort of being unable to confidently answer management questions.

Substantially enlightened managers who reach Level 3 will have become accustomed to percentile thinking. This often comes from their journey to understand the metrics they've adopted at Levels 1 and 2. The idea that the median isn't the most important number to track will cause a shift in the internal team dialogue. Questions like, "Was that the P50 number?" and "What does it look like at P75 and P90?" will become part of most metrics review meetings (which are now A Thing (™).

Percentiles and histograms become the only way to talk about RUM data in teams that reach Level 3. Most charts have three lines — P75, P90, and P95 — with the median, P50, thrown in as a vanity metric to help make things legible to other parts of the organisation that have yet to begin thinking in distributions.

Treating data as a distribution fundamentally enables comparison and experimentation because it creates a language for describing non-binary shifts. Moving traffic from one histogram bucket to another becomes a measure of success, and teams at Level 3 begin to understand their distributions are nonparametric, and they adopt more appropriate comparisons in response.

Level 3 managers and their teams are becoming scientists. For the first time, they will be able to communicate with confidence about the impact of performance work. They stop referring to "averages", understand that medians (P50) can tell a different story than the mean, and become hungry to explore the differences in system behavior at P50 and outlying parts of the distribution.

Significant effort is applied to the development and maintenance of custom metrics and tools. Products that do not report RUM data in more sliceable ways (e.g., by percentile, geography, device type, etc.) are discarded for those that better support an investigation.

Teams achieving this level of discipline about performance begin to eliminate variance from their lab data by running tests in "less noisy" environments than somewhere like a developer's laptop, a shared server, or a VM with underlying system variance. Low noise is important because these teams understand that as long as there's contamination in the environment, it is impossible to trust the results. Disaster is just around the corner when teams can't trust tests designed to keep the system from veering into a bad state.

Level 3 teams also begin to introduce a critical asset to their work: integration of RUM metrics reporting with their experimentation frameworks. This creates attribution for changes and allows teams to experiment with more confidence. Modern systems are incredibly complex, and integrating this experimentation into the team's workflow only intensifies as groups get ever-more sophisticated moving forward.

Teams can regress from Level 3 because the management structures that support consistent performance are nascent. Lingering questions about the quality of custom metrics can derail or stall progress, and some teams can get myopic regarding the value of RUM vs. lab data (advanced teams always collect both and try to cross-correlate, but this isn't yet clear to many folks who are new to Level 3). Viewing metrics with tunnel vision and an unwillingness to mark metrics to market are classic failure modes.

Level 4 teams are beginning to understand and manage the tolerances of their service.
Photo by Mastars

Strong Level 3 managers will realise that many performance events (both better and worse than average) occur along a user journey. This can be disorienting! Everything one thought they knew about how "it's going" is invalidated all over again. The P75 latency for interaction (in an evenly distributed population) isn't the continuous experience of a single user; it's every fourth tap!

Suddenly, the idea of managing averages looks naive. Medians have no explanatory power and don't even describe the average session! Driving down the median might help folks who experience slow interactions, but how can the team have any confidence about that without constant management of the tail latency?

This new understanding of the impact that variance has on user experiences is both revelatory and terrifying. The good news is that the tools that have been developed to this point can serve to improve even further.

Level 4 teams also begin to focus on how small, individually innocuous changes add up to a slow bleed that can degrade the experience over time. Teams that have achieved this sort of understanding are mature enough to forecast a treadmill of remediation in their future and recognise it as a failure mode. And failure modes are avoidable with management processes and tools, rather than heroism or blinding moments of insight.

Teams that achieve Level 4 maturity almost universally build performance ship gates. These are automated tests that watch the performance of PRs through a commit queue, and block changes that tank the performance of important user flows. This depends on the team having developed metrics that are known to correlate well with user and business success.

This implies all of the maturity of the previous levels because it requires a situated understanding of which user flows and scenarios are worth automating. These tests are expensive to run, so they must be chosen well. This also requires an investment in infrastructure and continuous monitoring. Making performance more observable, and creating a management infrastructure that avoids reactive remediation is the hallmark of a manager who has matured to Level 4.

Many teams on the journey from Level 3 to 4 will have built simpler versions of these sorts of gates (bundle size checks, e.g.). These systems may allow for small continuous increases in costs. Over time, though, these unsophisticated gates become a bad proxy for performance. Managers at Level 4 learn from these experiences and build or buy systems to watch trends over time. This monitoring ought to include data from both the lab and the field to guard against "metric drift". These more sophisticated monitoring systems also need to be taught to alert on cumulative, month-over-month and quarter-over-quarter changes.

Level 4 maturity teams also deputise tech leads and release managers to flag regressions along these lines, and reward them for raising slow-bleed regressions before they become crises. This responsibility shift, backed up by long-run investments and tools, is one of the first stable, team-level changes that can work against cultural regression. For the first time, the team is considering performance on longer time scales. This also begins to create organisational demand for latency budgeting and slowness to be attributed to product contributions.

Teams that achieve Level 4 maturity are cautious acquirers of technology. They manage on an intentional, self-actualised level and value an ability to see through the fog of tech fads. They do bake-offs and test systems before committing to them. They ask hard questions about how any proposed "silver bullets" will solve the problems that they have. They are charting a course based on better information because they are cognizant that it is both valuable and potentially available.

Level 4 teams begin to explicitly staff a "performance team", or a group of experts whose job it is to run investigations and drive infrastructure to better inform inquiry. This often happens out of an ad-hoc virtual team that forms in earlier stages but is now formalised and has long-term staffing.

Teams can quickly regress from Level 4 maturity through turnover. Losing product leaders that build to Level 4 maturity can set groups back multiple maturity levels in short order, and losing engineering leaders who have learned to value these properties can do the same. Teams are also capable of losing this level of discipline and maturity by hiring or promoting the wrong people. Level 4 maturity is cultural and cultures need to be defended and reinforced to maintain even the status quo.

Level 5 teams have understood the complexity of their environment and can make tradeoffs with confidence.
Photo by Colton Sturgeon

Teams that fully institutionalise performance management come to understand it as a strategic asset.

These teams build management structures and technical foundations that grow their performance lead and prevent cultural regressions. This includes internal training, external advocacy and writing5, and the staffing of research work to explore the frontier of improved performance opportunities.

Strategic performance is a way of working that fully embeds the idea that "faster is better", but only when it serves user needs. Level 5 maturity managers and teams will gravitate to better-performing options that may require more work to operate. They have learned that fast is not free, but it has cumulative value.

These teams also internally evangelise the cause of performance. Sibling teams may not be at the same place, so they educate about the need to treat performance as a commons. Everyone benefits when the commons is healthy, and all areas of the organisation suffer when it regresses.

Level 5 teams institute "latency budgets" for fractional feature rollouts. They have structures (such as managers or engineering leadership councils) that can approve requests for non-latency-neutral changes that may have positive business value. When business leaders demand the ability to ram slow features into the product, these leaders are empowered to say no.

Lastly, Level 5 teams are focused on the complete user journey. Teams in this space can make trades intelligently, moving around code and time within a system they have mastered to ensure the best possible outcomes in essential flows.

Level 3+ team behaviours are increasingly illegible to less-advanced engineers and organisations. At Level 5, serious training and guardrails are required to integrate new talent. Most hires will not yet share the cultural norms that a strategically performant organisation uses to deliver experiences with consistent quality.6

Strategy is what you do differently from the competition, and Level 5 teams understand their way of working is a larger advantage than any single optimisation. They routinely benchmark against their competition on important flows and can understand when a competitor has taken the initiative to catch up (it rarely happens through a single commit or launch). These teams can respond at a time of their choosing because their lead will have compounded. They are fully out of firefighting mode.

Level 5 teams do not emerge without business support. They earn space to adopt these approaches because the product has been successful (thanks in part to work at previous levels). Level 5 culture can only be defended from a position of strength. Managers in this space are operating for the long term, and performance is understood to be foundational to every new feature or improvement.

Teams at Level 5 degrade more slowly than at previous levels, but it does happen. Sometimes, Level 5 teams are poor communicators about their value and their values, and when sibling teams are rebuffed, political pressure can grow to undermine leaders. More commonly, enough key people leave a Level 5 team for reasons unrelated to performance management, like when the hard-won institutional understanding of what it takes to excel is lost. Sometimes, simply failing to reward continual improvement can drive folks out. Level 5 managers need to be on guard regarding their culture and their value to the organisation as much as the system's health.

It's possible for strong managers and tech leads to institute Level 1 discipline by fiat. Level 2 is perhaps possible on a top-down basis in a small or experienced team. Beyond that, though, maturity is a growth process. Progression beyond global baseline metrics requires local product and market understanding. TLs and PMs need to become curious about what is and isn't instrumented, begin collecting data, then start the directed investigations necessary to uncover what the system is really doing in the wild. From there, tools and processes need to be built to recreate those tough cases on the lab bench in a repeatable way, and care must be taken to continually re-validate those key user journeys against the evolving product reality.

Advanced performance managers build groups that operate on mutual trust to explore the unknown and then explain it out to the rest of the organisation. This means that advancement through performance maturity isn't about tools.

Managers who get to Level 4 are rare, but the number who imagine they are could fill stadiums because they adopted the technologies that high-functioning leaders encourage. But without the trust, funding to enquire and explore, and an increasingly fleshed-out understanding of users at the margins, adopting a new monitoring tool is a hollow expenditure. Nothing is more depressing than managerial cosplay.

It's also common for teams to take several steps forward under duress and regress when heroics stop working, key talent burns out, and the managerial focus moves on. These aren't fatal moments, but managers need to be on the lookout to understand if they support continual improvement. Without a plan for an upward trajectory, product owners are putting teams on a loop of remediation and inevitable burnout... and that will lead to regression.

Line engineers want to do a good job. Nobody goes to work to tank the product, lose revenue, or create problems for others down the line. And engineers are trained to value performance and quality. The engineering mindset is de facto optimising. What separates Level 0 firefighting teams from those that have achieved self-actualised Level 5 execution is not engineering will; it's context, space, and support.

Senior management sending mixed signals about the value of performance is the fastest way to degrade a team's ability to execute. The second-fastest is to use blame and recrimination. Slowness has causes, but the solution isn't to remove the folks that made mistakes, but rather to build structures that support iteration so they can learn. Impatience and blame are not assets or substitutes for support to put performance consistently on par with other concerns.

Teams that reach top-level performance have management support at the highest level. Those managers assume engineers want to do a good job but have the wrong incentives and constraints, and it isn't the line engineer's job to define success — it's the job of management.

Senior managers looking to help their teams climb the performance management maturity hill can begin by asking themselves a few questions:

  • Do we understand how better performance would improve our business?

    • Is there a shared understanding in the leadership team that slowness costs money/conversions/engagement/customer-success?
    • Has that relationship been documented in our vertical or service?
    • Do we know what "strategic performance" can do for the business?
  • What constraints have we given the team?

  • Have we developed a management fluency wth histograms and distributions over time?

    • Do we write OKRs for performance?
    • Are they phrased in terms of marginal device and network targets, as well as distributions?
  • What support do we give teams that want to improve performance?

    • Do folks believe they can appeal directly to you if they feel the system's performance will be compromised by other decisions?
    • Can folks (including PMs, designers, and SREs — not just engineers) get promoted for making the site faster?
    • Can middle managers appeal to performance as a way to push back on feature requests?
    • Are there systems in place for attributing slowness to changes over time?
    • Can teams win kudos for consistent, incremental performance improvement?
    • Can a feature be blocked because it might regress performance?
    • Can teams easily acquire or build tools to track performance?
  • What support do we give mid-level managers who push back on shiny tech in favour of better performance?

    • Have we institutionalised critial questions for adopting new technologies?
    • Are aspects of the product commons (e.g., uptime, security, privacy, a11y, performance) managed in a coherent way?
    • Do managers get as much headcount and funding to make steady progress as they would from proposing rewrites?
  • Have we planned to staff a performance infrastructure team?

    • It's the job of every team to monitor and respond to performance challenges, but will there be a group that can help wrangle the data to enable everyone to do that?
    • Can any group in the organisation serve as a resource for other teams that are trying to get started in their latency and variance learning journeys?

The answers to these questions help organisations calibrate how much space they have created to scientifically interrogate their systems. Computers are complex, and as every enterprise becomes a "tech company", becoming intentional about these aspects is as critical as building DevOps and Observability to avoid downtime.

It's always cheaper in the long run to build understanding than it is to fight fires, and successful management can create space to unlock their team's capacity.

Mature technology organisations may already have and value a discipline to manage performance: "Site Reliability Engineering" (SRE), aka "DevOps", aka "Observability". These folks manage and operate complex systems and work to reduce failures, which looks a lot like the problems of early performance maturity teams.

These domains are linked: performance is just another aspect of system mastery, and the tools one builds to manage approaches like experimental, flagged rollouts need performance to be accounted for as a significant aspect of the success of a production spike.

Senior managers who want to build performance capacity can push on this analogy. Performance is like every other cross-cutting concern; important, otherwise un-owned, and a chance to differentiate. Managers have a critical role to forge solidarity between engineers, SREs, and other product functions to get the best out of their systems and teams.

Everyone wants to do a great job; it's the manager's role to define what that means.

It takes a village to keep my writing out of the ditch, so my deepest thanks go to Annie Sullivan, Jamund Ferguson, Andy Tuba, Barry Pollard, Bruce Lawson, Tanner Hodges, Joe Liccini, Amiya Gupta, Dan Shappir, Cheney Tsai, and Tim Kadlec for their invaluable comments and corrections on drafts of this post.

FOOTNOTES

  1. High-functioning teams can succeed with any stack, but they will choose not to. Good craftsmen don't blame their tools, but neither do they wilfully bring substandard implements to a job site.

    Per Kellan Elliot-McCrea's classic "Questions for new technology", this means that high-functioning teams will not be on the shiniest stack. Teams choices that are highly correlated with hyped solutions are a warning sign, not an asset. And while "outdated" systems are unattractive, they also don't say much at all about the quality of the product or the team.

    Reading this wrong is a sure tell of immature engineers and managers, whatever their title.

  2. An early confounding factor for teams trying to remediate performance issues is that user intent matters a great deal, and thus the value of performance will differ based on context. Users who have invested a lot of context with a service will be less likely to bounce based on bad performance than those who are "just browsing". For example, a user that has gotten to the end of a checkout flow or are using a government-mandated system may feel they have no choice. This isn't a brand or service success case (failing to create access is always a failure), but when teams experience different amounts of elasticity in demand vs. performance, it's always worth trying to understand the user's context and intent.

    Users that "succeed" but have a bad time aren't assets for a brand or service, they're likely to be ambasassadors for any other way to accomplish their tasks. That's not great, long-term, for a team or for their users.

  3. Some prior art was brought to my attention by people who reviewed earlier drafts of this post; notably this 2021 post by the Splunk team and the following tweet by the NCC Group from 2016 (as well as a related PowerPoint presentation):

    Where are you on the #webperf maturity model? ow.ly/miAi3020A9G #perfmatters

    Image from Tweet

    It's comforting that we have all independently formulated roughly similar framing. People in the performance community are continually learning from each other, and if you don't take my formulation, I hope you'll consider theirs.

  4. Something particularly problematic about modern web development is the way it has reduced solidarity between developers, managers, and users. These folks now fundamentally experience the same sites differently, thanks to the shocking over-application of client-side JavaScript to every conceivable problem.

    This creates structural illegibility of budding performance crises in new, uncomfortably exciting ways.

    In the desktop era, developers and upper management would experience sites through a relatively small range of screen sizes and connection conditions. JavaScript was applied in the breach when HTML and CSS couldn't meet a need.7 Techniques like Progressive Enhancement ensured that the contribution of CPU performance to the distribution of experiences was relatively small. When content is predominantly HTML, CSS, and images, browsers are able to accelerate processing across many cores and benefit from the ability to incrementally present the results.

    By contrast, JavaScript-delivered UI strips the browser of its ability to meaningfully reorder and slice up work so that it prioritises responsiveness and smooth animations. JavaScript is the fuck it, we'll do it live way to construct UI, and stresses the relative performance of a single core more than competing approaches. Because JavaScript is, byte for byte, the most expensive thing you can ask a browser to process, this stacks the difficulty involved in doing a good job on performance. JavaScript-driven UI is inherently working with a smaller margin for error, and that means today's de facto approach of using JavaScript for roughly everything leaves teams with much less headroom.

    Add this change in default architecture to the widening gap between the high end (where all developers and managers live) and the median user. It's easy to understand how perfectly mistimed the JavaScript community's ascendence has been. Not since the promise of client-side Java has the hype cycle around technology adoption been more out of step with average usability.

    Why has it gone this badly?

    In part because of the privilege bubble. When content mainly was markup, performance problems were experienced more evenly. The speed of a client device isn't the limiting site speed factor in an HTML-first world. When database speed or server capacity is the biggest variable, issues affect managers and executives at the same rate they impact end users.

    When the speed of a device dominates, wealth correlates heavily with performance. This is why server issues reliably get fixed, but JavaScript bloat has continued unabated for a decade. Rich users haven't borne the brunt of these architectural shifts, allowing bad choices to fly under the radar much longer which, in turn, increase the likelihood of expensive remediation incidents.

    Ambush by JavaScript is a bad time, and when managers and execs only live in the privilege bubble, it's users and teams who suffer most.

  5. Managers may fear that by telling everyone about how strategic and important performance has become to them, that their competitiors will wise up and begin to out-execute on the same dimension.

    This almost never happens, and the risks are low. Why? Because, as this post exhaustively details, the problems that prevent the competition from achieving high-functioning performance are not strictly technical. They cannot — and more importantly, will not — adopt tools and techniques you evangelise because it is highly unlikely that they are at a maturity level that would allow them to benefit. In many cases, adding another tool to the list for a Level 1-3 team to consider can even slow down and confound them.

    Strategic performance is hard to beat because it is hard to construct at a social level.

  6. Some hires or transfers into Level 5 teams will not easily take to shared performance values and training.

    Managers should anticipate pushback from these quarters and learn to re-assert the shared cultural norms that are critical to success.

    There's precious little space in a Level 5 team for résumé-oriented development because a focus on the user has evacuated the intellectual room that hot air once filled. Thankfully, this can mostly be avoided through education, support, and clear promotion criteria that align to the organisation's evolved way of working.

    Nearly everyone can be taught, and great managers will be on the lookout to find folks who need more support.

  7. Your narrator built JavaScript frameworks in the desktop era; it was a lonely time compared to the clogged market for JavaScript tooling today. The complexity of what we were developing for was higher than nearly every app I see today; think GIS systems, full PIM (e.g., email, calendar, contacts, etc.) apps, complex rich text editing, business apps dealing with hundreds of megabytes worth of normalised data in infinite grids, and BI visualisations.

    When the current crop of JavaScript bros tells you they need increased complexity because business expectations are higher now, know that they are absolutely full of it. The mark has barely moved in most experiences. The complexity of apps is not much different, but the assumed complexity of solutions is. That experiences haven't improved for most users is a shocking indictment of the prevailing culture.

]]>