bdon.org

Getting to the bottom of web map performance

bdon.org — Mon, 11 Oct 2021 00:00:00 +0000

In 1993, two landmark PC games were released within months of each other: id Software’s Doom, and Cyan World’s Myst. Doom is a fast-paced action game viewed from a first-person perspective, with detailed weapons, demons and gore; Myst is a point-and-click puzzle adventure on a creepy island.

Doom was famously implemented using computer science techniques novel for games, such as binary space partitioning to determine the visibility of objects. In contrast, Myst was technically unsophisticated - with a first version created in HyperCard - but was still lauded for having “photorealistic” graphics that demanded an entire compact disc of data.

The core reason for Myst’s graphical fidelity is obvious to the millions that have seen both games: The visuals of Doom animate smoothly every few milliseconds, while the visuals of Myst are pre-rendered images that only change when the player clicks. In other words, Doom was a game with real-time rendering, which demanded orders of magnitude more computation and development effort than Myst.

Click Navigation in Myst

A slideshow of images, with a tiny cross-fade transition, gives the illusion of moving around the Myst island. The choppiness of this experience did not stop Myst from being a compelling game.

source: Cyan

Camera movement in Doom

Doom's smooth camera movement is a hard requirement for aiming and shooting at space demons; Doom's engine, id Tech 1, was re-used for other first person games like Chex Quest.

source: id software

1993’s PC gaming techniques today

30 years later, the core difference between Myst and Doom’s graphics reveals itself again via a more universal computing experience: digital maps on the web.

There are effectively two strategies to implementing interactive, zoomable maps in 2021. These are often referred to as raster maps and vector maps, but these names mostly describe the networking parts of a map system, and hide a spectrum of possible compromise designs. The key distinction is where rasterization - the conversion of shapes to pixels - happens. Unless you are using a vector monitor, you are seeing raster images on your display right now, but this might have happened on your device CPU, GPU or even on a totally different computer.

Vector rendering

A true vector graphics application: Asteroids implemented on an oscillograph. Unless you are using one of these, rasterization is happening somewhere.

source: Autopilot / Wikimedia

Modern Graphics Processing Unit (GPU) rendering

Vector features exist as arrays of vertices in GPU memory; these are rasterized to pixels efficiently by programmable shaders. Rasterization happens as late as possible, in parallel, without interacting with the CPU.

source: Learn OpenGL

The “Myst” approach to map rendering

So-called raster maps are best embodied by Leaflet, an extremely popular library for displaying zoomable, pannable maps with geographic overlays. The typical flow from geodata to pixels with a Leaflet map goes something like this:

Map data exists in vector form, in a format like GeoJSON, ESRI Shapefile, or PostGIS tables.
At some point - either on-demand or preprocessed - this data is rasterized into image tiles via a program like MapServer or Mapnik. Images can be PNGs, JPGs, whatever - they are all the same size, and correspond to regions of Earth based on the Web Mercator projection..
The map tiles are displayed in a web browser via img tags. Since images are everywhere on the web, browsers are designed to be very efficient at fetching images off-thread and animating them with GPU acceleration.

Some free or commercial services offer pre-rendered image tiles directly, like the default rendering of OpenStreetMap data on openstreetmap.orghttps://www.openstreetmap.org/, creative cartographic styles by Stamen Design, basemap tiles for visualization by CARTO, or Wikimedia maps embedded on Wikipedia. APIs may offer hi-DPI image tiles for Retina displays; the general conventions around these image map tiles are encoded in standards like WMTS and can be consumed by other libraries like OpenLayers or Pigeon Maps, a React-centric library.

The crucial compromise of this approach is tiles only exist at discrete zoom levels with no intermediate states. If rasterization happens on a server somewhere Out There on the Internet, the architecture is not responsive enough to truly show in-between images; systems like Leaflet use clever animations to hide this fact.

Leaflet + OpenStreetMap.org tiles

A popular map style displayed on openstreetmap.org; zooming in jumps between different zoom levels; a 250 millisecond scale transition is handled by Leaflet.

source: OpenStreetMap

Google Maps Fallback

Google Maps loads a stepped-zoom fallback map renderer initially, and then swaps to a WebGL-based real-time renderer if browser support is detected. It snaps between zooms with no animation.

source: Google Maps

Like Myst, a cross-fade animation helps achieve persistence of vision for what is actually a totally new image. “Cheating” in this way for maps can be acceptable if the blurry appearance of the map in transition is not important.

Fractional zoom levels

Are infinite, fractional zoom states possible by rounding to the nearest level? This works fine if your images are satellite photographs - the key distinction is that the features in your map are intended to be scaled in world space:

At zoom level 5, a feature is 20 px wide.
At zoom level 6, the feature is 40 px wide.
At zoom level 5.5, the feature is about 30 px wide (actually a bit less because of the exponential relationship)

Infinite zoom levels

Displaying satellite imagery works just fine with img tags and continuous zoom; the images are scaled smoothly via CSS transforms before the next or previous level is loaded.

source: Sentinel-2 cloudless - https://s2maps.eu by EOX IT Services GmbH (Contains modified Copernicus Sentinel data 2016 & 2017)</a>

OpenSeadragon is another general purpose library for “deep zoom” images that implements fractional zooming with simple browser APIs; it works great for museum collections and satellite imagery.

But this breaks down if your intention is to keep features consistent in screen space, like cartographic map labels with an assigned font size of 20px, or schematic roads with an assigned width of 5px:

At zoom level 5, the label is 20 px tall.
At zoom level 6, the label is 20 px tall.
At zoom level 5.49999, the label is about 40 px tall.
At zoom level 5.50000, the label is about 10 px tall.

Hacking it with Leaflet

Attempting to display fractional zoom levels with pre-rasterized schematic maps means that the map is jarringly pixelated at some levels, and label sizes swing between too small and too large - coherent cartographic design becomes impossible.

source: OpenStreetMap

The Myst approach for map display is unacceptable if you demand both screen-space rendering of features and fractional zoom levels. If you can compromise by having stepped zooms, non-realtime-rendered maps are significantly easier to implement.

The “Doom” approach to map rendering

The correct fractional-zoom approach to rendering is true real-time maps, where rasterization of map features happens Doom-style - multiple times a second, ideally 30-60. This is only practical to accomplish on GPUs, via a low-level API like OpenGL/WebGL, Direct3D or Metal.

The pinnacle of this approach for developers to reuse in web apps is the source-available Mapbox GL JS library or a fork which is still BSD-licensed. Google Maps for the web now uses its WebGL version for all browsers; there are also less widely adopted renderers like Harp.GL and Tangram JS. The flow of data is something like this:

Geographic features are projected into Web Mercator, sliced into tiles, and stored in a vector SVG-like format, like GeoJSON or MVT protocol buffers.
Tiles are fetched over the network and vertex data is moved to GPU memory. Polygons need to be converted into triangles, because GL-like graphics APIs only display triangles and textures. Text needs to be converted to textures or signed distance fields for map labeling.
Programmable shaders extrude linear features into screen-space lines - see Drawing Lines is Hard - and then composite them with triangle-ized polygons and texture-ized text on screen. A powerful map engine can allow user customization at any point in this pipeline.

Google Maps WebGL

Google Maps uses WebGL for real-time rasterization, which means that text labels are always sized correctly, and features are never blurry, even when data has not finished loading for the displayed zoom level.

source: Google Maps

The web is a difficult platform for real-time rasterization

Map applications on native platforms - desktop, iOS, Android - take the real-time rendering approach, because they have direct access to powerful APIs for doing mappy things - Metal, Direct3D for drawing, Core Text and Uniscribe for displaying fonts. Apps for the web are constrained by a weak platform and also the need to be cross-browser, leading rich design tools like Figma to ship their own implementation of low level graphics APIs.

How to build fast maps

Software developers and cartographers building mapping systems will describe an interactive map being fast or performant (sic) without any context as to what performance actually means. Performance, in the abstract, is used to justify building a real-time rendered system over the “obsolete” technology of pre-rendered tiles. By most quantifiable measures - like vertices rasterized per second - GL maps blow away the alternatives.

What end users of applications actually perceive as a high performance web app is low latency. A web map system should respond instantaneously to user input, zooming as soon as a wheel is scrolled, and panning perfectly in sync with mouse movements. The point here is that a non-real-time rasterized map system can be as fast or faster than a GL one, by choosing to show only discrete zoom levels, minimizing network latency, and limiting the total amount of data sent to the browser.

WebGL Map Latency at 2560x1600

MapLibre GL displaying a typical cartographic map style at full screen resolution on a 2016 MacBook Pro. Zooming is nice, with real-time label collisions, but panning the map is choppy. You can reproduce this experiment on this example page.

source: MapTiler, OpenStreetMap

Leaflet map at 2560x1600

Leaflet displays chunky zoom levels - objectively worse than a real-time rasterized system - but panning the map has very low latency, because the GPU is simply moving textures around.

source: OpenStreetMap

So which approach should I use?

It depends.

Decisions on mapping technology, and committing to tradeoffs, are wickedly cross-functional - spanning backend and frontend software engineering, graphic design, user experience and product management.

In the same way pre-rendered images were good enough to make Myst a best-selling game, non-realtime rendering of maps is good enough for some, maybe even the majority, of web map use cases today. If you’re building a whizbang mapping demo with 3D flythroughs to show your company execs, a real-time-rendered system - the Doom approach - is the best choice; if you’re building a site for helping people find the nearest COVID vaccine, the simplest solution is the best solution, and that usually means stepped-zoom maps.

Some good reasons to go real-time rendering:

Your application is built around a full-screen interactive map, and smooth zooming is a core part of the experience.
You need to display 3D geometry or rotate the map.
You intend to display a high density of labels, and need real-time collision between them.

Some good reasons to use Leaflet-style maps:

The map is just a part of the application, like an inset or locator map.
The map is never zoomed, only panned or completely static.
Chunky, stepped zooms are an acceptable user experience.

Web maps without centralized services

bdon.org — Mon, 21 Dec 2020 00:00:00 +0000

How many web applications or mobile apps do you use that have a map component? Rideshare apps display the location of drivers and passengers; ratings sites show the locations of restaurants; Craigslist and Wikipedia show contextual location for classifieds or encyclopedia articles. High quality, interactive maps are pervasive and simply fade into the background for end users.

The COVID-19 pandemic has created a surge of new mapping applications. Beyond websites that visualize the case count in different regions, maps have become critical computing infrastructure for delivering health services.

Maps as Infrastructure

The New York City government uses Google Maps to help residents find a COVID-19 testing location near them.

source: coronavirus.health.ny.gov

Maps and Citizen Tech

The Taiwan government rationed face masks at pharmacies and published inventories via a public API; this enabled citizen developers, in combination with geocoding services and third-party basemaps, to create a variety of web and smartphone applications.

source: facebook.com/g0v.tw, 江明宗

For a developer who wants to integrate a map into their application, the choices are almost always to use a hosted, centralized map service; developer map mindshare is dominated by multinational companies. Some hosted map examples:

The embeddable native maps on Google’s Android or Apple’s iOS/iPadOS/macOS/watchOS operating systems
Google’s commercial web map offering (Google Maps JavaScript API)
Microsoft’s recent release of Azure Maps and Amazon’s AWS Location Service
Commercial map GIS companies such as ESRI, MapQuest, CloudMade, HERE and Mapbox; free hosted maps like OpenStreetMap Carto or maps by Stamen Design

What is a basemap?

Typical parts of a map application include:

Translation from search tokens or addresses to locations (geocoding)
Determining the place name or administrative division of a location (reverse geocoding)
Finding directions between two locations (routing)
Overlaying of domain-specific geographic features, such as COVID-19 testing sites
Showing physical and cultural context on maps, such as city names, nearby points of interest, and roads

It’s this last component that we usually call a basemap - especially if it displays map labels, and is zoomable or multi-resolution, implying the application of cartographic generalization.

Map Labels

A basemap is most useful for context if it displays map labels. Labels may be in a single language, but might display more than one language, alternate names, or transcriptions: for example, Romaji labels on maps of Japan.

source: OpenStreetMap

Generalization - More Art than Science

When the map scale is decreased - in this case by a factor of 2 - less relevant geographic features can be removed from the map. The ideal generalization keeps the visual density of the map constant as scale is decreased, even though the number of geographic features existing within those bounds increases exponentially.

source: OpenStreetMap

Single-scale wayfinding maps you see in the real world are labeled and generalized by cartographers; multi-scale basemap services accomplish this in an automated way. The sophistication of computer programs to do this - especially at whole earth scale - can be grossly underestimated, even by experienced programmers. Good labeling and generalization are taken for granted, like good font rendering and typesetting - invisible when successful, glaringly flawed when unsuccessful.

This also illustrates the key difference between a basemap and a dataset of geographic features, such as OpenStreetMap. OSM is not a cartographic product; it does not attempt any generalization or ranking of features. Natural Earth is an example of a cartographic product - it include fields such as scalerank to assist in generalization, curated by human editors.

In theory, basemaps are fungible - the cartographic context required is similar among diverse applications. So why do developers - and as a consequence, the World Wide Web - gravitate towards a small handful of hosted map platforms?

Mapmakers: Victims of Lofty Expectations

The basemap status quo is defined by the cutting edge commercial product, and right now that’s Google Maps. Users are accustomed to, and will demand, a seamless abstraction of the physical world. Map panning and zooming should be instantaneous, so designers of a high performance cartographic system must take into account:

The completeness and quantity of the underlying geodata
The arrangement and indexing of geodata to achieve low latency
Tile schemes and partitioning for generalized data
Protocols for fetching and caching those tiles over the network
Choice of a rasterization API like Cairo or OpenGL and the flow of vertex data to them

The desired end product is a videogame-like, sixty-frames-per-second experience, and map systems often make the same technical compromises as games: for example, a map system that smoothly resizes labels in real time may choose to precompute signed distance fields for text glyphs on the server and fetch them by codepoint ranges. This works beautifully for languages that use at most a few hundred Latin characters, but navigating to places that use Han characters such as China or Japan will degrade performance. Thus a cartographic system demands holistic design around tradeoffs such as latency or localization capabilities.

Not Good Enough Anymore

Mapquest in the early 2000s required the user to click arrow buttons to pan and zoom; maps in 2020 are expected to respond instantly to mouse wheel scroll and finger pinch gestures.

source: MapQuest, peterme.com

Serving Maps: best left to BigCorps?

Given the demanding technical requirements above, it’s easy to be led into thinking maps are too hard for individual developers to solve and should be outsourced to a third-party hosted service. A cartographic system is very “full-stack” in nature, but the commoditization of some potential components, like PostGIS or HTTP accelerators, makes the overall task less daunting. Bigness isn’t really a constraint; the OpenStreetMap dataset as XML is about 100 gigabytes. Let’s estimate 10x expansion to make a cartographic product; that’s still only a terabyte of data to put on a disk somewhere.

Developers believe something is too hard, until it isn’t. Issuing of SSL Certs was in practice limited to a few Certificate Authorities until Let’s Encrypt; Hosting of seekable video content was only done by big sites in concert with CDNs, but now we have the standardized, hardware-accelerated HTML video element and ubiquitous broadband. Email is a notable counterexample; you’ve been able to run your own Mail Transfer Agent since the beginning, but the Hobbesian underworld of spam and phishing drives users into the oligopoly of a few providers.

Serving maps is hard but not insurmountable; I’ll suggest a factor limiting the landscape of web maps is not data bigness or the speed of slinging geodata down tubes, but licensing and realities of the market.

Intellectual Property, Fine Print

Firstly, mapmaking businesses rightly assert copyright over their cartography. This far predates webmaps: Trap Streets are a longstanding practice on print maps. This is often true even if the map is built from open data such as OSM - the derivation of a cartographic product, and quality of output, is “secret sauce” to achieve product differentiation.

Secondly, map serving easily admits a Software-As-A-Service (SaaS) business model. Shrink-wrapped software, or creating a piece of software and “selling it in a box”, has been out of vogue for decades, possibly over piracy concerns, or because it has less extractive potential on enterprise budgets. In addition to copyright, Terms of Service for hosted maps often prohibit combination with other geographic services, exclude specific use cases, or forbid caching. These contracts work against developers by inventing scarcity: even if you need a small subset of map tiles, they must be fetched and billed; don’t scrape the data and put it in version control.

The freemium, ToS-constrained nature of hosted map services disincentivizes innovation. Inability to swap out the Google Maps frontend means interesting problems like labeling and generalization can’t be attacked with piecemeal incrementalism. This tips developer’s scales toward Buy rather than Build, especially when the hosted pay-per-hit product starts out free, often as a loss leader for a bundle of geospatial services. These market realities leave some mapping use cases underserved by existing technology.

Map applications ought to outlive the SaaS business lifecycle

Accessing a web map over the network isn’t always practical. Maps may need to be displayed on air-gapped internal applications, or work offline without cell tower reception.

Also, websites and mobile applications are often built by organizations separate from their long-term maintainers (e.g. universities, museums, cultural institutions); ambitious visualization projects may be deterred by clients not wanting to foot a pay-per-hit subscription. An application depending on a hosted map service may be permanently disabled if the provider deprecates their API. Hard dependencies on hosted services make software worse - if your data visualization uses Google Maps, don’t count on it still working in a couple years.

Small-Scale Map Apps

80s.nyc is a streetview-like application for navigating historical images of New York City. It can be useful for discovering lost pictures of your great-grandmother. Depending on a hosted basemap API would expose non-commercial projects to the whims and business models of SaaS providers.

source: 80s.nyc, NYC Department of Records / Municipal Archives

Developers ought to be able to control their map content

Self-hosted has some generic advantages - less phoning home means better security and privacy. For maps in particular, designers may want to customize the visual appearance of map features, exclude certain layers, or change the language of labels. These are all achievable to some degree with hosted vector map backends and client-side rendering, so DIY maps don’t necessarily win here.

A more abstract ideal for DIY maps is control over the underlying geodata. Google Maps data has adjudicated on neighborhood names, administrative boundaries, even the display of particular tourist attractions when zoomed out. Compare to OpenStreetMap: In theory, anyone can edit the map and communities can determine for themselves, say, whether a road’s bike lane is sufficiently hazard-free to be highlighted in green. Its availability under an open license (the copyleft ODbL) means developers could exert total control over their maps, even remixing additional datasets into basemaps. Cartographic authority was long ago exercised by governments; commoditization of printing technology transferred this authority to mapmaking and surveying firms; mobile computing has shifted this to Google and Apple Maps. What tools must be built to put mapmaking in the hands of individuals?

14th Century Maps

Maps were once so resource-demanding to only be commissioned by emperors. Centuries-old maps are often expressions of state power; white rectangles on the Da Ming Hunyi Tu are paper labels overwriting Chinese placenames with Manchu ones.

source: 大明混一圖

Which Name?

Google maps displays "New Zealand", while OpenStreetMap-based maps that default to the name tag display "New Zealand / Aotearoa"

source: Google Maps, OpenStreetMap

To follow along, subscribe via Atom/RSS.

bdon.org

Getting to the bottom of web map performance

Click Navigation in Myst

Camera movement in Doom

1993’s PC gaming techniques today

Vector rendering

Modern Graphics Processing Unit (GPU) rendering

The “Myst” approach to map rendering

Leaflet + OpenStreetMap.org tiles

Google Maps Fallback

Fractional zoom levels

Infinite zoom levels

Hacking it with Leaflet

The “Doom” approach to map rendering

Google Maps WebGL

The web is a difficult platform for real-time rasterization

How to build fast maps

WebGL Map Latency at 2560x1600

Leaflet map at 2560x1600

So which approach should I use?

Further Reading

Web maps without centralized services

Maps as Infrastructure

Maps and Citizen Tech

What is a basemap?

Map Labels

Generalization - More Art than Science

Mapmakers: Victims of Lofty Expectations

Not Good Enough Anymore

Serving Maps: best left to BigCorps?

Intellectual Property, Fine Print

Map applications ought to outlive the SaaS business lifecycle

Small-Scale Map Apps

Developers ought to be able to control their map content

14th Century Maps

Which Name?

Recommended Further Reading