Jeffrey Paul

Megalopolis

2026-02-27T00:00:00+00:00

I’ve been in Bangkok the last two weeks, and three times in the last six months. The city is incredible. It’s well over 10 million people, approaching that of Istanbul, which is the largest city in all of Europe. After a certain threshold size, you get to a point where anything humans do on Earth happens here. Every instrument that exists is played, every language that is spoken is spoken, every cuisine that is cooked is cooked. Everything happens here.

The density in Bangkok is on par with any place I’ve ever been, Tokyo included. The breadth of activities in Bangkok far surpasses any other place I’ve ever been. I’m not euphemistically talking about the infamous sex work (I get my kicks above the waistline, sunshine), but the relative freedom here compared with places of equivalent density that I’ve visited, such as Tokyo or Beijing or London. The general practical permissiveness of this city was a surprise to me, given just how tightly controlled most cities of this type are, and how conservative and authoritarian I know Thailand to be.

I can see why people come and live here.

It’s incredibly convenient to exist in such a place. Last night, I had Georgian food delivered an hour before midnight, cooked and brought to my hotel in less than 40 minutes. I believe the entire (priority direct) delivery fee was around 100 baht (~$3), maybe less. I always tip the delivery drivers another 100 baht (~$3), which is the maximum the tip selector UI gives me. That’s around $7 all in.

The climate in Bangkok lends itself exceptionally well to plants. Plant matter here is bordering on too cheap to meter. The quality of fruits and vegetables is off the charts—it’s as good as any place I’ve ever been, the Mediterranean included. Greek salads actually work here. Pasta sauces made well in the Italian style are indistinguishable from those made on the Mediterranean coast.

Thai people (or cooks, at least) seem to resemble Americans in the sense that they’re more than willing to deep fry anything or cover it in cheese or put way too much refined sugar into a dish or a drink. Almost all the drinks that I get here are way sweeter than I would expect them to be, and there seems to be a trend of putting cream cheese (simply called “cheese”) on everything, even things it probably shouldn’t go on. This matches my palate exactly as a Midwestern American fat kid, but is somewhat surprising in a city this large and cosmopolitan.

This is definitely the most westernized place I’ve ever been in Asia, and it’s a gigantic city at that. I’m staying in a neighborhood called Phrom Phong, which has three of the largest and most well-appointed shopping centers that I’ve ever seen. One has a dentist and a nail salon and an indoor trampoline place inside of it. There are three of them all within a five-minute walk of each other. The food choices are mind-bending in their quantity.

I’ve never seen this diversity of food available in such a small area in my entire life. Parts of Tokyo approximate it, but Japanese people seem to be somewhat more fixated on specific types of cuisine—French pastries, for example. Bangkok doesn’t give a fuck. They’ve simply got everything here. There are tons of Japanese immigrants, and so there’s tons of really good Japanese restaurants.

In one building, you can get a tooth descaling, see a movie in IMAX, buy a Coach handbag or a new iPhone, eat incredibly good authentic Italian food, buy an array of macarons that would rival any place I’ve ever seen, and eat Chicago’s finest Garrett Caramel Corn. (IYKYK - Chicago mix all the way.)

In most cities of this echelon, you find yourself coping under the cast of a sort of economic damage-over-time spell. A few days in Paris or Hong Kong and you realize that your budget for your ten-day trip was, well, optimistic. A coffee here, a dinner there, and suddenly you’re in the red in a day. No such background preoccupations loom here, as even in the glittering center nothing exceeds manageable prices (for westerners, anyway—Thai people tell me Bangkok’s ridiculously expensive). The nonstop construction of luxury condos everywhere threatens to imminently alter this upwards, just like Beijing and Shanghai and Seoul before it.

I’ve played now in two underground poker games here that were run better than most legitimate casinos. The games here also run a little bit larger than I was expecting—about twice the size that I normally play in Las Vegas, which is surprising considering that the average daily labor income here is somewhere in the $15-$20 USD equivalent range. The cash games that I’ve played have blinds of 100/200 baht (~$3/$6). The level of income inequality here is breathtaking.

I’ve heard of such insane disparities in places like New Delhi, and of course in China, but in China, you don’t really seem to have it so much in one city. The Chinese government keeps huge masses of people from moving into places like Beijing or Shanghai. There’s a long waiting list, and I believe a lottery system for people who want to move to Beijing. The Thai government, as far as I understand, does not have any such restrictions, and anyone who wants can come (or move) to the city.

In most other places in the world where you’d see a construction site of 20–30 people, you’d have 10 of them working and a few standing around per worker. In Thailand, you see 50 people on the project all working their ass off covered in sweat. I don’t think I’ve ever seen a place with a work ethic as incredible as Thailand. This is not to disparage the salarymen of Japan or the 996ers in the Pearl River Delta, but the amount of labor that gets done in this city simply astounds me.

I can’t recommend it enough. The humidity is famously oppressive, but if you can get past that, the weather is always perfect. I was swimming in a rooftop pool on Christmas day here—it was somewhere around 80°F. That trip I actually came here from my winter home, Las Vegas, specifically because the weather is better. I think Berlin was under a nice dusting of snow.

I think it would be dangerously easy to move here and get caught up in life here, given the leverage of the huge disparity between western incomes and far east prices. Life here is phenomenally easy and exceptionally high quality. I see why so many Westerners move here. I can’t help but feel that it’s all built on the backs of a tremendous and invisible underclass, slaving away for pennies an hour to support the tip of the iceberg that appears in pristine retail towers.

The sheer extent of this place, though, is staggering. Just walking around, you can go from an exposed sewer in a back alley to a gleaming retail outlet of a global brand, sparkling and spotless, in 100 seconds of walking. I don’t know any other place in the world like this.

My favorite places in the world are the A-list world cities: Paris, London, Hong Kong, Tokyo, New York, Los Angeles, Moscow, Istanbul, Seoul. I don’t know all of them — I haven’t been able to come up with an authoritative heuristic. It’s not just the so-called primate cities, although Bangkok is indeed a primate city. I don’t know why Bangkok escaped my notice for so long, given my true appreciation for A-list cities. Perhaps the stigma of Thailand being a “poor country” (and it is, indeed, a poor country) made me assume that Bangkok was going to be shabby. That was a mistake.

I should have come here years ago. Visit it if you can.

Build / Buy / Bot

2025-12-24T00:00:00+00:00

In the old days when you ran your startup, you had to focus on your core objectives very closely in the early days. You had a limited number of innovation tokens, and beyond that you had to focus on your core product exclusively. No sysadminning (Heroku/Lambda), no hard drives (S3), no databases (RDS), no CDN (Cloudflare), no geth (Infura).

As a result, there are many SaaS startups that offer relatively straightforward services: things like APIs, API caching/aggregation/translation layers, industry-specific CRUD apps, image or video transcoding and resizing, analytics, webhook gateways, etc. They’re not hard to build, they’re just time-consuming, and adjacent to your startup’s core mission. You can pay them some small usage-based fee now, and if you get larger, you can build a replacement or renegotiate or change vendors.

A lot of SaaS startups’ secret sauce was simply having faster devs than their customers and perhaps on-call sysadmins, not some specific insight or unique knowledge about the problem space they are solving.

That’s all out the window now with SOTA AI models. Disparage “vibe coding” all you like, it’s here and it works and it’s going to change the world.

The build/buy equation just got an anvil dropped on one side of the seesaw, and your stupid SaaS startup idea is pulling a Wile E. Coyote midair at 30,000 feet. If all of the edge you had was that you could build it faster than your customers, you’re toast.

Competent engineers can now crank out a prototype supporting api/service (unrelated to their startup’s raison d’être) in a day or two, something that used to take a few people a month. Will it be as good and featureful and polished? No. Will it be as reliable? No, but serverless exists and bugfixes and test coverage will take minutes now instead of hours. It’s a startup, and it’ll probably be good enough, and close enough in perf to your free/startup tier that they’ll do that instead of paying you $49/month.

Furthermore, they’ll probably open source it, because it is unrelated to their profit model; remember, we’re talking about tangential supporting services here. If they don’t, someone else scratching that itch will. People will add features to it to make it more broadly useful on an as-needed basis. Bullshit “open core” startups will thankfully be toast.

The barrier to entry for starting a useful SaaS before was that you could actually ship working software faster than most. That’s gone now.

I expect:

many small orgs/startups choosing “build” much more often than “buy”
large orgs assigning small teams to build internal replacements for medium-to-large SaaS products that they are paying a vendor big recurring dollars for, because of the force multipliers that are LLMs
boring/basic CRUD b2b SaaSes to die even faster than they already are
a slight shift back toward b2c due to the decreased cost of platform-specific mobile client development
tons of SaaSes being replaced by f/oss projects
- …many of which will sadly be fake open source
“open core” projects to have their data retention/SSO/2FA enterprise carrot nonfree plugins/libraries to be either replaced by vibe coded f/oss replacements that slot in easily to the “community edition”, or simply forked, as the cost of reviewing PRs and merging in upstream changes drops dramatically
increased frequency of new working and useful f/oss projects, because scratching an itch becomes so much easier and faster
- I even have a few of my own. I imagine many others are tagging 1.0.0 of little side projects that have been pending for years.

Show Candidates Your Cap Table.

2025-08-01T00:00:00+00:00

It’s hard enough trying to decide if you should take a job with an early-stage startup. There’s so much uncertainty: the product, your reputation, salary (as always), and, of course, equity.

The generally accepted wisdom is that you should treat equity like a lottery ticket. It’s nice to have, but it probably won’t be worth anything. This isn’t bad advice, but when you buy a lottery ticket, you know the exact number the jackpot represents.

Most employee equity grants (which almost always take the form of stock options) typically only specify the number of options being granted to the employee and the associated strike price (and a vesting schedule). They can’t indicate what percentage of the company at a potential exit those options represent without also providing the number of total shares outstanding and a rough estimate of future dilution as the company raises more money. Even with those figures, it is not possible to assess the potential value of an options grant without additional key inputs—most critically, the post-money valuation of the most recent financing round. This isn’t even taking into account any liquidation preferences.

Most employee option grants vest over time, typically with a one-year cliff and a four-year schedule. This means you won’t have access to most of your options until you’ve worked at the company for several years. Depending on the stage of the company, that timeframe may encompass one or more additional financing rounds before it would be reasonable to exercise the portion of your options that are vested, let alone participate in any kind of secondary market. Additional dilution affecting all equity holders (including founders) is almost certain as further funding rounds occur. This adds even more uncertainty to your calculations, driven by variables both beyond your control and presently unknowable with any precision.

It makes a hard problem even harder when trying to estimate what one’s options might be worth in the happy path of a successful company and liquidity event. There are already many unknowns around the product, the company, future staffing decisions that will determine success or failure, the market for employee equity in private companies (naturally the VC market ebbs and flows as money becomes cheaper or more expensive), future required funding rounds, and salary, which is often subject to wide negotiation in early-stage companies.

When attempting to estimate the expected value of options, a candidate has to juggle the probabilistic distributions of several different variables. Even if you regard it as a lottery ticket, you want to calculate the approximate value at the two poles: zero, and the value in a successful exit. Only then can you get into the even messier work of calculating EV based on your actual opinion of success probability.

This is difficult enough without the founders withholding the most important part of the calculation: the denominator. Without any understanding of how many shares might be outstanding when your options mature, it transforms an extremely difficult task into a literally impossible one.

The current practice of most startup founders is to treat their cap table as somewhat of a secret. I’ve often encountered pushback when asking founders for access to it. But the cap table (or at least the critical figures: total shares outstanding, fully diluted share count, option pool size, and the post-money valuation of the most recent funding round or two) should be available to all employees who hold or may be granted equity. These figures establish the basis for understanding how much of the company your grant represents, what price investors recently paid, and what kind of outcomes are required for your equity to be worth anything. It is as critical to understanding your total compensation as your salary offer. They are also relevant during employment, as it’s not uncommon for employees to negotiate for more options paired with a reset multi-year vesting schedule, this being a normal occurrence during tough times between rounds to retain employees who might be beginning to update their resume.

A clearly defined subset of those figures should always be shared with potential hires alongside any discussion of equity grants included in an offer. This is not the practice today, and it well should be. Make them sign a limited NDA to receive the offer if you must due to backend contractual obligations, but don’t skip the facts of the matter.

Founders should care a lot about this. Withholding critical cap table details from the people you’re asking to take a significant personal and professional risk on your company feels like a thinly veiled attempt at distracting candidates from the real meat of the deal. It doesn’t read as inexperience, instead, to those of us who know math, it reads as subterfuge. It repels high-agency, detail-oriented talent, precisely the kind of professionals who ask hard questions and tend to hold the line under pressure, i.e. the motherfuckers you really want. Worse, it misrepresents the economic reality of the offer. Founders typically frame equity as a meaningful part of the total compensation package. This framing implies value, and in many cases is used to justify a below-market cash salary. A grant of “100,000 options” can sound impressive. But 100,000 options in a 10-million-share company with a $20M post-money valuation (implying potentially $200k in value) is an order of magnitude more than in a 100-million-share cap table with an identical $20M valuation (implying $20k). Omitting the denominator, strike price-to-preferred price delta, and liquidation preferences allows founders to imply upside that simply doesn’t exist. Equity without numbers isn’t a good faith negotiation, it’s just theater.

The recent shift toward extending the post-employment vested option exercise window from 90 days to 10 years (spearheaded by a16z in 2016) is a big step in the right direction, and a good signal that at least some VCs and founders are actually willing to question “this is how we’ve always done things” when the status quo is plainly unfair to early employees, but there are more traditions yet that we should retire. If you’re asking people to work nights and weekends making a last-ditch attempt to lift off before the runway ends, you owe them more than vague promises and hand-waving: you owe them the numbers.

There’s an implicit pressure on candidates to not push harder for such figures, as if wanting the actual details about their compensation package means they might not sufficiently believe in the vision or presumed success. Somehow, inconsistently, this implied perception of skepticism isn’t applied to honest discussions about the salary. The situation is ultimately ridiculous. You want serious people? Have serious conversations.

Indeed, founders take on significant risk when they start a company. However, early-stage employees do as well, and the risk to them is primarily the same risk as it is to the founders: many years of their prime working life. After all, it’s usually not the founders’ money that’s on the line if the company fails, but the VCs’. In this regard, employees and founders have drastically different reward structures balanced against approximately the same amount of personal risk. This imbalance is not unfair, as it correctly rewards initiative, vision, and a near-pathological amount of gumption. These are exceptionally rare and valuable things that should always be hugely incentivized. It is, however, a little bit patronizing to further skew this already asymmetrical relationship against the potential or current employee while they are undertaking one of the potentially largest financial decisions of their life. While the compensation may remain asymmetrical, the access to information should not be so.

As always, you are invited to discuss this post on the BBS (signup required).

Minio Are Assholes

2025-07-20T00:00:00+00:00

I use the fake open source project Minio to store data. It’s an AGPL-licensed S3-compatible server, marketed under the term “open source” for marketing/street cred (despite the AGPL being a nonfree license).

The version RELEASE.2025‑04‑22T22‑12‑26Z includes an administration GUI. The following version does not. It’s of course still available in their paid fork. People are not happy.

They explicitly hobbled their “open source” version to coerce people who don’t want to use their arcane CLI (which is called mc so conveniently name-conflicts with the mc cli utility that many people have installed) to do simple admin tasks.

This sort of shitty behavior from wannabe open source maintainers is user hostile and should be called out regularly. The spirit of open source is so that improvements can be made by anyone and benefit everybody.

Now, Minio have claimed in that thread they’d welcome community contributions to maintain this, and they might even be telling the truth, but they locked the thread after someone expressed disappointment at them vandalizing a working feature in the upstream repository. Anyone who wants to continue to use this working code would have to fork the repository and merge all of the future changes. Or, of course, simply pay Minio, which is what I’m sure they’d prefer.

Many other fake open source projects, however, straight up refuse to merge community implementations of SSO or audit logging because they have a proprietary product alongside that has those features paywalled. I believe a project, at that point, regardless of licensing, should not be called open source. Open source (of course more accurately called “free software”, or “libre software”) is not just a license, it’s an ideology.

Open core is not open source, regardless of what license they use. It’s what I’ve taken to calling “open source cosplay”.

If a project is asking you to assign them copyright in a CLA (GNU notwithstanding), they’re probably doing so to dual license their code. People who believe in software freedoms don’t dual license their code. People who believe in software freedoms don’t release software, any software, under nonfree licenses, such as “Enterprise Editions”.

Open core is bullshit.

Other major offenders:

Microsoft’s VS Code (all of the good features are in packages that aren’t free software and aren’t allowed to be modified or redistributed)
Google’s Android (doesn’t work without Play Services, and they’re not even pretending anymore, as future versions won’t even cosplay being open)
Docker (the commonly used Docker Desktop isn’t even source available, it’s entirely proprietary)
Redis (switched to RSAL (nonfree), SSPL (nonfree), and AGPL (nonfree))
Mattermost

Stop using these tools.

Before I’m accused of doing the standard open-source-user-entitlement thing, I want to be clear that I’m very much not asking for free software. I’m asking for these companies that are materially benefitting from marketing themselves using the term “open source” to walk the walk. You simply don’t believe in software freedoms if you gate features behind a paywall, or if you (predictably) refuse to merge those same community-implemented features into the “community edition” because you’re anticompetitive and know you can’t compete with free-as-in-beer. If you want to make proprietary software, do so, but stop pretending to be involved in the free software community by using terms like “community” or “project” or “open source” when we all know you’re not.

Mongo (switched to SSPL) and Hashicorp (switched to BSL) and ElasticSearch (SSPL) all did so and, critically, stopped pretending to be open source. The community forked Redis (Valkey) and Terraform (OpenTofu) when the upstreams did this. (Anyone smart enough to fork and maintain a project like this is smart enough to know MongoDB is and has always been shit only that is used by fools who don’t know any better.)

Most projects didn’t take this path, and still call themselves open source when they clearly don’t respect software freedoms at all.

This is dishonest and greedy. Stop pretending to be open source projects when you restrict software freedoms. Admit to yourselves and the world what your ideals and values actually are. You plainly do not believe in the freedom for others to use source code that you have written and published for any purpose they can think up.

The AGPL License Is Nonfree

2025-07-20T00:00:00+00:00

I frequently assert that the Affero GNU Public License (AGPL) is a nonfree license. People always seem surprised that I say this, or say I’m wrong, because the anticapitalist zealots at the Free Software Foundation have endorsed it and have got other organizations to certify it as free software.

It’s very obviously not. Don’t believe the hype. Let’s review:

Here’s the actual text of the AGPL.

(The part you want is section 13, “Remote Network Interaction”.)

The four freedoms, the fundamental conceptual basis of software freedom, as originally defined and espoused by the Free Software Foundation:

Freedom 0: The freedom to run the program as you wish, for any purpose.
Freedom 1: The freedom to study how the program works, and change it to make it do what you wish. Access to the source code is a precondition for this.
Freedom 2: The freedom to redistribute copies so you can help others.
Freedom 3: The freedom to distribute copies of your modified versions to others. By doing this you can give the whole community a chance to benefit from your changes. Access to the source code is a precondition for this.

The FSF and many others regard an important consequence of freedom 0 as a bug. That consequence being that you can take publicly available free software, make modifications to it, and run it privately in service of your business.

They call this the “Application Service Provider” (ASP) loophole, and they really don’t like it, because capitalists leverage software that they so graciously gave away as a gift to, you know, make money. Many other businesses that use the term “open source” for street cred (such as MongoDB and now famously Hashicorp and previously Redis) also don’t like it, because they are engaging in open source cosplay as marketing for their business and don’t actually give a shit about software freedom.

The problem is, Freedoms 2 and 3 are negative rights - they are freedoms to have the option to redistribute without interference from the copyright holder, not an obligation to do so. Using free software, even modified free software (freedom 1), to run a business is a clearly protected activity under freedom 0.

(Of course, licenses like the GPL can create obligations to distribute source code - but only if you are actually distributing software first, but that’s not comparable, because software and services simply are not the same thing, despite the common usage of the term “SaaS”. There’s a fundamental and material difference between me handing you a calculator and me telling you that the answer to 2+2 is 4.)

They see this as bad, because they think you should be forced to publish your modifications to the software you use internally, even if you don’t want to. This is, among other things, a major violation of your personal privacy (this is actually the main reason that I personally hate the AGPL - it’s compelled speech).

The AGPL is a license that attempts to force you to publish your modifications to the software you use internally, by requiring you to publish the source code of the software you run on a server (not strictly true, more on this later), if you make it available to users as a network service. The problem is, you can’t actually do this in a software license. A software license deals in permissions, not obligations. You can’t force people to do things with a software license, you can only grant them permission to do things. The AGPL is a license that grants you permission to run the software, but only if you agree to use that same software, in process, to publish your modifications to the software you run on a server. This is not a software license, but what is commonly referred to as an “end user license agreement” (EULA), because it has more than just a copyright license in it. EULAs are very distinct from software licenses (though they frequently include a software license within them).

Free software is and has always been against EULAs, because of freedom 0.

They’re trying now to square the circle, because freedom 0 is fundamentally incompatible with forcing people to use software a certain way (that is, to make it a requirement that the service you provide over a network to your customers also provides them with the modified software source code to that same service).

It’s a EULA pretending to be a copyright license.

The AGPL is a nonfree license, because it violates freedom 0 by restricting the purposes for which you can use the software (such as keeping its own source code private). It violates freedom 1 as well, by restricting the set of modifications you can make to the software (such as removing the anti-privacy misfeature the AGPL requires that the software furnish its own source code to users over the network).

It restricts how you may use and modify the software to prevent uses that the FSF doesn’t like. The FSF is, at the core, being dishonest about this because they are trying to scam people into believing that it isn’t a EULA, because “everyone knows” that EULAs are violations of freedoms 0 and 1.

It’s not just me saying this. Hector Martin (@marcan), founder and former lead dev of the Asahi Linux project (Linux on M-series Macbooks), wrote the following:

I really wish the FSF would stop pushing this nonsensical license and misleading everyone into thinking it does things it doesn’t. Their PR around it absolutely does not reflect the actual wording and consequences of the license. It’s deceptive and harmful to the free software ecosystem.

Now, let’s get to the actual nitty gritty: the AGPL as written is impossible to actually comply with in normal everyday use, and I would venture to say that every single developer working on AGPL software is violating the license every single time they touch it.

As a consequence of this I suspect it’s unenforceable in practice. (To date, it has never been adjudicated at trial.) I’m not a lawyer, and this is not legal advice.

To wit, AGPL Section 13, “Remote Network Interaction”:

Notwithstanding any other provision of this License, if you modify the Program, your modified version must prominently offer all users interacting with it remotely through a computer network (if your version supports such interaction) an opportunity to receive the Corresponding Source of your version by providing access to the Corresponding Source from a network server at no charge, through some standard or customary means of facilitating copying of software. This Corresponding Source shall include the Corresponding Source for any work covered by version 3 of the GNU General Public License that is incorporated pursuant to the following paragraph.

This was obviously written with webapps in mind; they’re trying to compel people who run such software to link to a download of the source code used to provide the service. There are several problems with this, both conceptual and practical.

Conceptually, this exists to allow end users to download the source and do something meaningful with it, such as modify it further for their own purposes. Let’s look at the biggest and most used SaaS in the world for an example. If I gave you the complete source code to Google, would that allow you to do something meaningful with it, such as run your own search engine with your local modifications? It’s a ludicrous proposition, and the logical basis for such a requirement is fundamentally damaged. Software isn’t services and services aren’t software.

But, let’s skip that for a moment. Let’s say you’re a developer and you clone an AGPL repo, and run the server locally. You’re all good, the in-app link still points to the source code. Then you make a change and “npm run” to try it out, but now the link in the app points to the original version upstream, not your version running on your local computer. License violation! So you update the link to point to your modified copy. Oops, there’s no way to do that - your version only exists on your laptop, which isn’t running a webserver or is behind NAT; no URL to your modified copy exists to link to. So you push your code up to a public git repo, and change the link to point to that. Now you’re in compliance again, but only for a moment. The first moment you make a change, and re-run the server (without committing and pushing your change to the remote public repository first), you’re violating the license again as the network service is running but the source code is not available to any users thereof.

This is a fundamental problem with the AGPL. It’s impossible to use AGPL software in a normal development process that is compliant with the license terms. The only way you could do it, I think, is to embed the complete source code of the program into the binary itself, and serve it directly from there. This works for webservers, but what about, say, a database server, that has no way of serving arbitrary files to end users? Or must we include a listening HTTP port in every AGPL program? What about that pesky freedom 1 that allows us to modify the program in any way we see fit (such as removing an unnecessary web server component)?

The AGPL sucks in every way you look at it. It’s damaging to the free software movement, because it undermines the credibility of the FSF and enables charlatans like Minio, Matrix/Element, Proxmox, Bitwarden, Anki, Mastodon, Mattermost, and many many others to get all of the public credibility of being an open source project, without actually being free software, or respecting software freedoms.

Credit where credit is due: I’ve harbored these feelings for a long time, but it wasn’t until marcan really laid it out specifically that I was able to crystallize these thoughts into a fully formed opinion.

Finally

The core issue here is that people now seem to believe that releasing software under free software licenses morally obligates the recipients to release their modifications in turn (which is more readily encapsulated in the viral copyleft provisions of the original GPL, tellingly one of the least popular open source licenses).

The problem is, it’s simply not reasonable. It only got amplified because people were using modified free software to make gobs of money, to which the original authors then felt entitled. They believed that because they spent time, effort, and money making software, that the changes that others made should correspondingly be “contributed back” to them, because “big rich corporation” and “little guy”.

Gifts absolutely do not work like that. To release free software is to give a gift to the world, freely. It is not a transaction, and it does not confer any requirement on the recipients of those gifts to “give back”. Amazon taking Redis and modifying it and selling it as a service doesn’t cost the original authors a single thing or harm them in any way, and the idea that they are entitled to anything in return is simply a delusion.

When you give free gifts to the whole world, you are also inseparably and simultaneously giving free gifts to the rich and powerful. You are giving gifts to people who will use your software to do harm, to violate the privacy of their users, to make money off of your work, and to otherwise personally enrich themselves in any way they can possibly imagine. This is the value of free software to the world. Freedom 0 is fucking important.

Famously, Douglas Crockford used the MIT license with a custom modification when releasing JSLint:

The Software shall be used for Good, not Evil.

This was a little bit of a joke, but notoriously rendered the license nonfree, as such a restriction plainly violates freedom 0.

We explicitly have the right to use free software to do Evil. (“Evil” in the eyes of the licensor, of course.)

I support this wholeheartedly. It is analogous to supporting free speech for those who say things you don’t like.

Other nonfree license examples that restrict use are the Hippocratic License (prohibitions against violating human rights, war, surveillance, etc), Peace Public License (no military or law enforcement use), and the Do No Harm License (prohibitions against use for war and weapons and fossil fuels).

The FSF and others now see Freedom 0 as a “loophole” that needs to be closed, that their gifts should come with strings attached if you’re going to use them to run a service to make money. This is a fundamentally broken and toxic view of software freedoms. “Use it for anything, but not like that” renders your position fundamentally unserious.

There’s a common trope of calling open source software users “entitled” because they file snippy bug reports, or demand features, and otherwise act ungrateful for the gifts that they receive. This is a valid criticism of the community.

The problem is that open source developers are acting even more entitled with their embrace of the AGPL: entitled to violate the privacy of their users who use their (freely given!) source code because they’re modifying it to make money. If you want a EULA on your software, stop giving it away as free software, duh. You don’t get it both ways.

Subject Lines and File Names: You’re doing it wrong.

2025-04-30T00:00:00+00:00

I got an alert today for a meeting I have 24 hours from now.

The person who named this just typed some words into the box, the first ones that came to mind. Don’t be that guy: take 10 seconds to think about what this field is going to be used for and why.

The assumptions built in to a title such as “Project kickoff call”:

I know which project it’s supposed to be (I don’t)
I know what “kickoff” implies in terms of agenda (I don’t)
I remember who’s going to be on the call (I don’t)

Be specific when you name things!

If it’s a 1-on-1, name both parties in the title. You know who you are, I don’t.

Include an agenda in the description if it’s something that is safe to be sent via email.

For file names, it might be a good idea to include the date the file was authored in the format YYYY-MM-DD at the beginning of the file, e.g. 2025-04-30.Project.Dragonfly.Agenda.md. This way, it will sort chronologically in a directory listing. Documents should always, always, always have a date on them (otherwise, how does a reader know if it’s a day or a year or a decade old?), and a date in the filename serves the purpose.

Readme Howto

2024-12-24T00:00:00+00:00

The United States Virgin Islands: Notes

2024-12-24T00:00:00+00:00

I just visited the Lesser Antilles for the first time, specifically the island of St. Thomas in the US Virgin Islands, during the first week of December 2024.

I’m fairly well-traveled for an American (at least in the industrialized world), having been to places like Russia, China, North Korea, Good Korea, et c. The following is a list of things that I was not expecting about the USVI.

For historical reasons, they drive on the left, despite being part of the United States.
For presumably economic and legal reasons, they use standard mainland US vehicles, which are the same left-hand drive types used in drive-on-the-right places. This is, of course, insane, and I suspect relatively unsafe.
Like China, everyone drives with their high beams on 100% of the time at night without regard for other drivers.
Despite being a pretty significant distance from the equator, they’re far enough south and thermally buffered by ocean that the water temperature even in December doesn’t dip under 70F. It’s full bore suntan-and-swimming weather even at the dead “coldest” part of the year.
The electric grid is expensive, and trash. I understand it’s the same in Puerto Rico. All of the big resorts have HUGE generators.
The roads are in the worst shape of anywhere I’ve ever been. Steep terrain, and foot-deep potholes suggest that vehicles don’t last very long there.
Gasoline and diesel are very expensive at retail.
There’s a big lack of fine dining; the island seems to be more geared toward discount vacationers and those arriving on cruise ships.
A US Amazon Prime membership does not include free shipping to USVI. Shipping charges for most items looked to be $10-20 each. Shipping estimates were over 10 days out, which means that if you visit for 7 days as I did, you can’t use Amazon at all while you are there. Larger packages can’t be ordered from Amazon as they simply won’t ship them to those ZIP codes, period.
Phone calls to the USVI are toll calls. The rest of the continental US effectively abolished long-distance toll charges for phone calls well over a decade ago. I went to call a NANP (+1-NNN-XXX-XXXX) number from my Google Voice account (which allows unlimited free calls to US numbers even with a zero balance) and received an error that my account didn’t have enough credit balance to make the call. I was floored. I suspect without evidence that this anachronistic state of affairs is due to deeply-rooted telco monopoly corruption, but I have done absolutely no research whatsoever on this matter.
There are insanely strict firearms laws in the USVI. There is repeated and very insistent signage all over the arrivals area demanding that people declare any firearms or ammunition. It feels in this regard like visiting a foreign country.
Outside of the shiny corporate resorts, the island is a lot more rural and poor and in disrepair than I anticipated. I understand that these days it gets proper fucked by massive hurricanes on a very regular basis.
Flying back to the mainland US (ATL) from St Thomas (STT), an ostensibly domestic flight, one must pass through a US Customs and Border Patrol (CBP) interrogation stop prior to the TSA security search checkpoint, same as when entering the US from a foreign country. All passengers passing through the departure hall are funneled into this area, regardless of destination. I presume this is a result of the fact that CBP claims jurisdiction anywhere within 50(?) miles of the US border, which encompasses most major cities in the US as well as the entirety of the USVI. They asked me the usual nosy (and optional, for a US citizen) questions. They commanded(!) me to stand for a biometric-capturing photo, and responded punitively when I inquired if it was optional (they said yes) and I declined: standard mainland CBP asshole behavior, just with an island accent. (I’ve had the same kind of aggressive, indignant, rude, and unprofessional treatment from CBP upon exercise of 5A and 6A rights before, over and over and over again for many years now. The US supreme court, in their infinite wisdom, has decided that 4A rights do not apply to US citizens at the US border.)
Everywhere I visited seemed to be a tourist trap, though I did not very dilligently seek any non-tourist-trap activities, areas, or excursions, as I was there on a bog-standard beach tourist holiday.
The weather, while a bit humid for my Mojave-adapted preferences, is completely fantastic.
There are very nice beaches and very nice swimming, something I have not often experienced on coastlines of Big Ocean. I’m told that the beaches are all public, though the usual issue remains in such places: many are walled off so you can only access them via secured private property, or boat.
It seems like a nice place to go diving, or sailing. The water is nice and I got to snorkel with some sea turtles. There are tons of boats around, from star-destroyer-sized mega cruise ships (multiple per day), to the superyachts of billionaires (I saw “Kaos”, the yacht of the Walmart heiress, among others), to little family fulltime sailboats. It’s as active a port as I’ve ever seen.
Like a lot of small tourist islands, the airport is filthy, cramped, over-crowded and under-air conditioned. (Unrelated: I observed that JTR fixed this about theirs within the last few years.)

Discussion

This, as all posts on this blog, can be discussed on the BBS.

My 2024 Code Styleguide

2024-06-10T00:00:00+00:00

I have documented and published, for the first time, my personal code style guide. It is a living document (which is why it’s in git) that I will update periodically as I consciously notice more of my longstanding habits and techniques. I estimate it’s 70-80% complete (at least for Go) presently; the other languages included are just stuff that’s off the top of my head.

Half the trouble with documenting things like this for publication is that a ton of my experience and methods are so ingrained that I don’t even notice them anymore.

It is my hope that it is useful and instructive to less experienced programmers. I will do my best to amend it.

It can be found at: https://git.eeqj.de/sneak/styleguide.

Feedback is welcome. I can, as always, be reached at [email protected]">[email protected].

The comment thread for this post can be found on the BBS.

Apple OSes Are Insecure By Design To Aid Surveillance

2023-10-05T00:00:00+00:00

I have a theory that I believe is supported by enough evidence for you to believe it, as well.

Mind you, this is definitionally a conspiracy theory; please don’t let the connotations of that phrase bias you, but please feel free to read this (and everything else on the internet) as critically as you wish.

I believe that Apple is preserving unencrypted server connections in their operating systems in an effort to enable global location tracking of their userbase by passive monitoring of major internet backbones.

This is supported by timelines and context, which will be provided.

Several important connections (TSS, OCSP) are made from Apple devices in plaintext (that is, completely unencrypted). This began for historical reasons, but has been repeatedly reported to Apple. They have not fixed it.

TSS checks happen on update. OCSP checks happen, among other times, on app launch.

Apple committed in writing a few major versions (i.e. ~3 years ago) to providing a preference setting for disabling online OCSP checks in macOS when I made a stink about it, within one year. Not only did this not happen within a year (a rare instance of Apple actually outright lying), but someone was kind enough to write me and tell me that Apple has edited the webpage to remove this promise. Presumably there are no plans to offer users ability to disable OCSP checking, which leaks which apps are being launched on your system, when you launch them.

Not only can you not disable it, they’re still not happening over encrypted (https:) connections. This is straightforward to fix, but it hasn’t happened.

Apple’s webpage says:

We have never combined data from these checks with information about Apple users or their devices.

We do not use data from these checks to learn what individual users are using on their devices.

These security checks have never included the user’s Apple ID or the identity of their device. To further protect privacy, we don’t log IP addresses associated with Developer ID certificate checks, and we make sure that any collected IP addresses are removed from logs.

The problem is that the connections are still unencrypted! Anyone in the world who can watch the internet traffic to or from your computer, or to and from Apple, or between your computer and Apple (this is a lot of people), can make their own logs, with all of the IP addresses and all of the launched apps. If you use enough apps, the specific constellation of your apps is probably pretty close to uniquely identifying you.

The OCSP checks (“Gatekeeper”, in Apple’s terminology) are not the big deal, however.

When you update a modern mac, it needs something called a boot “ticket” for the new OS. This ticket is cryptographically signed by Apple, and is unique to your specific Mx Apple Silicon CPU/SoC (or your specific T1/T2 security chip, if you are using an Intel mac).

The request to Apple for this boot ticket is via an API (called TSS), and includes specific unique identifying serial nubmers of your computer (such as your chip’s ECID) that never, ever change. It’s done on every major OS update, and, you guessed it, it’s done totally unencrypted. Anyone watching the backbone traffic on the internet will be able to pair ECIDs with client IP addresses on every major macOS update. (Client IP addresses identify city-level location. With the information available to DHS/FBI from the carriers and cable companies, they identify a specific building and subscriber name.)

This is a pcap taken today of the OS updater included in 13.4, released in May of this year, transmitting my ECID in plaintext.

I screamed loudly about this in April 2022 and ended my post with “Continued transmission of plaintext identifiers will be assumed as malicious intent. Fix it.”

It was not fixed. Not in the next release, not in the next major version.

It’s still not fixed in Ventura 13.x, at least as of 13.4 (May 2023). Apple has been leaking this customer PII across the internet unencrypted for seven years, ever since the introduction of the first T1 chip in the original Touch Bar MBP.

First, during the update, Apple’s updater has to send your activity data back to the mothership even when analytics transmission is explicitly disabled. (This is why xp.apple.com is in so many hosts file privacy blocklists.)

Then the updater can get to the important work of leaking your plaintext ECID across the whole internet.

I would be likely to give Apple the benefit of the doubt here, if not for two very important mitigating factors:

Apple does not allow plaintext server communications in apps released by developers in the App Store. This is explicitly against the rules, and they have tools available for app developers to use that make 100% encrypted connections the unavoidable status quo (App Transport Security). But for some reason for seven years and counting they didn’t mandate this for their own OS updates.
Apple has a documented history of preserving cryptographic backdoors to aid US government surveillance.

You might argue that the latter is now invalid, given that there is now an option to enable end to end encryption (on this page, hereinafter referred to as E2EE, but called “Advanced Data Protection” by Apple) for iCloud and thus iCloud Backup. That’s also not valid, and I’ll explain why.

First, iCloud E2EE is opt-in. The setting is buried, and there are no prompts to enable it, so approximately 0% of iCloud users have turned it on. It might as well not exist.

Second, iCloud E2EE is woefully incomplete. When you iMessage with someone, they have iCloud Backup on by default, and non-E2EE by default, which means that approximately all of your iMessages (including all image and document attachments) will still be readable by Apple and the FBI because they are backed up twice: once from each end of the conversation. Unless you and ALSO everyone you iMessage with has enabled E2EE (which, today, is never, ever true) then your iMessages are subject to surveillance by Apple and the whole of the US government that can force them to turn them over.

Furthermore, the E2EE for iCloud Photos is not designed to preserve privacy. Even though iCloud Photos now supports E2EE for the content of the photos and videos stored, the file metadata is not E2EE, and the metadata includes the FILENAME and also a unique hash of the unencrypted file content. This means that if you make a first-of-its-kind Winnie the Pooh meme and save it to your camera roll (hooked up to an E2EE-enabled iCloud Photos account), then send it via secure means (Signal, or in-person AirDrop, or whatever) to another person who has iCloud Photos enabled (also with E2EE) and they save it, Apple can see that you both have the same file, the only two people in the world with it.

They can see who had it first. They can see who had it next, and when. They can see where it went after that. This is with full “E2EE” turned on, and without even using Apple messaging apps. This leaks your social graph, too, and it provides a nice list of dissidents who all have the same file.

This applies to all files in iCloud Drive, too, not just your photos. Share a file securely within a group of people who all have E2EE enabled for their iCloud Drive? Guess what, Apple now knows they’re a group, and by extension the US and Chinese governments can, too.

If you enabled E2EE for iCloud Drive, would you expect that Apple sysadmins can read all of your filenames? How about the FBI without a warrant?

Update 2024-03-11

As of at least macOS 14.3.1 (and, if I recall correctly, several versions prior, for at least a few weeks now) the update process (UpdateBrain) that obtains the hardware-specific boot ticket signatures now uses TLS to contact gs.apple.com. I assume it finally happened due to this webpage.

“Privacy, that’s Apple.”

Any repressive government can now go to Apple with an image file or document (or its hash) and demand a list of every single phone number, payment card number, full name, and last known device location of everyone in iCloud who has a copy of the file, even if all of those people have opted in to Apple’s false-sense-of-security E2EE system.

Note that this was always possible before, too. But it’s still possible, even under the current E2EE system. So far, it’s farce.

Why Bother?

What exactly is the point of rolling out this E2EE feature if:

you’re not going to prompt to migrate people to it
you still enable authoritarian repressive surveillance of your users even when it’s turned on, because its design sucks
you still leak the social graph of the users based on the path and time of spread of unique files

Apple has designed and operated truly private systems before. iCloud Keychain and Health are two bits that have been E2EE and are, as far as I can tell, immune to surveillance orders or corrupt governments (such as those in Apple’s two biggest markets).

(Please don’t write me about how the iCloud plaintext content hashes are to support deduplication. Apple already did in HT202303. That’s not the point, and it’s irrelevant. Also please don’t write me about how E2EE is bad for most users because users will lose all their keys and then lose all their photos and be sad. It’s called sneak’s law and I wrote it. It’s irrelevant to the topic today.)

Apple says in HT202303 that they are “committed to ensuring more data, including this kind of metadata, is end-to-end encrypted when Advanced Data Protection is enabled”. Maybe if they hadn’t dragged their feet for the FBI years ago then privacy would actually be baked into this product by now.

Note well: You can’t use Homepods, Apple Pay, Handoff, or Passkeys without opting in to iCloud; each year choosing not to use iCloud comes with a larger list of disabled functionality on your device. I imagine the Apple Vision will be similarly hobbled without sending Apple and the FBI a constant realtime stream of your life’s activity in the form of iCloud metadata. A ton of the functionality of Apple devices is missing if you don’t submit to the ability to be surveilled.

Wrapping Up

iCloud is a privacy nightmare. iMessage is not end to end encrypted due to both endpoints escrowing their secret keys to Apple by default in the non-E2EE iCloud Backup. Even if you turn on E2EE on one end, it’s ineffective because the other end still has it off.

On every macOS update, you transmit plaintext TSS API requests which include your unchanging ECID, alerting everyone on the internet backbones of your ECID-to-IP mapping, allowing your movement to be tracked over time.

On many app first launches (not every launch), you transmit plaintext identifiers leaking what developer’s app you have launched and when (and client IP, again). (Note that most developers only publish a single app, so this aliases to which app you have launched.)

Furthermore, Apple knows all of these things, and has opted to do nothing meaningful to mitigate the threats.

I think that macOS has too many plaintext network privacy leaks, for far too long (in the context of everything else I’ve enumerated here) for this to be carelessness, coincidence, or deprioritization.

Finally

Please take the time to go to Berlin, and visit the Stasi museum, if you think my warnings about the potential for abuse by the FBI when they are not constrained by the need for probable cause or search warrants are overblown.

This is the same FBI that wrote Martin Luther King Jr. anonymous letters telling him to kill himself.

Presently, the US government is accessing the full content of around seventy thousand(!) Apple accounts per year (as of 2022) without a search warrant, per Apple’s own transparency report. The numbers are much higher when you include warrants issued with probable cause.

My concern is not so much that Apple is the threat, as Apple is not in the business of oppression or imprisoning journalists, but the governments that can meaningfully compel Apple: the United States, and the People’s Republic of China. (Google could pull out of China and lose their customers; however approximately 100% of everything Apple sells is made in China, by Chinese people, in factories subject exclusively to Chinese law. China has more control over Apple presently than the USA does.) Anything Apple can know, the CCP or USG can know.

A society that is under constant and total suspicionless police surveillance is a society that is, given enough time, actually doomed.

Apple is complicit in building this society currently, and it poses an existential threat to freedom worldwide if it is not rectified.

Finally, if you still don’t believe that Apple would be game to play ball in such a manner, read this. There isn’t much wiggle room for large corporations to refuse the direct demands of the military in the USA, there’s simply too much asymmetry in terms of organizational goals. Corporations are ultimately extremely fragile and far too vulnerable to the state’s will.

Footnote

I’m experimenting with blogging more quickly and with less time spent editing. Please provide feedback on this or any other post on the BBS or via email.