rebecca®the digital home of rebecca turner, software engineer & mathemagicianZola2025-11-10T00:00:00+00:00https://becca.ooo/atom.xmlVertical Integration is the Only Thing That Matters2025-11-10T00:00:00+00:002025-11-10T00:00:00+00:00
Rebecca Turner
https://becca.ooo/blog/vertical-integration/<p>On the subject of developer tooling, or perhaps computer programs more broadly,
I have become increasingly convinced that vertical integration is the only
thing that matters. I also think that the inability of developer productivity
startups to vertically integrate their offerings has hindered their adoption
and utility. I’d like to talk about what I mean by “vertical integration” and
why we don’t have it today.</p>
<h2 id="what-is-vertical-integration"><a class=anchor href="#what-is-vertical-integration" aria-label="Link to section what-is-vertical-integration">#</a>What is vertical integration?</h2>
<p>When I say “vertical integration”, I am referring to tight integration between
different tools in a stack.</p>
<p>Here are some workflows that are possible with a vertically-integrated stack:</p>
<ol>
<li>
<p>You create a new branch and build the project you’re working on. You haven’t
made any changes yet, so the build finishes in under a second because your
build system shares artifacts with builds from CI and deploys. This build
system also caches artifacts from builds you initiate locally, so a coworker
who checks out your branch (also a single click from the PR) will be able to
run your code immediately.</p>
</li>
<li>
<p>A test fails in CI. You click a button to open the failing test in your
editor, which gives you an interactive call-stack (potentially spanning
multiple binaries and programming languages) to explore.</p>
</li>
<li>
<p>A check on your PR fails because your code wasn’t formatted correctly. You
click a button to apply the formatting changes directly to your code, and
they’re immediately reflected in your editor.</p>
</li>
<li>
<p>You merge a PR which breaks a service. Your PR is initially deployed as a
canary to a small portion of clients. The deploy system detects an elevated
failure rate, and rolls back your PR automatically. This adds a note to your
PR and messages you on your Slack-equivalent. The note indicates which
checks failed, and you can click them to see a history of those checks.</p>
</li>
<li>
<p>You click on a function in your editor and are presented with a list of
usages of that function. The list includes entries from other projects,
written in other languages, and across RPC boundaries.</p>
</li>
<li>
<p>A service crashes in production. A ticket is automatically opened and routed
to your team (or a note is added to an existing ticket). You click a button
to open the line of code that crashed in your editor, and again you’re shown
an interactive call-stack for the failure.</p>
</li>
</ol>
<p>All of these workflows span multiple parts of the stack: local builds and CI
builds, the test runner and the editor, the code review system and CI, the
deploy system and version control. These features are often derisively referred
to as “glue code”. <strong>The glue code <em>is</em> the vertical integration work.</strong></p>
<p>I’d like to draw your attention to a couple different themes here:</p>
<ol>
<li>
<p>None of the features here are particularly shocking, but they all require
cooperation between tools that aren’t used to cooperating. Your test runner
knows the call stack of a failing test, but it can’t make that information
available in a format your editor or terminal is able to consume. Your
deploy system runs an optimized build and then throws away all the
artifacts, so if you want to build the same commit you need to start from
scratch. The compilation was already run, but your build system isn’t able
to grab artifacts from CI because your build system doesn’t know that you
<em>have</em> CI.</p>
</li>
<li>
<p>IDEs make some of these workflows possible, albeit scoped to a single
project. This is because IDEs implement glue integrations between a bunch of
different test runners, build systems, languages, and so on.</p>
</li>
<li>
<p>Many of these workflows require infrastructure to be running independently
of your CI, deploy system, and developer workstations. If you want CI to
tell you if a test has failed recently in other builds, you need a system
that knows when that test was run and what the results were. If you want to
share build artifacts between CI and developer builds, you need a system to
cache those artifacts and accurately identify when they can be reused.</p>
</li>
</ol>
<h2 id="why-isn-t-a-vertically-integrated-stack-the-default"><a class=anchor href="#why-isn-t-a-vertically-integrated-stack-the-default" aria-label="Link to section why-isn-t-a-vertically-integrated-stack-the-default">#</a>Why isn’t a vertically-integrated stack the default?</h2>
<p>The reasons why vertically-integrated stacks aren’t common are a bit different
for open source and industrial users, but industry funds open source so there’s
a fair amount of overlap.</p>
<h3 id="vertical-integration-in-open-source"><a class=anchor href="#vertical-integration-in-open-source" aria-label="Link to section vertical-integration-in-open-source">#</a>Vertical integration in open source</h3>
<p>Open source projects tend to be miniscule. Even projects with hundreds of
thousands of lines of code are uncommon, while codebases of that size seem
quaint to industrial engineers.</p>
<p>While I’m sure that many engineers would enjoy having access to the workflows I
described at the start of the post, the motivation is a lot less pressing. Who
needs shared build artifacts when a clean optimized build takes a couple
minutes? Who needs integration between the test suite and the code editor when
the entire codebase fits in the working memory of the only engineer who
seriously works on it? And gradual rollouts don’t even make sense when the
project doesn’t publicly provide a hosted instance of the software.</p>
<p>Integrated stacks also present a coordination issue for open source projects.
Does the integration between the test runner and the code editor live in the
test runner or the code editor? How is it tested? Open source is full of petty
kings and overgrown forum moderators, so this sort of collaboration is rarely
feasible in practice.</p>
<p>Open source is also full of software freedom acolytes who insist that each tool
must “do one thing well.” To these engineers, project A maintaining an
integration with project B is a threat to the ability of users to swap out
project B for a different tool; the best approach, to them, is for every tool
to behave as if no other tool exists. The fact that this results in strictly
less-capable tools seems to be lost on these engineers.</p>
<p>At the same time, many open source projects are owned or funded largely by a
single corporation with no motivation (or ability) to make their internal stack
available externally. Any integrations in the project must therefore be
compatible with the stack used by those corporations internally. For similar
reasons, it is also common to see projects with test suites or build systems
that cannot be used outside of the organization that funds them.</p>
<h3 id="vertical-integration-in-industry"><a class=anchor href="#vertical-integration-in-industry" aria-label="Link to section vertical-integration-in-industry">#</a>Vertical integration in industry</h3>
<p>Industrial users aren’t interested in a smooth developer experience because it
makes engineers’ lives easier. Industrial users want a vertically-integrated
stack because engineer time is <em>expensive.</em> This means that a
vertically-integrated stack must save time in an absolute sense; that is,
<em>including</em> the time it takes to implement and adopt the system.</p>
<p>These migration costs push industrial users away from tools that span the
stack. When startups are small, the overhead of implementing such a system
increases the likelihood that the startup will fail. An organization must be
stable enough in order for spending time on integration work to be justifiable.
But once an organization <em>is</em> stable enough to want to build out a smoother
developer experience, they’ve already developed their own bespoke build and
deploy system, often spanning multiple repos with complex dependencies and
subtle quirks. At this point, the costs of switching to an off-the-shelf
vertically-integrated stack are prohibitively large.</p>
<p>Organizations that investigate e.g. switching to Bazel will hear horror stories
of migration efforts that went on for years before being scrapped. This is
another issue with implementing tight vertical integrations across the stack:
deep investment is needed, over an extended period of time, in order for
benefits to materialize. Many younger organizations used to bringing features
from planning to production in one or two quarters find this sort of investment
very challenging to justify.</p>
<h3 id="vertical-integration-as-a-product"><a class=anchor href="#vertical-integration-as-a-product" aria-label="Link to section vertical-integration-as-a-product">#</a>Vertical integration as a product</h3>
<p>Could we build a vertically-integrated development environment as a product and
sell it to companies who would otherwise build a similar but less-capable
system internally? On the surface, this sounds like a great idea: lots of
organizations are devoting a lot of effort to building out pretty similar
systems for building, deploying, and editing code. It would be fantastic if we
could centralize that effort, saving each organization valuable engineering
time while delivering a better experience.</p>
<p>But remember what we’re talking about: integrations between a code forge, build
system, deploy system, CI, and code editor. Large organizations are either
unwilling or unable to adopt radically different tools. Your developer tools
startup will then need to build and maintain integrations between <em>N</em> products
for each of your <em>M</em> large clients. At the same time, you’ll have to compromise
on features; if you can’t ship a patch to GitHub, you can’t build a link to
open a failing test case locally into their UI.</p>
<p>Selling integrations is also a risky position for a company to be in. There’s a
natural incentive for the companies being integrated to write replacement
integrations in-house and claw back some of the profits that are going to the
external competitor. Vendors selling integrations may also find themselves in
an uncomfortable position when the services they’re integrating with raise
their prices or restrict API access. Cursor, for example, had to <a href="https://techcrunch.com/2025/07/07/cursor-apologizes-for-unclear-pricing-changes-that-upset-users/">hastily
change their pricing</a> after Anthropic changed their prices.
Cursor also had to <a href="https://devclass.com/2025/04/08/vs-code-extension-marketplace-wars-cursor-users-hit-roadblocks/">beg Microsoft to unban Cursor</a> from the
VS Code extension marketplace. Anthropic and Microsoft both view Cursor as a
competitor and would rather they did not exist at all. This is not a foundation
for a dynamic of productive collaboration between these companies on
integrations between their services!</p>
<p>For smaller organizations, adopting an integrated developer environment
requires a huge leap of faith. GitHub is a very well-known and mature product.
Every enterprise developer tool provides a GitHub integration (not as good as
first-party support, but much better than nothing). What if the organization
providing your developer tooling shuts down? It will be extremely challenging
to convince early adopters that the costs are worth it. The migration costs we
discussed earlier also apply to <em>leaving</em> such a system, except that users may
find themselves <em>forced</em> to leave such a system with little notice.</p>
<p><a href="https://www.scottkennedy.us/developer-cloud.html">Scott Kennedy notes in “It’s not just an IDE: building the developer cloud is
hard”</a> that it’s possible to build a <em>piece</em> of the vision
successfully, but benefits remain limited without cooperation from the rest of
the stack:</p>
<blockquote>
<p>Startups or open source historically focused on one piece of the puzzle.
Sourcegraph has been working hard for a decade just to get the CodeSearch
piece right. Bazel might be open source, but it has a “batteries not
included” feel. The full experience requires putting many pieces together.
It’s hard.</p>
</blockquote>
<p>When (e.g.) code search and the build system are produced by different
corporations, the profit motive discourages corporations from building tight
integrations with other products in the same space.</p>
<p>The vertical integration, the glue code linking these different products
together, really is the thing that creates the value here. And the glue code is
inherently tied to the quirks of the infrastructure of the corporation that
built it:</p>
<blockquote>
<p>We’ve also seen dozens of companies successfully take Google/Facebook’s
internal tooling ideas and build billion dollar B2B SaaS companies. So where
is my developer cloud?</p>
<p>It doesn’t exist because it’s really fucking hard.</p>
<p>Google can’t make theirs available because they can’t disentangle from their
internal architecture without damaging their internal productivity. I know
because I was there when they tried. It’s hard.</p>
</blockquote>
<p>The value comes from the whole stack being tightly integrated. If you can’t
ship the whole stack, you’re very limited in the features you can provide. Glue
code is very challenging to sell. Vertical integration is the only thing that
matters.</p>
(Dark Souls Font) WiFi Restored2025-09-26T00:00:00+00:002025-09-26T00:00:00+00:00
Rebecca Turner
https://becca.ooo/blog/wifi-restored/<p><em>Previously:</em> <a href="https://becca.ooo/blog/my-iphone-hates-my-wifi/">My iPhone Hates My WiFi</a>.</p>
<p>For about a year, iPhones in my household would frequently disconnect from the
wifi, with no clear explanation or cause. After a month of frustration I got
desperate enough to spend $300 on a Ruckus R350 access point that didn’t help
(NB: an access point is “the thing that makes the wifi”). And then I just… gave
up for almost a year, I guess. Well, my blog post a couple weeks ago got me
interested in doing a little more debugging. The next time my phone claimed to
be connected-but-not to the wifi (with a connection listed in the system
settings but not in the status bar at the top of the screen), I went into the
connection details to see… nothing in the DHCP section!</p>
<p>I confirmed that DHCP seemed messed up the next couple times the connection
faltered. Sometimes the whole DHCP section was blank, and sometimes I had a
garbage address in 169.254/16. My good friend <a href="https://selectric.space/">Liz</a>, who knows much
more about networking than I, informed me that the 169.254/16 block is reserved
for <a href="https://en.wikipedia.org/wiki/Link-local_address">link-local addresses</a> and generally indicates a DHCP failure.</p>
<p>My landlord has graciously included a $50 surcharge on my monthly rent for
mandatory Xfinity internet service, so I’ve been using the Xfinity-provided
modem up until now. I’ve never really liked this (there is something very
unnerving about Xfinity having my wifi password) and the DHCP logs available to
me in the modem’s web interface were nonexistent, so at this point I was forced
to accept the conclusion I’d been trying desperately not to reach for almost a
year: after spending $300 on a new access point, I needed to pony up even more
cash for a new modem. For reasons I don’t really understand, your ISP needs to
approve of the specific modem you’re using; you can’t just get a random DOCSIS
3.1 modem. So I went to dig up the list of <a href="https://www.xfinity.com/support/articles/list-of-approved-cable-modems">Xfinity-approved
modems</a> and saw the <a href="https://store.ui.com/us/en/category/internet-solutions/collections/pro-internet-solutions/products/uci">Unifi UCI</a> listed as an option.
Well, the dolls seem to like the Ubiquiti hardware, so I went for it. Who can
resist the allure of bead-blasted galvanized steel and those adorable little
touch screens?</p>
<p>A couple days later, my shiny new modem arrived, so I unboxed it and plugged it in:</p>
<blockquote>
<p><time aria-hidden=true>5:22pm</time> <strong>Me:</strong> might be time to set up a modem<br>
<time aria-hidden=true>5:22pm</time> <strong>Me:</strong> how hard could it possibly be<br>
<time aria-hidden=true>5:26pm</time> <strong>Me:</strong> wait do i need a gateway -_-<br>
<time aria-hidden=true>6:10pm</time> <strong>Liz:</strong> …yes.<br>
<time aria-hidden=true>6:10pm</time> <strong>Me:</strong> Fuck<br>
<time aria-hidden=true>6:11pm</time> <strong>Me:</strong> liz…<br>
<time aria-hidden=true>6:11pm</time> <strong>Me:</strong> what is a gateway<br>
<time aria-hidden=true>6:11pm</time> <strong>Liz:</strong> some kind of main router, y’know<br>
<time aria-hidden=true>6:11pm</time> <strong>Liz:</strong> idk if the uniif <em>[sic]</em> cable modem does that! or if it’s just a modem!<br>
<time aria-hidden=true>6:11pm</time> <strong>Liz:</strong> but knowing unifi it’s probably just a modem.<br>
<time aria-hidden=true>6:11pm</time> <strong>Me:</strong> i have switches. i have an access point.<br>
<time aria-hidden=true>6:11pm</time> <strong>Liz:</strong> but you need a ROUTER<br>
<time aria-hidden=true>6:11pm</time> <strong>Me:</strong> <span class=visually-hidden>fuck</span><span aria-hidden=true class=imessage-bloom><span>f</span><span>u</span><span>u</span><span>u</span><span>u</span><span>u</span><span>u</span><span>u</span><span>u</span><span>u</span><span>u</span><span>c</span><span>k</span></span><br>
<time aria-hidden=true>6:12pm</time> <strong>Me:</strong> g-d dammit<br>
<time aria-hidden=true>6:12pm</time> <strong>Liz:</strong> lmao</p>
</blockquote>
<p>So I bought <a href="https://store.ui.com/us/en/category/all-cloud-gateways/products/ux7">a Unifi gateway</a>. In for a penny, in for a pound, right? At
this point in my adventure a $200 gateway-cum-access-point seemed downright
thrifty to me. I got the gateway set up and… everything has worked perfectly.
Nobody in my home has found themselves unexpectedly disconnected from the
internet. I can finally stop thinking about it!! The Ubiquiti management
interface and app are extremely slick, surfacing tons of useful information:
latency and throughput graphs, alerting when the internet goes down, and, of
course, an (empty!!!) log of DHCP failures.</p>
<p>Thus ends the 2024–2025 Bad WiFi Saga!</p>
My iPhone Hates My WiFi2025-09-11T00:00:00+00:002025-09-11T00:00:00+00:00
Rebecca Turner
https://becca.ooo/blog/my-iphone-hates-my-wifi/<p>Sometimes, my iPhone disconnects from my wifi. The little icon disappears from
the status bar. Then, when I go to the settings app to reconnect, it claims
that it’s still connected to the wifi. Maddening. What follows is an
infuriating ritual of flicking the airplane mode and wifi switches on and off
until it will stay connected. Sometimes it will connect, showing a strong
signal in the status bar, only for it to immediately disappear. It probably
takes 3-5 iterations on average to get it to stick. I am not aware of any way
to reproduce the issue, which has greatly hindered my ability to resolve it. I
have not been taking notes as thoroughly as I should have. The bug is
intermittent. On a ‘bad day’ I will be disconnected from the wifi a handful of
times.</p>
<p>It started, I think, with iOS 18 (released September 2024). This is one of the
things I really wish I’d written down. The bug impacts my partner’s iPhone,
too.</p>
<p>The network itself isn’t going down. My partner will tell me their phone has
disconnected from the wifi while my phone remains perfectly connected. Some
Android phones have issues with the network. My MacBook seems largely immune to
its issues. Ethernet, of course, remains reliable as ever.</p>
<p>I bought a new Ruckus R350 access point in November 2024 to see if that would
help. It didn’t. I have not tried replacing my modem, but my ISP did it anyways
unprompted. I’m not sure if I’m allowed to <abbr title="Bring Your Own
Device">BYOD</abbr>; it’s a big box in a frankly inconvenient spot in my living
room with one of those big “it’s a felony to look at this” stickers on it.</p>
<p>Do you, reader, have any ideas for debugging this issue, or even reproducing
it? Are you trapped in this strange predicament with me? <a href="/contact">Drop me a
line.</a></p>
My Student Loan Servicer took $560 and kept it for a year2025-09-08T00:00:00+00:002025-09-08T00:00:00+00:00
Rebecca Turner
https://becca.ooo/blog/an-interest-free-loan-2/<p>A quick update on last year’s <a href="/blog/an-interest-free-loan">“Giving My Student Loan Servicer a $560
Interest-Free Loan”</a>: they gave me my money back!</p>
<h2 id="timeline-recap"><a class=anchor href="#timeline-recap" aria-label="Link to section timeline-recap">#</a>Timeline recap</h2>
<p>Here’s the story I covered in my post last year:</p>
<ul>
<li>2024-04-08: I finish paying off my student loans.</li>
<li>2024-04-08: Aidvantage processes my monthly autopay despite my loans having a
balance of $0, taking an extra $560 from my bank account and leaving me with
a negative balance on my loan.</li>
<li>2024-04-15: I submit a support request asking for my money back.</li>
<li>2024-04-17: Aidvantage responds and says they’ll return the money in 8-12
weeks.</li>
<li>2024-06-24: A refund check is created 68 days after my request (I only find
out about this after the fact).</li>
<li>2024-07-10: I call Aidvantage to ask where my money is. The agent says that a
check was sent out and should have arrived. They do not have any proof that a
check was sent out. The agent offers to ACH the refund instead, and says it
should take 3-4 weeks.</li>
<li>2024-08-15: I call Aidvantage to ask where my money is. The agent says that
the ACH was never a possibility, because they have to wait for the check they
“sent out” to expire. I am informed it takes 14 months for a check to expire
and told to call back on August 25th, 2025.</li>
</ul>
<h2 id="new-developments"><a class=anchor href="#new-developments" aria-label="Link to section new-developments">#</a>New developments</h2>
<p>At some point after all these interactions, the negative balance on my loan
disappeared from the website. Luckily, this didn’t turn out to be a problem,
but I wasn’t sure I would be able to convince them that they owed me money if
it wasn’t visible in their database.</p>
<p>I recently received a letter from Aidvantage dated July 3rd, 2025 titled
“<strong>ACTION REQUIRED—YOU MAY BE OWED A REFUND</strong>”, asking me to verify my address
before a refund check could be created. The letter did not explain <em>how</em> much
of a refund I was owed. I called up Aidvantage, verified my address, and they
sent me a check. Thankfully, the check actually arrived this time.</p>
<p>On July 25th, 2025, the check for $560 cleared and my student loans are
finally, <em>truly</em> complete. If I’d been able to invest that money in the S&P 500
over that period, I would have $714 instead of $560. Unfortunately for me, if
you have someone’s routing and account numbers, you can just take $560 from
their account and hold on to it interest-free for a year. What are they going
to do, consult with a lawyer for $600 an hour?</p>
How to turn an iPhone's ringer on2025-08-15T00:00:00+00:002025-08-15T00:00:00+00:00
Rebecca Turner
https://becca.ooo/blog/how-to-turn-an-iphones-ringer-on/<h2 id="foreword-missed-calls"><a class=anchor href="#foreword-missed-calls" aria-label="Link to section foreword-missed-calls">#</a>Foreword: Missed calls</h2>
<p>Today I had a delivery scheduled for a piece of furniture, so I had my iPhone’s
ringer on. This is pretty unusual for me; my policy is that if you want to get
on the phone with me, you need to either tell me in advance when you’ll call or
be in my contacts. Otherwise, unknown numbers are silenced.</p>
<p>I don’t think this pattern of phone usage is that unusual. A <a href="https://www.pewresearch.org/internet/2025/07/31/online-scams-and-attacks-in-america-today/">2025 Pew
survey</a> survey found that 68% of adult Americans get scam calls at least
weekly and 31% of Americans get scam calls daily. Here’s a Redditor who
<a href="https://www.reddit.com/r/AskOldPeople/comments/1dhlytr/how_many_scam_callers_do_you_get_in_a_day/">received 47 scam calls in a single day</a>, and lots of the replies are
from people who have settled on my strategy as well:</p>
<blockquote>
<p>i do not have a landline and my cellphone only rings for numbers in my contact list.</p>
</blockquote>
<blockquote>
<p>If I know the caller, they’re on my contacts list. If they’re calling from an
unfamiliar number for some reason, they’ll leave a message and I’ll call
right back.</p>
</blockquote>
<blockquote>
<p>I don’t answer unless I know who’s calling.</p>
</blockquote>
<blockquote>
<p>I don’t answer my phone, so it’s not an issue.</p>
</blockquote>
<p>Sometimes, though, you <em>do</em> need to answer a phone call from a stranger. Maybe
you have a bulky delivery scheduled, like me. Perhaps you’re waiting on a call
from a potential employer. Or you could be waiting to hear the results of a
medical test. Something all of these situations have in common is that missing
the phone call has consequences.</p>
<p>So you turn your ringer on. How hard could it be?</p>
<h2 id="settings-that-control-iphone-call-notifications"><a class=anchor href="#settings-that-control-iphone-call-notifications" aria-label="Link to section settings-that-control-iphone-call-notifications">#</a>Settings that control iPhone call notifications</h2>
<p>I’d been burned before, so after I turned my ringer on, I went to Settings →
Apps → Phone and disabled Silence Unknown Callers. Do you know what happened
next? After work, I noticed a notification for a missed call. It turns out
there’s way more settings that can silence calls than I thought. Here’s a list
of as many as I can find. If you learn about more settings, please <a href="/contact">let me
know</a> and I can add them here.</p>
<p>Please note that this article does not cover turning an iPhone’s ringer <em>off</em>,
which irritatingly involves a slightly different list of settings. For example,
in the Contacts app you can add individual contacts to an “Emergency Bypass”
list which causes calls from those contacts to <em>always</em> ring audibly.</p>
<h3 id="the-physical-silence-switch"><a class=anchor href="#the-physical-silence-switch" aria-label="Link to section the-physical-silence-switch">#</a>The physical silence switch</h3>
<p>On the side of the phone is a physical switch to silence notifications and
calls (and also, like, sound effects in games? I don’t really know why it does
double-duty like this). Make sure that’s not set to “silent”.</p>
<h3 id="settings-sounds-haptics"><a class=anchor href="#settings-sounds-haptics" aria-label="Link to section settings-sounds-haptics">#</a>Settings → Sounds & Haptics</h3>
<p>Set the <strong>Ringtone and Alerts</strong> volume to the <strong>maximum</strong> (or whatever you
want). If vibration is important as well, set <strong>Haptics</strong> to <strong>Always Play</strong>.</p>
<p>If you have <strong>Change with Buttons</strong> enabled, you can use the physical volume up
button to change the ringtone volume. Make sure that’s not silent either!</p>
<p>If <strong>Change with Buttons</strong> is <em>not</em> enabled, then the physical volume buttons
will have <em>no effect</em> on the ringtone volume. Many users find this unintuitive,
to say the least.</p>
<p>I saw someone say to make sure you have a sound assigned to <strong>Ringtone</strong>, but I
couldn’t see a way to <em>not</em> have a sound assigned, so that might be a relic
from a previous iOS version.</p>
<h3 id="settings-apps-phone"><a class=anchor href="#settings-apps-phone" aria-label="Link to section settings-apps-phone">#</a>Settings → Apps → Phone</h3>
<p>Set <strong>Silence Unknown Callers</strong> to <strong>Off</strong>.</p>
<p>Under <strong>Call Blocking & Identification</strong>, set <strong>Silence Junk Callers</strong> to
<strong>Off</strong>.</p>
<p>Under <strong>Blocked Contacts</strong>, make sure the phone number you’re expecting a call
from (lol, lmao, they never tell you ahead of time) isn’t present.</p>
<p>Also, make sure that <strong>Call Forwarding</strong> is set to <strong>Off</strong>, although that
setting doesn’t exist on my phone.</p>
<p>I saw someone online say that they needed to set <strong>Announce Calls</strong> to
<strong>Always</strong>, but I think that setting just makes Siri read information about the
incoming caller out loud and it’s not actually related to the ringer (I tested
this).</p>
<h3 id="do-not-disturb"><a class=anchor href="#do-not-disturb" aria-label="Link to section do-not-disturb">#</a>Do Not Disturb</h3>
<p>Open the <strong>Control Center</strong> and make sure <strong>Do Not Disturb</strong> is not enabled.
You should see a gray button that says “Focus” with a gray moon icon next to
it. If <strong>Do Not Disturb</strong> is enabled, the button will say “Do Not Disturb” and
have a blue moon icon next to it.</p>
<p>If it’s in a different focus mode, you may want to disable that. Focus modes
can be customized under <strong>Settings → Focus</strong>:</p>
<ul>
<li>For a given focus mode, go to <strong>Notifications → Apps</strong> and make sure that if
it’s in “Silence Notifications From” mode that the “Phone” app is <em>not</em>
present, and if it’s in “Allow Notifications From” mode that the “Phone” app
<em>is</em> present.</li>
<li>Under <strong>Notifications → People</strong>, make sure that <strong>Allow Calls from Silenced
People</strong> is enabled.</li>
<li>Under <strong>Focus Filters</strong>, make sure that <strong>Silent Mode</strong> is either not present
or set to <strong>Turn Off</strong>.</li>
</ul>
<p>Note that focus modes (including Do Not Disturb) can be activated automatically
on a schedule, so make sure that no focus modes will be enabled while you’re
waiting for a call. In particular, the <strong>Sleep</strong> focus mode is automatically
activated if you have a sleep schedule set in the <strong>Clock</strong> app under the
<strong>Alarms</strong> tab. The <strong>Sleep</strong> focus mode schedule will show up in the settings
like the other focus modes, though, so you don’t need to go to the <strong>Clock</strong>
app specifically to turn it off.</p>
<h3 id="settings-screen-time-communication-limits"><a class=anchor href="#settings-screen-time-communication-limits" aria-label="Link to section settings-screen-time-communication-limits">#</a>Settings → Screen Time → Communication Limits</h3>
<p>Set <strong>During Screen Time</strong> to <strong>Everyone</strong>. Set <strong>During Downtime</strong> to <strong>Everyone</strong>.</p>
<h3 id="settings-face-id-passcode-attention-aware-features"><a class=anchor href="#settings-face-id-passcode-attention-aware-features" aria-label="Link to section settings-face-id-passcode-attention-aware-features">#</a>Settings → Face ID & Passcode → Attention Aware Features</h3>
<p>Make sure that <strong>Attention Aware Features</strong> is <strong>disabled</strong>. If these are
enabled, your iPhone may silence calls if it notices you looking at it (!!!!!).</p>
<h3 id="settings-accessibility-touch-call-audio-routing"><a class=anchor href="#settings-accessibility-touch-call-audio-routing" aria-label="Link to section settings-accessibility-touch-call-audio-routing">#</a>Settings → Accessibility → Touch → Call Audio Routing</h3>
<p>Under the <strong>Touch</strong> section, for some fucking reason, you can set <strong>Call Audio
Routing</strong> to be “Automatic”, “Bluetooth Headset”, or “Speaker”. If you will not
hear your phone’s speaker, or if you will not hear a connected Bluetooth
headset, pick the other option.</p>
<p>Also under <strong>Call Audio Routing</strong>, set <strong>Auto-Answer Calls</strong> to <strong>Off</strong>. (Why
is this under “Call Audio Routing”??)</p>
<h3 id="settings-accessibility-hearing-devices"><a class=anchor href="#settings-accessibility-hearing-devices" aria-label="Link to section settings-accessibility-hearing-devices">#</a>Settings → Accessibility → Hearing Devices</h3>
<p>I think there’s some relevant settings for <strong>Hearing Devices</strong>, but I don’t
have any of those, so I can’t help you here. Here’s <a href="https://support.apple.com/guide/iphone/use-hearing-devices-iph470b1833/ios">a page on using hearing
devices from the iPhone User Guide.</a></p>
<h3 id="headset-mode"><a class=anchor href="#headset-mode" aria-label="Link to section headset-mode">#</a>Headset mode</h3>
<p>Maybe some phones can get stuck in “headphone mode” or “headset mode”? Unclear.
In the <strong>Control Center</strong>, tap the <strong>AirPlay icon</strong> and make sure you’re not
playing audio to headphones unless you want to be doing that.</p>
<h3 id="notification-settings-on-your-mac"><a class=anchor href="#notification-settings-on-your-mac" aria-label="Link to section notification-settings-on-your-mac">#</a>Notification settings on your Mac</h3>
<p>In the Settings app on your Mac, under <strong>Focus</strong>, if you have <strong>Share across
devices</strong> enabled (NB: setting names are lowercase on macOS?), then enabling
(e.g.) Do Not Disturb on your Mac will also enable it on your phone.</p>
<h2 id="afterword-build-predictable-systems"><a class=anchor href="#afterword-build-predictable-systems" aria-label="Link to section afterword-build-predictable-systems">#</a>Afterword: Build predictable systems</h2>
<p>I could say a lot more about this, but I need to feed my cats, so I’ll keep it
brief. One of the biggest problems with the status quo here is that it’s not
easy to <strong>predict</strong> if a given call will be silenced or not. You’ve disabled
“Silence Unknown Callers”. Will your phone ring when the deliver person calls?
Until it happens, there’s no way to know. You should be able to type in a phone
number and have your phone tell you if it will ring or not. Nobody should ever
be <em>surprised</em> by the behavior of their phone’s <em>ringer</em>.</p>
<p>Also, give me a big button that says “if the ringer does not go off for my next
phone call I will do something drastic and unreasonable.” It could even be one
of those fancy new Control Center widgets they added in iOS 18. Call me, Apple,
let’s link and build this.</p>
macOS dotfiles should not go in ~/Library/Application Support2025-03-17T00:00:00+00:002025-03-17T00:00:00+00:00
Rebecca Turner
https://becca.ooo/blog/macos-dotfiles/<p>One of my pet peeves is when command-line tools look for user configuration
files in <code>~/Library/Application Support</code> when running on macOS. In addition to
offering poor ergonomics for users, I believe this behavior is <em>incorrect</em>
according to the documentation which is cited to justify it. Instead,
command-line tools should implement the <a href="https://specifications.freedesktop.org/basedir-spec/latest/">XDG Base Directory Specification</a>
and look for configuration files in <code>$XDG_CONFIG_HOME</code>, which defaults to
<code>~/.config</code>.</p>
<p>Usually, when a program looks for configuration files in <code>~/Library/Application Support</code>, it’s not because of an intentional design decision. Instead, the
author delegated that decision to a library, and several popular libraries for
determining platform-specific configuration directories use
<code>~/Library/Application Support</code> as the default configuration directory on
macOS, including <a href="https://pypi.org/project/platformdirs/">Python’s <code>platformdirs</code> package</a> (<a href="https://pypistats.org/packages/platformdirs">242
million downloads / month</a>), <a href="https://www.npmjs.com/package/env-paths">JavaScript’s
<code>env-paths</code></a> (95 million downloads / month), <a href="https://lib.rs/crates/dirs">Rust’s <code>dirs</code>
crate</a> (4.8 million downloads / month), and <a href="https://pkg.go.dev/github.com/adrg/xdg">Go’s <code>adrg/xdg</code>
package</a> (used in 913 packages). While <code>~/Library/Application Support</code>
may be the correct configuration directory for <em>GUI apps</em> which manage their
configuration files for the user automatically, the vast majority of
applications using these libraries are command-line utilities whose
configuration belongs in <code>~/.config</code>.</p>
<p>The main reason I dislike programs looking for dotfiles in
<code>~/Library/Application Support</code> is that it’s <em>unexpected.</em> As <a href="https://github.com/nushell/nushell/issues/893#issuecomment-1736736555">Ashlin Eldridge
explained on GitHub</a>:</p>
<blockquote>
<p>As a new user, it’s extremely surprising to me to that a modern tool like
<code>nu</code> would put config files under Application Support on macOS. It’s even
more surprising to see that some people are
<a href="https://github.com/nushell/nushell/discussions/5279">actively</a>
<a href="https://news.ycombinator.com/item?id=33425312">against</a> adopting the XDG
standard.</p>
<p>The origins of XDG no longer matter at this point. No one really cares what
the X, D, and G stand for. The fact is that
<a href="https://wiki.archlinux.jp/index.php/XDG_Base_Directory">hundreds</a> of tools
support it (including Git, Emacs, Neovim, Tmux) and users have come to expect
support for it. It’s the closest thing we have to a standard that allows
users to control the placement of their config files (which facilitates
users’ ability to source control their files) and it also includes support
for files that shouldn’t be source controlled such as cache and data files.</p>
</blockquote>
<p>If a user explains that they found a program’s behavior surprising, many
engineers will respond by telling the user that their expectations were wrong.
However, I find that the reason users form an incorrect mental model of a
program’s behavior is usually because the program’s design violated an
established convention from some broader context. In accordance with <a href="https://en.wikipedia.org/wiki/Principle_of_least_astonishment">the
Principle of Least Astonishment</a>, I prefer to change the program to
behave how users expect rather than futilely attempt to change what users
expect.</p>
<p>Users expect macOS CLI tools to look for configuration files in <code>~/.config</code> is
because <strong>that’s what almost every other program does.</strong> Your program should
probably not disagree with Bash and Vim and Git about where configuration files
are supposed to go!</p>
<p>These conventions help us explore unfamiliar systems by letting us apply
knowledge about one part of the system to other parts of the system.
Consistency is predictability. When every tool works the same except for a
small handful, users get justifiably annoyed at the exceptions for wasting
their time.</p>
<h2 id="what-do-dotfile-managers-do"><a class=anchor href="#what-do-dotfile-managers-do" aria-label="Link to section what-do-dotfile-managers-do">#</a>What do dotfile managers do?</h2>
<p>Many engineers use (or maintain) some sort of dotfile manager to handle
tracking these configuration files. If configuration files are really supposed
to go in <code>~/Library/Application Support</code>, we would expect dotfile managers to
put configuration files there by default when running on macOS. After all, the
whole point of a dotfile manager is to simplify managing configuration files
across multiple machines and platforms. Let’s take a quick survey to see how
some popular dotfile managers behave on macOS:</p>
<ul>
<li><a href="https://www.chezmoi.io/">chezmoi</a> (14.3k stars) makes no effort to link
configuration to <code>~/Library/Application Support</code>, despite including
<a href="https://www.chezmoi.io/user-guide/machines/macos/">documentation for using chezmoi on macOS
specifically</a>.</li>
<li><a href="https://github.com/anishathalye/dotbot">dotbot</a> (7.3k stars) ignores
<code>~/Library/Application Support</code>. Despite including macOS-specific
configuration files in the examples, the README shows a configuration for VS
Code that will be completely ignored on macOS.</li>
<li><a href="https://github.com/yadm-dev/yadm">yadm</a> (5.5k stars) makes no effort to
support <code>~/Library/Application Support</code>. The example configurations include
macOS-specific portions but also do not reference <code>~/Library/Application Support</code>.</li>
<li><a href="https://github.com/thoughtbot/rcm">rcm</a> (3.2k stars) makes no effort to
support <code>~/Library/Application Support</code>.</li>
<li><a href="https://www.gnu.org/software/stow/">GNU Stow</a> makes (unsurprisingly) no
effort to support <code>~/Library/Application Support</code>.</li>
</ul>
<p>Most of these tools are <em>able</em> to link configuration files to
<code>~/Library/Application Support</code> if you ask them to explicitly (although macOS
will regularly replace your symlinks with copies of the destination files), but
the fact that they don’t bother to do so by default is revealing. Users only
want this behavior for specific misbehaving programs.</p>
<h2 id="an-appeal-to-logic"><a class=anchor href="#an-appeal-to-logic" aria-label="Link to section an-appeal-to-logic">#</a>An appeal to logic</h2>
<p>Perhaps “consistency” isn’t a good enough reason for you to place your
program’s configuration files in <code>~/.config</code> like everybody else. Maybe you
think that programs should follow the platform guidelines, even if they don’t
match developer expectations. <a href="https://github.com/dirs-dev/directories-rs/issues/47#issuecomment-478337412">@soc on GitHub</a> phrases it like this:</p>
<blockquote>
<p>As a general advice for macOS devs: If you are developing on their platform,
do what the platform owners tell you. They are the lords, you are the
share-croppers.</p>
</blockquote>
<p>Elsewhere, <a href="https://github.com/nushell/nushell/issues/12103#issuecomment-1982144513">Reilly Wood rebukes</a> a user asking if there would “be any
downside to just not using <code>Application Support</code> ever” by saying that “[Nushell
would] no longer be following the <a href="https://developer.apple.com/library/archive/documentation/FileManagement/Conceptual/FileSystemProgrammingGuide/FileSystemOverview/FileSystemOverview.html#//apple_ref/doc/uid/TP40010672-CH2-SW6">macOS Standard Directories
guidelines</a>.”</p>
<p>We’ll read through those guidelines in a minute, but it’s not entirely clear to
me that they’re relevant in the first place. The <a href="https://specifications.freedesktop.org/basedir-spec/latest/">XDG Base Directory
Specification</a> mentions Unix a few times but lists no carveouts for macOS
or any other operating system. If <code>~/.config</code> is accepted as the standard
location for configuration files on Unix-like operating systems, then surely it
would be the standard location on macOS as well, given that macOS is a Unix by
way of BSD.</p>
<p>But suppose we accept that the XDG specification only applies to <em>some</em> Unix
operating systems, despite making no mention of this. The <a href="https://developer.apple.com/library/archive/documentation/FileManagement/Conceptual/FileSystemProgrammingGuide/FileSystemOverview/FileSystemOverview.html#//apple_ref/doc/uid/TP40010672-CH2-SW6">macOS Standard
Directories</a> documentation starts by stating that
“[w]hether provided by the system or created by your app, every file has its
place in macOS”, and honestly we could stop reading right there, because <strong>a
command-line tool is not the system or an app.</strong></p>
<p>While this might seem like a pedantic nitpick, in reality it’s <em>critical</em> to
correctly interpreting this documentation! I suspect that many of the
developers who cite these guidelines do not know very much about macOS and
think that “app” is just what macOS calls executables, but that’s not true. A
couple screens down, the same page says that <code>/bin</code> “[c]ontains essential
command-line binaries”, so we can be sure the authors aren’t conflating apps
and command-line tools. Apps on macOS are installed in <code>/Applications</code> and are
subject to <a href="https://developer.apple.com/documentation/xcode/preparing-your-app-for-distribution">a number of additional requirements</a>, from a bundle ID
(a unique reverse-DNS identifier for an app, like <code>com.apple.Preview</code> for
<code>Preview.app</code>) to application icons, launch screens, code signing,
notarization, app sandboxing, and runtime hardening. Needless to say, none of
the developers shipping command-line utilities are following any of <em>these</em>
guidelines.</p>
<p>Moving on, <a href="https://developer.apple.com/library/archive/documentation/FileManagement/Conceptual/FileSystemProgrammingGuide/MacOSXDirectories/MacOSXDirectories.html#//apple_ref/doc/uid/TP40010672-CH10-SW1">the documentation for <code>~/Library/Application Support</code></a> says that it:</p>
<blockquote>
<p>Contains all app-specific data and support files. These are the files that
your app creates and manages on behalf of the user and can include files that
contain user data.</p>
</blockquote>
<p>Again, command-line tools do not have <em>any</em> app-specific data because they are
not apps. Further, dotfiles are usually written by hand and not managed <em>on
behalf of</em> the user.</p>
<p>The next paragraph makes it very clear that <code>~/Library/Application Support</code> is
for apps only:</p>
<blockquote>
<p>By convention, all of these items should be put in a subdirectory whose name
matches the bundle identifier of the app. For example, if your app is named
<code>MyApp</code> and has the bundle identifier <code>com.example.MyApp</code>, you would put your
app’s user-specific data files and resources in the <code>~/Library/Application Support/com.example.MyApp/</code> directory. Your app is responsible for creating
this directory as needed.</p>
</blockquote>
<p>Command-line tools do not have a bundle identifier, including many command-line
tools shipped with macOS by Apple (such as <code>ls</code> or <code>vim</code>). And even though
<a href="https://git.lix.systems/lix-project/lix/issues/515">Apple feels very comfortable shipping command-line tools that behave in a
subtly different manner from their equivalents on other modern Unix-descendent
operating systems,</a> the macOS variants of <code>bash</code>, <code>zsh</code>, <code>git</code>, and
<code>vim</code> all look for their configuration files in the same place: <code>~/.config</code>.
Unless your tool is installed in <code>/Applications</code>, there’s no reason for its
configuration to live in <code>~/Library/Application Support</code>.</p>
<h2 id="when-should-you-use-library-application-support"><a class=anchor href="#when-should-you-use-library-application-support" aria-label="Link to section when-should-you-use-library-application-support">#</a>When <em>should</em> you use <code>~/Library/Application Support</code></h2>
<p>With the guidelines and conventions listed above in mind, your application
should store its configuration files in <code>~/Library/Application Support</code> instead
of <code>$XDG_CONFIG_HOME</code> if both of the following conditions apply:</p>
<ol>
<li>It is a GUI application installed in <code>/Applications</code> or <code>~/Applications</code>.</li>
<li>It manages its configuration files automatically on behalf of the user,
rather than expecting the user to write their configuration in a text file
themself.</li>
</ol>
<h2 id="tl-dr"><a class=anchor href="#tl-dr" aria-label="Link to section tl-dr">#</a>TL;DR</h2>
<p>Users do not expect command-line tools to look for configuration files in
<code>~/Library/Application Support</code> on macOS. Dotfile managers do not place
configuration files in <code>~/Library/Application Support</code> on macOS. The most
commonly-cited justifications for placing configuration files in
<code>~/Library/Application Support</code> are not meant to apply to command-line tools at
all. Even command-line tools like <code>bash</code> and <code>git</code> shipped by Apple look for
their configuration files in <code>~/.config</code>.</p>
<p>Please, please, <em>please</em> just use the <a href="https://specifications.freedesktop.org/basedir-spec/latest/">XDG Base Directory Specification</a>.</p>
Global State in Haskell with IORef2025-01-02T00:00:00+00:002025-01-02T00:00:00+00:00
Rebecca Turner
https://becca.ooo/blog/global-state-in-haskell-with-ioref/<p>AKA “The <a href="https://hackage.haskell.org/package/base/docs/Data-IORef.html"><code>IORef</code></a> trick”.</p>
<p>In Haskell, functions are pure and there’s no global state. But sometimes
global state is convenient! If you want to add some extra bit of configuration
that you would need to thread through some large portion of code in a fairly
mechanical fashion (for example, “are we running in verbose mode” or “is this a
dry run”) and your program is written in <code>IO</code> or some other context that’s not
very amenable to being extended with additional data, or if you’re prototyping
a solution before implementing it “the right way”, then a little bit of mutable
global state can be a very useful thing indeed. To be clear, I’m not saying you
should add global state to your programs. In fact, I will caution you against
it: this is the sort of thing that gets your PRs rejected.</p>
<p>This technique, which I will call “the <a href="https://hackage.haskell.org/package/base/docs/Data-IORef.html"><code>IORef</code></a> trick”, allows us to
implement a global variable that can be read or written in any <code>IO</code> action. I
believe the <code>IORef</code> trick is fairly well-known, but not talked about much — I
couldn’t find it documented anywhere when I needed it (and of course I didn’t
<em>really</em> need it). I suspect that this is because your coworkers are worried
you will start littering global mutable state everywhere you go, if only you
knew how, and they would rather you do things “the right way” and. Admittedly,
they’re probably correct, so be responsible with this technique!</p>
<h2 id="the-ioref-trick"><a class=anchor href="#the-ioref-trick" aria-label="Link to section the-ioref-trick">#</a>The <code>IORef</code> trick</h2>
<p>Here’s how the trick works:</p>
<pre data-lang="haskell" class="language-haskell "><code class="language-haskell" data-lang="haskell">module MyParameter (getMyParameter, setMyParameter) where
import Data.IORef (IORef, atomicModifyIORef', newIORef)
import System.IO.Unsafe (unsafePerformIO)
{-# NOINLINE myParameterRef #-}
myParameterRef :: IORef Int
myParameterRef = unsafePerformIO (newIORef 0)
setMyParameter :: Int -> IO ()
setMyParameter newValue =
atomicModifyIORef' myParameterRef (\_ -> (newValue, ()))
getMyParameter :: IO Int
getMyParameter =
atomicModifyIORef' myParameterRef (\value -> (value, value))
</code></pre>
<p>First, we create a new <a href="https://hackage.haskell.org/package/base/docs/Data-IORef.html"><code>IORef</code></a> called <code>myParameterRef</code> to store our
global variable and initialize it with its default value. This is where the
naughtiness lives: we have to use <a href="https://hackage.haskell.org/package/base/docs/System-IO-Unsafe.html#v:unsafePerformIO"><code>unsafePerformIO</code></a> to
create the <code>IORef</code>. It’s very important to mark this value as
<a href="https://www.haskellforall.com/2024/11/the-haskell-inlining-and-specialization.html#noinline"><code>NOINLINE</code></a> or GHC may end up running <code>newIORef</code> multiple times,
causing you an enormous headache. If your global variable has no sensible
default value, you may want to adapt this code to work with an <a href="https://hackage.haskell.org/package/base/docs/Control-Concurrent-MVar.html"><code>MVar</code></a>
instead.</p>
<p>Next, we write a setter for our variable using <code>atomicModifyIORef'</code>. Evaluating
<code>atomicModifyIORef' ref (\old -> (new, result))</code> will set <code>ref</code> to <code>new</code> and
return <code>result</code>. Note that the supplied lambda has access to the previous value
as well. (This is explained in the documentation for <code>atomicModifyIORef</code>, but
only in the <em>seventh</em> paragraph, so I thought it might be worth calling
attention to.)</p>
<p>We could simplify <code>setMyParameter</code> by writing it in terms of <code>atomicWriteIORef</code>,
but there’s no corresponding <code>atomicReadIORef</code> function so we’ll need to import
<code>atomicModifyIORef'</code> to write the getter.</p>
<p>Finally, we write a getter in terms of <code>atomicModifyIORef'</code>, which re-stores
the old value in the <code>IORef</code> again and then returns it.</p>
<h2 id="strictness"><a class=anchor href="#strictness" aria-label="Link to section strictness">#</a>Strictness</h2>
<p>The single quote at the end of <code>atomicModifyIORef'</code> means (in this context)
that it’s the strict version of <code>atomicModifyIORef</code>. (You may hear this symbol
referred to as a tick mark or a <a href="https://en.wikipedia.org/wiki/Prime_(symbol)">prime</a>.) Strictness here means
that running <code>setMyParameter newValue</code> will evaluate <code>newValue</code> to <a href="https://stackoverflow.com/a/6889335">weak head
normal form</a>. One alternative is to leave the value unevaluated, which
may cause <a href="https://stackoverflow.com/a/5892699">lazy IO</a> problems, <a href="https://stackoverflow.com/a/61670171">memory leaks</a>, or <a href="http://blog.ezyang.com/2011/05/anatomy-of-a-thunk-leak/">thunk
buildup</a>. Another alternative is to use <a href="https://hackage.haskell.org/package/deepseq/docs/Control-DeepSeq.html"><code>deepseq</code></a> to
fully force the value. I think that <code>atomicModifyIORef'</code> is a good default but
the correct choice will depend on the semantics of your particular application.</p>
<h2 id="thread-safety"><a class=anchor href="#thread-safety" aria-label="Link to section thread-safety">#</a>Thread safety</h2>
<p>The “atomic” in <code>atomicModifyIORef'</code> means that a <a href="https://stackoverflow.com/a/286705">memory fence</a>
is added when executing this function, which will make calls to the getter and
setter return consistent values even with <a href="https://mortoray.com/cpu-reordering-what-is-actually-being-reordered/">instruction
reordering</a> on multiple threads at the cost of some
performance. If you don’t require thread safety, you can rewrite the getter and
setter in terms of <code>readIORef</code> and <code>writeIORef</code>.</p>
Reading git range-diff output2024-12-11T00:00:00+00:002024-12-11T00:00:00+00:00
Rebecca Turner
https://becca.ooo/blog/git-range-diff/<p>The <a href="https://git-scm.com/docs/git-range-diff"><code>git range-diff</code></a> command compares two commit ranges. You
can use this to check how a branch changed before and after a rebase, or to
find differences between a commit on your <code>main</code> branch and a corresponding
backport on a long-term-support branch.</p>
<p>If you’ve ever tried to use <code>git diff</code> to figure out the changes introduced by
resolving merge conflicts, you might understand the pain point this helps
address — <code>git diff</code> shows you <em>all</em> the changes between two commits, so you
get all the changes introduced by the merge as well as all the conflict
resolutions. <code>git range-diff</code> factors these changes out and makes it a lot
easier to see how a branch changed over time. (It is an endless pet peeve of
mine that <a href="https://mitchellh.com/writing/github-changesets">GitHub does not give any visibility into how a PR changes when you
push commits to it</a>. As a result, many developers who have
only used version control through GitHub have no idea that this sort of tool is
even possible, despite being trivial to construct.)</p>
<p>I’ve started to use <code>git range-diff</code> more lately, but I’ve been somewhat
baffled by the output format, which the man page considers unworthy of
explanation. Last night I read some of the Git source code to figure out how
<code>git range-diff</code> works — here’s what I found.</p>
<h2 id="diffs-of-diffs"><a class=anchor href="#diffs-of-diffs" aria-label="Link to section diffs-of-diffs">#</a>Diffs of diffs</h2>
<p><strong><code>git range-diff</code> runs two <code>git diff</code> operations and then <code>diff</code>s the
results.</strong> The output syntax is not very special — it’s <em>literally</em> just a
diff of diffs. The first column of diff markers shows you how the diffs
changed, and the second column shows you the diff itself.</p>
<p>Let’s walk through a set of changes to learn how to read diffs of diffs.
Suppose we have a shell script which finds YAML files:</p>
<pre data-lang="bash" class="language-bash "><code class="language-bash" data-lang="bash">find \
-path '*.yaml'
</code></pre>
<p>You notice that it’s not picking up files with the <code>.yml</code> extension, so you add
another clause:</p>
<pre data-lang="bash" class="language-bash "><code class="language-bash" data-lang="bash">find \
-path '*.yaml' \
-o -path '*.yml'
</code></pre>
<p>Meanwhile, your colleague runs <a href="https://www.shellcheck.net/">ShellCheck</a> on the script, notices
that <a href="https://www.shellcheck.net/wiki/SC2185">some implementations of <code>find</code> have no default path</a>
and adds one to fix the script on macOS and other BSDs:</p>
<pre data-lang="bash" class="language-bash "><code class="language-bash" data-lang="bash">find \
. \
-path '*.yaml'
</code></pre>
<p>You merge in your colleague’s changes, producing this result:</p>
<pre data-lang="bash" class="language-bash "><code class="language-bash" data-lang="bash">find \
. \
-path '*.yaml' \
-o -path '*.yml'
</code></pre>
<p>Then, you run <code>git range-diff main BEFORE_MERGE AFTER_MERGE</code>, perhaps using
<a href="https://git-scm.com/docs/git-reflog"><code>git reflog</code></a> to find the commit hashes before and after merging
in the changes from <code>main</code>. Here’s the diff output:</p>
<pre data-lang="diff" class="language-diff "><code class="language-diff" data-lang="diff"> ## find-yaml.sh ##
@@
- #!/usr/bin/env bash
find \
+ . \
- -path '*.yaml'
+ -path '*.yaml' \
+ -o -path '*.yml'
</code></pre>
<p>First, note the <code>- #!/usr/bin/env bash</code> line. Due to the extra argument to
<code>find</code> added in <code>main</code>, the changes in the <code>AFTER_MERGE</code> commit start one line
later in the file, so the diff’s <em>context</em> starts one line later as well. This
shows up as the context line being <em>removed</em>, even though it wasn’t changed on
either side!</p>
<p>This is a rather confusing result of <code>git range-diff</code>’s extremely naïve
implementation — it makes no effort to distinguish between context lines and
actual meaningful elements of the diff. This is the sort of obvious UX pitfall
that made me assume that <em>surely</em> <code>git range-diff</code> would do something
<em>slightly</em> smarter than <em>literally</em> diffing two diffs, but I don’t know why I
expected that. <a href="https://stevelosh.com/blog/2013/04/git-koans/">It’s Git, after all.</a></p>
<p>Similarly, the next <code>+</code> line shows us that the <code>.</code> argument is present in the
<code>AFTER_MERGE</code> commit, but that it wasn’t added by the <code>AFTER_MERGE</code> commit.
Unlike <code>git diff</code>, which shows you all changes between two commits, <code>git range-diff</code> will only show you changes from previous commits when they’re
present as context in the diffs of the commits you actually care about.</p>
<p>The rest of the changes are present in both versions of the diff.</p>
<p>To help make it easier to distinguish between a diff marker in the first column
and one in the second column, <code>git range-diff</code> applies <a href="https://git-scm.com/docs/git-range-diff#Documentation/git-range-diff.txt---no-dual-color">“dual
coloring”</a>, which colors the background (instead of the text) of
the first column of diff markers.</p>
<p>Suppose you want to make your intent clearer, so you go and add a comment to
explain your change:</p>
<pre data-lang="bash" class="language-bash "><code class="language-bash" data-lang="bash"># Make sure to find `.yaml` and `.yml` files!
find \
. \
-path '*.yaml' \
-o -path '*.yml'
</code></pre>
<p>You amend your commit and run the range-diff again. Now, we see this line in the output:</p>
<pre data-lang="diff" class="language-diff "><code class="language-diff" data-lang="diff">++# Make sure to find `.yaml` and `.yml` files!
</code></pre>
<p>This is showing us that we’ve added a line in the <code>AFTER_MERGE</code> commit only. If
you’re comparing a commit before and after fixing merge conflicts, this could
indicate a new change that was added unintentionally, so pay attention to these!</p>
<p>Exercise for the reader: What do <code>--</code>, <code>-+</code>, and <code>+-</code> lines indicate?</p>
<h2 id="diffs-of-logs"><a class=anchor href="#diffs-of-logs" aria-label="Link to section diffs-of-logs">#</a>Diffs of logs</h2>
<p><code>git diff</code> only deals with the contents of files, but <code>git range-diff</code> deals
with ranges of commits, so those need to be compared as well!</p>
<p>Here’s how Git shows a series of commits being compared:</p>
<pre><code> 1: d17004e2f4 = 1: 92954fddce rts: Tighten up invariants of PACK
2: 42f1801df4 < -: ---------- testsuite: Fix badly escaped literals
-: ---------- > 2: cdfd86e951 testsuite: Fix badly escaped literals
3: 4d7afaaa7f = 3: 2dbf88daed rts/Interpreter: Improve documentation of TEST*_P instructions
4: 5f7c2d3e99 ! 4: e646db18c4 rts: Annotate BCOs with their Name
</code></pre>
<p>From left to right, we have:</p>
<ol>
<li>
<p>The position of the left-hand side’s commit in the sequence being compared.
For example, <code>1:</code> indicates this is the first commit in the range of commits
on the LHS, and <code>-:</code> indicates that no corresponding commit was found on the
left-hand side.</p>
</li>
<li>
<p>The (abbreviated) commit hash of the LHS commit, like <code>d17004e2f4</code>.</p>
</li>
<li>
<p>A marker indicating if the commits are identical (<code>=</code>), only present on the
LHS (<code><</code>), only present on the RHS (<code>></code>), or different (<code>!</code>).</p>
</li>
<li>
<p>The position of the RHS’s commit in the sequence being compared.</p>
</li>
<li>
<p>The (abbreviated) commit hash of the RHS commit, like <code>92954fddce</code>. Note
that this can be different than the LHS commit hash even if the diffs are
identical, because the parent commits can be different.</p>
</li>
<li>
<p>The subject line of the commit message. If the message differs between the
LHS and RHS, the RHS’s commit message is used here and a diff between the
two commit messages is shown.</p>
</li>
</ol>
<p>Note that in the middle of the log, we have two commits that Git can’t find a
match for in the other side of the sequence:</p>
<pre><code> 2: 42f1801df4 < -: ---------- testsuite: Fix badly escaped literals
-: ---------- > 2: cdfd86e951 testsuite: Fix badly escaped literals
</code></pre>
<p>Similar to <a href="https://git-scm.com/docs/git-diff#Documentation/git-diff.txt---find-renamesltngt"><code>git diff</code>’s rename detection</a>, <code>git range-diff</code>
sets a threshold beyond which large changes between diffs will be considered
entirely different commits. We can adjust these costs by setting
<code>--creation-factor=90</code> (a percentage where higher numbers are more forgiving of
large changes) to force the comparison of the two diffs to be shown.</p>
<h2 id="parting-thoughts"><a class=anchor href="#parting-thoughts" aria-label="Link to section parting-thoughts">#</a>Parting thoughts</h2>
<p>I felt pretty silly once I realized that the <code>git range-diff</code> output is
<em>literally</em> just a diff of diffs. In fact, when reading the Git source code to
figure out how it was formatting the output, <a href="https://jade.fyi/">Jade</a> discovered that <a href="https://github.com/git/git/blob/caacdb5dfd60540ecec30ec479f147f3c8167e11/range-diff.c#L52-L66"><code>git range-diff</code> actually runs <code>git log --patch</code></a> to get
the diffs it feeds into the meta-diff. Perhaps the authors assumed the
implementation was so obvious that the output needed no explanation!</p>
<p>Even if you don’t know quite how the <code>git range-diff</code> output works, it’s still
a useful tool for comparing ranges because it filters out a lot of the noise,
and if you know the before/after of both ranges it’s usually pretty easy to
reconstruct the context.</p>
<p>If you haven’t used <code>git range-diff</code> before, give it a try! I’ve been finding
it very handy to have in my toolkit lately and wanted to help make it more
accessible for more people. Happy rebasing!</p>
Bisecting GHC is easier than I thought2024-12-05T00:00:00+00:002024-12-05T00:00:00+00:00
Rebecca Turner
https://becca.ooo/blog/bisecting-ghc/<h2 id="introduction"><a class=anchor href="#introduction" aria-label="Link to section introduction">#</a>Introduction</h2>
<p>At work, I’m busy upgrading our compiler from <a href="https://www.haskell.org/ghc/" title="GHC, the Glasgow Haskell Compiler">GHC</a> 9.6 to GHC 9.10. A
couple days ago I found some failing tests which eventually turned out to be <a href="https://gitlab.haskell.org/ghc/ghc/-/issues/25529">a
compiler bug</a>! Simon Peyton Jones took the time to reply with <a href="https://gitlab.haskell.org/ghc/ghc/-/issues/25529#note_597713" title="Simon Peyton Jones left a detailed comment explaining the bug">a very
detailed comment</a> explaining how GHC’s constraint solving logic
led to the bug, so be sure to check that out if you’re curious about how the
bug works.</p>
<p>To find the commit that introduced the bug I used <a href="https://git-scm.com/docs/git-bisect"><code>git bisect</code></a> to
perform a binary search over the commits. The principle behind <code>git bisect</code> is
simple, but building a compiler is a thorny problem in practice.</p>
<p><strong>It turns out that building GHC only takes about 12 minutes.</strong> This means that
you can search through 1000 commits in about two hours or one million commits
in four hours, which is much faster feedback than I assumed before I started!
It happens that there’s just about 1000 commits between GHC 9.6 and 9.8, so I
was able to get this done in about a day, including the time it took me to
write and iron out the bisection script. (Note that the 12 minute figure is
for a <a href="https://en.wikipedia.org/wiki/Bootstrapping_(compilers)">stage 1</a> build; it’s probably not optimized but it was
good enough to find my bug! I also tried building GHC on my laptop and it took
19 minutes, so your mileage may vary.)</p>
<p>Here’s how I performed the bisection, from start to finish. This is a wandering
account featuring a lot of errors that are more-or-less irrelevant to the final
product, so feel free to <a href="https://becca.ooo/blog/bisecting-ghc/#putting-it-all-together">skip to the end if you want to see the finished
bisection test script</a>. However, if you’re planning
on performing a similar bisection yourself, you may find the errors and the
methods I used to work around them helpful.</p>
<h2 id="key-takeaways"><a class=anchor href="#key-takeaways" aria-label="Link to section key-takeaways">#</a>Key takeaways</h2>
<p>Build systems and dependencies for compilers are constantly changing. Bisecting
a large range of commits means you have to be able to build a large range of
commits. Choosing a set of tools that can build every commit in the range is
not always possible.</p>
<p>Relatedly, the GHC build system is <em>very</em> sensitive to the particular set of
tools in use. Attempting to build and run GHC with a boot GHC from
<a href="https://nixos.org/manual/nixpkgs/stable/#preface">Nixpkgs</a>, I found a wide variety of different errors depending on
which revision of Nixpkgs I used.</p>
<p>Building old revisions of software with old versions of tools can be quite
frustrating. You will encounter bugs that have been fixed in newer revisions of
the software, or revisions of the software which only work with old versions of
the tools. You may have to cherry-pick or conditionally apply patches to work
around these issues.</p>
<p>Tools like Nixpkgs are indispensable for large-scale integration
work like this. Being able to easily build, patch, and cache a wide range of
versions of software makes it so much easier to find a set of tools that can
build the relevant versions of GHC.</p>
<h2 id="what-to-bisect"><a class=anchor href="#what-to-bisect" aria-label="Link to section what-to-bisect">#</a>What to bisect?</h2>
<p>First, we want to determine the range of commits to bisect on. I used
<a href="https://www.haskell.org/ghcup/">ghcup</a> to download different versions of GHC and run my reproducer with
them, which allowed me to determine that the error was introduced somewhere
between GHC 9.6 and GHC 9.8. I checked out <a href="https://gitlab.haskell.org/ghc/ghc">the GHC repository</a> and
listed the tags, eventually determining that I wanted the range of commits from
the <code>ghc-9.7-start</code> tag to the <code>ghc-9.8.1-release</code> tag. (Note that there is no
publicly released GHC 9.7, so this is rather unintuitive unless you already
follow the GHC development process!)</p>
<h2 id="submodules-to-whatever-the-opposite-of-a-rescue-is"><a class=anchor href="#submodules-to-whatever-the-opposite-of-a-rescue-is" aria-label="Link to section submodules-to-whatever-the-opposite-of-a-rescue-is">#</a>Submodules to whatever the opposite of a rescue is</h2>
<p>If you’ve touched the GHC repository at all, you might have modified files,
untracked files which exist in the release you want to check out, or — worst
of all — modified submodules. To clean out your working tree, you need to run
the following commands:</p>
<pre data-lang="bash" class="language-bash "><code class="language-bash" data-lang="bash"># Remove untracked files:
# -d is for "recurse into directories", -x is for "remove ignored files".
git clean --force -dx
# Undo changes to tracked files:
git reset --hard
# Unregister submodules and remove their worktrees:
git submodule deinit --all --force
</code></pre>
<p>The submodules make working in the GHC repo quite frustrating, but I recently
read <a href="https://www.cyberdemon.org/2024/03/20/submodules.html">“Demystifying git submodules”</a> by Dmitry Mazin, which helped
me build out my mental model. Except… <code>git submodule update</code>, which <a href="https://git-scm.com/docs/git-submodule#Documentation/git-submodule.txt-update--init--remote-N--no-fetch--no-recommend-shallow-f--force--checkout--rebase--merge--referenceltrepositorygt--ref-formatltformatgt--depthltdepthgt--recursive--jobsltngt--no-single-branch--filterltfilter-specgt--ltpathgt82308203">the
manual page promises</a> will “update the registered
submodules to match what the superproject expects by […] updating the working
tree of the submodules”, doesn’t seem to work like I expect it to.</p>
<p>Here’s what happens. After running a build on <code>ghc-9.7-start</code>, Git
tells me that content has been modified in the <code>libraries/unix</code> submodule:</p>
<pre><code>$ git status
HEAD detached at ghc-9.7-start
Changes not staged for commit:
(use "git add <file>..." to update what will be committed)
(use "git restore <file>..." to discard changes in working directory)
(commit or discard the untracked or modified content in submodules)
modified: libraries/unix (modified content)
</code></pre>
<p>I update the submodules:</p>
<pre><code>$ git submodule update --init --recursive --force
...
Submodule path 'utils/haddock': checked out '261a7c8ac5b5ff29e6e0380690cbb6ee9730f985'
...
$ git status
HEAD detached at ghc-9.7-start
nothing to commit, working tree clean
</code></pre>
<p>But then, when I go to checkout another branch, Git tells me that untracked
files in <code>utils/haddock</code> are getting in the way:</p>
<pre><code>$ git switch master
error: The following untracked working tree files would be overwritten by checkout:
utils/haddock/.github/mergify.yml
...
Aborting
</code></pre>
<p><strong>UPDATE:</strong> I sent this article to my friend who works as a firmware engineer
— and, therefore, uses submodules gleefully — and she immediately suggested
that a submodule might have been <em>removed</em>. I checked and that’s what’s going
on here. It had never occurred to me because I was checking out a later commit
starting at an earlier one, and I had assumed that a repository with 31
submodules didn’t get there by <em>removing</em> them. What follows are my flailing
attempts to fix this without removing all the submodules. Skip ahead to <a href="https://becca.ooo/blog/bisecting-ghc/#getting-ready">“Getting
ready”</a> if you don’t want to read that.</p>
<p>I try to remove the untracked files, but it doesn’t help:</p>
<pre><code>$ git clean -fdx
$ git submodule foreach git clean -fdx
...
Entering 'utils/haddock'
...
$ git switch master
error: The following untracked working tree files would be overwritten by checkout:
utils/haddock/.github/mergify.yml
...
Aborting
</code></pre>
<p>In fact, Git thinks that the working tree for <code>utils/haddock</code> is clean and that
<em>all</em> of those files are tracked:</p>
<pre><code>$ pushd utils/haddock
$ git status
HEAD detached at 261a7c8a
nothing to commit, working tree clean
$ git ls-files .github/mergify.yml
.github/mergify.yml
$ popd
</code></pre>
<p>Removing the checkouts for the submodules entirely seems to work, but I can’t
figure out why:</p>
<pre><code>$ git submodule deinit --all --force
...
Cleared directory 'utils/haddock'
Submodule 'utils/haddock' (https://gitlab.haskell.org/ghc/haddock.git) unregistered for path 'utils/haddock'
...
$ git switch master
Previous HEAD position was 261a7c8a Bump GHC version to 9.7
Switched to branch 'master'
Your branch is up to date with 'origin/master'.
</code></pre>
<h2 id="getting-ready"><a class=anchor href="#getting-ready" aria-label="Link to section getting-ready">#</a>Getting ready</h2>
<p>Now that we know the range of commits we want to bisect, let’s get GHC
building. If you haven’t built GHC before (I hadn’t!), <a href="https://gitlab.haskell.org/ghc/ghc/-/wikis/building/preparation">the GHC Wiki has a good
article on setting up your system for building GHC</a>.</p>
<p>After getting everything set up, the build process looks roughly like this:</p>
<pre data-lang="bash" class="language-bash "><code class="language-bash" data-lang="bash"># Clone and update the submodules.
git submodule update --init --recursive
# Run `autoreconf`, copy `config.sub` into submodules.
./boot
# Configure for building; `$CONFIGURE_ARGS` is set by `ghc.nix` and contains
# paths to the `gmp` and `ncurses` libraries.
./configure $CONFIGURE_ARGS
# Build GHC with the bespoke Hadrian build system:
./hadrian/build -j --flavour=Quick
</code></pre>
<p>I chose to use <a href="https://gitlab.haskell.org/ghc/ghc.nix">ghc.nix</a> to install the dependencies. This worked
really well, but when I attempted to reproduce my builds for this writeup
everything fell apart! It turned out that I had been using <a href="https://github.com/alpmestan/ghc.nix">an old version of
ghc.nix</a> from a GitHub repository that hadn’t been updated since
October 2023. By pure coincidence, this was perfectly suited for building the
revisions of GHC I was interested in, which were authored between December 2022
and October 2023.</p>
<p>I also made a couple tweaks to the <code>userSettings</code> attribute-set in ghc.nix’s
<code>flake.nix</code>:</p>
<ul>
<li>I set <code>withIde = false;</code> to disable building the huge and expensive
<code>haskell-language-server</code>, which I wouldn’t be using.</li>
<li>I set <code>bootghc = "ghc96";</code> because I’m building older versions of GHC,
which isn’t necessarily supported with a newer GHC. For bisecting larger
ranges or historical versions, it might be nice to build some tooling to
dynamically chose a boot GHC based on the commit being built.
(Also, I would learn later that GHC 9.4 would have been a better choice.)</li>
</ul>
<h2 id="fixing-up-ghc-nix-a-potpourri-of-build-errors"><a class=anchor href="#fixing-up-ghc-nix-a-potpourri-of-build-errors" aria-label="Link to section fixing-up-ghc-nix-a-potpourri-of-build-errors">#</a>Fixing up ghc.nix & a potpourri of build errors</h2>
<p>I suspect that the smarter move here would have been to just point at an old
commit of ghc.nix and call it a day, but instead I’ve spent my downtime for the
last couple days fixing up the latest revisions of ghc.nix to make it easier to
perform these builds in the future.</p>
<p>With <a href="https://github.com/alpmestan/ghc.nix/commit/c31bcab7a74569e6bd37dbe7de94c97cad1e35e1">the old revision of ghc.nix</a>, I got a hash mismatch
error:</p>
<pre><code>$ nix develop
error: NAR hash mismatch in input
'github:commercialhaskell/all-cabal-hashes/bd2d976d126b7730d82c772a207cf34e927aa69d'
(/nix/store/i19cc0hifz3f7iayz477v6s2aajyl1ii-source),
expected 'sha256-c6R3PkzqDCeAIqB+aygnjIMOmnkAmepyakOqtb8oQrg=',
got 'sha256-7wwhcWxDLTKHghemMh30Le7D5pi7+eSUCg4jj7yS+jM='
</code></pre>
<p>I fixed this by updating the relevant input with <code>nix flake update all-cabal-hashes</code>.</p>
<p><code>./boot</code> and <code>./configure</code> worked fine, but I ran into trouble when I attempted
to run the actual build process:</p>
<pre><code>$ ./hadrian/build -j --flavour=Quick
Resolving dependencies...
Error: cabal: Could not resolve dependencies:
[__0] trying: hadrian-0.1.0.0 (user goal)
[__1] trying: base-4.18.0.0/installed-4.18.0.0 (dependency of hadrian)
[__2] next goal: Cabal (dependency of hadrian)
[__2] rejecting: Cabal-3.10.1.0/installed-3.10.1.0 (conflict: hadrian =>
Cabal>=3.2 && <3.9)
...
</code></pre>
<p>In retrospect, this should have been a signal to find a better version of GHC
and Cabal to build with. According to the (<em>very</em> well-hidden) <a href="https://web.archive.org/web/20241203234539/https://gitlab.haskell.org/ghc/ghc/-/wikis/commentary/libraries/version-history">GHC Boot
Library Version History</a> page on the GHC wiki, GHC 9.4
was distributed with Cabal 3.8, so that’s what we likely want.</p>
<p>I decided to charge on, though. Here’s what I needed to do to get GHC to build
with “close enough” versions of the build tools!</p>
<p>First, you need to tell <a href="https://gitlab.haskell.org/ghc/ghc/-/blob/fc3a2232da89ed4442b52a99ba1826d04362a7e8/hadrian/build-cabal#L5">the Cabal that’s building Hadrian</a> to
ignore the dependency version bounds with <a href="https://cabal.readthedocs.io/en/latest/cabal-project-description-file.html#cfg-flag---allow-newer"><code>--allow-newer</code></a>:</p>
<pre><code>CABFLAGS=--allow-newer ./hadrian/build -j --flavour=Quick
</code></pre>
<p>This will build Hadrian and then GHC. However, it won’t get very far:</p>
<pre><code># cabal-configure (for _build/stageBoot/linters/lint-whitespace/setup-config)
Error: hadrian: Encountered missing or private dependencies:
Error: hadrian: Encountered missing or private dependencies:
mtl >=2.1 && <2.3
mtl >=2.1 && <2.3
</code></pre>
<p>For most command-line options, we can use the <a href="https://gitlab.haskell.org/ghc/ghc/-/blob/c3fc9b861fd00a85a4fcbd9960b8242d9fabe04b/hadrian/doc/user-settings.md#key--value-and-key--value-style-settings">Hadrian user
settings</a> to add Cabal options (e.g., <code>echo ".*.cabal.configure.opts += ..." >> _build/hadrian.settings</code>), but the Hadrian
build system works with <a href="https://cabal.readthedocs.io/en/stable/setup-commands.html">Cabal’s lower-level <code>Setup.hs</code> interface</a>,
which doesn’t support the <code>--allow-newer</code> flag, so we have to patch the
<code>.cabal</code> files manually.</p>
<p>I used a tool called <a href="https://github.com/NixOS/jailbreak-cabal"><code>jailbreak-cabal</code></a> to remove version
bounds from dependency specifications in <code>.cabal</code> files. Here’s the command to
run, which skips the hundreds of <code>.cabal</code> files in integration tests:</p>
<pre data-lang="bash" class="language-bash "><code class="language-bash" data-lang="bash">find . \
! '(' \
-path ./testsuite -prune \
-o -path ./libraries/Cabal/Cabal-tests/tests -prune \
-o -path ./libraries/Cabal/cabal-install/tests -prune \
-o -path ./libraries/Cabal/cabal-testsuite/PackageTests -prune \
')' \
-path '*.cabal' \
-exec jailbreak-cabal '{}' ';'
</code></pre>
<p>With that in place, my builds completed successfully, but I still needed to
make a few other tweaks to get the build working with the new canonical ghc.nix
repository.</p>
<p>First, I needed to <a href="https://gitlab.haskell.org/ghc/ghc.nix/-/merge_requests/24">remove <code>gdb</code> from the included tools</a> because
<code>gdb</code> isn’t available on macOS.</p>
<p>Then, I needed to <a href="https://gitlab.haskell.org/ghc/ghc.nix/-/merge_requests/25">export <code>CXX</code> with an absolute path</a> so that the
<code>./configure</code> script wouldn’t fail due to a quoting issue. This bug is
fixed on recent versions of GHC, but it’s present on <code>ghc-9.7-start</code>, which we
want to build!</p>
<p>After fixing the <code>./configure</code> step, I could attempt the build, but it didn’t
get very far:</p>
<pre><code>| Run GhcPkg Update (Stage0 InTreeLibs): _build/stage0/libraries/ghc-boot-th/inplace-pkg-config => none
dieVerbatim: user error (Error: hadrian:
'/nix/store/3dahn98q1m46ickndsg797zp8gv801b2-ghc-9.6.6-with-packages/bin/ghc'
exited with an error:
ghc-9.6.6: can't find a package database at
_build/stage0/libraries/ghc-boot-th/build/package.conf.inplace
)
Build failed.
</code></pre>
<p>At this point, I suspected my GHC / Cabal versions were to blame, so I started
to look for more historically-accurate versions of the tools. First I found the
date of the commit I was trying to build:</p>
<pre><code>$ git show
commit fc3a2232da89ed4442b52a99ba1826d04362a7e8 (HEAD, tag: ghc-9.7-start)
Author: Ben Gamari <[email protected]>
Date: Thu Dec 22 13:45:06 2022 -0500
Bump GHC version to 9.7
</code></pre>
<p>Then I asked for <a href="https://github.com/nixos/nixpkgs/commit/012700eae502f6054a056cf7b94f78ff549e278d">a commit around that time in Nixpkgs</a>:</p>
<pre><code>$ git log --before='Thu Dec 22 13:45:06 2022' nixos-unstable
commit 012700eae502f6054a056cf7b94f78ff549e278d
Merge: 0550dfc0228a 5f1760cb902c
Author: Fabian Affolter <[email protected]>
Date: Thu Dec 22 22:38:20 2022 +0100
Merge pull request #207299 from r-ryantm/auto-update/python3.10-python-crontab
python310Packages.python-crontab: 2.6.0 -> 2.7.1
</code></pre>
<p>If you’ve read <a href="https://git-scm.com/docs/git-rev-parse#_specifying_revisions">the <code>git rev-parse</code> documentation</a>, you might
be tempted to try <code>git log 'nixos-unstable@{Thu Dec 22 13:45:06 2022}'</code>, but
that will attempt to find what <em>your</em> checkout had <code>nixos-unstable</code> pointing to
on that date.</p>
<p>Attempting to build GHC from that revision gave me a compiler error:</p>
<pre><code>ghc: panic! (the 'impossible' happened)
GHC version 9.4.2:
Template variable unbound in rewrite rule
Variable: sg_shgu :: WasmTypeTag 'I32 ~R# WasmTypeTag w_sgB4
Rule "SC:$j0"
Rule bndrs: [sg_shgu]
LHS args: [ty_word_X16]
Actual args: [ty_word_X16]
Call stack:
CallStack (from HasCallStack):
callStackDoc, called at compiler/GHC/Utils/Panic.hs:182:37 in ghc:GHC.Utils.Panic
pprPanic, called at compiler/GHC/Core/Rules.hs:619:10 in ghc:GHC.Core.Rules
</code></pre>
<p>I jumped ahead and tried <a href="https://github.com/nixos/nixpkgs/commit/33650d3ae02cac67b6e8f75f2b997b1e0556b676">a commit from a year later</a>. That
built successfully, but running any of the executables it produced failed:</p>
<pre><code>$ _build/stage1/bin/ghc
dyld[12621]: symbol not found in flat namespace '_unixzm2zi8zi3zi0zminplace_SystemziPosixziFiles_getFileStatus_closure'
fish: Job 1, '_build/stage1/bin/ghc' terminated by signal SIGABRT (Abort)
</code></pre>
<p>It was around this time that I finally looked at the old ghc.nix repository
again and noticed it was pointing to <a href="https://github.com/nixos/nixpkgs/commit/5cfafa12d57374f48bcc36fda3274ada276cf69e">a commit on the <code>nixos-23.05</code> branch of
Nixpkgs.</a> A stable release branch! Why hadn’t I thought of
that earlier? I only needed one more tweak to <a href="https://gitlab.haskell.org/ghc/ghc.nix/-/merge_requests/26">fix ghc.nix with old versions of
Nixpkgs that don’t include a top-level <code>pkgs.happy</code>
attribute</a> and my builds started completing successfully.</p>
<p>Now we’re ready to start bisecting!</p>
<h2 id="starting-the-bisection"><a class=anchor href="#starting-the-bisection" aria-label="Link to section starting-the-bisection">#</a>Starting the bisection</h2>
<p>First, mark a commit you know is bad and a commit you know is good:</p>
<pre data-lang="bash" class="language-bash "><code class="language-bash" data-lang="bash">git bisect start ghc-9.8.1-release ghc-9.7-start
</code></pre>
<p>Then run the bisection using a script to determine which commits are good:</p>
<pre data-lang="bash" class="language-bash "><code class="language-bash" data-lang="bash">git bisect run ../test.sh
</code></pre>
<p>Note that the <code>test.sh</code> script is in the <em>parent</em> directory so it doesn’t get
deleted when we run <code>git clean</code>.</p>
<h2 id="the-bisection-script"><a class=anchor href="#the-bisection-script" aria-label="Link to section the-bisection-script">#</a>The bisection script</h2>
<p>We’ll need to build GHC repeatedly across a variety of different versions, so
we’ll need some tweaks to get things working smoothly.</p>
<p>First, we’ll want to remove any previous build artifacts and get the submodules
set up correctly:</p>
<pre data-lang="bash" class="language-bash "><code class="language-bash" data-lang="bash"># Log commands as we run them.
set -x
# Reset the submodules in case of changes (e.g., to generated files).
# This takes about 5 seconds, or a minute for a fresh clone.
git submodule update --init --force --recursive || exit 128
# Remove existing build products.
# This takes roughly no time.
time rm -rf _build || exit 128
</code></pre>
<p>Why exit with code 128 if these commands fail? According to <a href="https://git-scm.com/docs/git-bisect#_bisect_run">the <code>git bisect</code>
man page</a>:</p>
<blockquote>
<p>Note that the script […] should exit with code 0 if the current source code
is good/old, and exit with a code between 1 and 127 (inclusive), except 125,
if the current source code is bad/new.</p>
<p>Any other exit code will abort the bisect process. It should be noted that a
program that terminates via <code>exit(-1)</code> leaves <code>$? = 255</code>, (see the <code>exit(3)</code>
manual page), as the value is chopped with <code>& 0377</code>.</p>
<p>The special exit code 125 should be used when the current source code cannot
be tested. If the script exits with this code, the current revision will be
skipped (see <code>git bisect skip</code> above). 125 was chosen as the highest sensible
value to use for this purpose, because 126 and 127 are used by POSIX shells
to signal specific error status (127 is for command not found, 126 is for
command found but not executable—these details do not matter, as they are
normal errors in the script, as far as <code>bisect run</code> is concerned).</p>
</blockquote>
<p>The first script I tried looked like this:</p>
<pre data-lang="bash" class="language-bash "><code class="language-bash" data-lang="bash">function build {
git submodule update --init --force --recursive
rm -rf _build
./boot
# ...
}
set -ex
build || exit 128
</code></pre>
<p>I hoped that this would run the commands in <code>build</code>, returning 1 from <code>build</code>
if any of the commands failed, and then exiting with <code>128</code> as a result. It
turns out this is not the case! According to <a href="https://www.gnu.org/software/bash/manual/html_node/The-Set-Builtin.html">the Bash manual</a>, using
<code>set -e</code> will cause the script to:</p>
<blockquote>
<p>Exit immediately if a pipeline, which may consist of a single simple command,
a list, or a compound command returns a non-zero status. The shell does not
exit if the command that fails is […] part of any command executed in a <code>&&</code>
or <code>||</code> list except the command following the final <code>&&</code> or <code>||</code></p>
<p>If a compound command or shell function executes in a context where <code>-e</code> is
being ignored, none of the commands executed within the compound command or
function body will be affected by the <code>-e</code> setting, even if <code>-e</code> is set and a
command returns a failure status.</p>
</blockquote>
<p>Therefore, because we run <code>build || exit 128</code>, <code>build</code> will <em>keep going</em> if one
of the commands in it fails! I googled “bash errexit site:gnu.org” to find the
documentation I’ve linked above, but the first result was <a href="https://lists.gnu.org/archive/html/bug-bash/2012-12/msg00093.html">a message sent to
the Bash mailing list titled “why does errexit exist in its current utterly
useless form?”</a>, complaining about this exact problem.</p>
<p>To work around this, we have to append <code>|| exit 128</code> to each command in the
build process. To state the obvious, this is tedious and error-prone.</p>
<p>Anyways, now we’re ready to run the build! Booting the repository takes about
two seconds:</p>
<pre><code>./boot || exit 128
</code></pre>
<p>Configuring it takes about 30 seconds:</p>
<pre><code>./configure $CONFIGURE_ARGS || exit 128
</code></pre>
<p>And building Hadrian and then GHC takes about 12 minutes on my desktop (M1
Ultra with 20 cores and 64GB RAM) or about 19 minutes on my laptop (M2 Max with
12 cores and 32GB RAM):</p>
<pre><code>CABFLAGS=--allow-newer ./hadrian/build -j --flavour=Quick || exit 128
</code></pre>
<p>When we’re done, make sure to reset any changes we made to files in the working
tree so that Git can check out the next revision:</p>
<pre><code>git reset --hard || exit 128
</code></pre>
<p>Double-check that we’ve actually produced a <code>runghc</code> executable, and then we
can use the built compiler to run our tests:</p>
<pre><code># Make sure we actually built a compiler:
if [[ ! -e _build/stage1/bin/runghc ]]
then
exit 128
fi
tmp=$(mktemp)
_build/stage1/bin/runghc ../Main.hs > "$tmp" || exit 128
# If the output doesn't contain 'action', the bug is present in this commit.
grep --quiet action "$tmp"
</code></pre>
<p>And that’s it! The finished product isn’t <em>too</em> complex, but the road to get
there involved a lot of subtle pitfalls I wanted to write down.</p>
<h2 id="putting-it-all-together"><a class=anchor href="#putting-it-all-together" aria-label="Link to section putting-it-all-together">#</a>Putting it all together</h2>
<p>Here’s what the bisection test script looks like when it’s all assembled
together! <a href="https://gist.github.com/9999years/9ad5e0929c44b1908d66473cf66ddfc4">The script is also available as a GitHub Gist.</a></p>
<pre><code>#!/usr/bin/env bash
# Log commands as we run them, and exit if any command produces an error.
set -x
# Attempt to build GHC.
#
# If we can't build GHC, exit with 128. This will abort the entire `git bisect`
# instead of erroneously marking the commit as 'bad'.
#
# From `man git-bisect`:
# > Note that the script [...] should exit with code
# > 0 if the current source code is good/old, and exit with a code between 1
# > and 127 (inclusive), except 125, if the current source code is bad/new.
# >
# > Any other exit code will abort the bisect process. It should be noted that
# > a program that terminates via exit(-1) leaves $? = 255, (see the exit(3)
# > manual page), as the value is chopped with & 0377.
# 5 seconds:
# Reset the submodules if there's changes like generated files.
time git submodule update --init --force --recursive || exit 128
# Remove existing build products.
time rm -rf _build || exit 128
# Now we run the actual build process.
# See: https://gitlab.haskell.org/ghc/ghc/-/wikis/building/preparation
# 2 seconds:
time ./boot || exit 128
# 30 seconds:
time ./configure $CONFIGURE_ARGS || exit 128
# ~12 minutes (M1 Ultra, 20 cores), ~19 minutes (???, 12 cores):
time CABFLAGS=--allow-newer ./hadrian/build -j --flavour=Quick || exit 128
# Reset any modified files or checking out the next commit will fail:
time git reset --hard || exit 128
# Make sure we actually built a compiler:
if [[ ! -e _build/stage1/bin/runghc ]]
then
exit 128
fi
tmp=$(mktemp)
_build/stage1/bin/runghc "../Main.hs" > "$tmp" || exit 128
# If the output contains 'action', we're all OK.
# If it just says 'run', we have a problem!
grep --quiet action "$tmp"
</code></pre>
Announcing git-prole2024-10-22T00:00:00+00:002024-10-22T00:00:00+00:00
Rebecca Turner
https://becca.ooo/blog/announcing-git-prole/<p><strong>TL;DR:</strong> I’ve built <a href="https://github.com/9999years/git-prole"><code>git-prole</code></a>, an ergonomic
<a href="https://git-scm.com/docs/git-worktree"><code>git-worktree</code></a> manager. Check out the <a href="https://9999years.github.io/git-prole/">user manual</a> to
get started! If you don’t know what a Git worktree is or why you might want
one, read on.</p>
<p>If you’re anything like me, you have a couple repositories cloned that are
cumbersome enough to clone more than once:</p>
<pre><code>$ ls -l
drwxr-xr-x - rebeccat 15 Oct 11:13 mwb
drwxr-xr-x - rebeccat 11 Oct 14:19 mwb2
drwxr-xr-x - rebeccat 30 Sep 10:06 mwb3
drwxr-xr-x - rebeccat 19 Oct 13:31 mwb4
drwxr-xr-x - rebeccat 2 Aug 16:45 mwb5
drwxr-xr-x - rebeccat 3 Oct 12:10 mwb6
</code></pre>
<p>This feels a little bit silly, but there’s some good reasons for it:</p>
<ul>
<li>If builds are expensive, then switching branches can incur a significant time
cost.</li>
<li>You might want to check out a colleague’s branch, but you have uncommitted
work you’re hesitant to commit or stash.</li>
<li>It’s often convenient to compare your code against the repository’s main
branch. <code>git diff</code> can do this, of course, but chances are it’s not very well
integrated into your editor, or you need to be able to run the code in
addition to reading it.</li>
</ul>
<p>At the same time, there’s some costs to this approach:</p>
<ul>
<li>The repositories have to be cloned separately, which is annoying. You can
clone from a local repository, but then you have to be careful to add the
correct remote, remove the old remote, and set up a bunch of remote-tracking
branches again.</li>
<li>You can’t easily share work between the repositories. I often want to split a
couple of small fixes off from a current branch, but there’s not a great way
to do this without generating a patch and applying it manually or pushing
your work to a remote.</li>
<li>The clones are separate repositories, so it’s not easy to add new checkouts
or remove unused ones. You effectively have to decide how many branches you
want to be able to check out at the same time in advance.</li>
</ul>
<p>Git worktrees, then, are a feature that allows you to associate multiple
checkouts with a single repository, like this:</p>
<pre><code>my-repo/
main/ # A checkout for the main branch
.git/ # The main .git directory.
README.md
feature1/ # A checkout for work on a feature
.git # A file containing `gitdir: /Path/to/my-repo/main/.git`
README.md
</code></pre>
<p>And, as it turns out, with a little fiddling, you can also use a bare checkout
for the <code>.git</code> directory; this means that none of the worktrees are “special”
and they can all be deleted (if you try to set up a repository like this
yourself, you will have problems; more on this later).</p>
<pre><code>my-repo/
.git/ # A bare repository
main/ # A checkout for the main branch
.git # A file with the path to the bare repository
README.md
feature1/ # A checkout for work on a feature
.git
README.md
</code></pre>
<h2 id="features"><a class=anchor href="#features" aria-label="Link to section features">#</a>Features</h2>
<p><a href="https://9999years.github.io/git-prole/"><code>git-prole</code></a> includes a number of tools to make using multiple
worktrees as ergonomic as possible:</p>
<ul>
<li>
<p><code>git prole clone URL [DESTINATION]</code> will clone a repository into the
multiple-worktrees layout described above.</p>
</li>
<li>
<p><code>git prole convert</code> will convert the current repository into a worktree
layout. (Use <code>--dry-run</code> to see what will happen.)</p>
</li>
<li>
<p><code>git prole add</code> will add a new worktree:</p>
<ul>
<li>
<p><code>git prole add feature1</code> will create a <code>feature1</code> directory next to the
rest of your worktrees; <code>git worktree add feature1</code>, in contrast, will
create a <code>feature1</code> subdirectory nested under the current worktree.</p>
</li>
<li>
<p>Branches created with <code>git prole add</code> will start at and track the
repository’s main branch by default.</p>
</li>
<li>
<p><code>git prole add</code> will copy untracked files to the new worktree by default,
making it easy to start a new worktree with a warm build cache.</p>
</li>
<li>
<p><code>git prole add</code> can run commands when a new worktree is created, so that
you can warm up caches by running a command like <code>direnv allow</code>.</p>
</li>
<li>
<p><code>git prole add</code> can perform regex substitutions on branch names to compute
a directory name, so that you can run <code>git prole add -b myname/team-1234-my-ticket-with-a-very-long-title</code> and get a directory name
like <code>my-ticket</code>.</p>
</li>
<li>
<p><code>git prole add</code> respects the <code>-c</code>/<code>--create</code> option (to match <code>git switch</code>); <code>git worktree add</code> only allows <code>-b</code> (with no long-form option
available).</p>
</li>
</ul>
</li>
</ul>
<h2 id="why-hadn-t-i-heard-of-worktrees-before"><a class=anchor href="#why-hadn-t-i-heard-of-worktrees-before" aria-label="Link to section why-hadn-t-i-heard-of-worktrees-before">#</a>Why hadn’t I heard of worktrees before?</h2>
<p>This is a pretty cool feature, and once we have some tooling like <code>git-prole</code>
to paper over the ergonomic challenges it makes a great workflow! So why
haven’t you heard of worktrees before?</p>
<p>For reasons I can’t fully determine, worktrees have received basically zero
fanfare except from your coworker who runs <code>nix-darwin</code>. The <code>git-worktree</code>
command was <a href="https://github.com/git/git/commit/7783eb2e59684492e75068443e1f77f64fe37cc9">first present</a> in Git 2.5.0, released in 2015.
However, <a href="https://github.com/git/git/blob/master/Documentation/RelNotes/2.5.0.txt">the release notes for Git 2.5.0</a>, which
include dozens of updates like “Clarify in the <code>Makefile</code> a guideline to decide
use of <code>USE_NSEC</code>”, have nothing to say about worktrees. GitHub actually
published a blog post titled <a href="https://github.blog/open-source/git/git-2-5-including-multiple-worktrees-and-triangular-workflows/">“Git 2.5, including multiple worktrees and
triangular workflows”</a>, which received, as far as I
can tell, <a href="https://www.reddit.com/r/git/comments/3f5shh/git_25_including_multiple_worktrees_and/">one Reddit post</a> with 35 upvotes and two top-level comments,
neither of which were excited for the new feature.</p>
<p>Recently, worktrees have started to garner a little more attention (see, for
example, matklad’s <a href="https://matklad.github.io/2024/07/25/git-worktrees.html">“How I Use Git Worktrees”</a>), but the second search
result for “git worktrees” is a Stack Overflow question titled <a href="https://stackoverflow.com/questions/31935776/what-would-i-use-git-worktree-for">“What would I
use <code>git-worktree</code> for?”</a></p>
<h2 id="trying-and-failing-with-worktrees"><a class=anchor href="#trying-and-failing-with-worktrees" aria-label="Link to section trying-and-failing-with-worktrees">#</a>Trying and failing with worktrees</h2>
<p>A couple weeks ago, I finally decided to take a couple hours and figure out how
to use worktrees. Let’s create a worktree for a new branch, which I’ll call <code>feature1</code>:</p>
<pre><code>$ git worktree add feature1
Preparing worktree (new branch 'feature1')
branch 'feature1' set up to track 'main'.
HEAD is now at b1b2888 Fix static binary builds in releases (#86)
</code></pre>
<p>Great, that was easy! Now, let’s just type <code>git status</code>, which I enter
reflexively after nearly any Git command:</p>
<pre><code>$ git status
On branch main
Your branch is up to date with 'origin/main'.
Untracked files:
(use "git add <file>..." to include in what will be committed)
feature1/
nothing added to commit but untracked files present (use "git add" to track)
</code></pre>
<p>…huh. The first argument to <code>git worktree add</code> is used as a branch name, but
it’s actually a path, and it will happily create a worktree nested in another
worktree, despite (presumably) nobody ever wanting to do this.</p>
<p>The actual invocation you want is something like this, where Git uses the last
<em>component</em> of the path as the branch name:</p>
<pre><code>$ git worktree add ../feature1
</code></pre>
<p>And then we get to the next ergonomic issue with Git worktrees: what do you
name them? If you’re not a worktree user already, you probably have your
repository cloned in some “projects” directory, containing a whole bunch of
repositories. So when you start working on <code>feature1</code>, you should really
disambiguate it:</p>
<pre><code>$ git worktree add ../my-repo-feature1
</code></pre>
<p>This is a bad experience! You have to type a path manually, prefix the path
with the repository name, and then you have to type the repository name again
when you <code>cd</code> to the checkout later. Can we do better? Of course!</p>
<h2 id="a-better-layout-for-a-worktree-repository"><a class=anchor href="#a-better-layout-for-a-worktree-repository" aria-label="Link to section a-better-layout-for-a-worktree-repository">#</a>A better layout for a worktree repository</h2>
<p>If we look at the example workflow from the GitHub blog post above, we can see
that they expect you to have the workflow layout I showed you at the start of
the post:</p>
<pre><code>$ cd ../main
$ rm -rf ../hotfix
</code></pre>
<p>GitHub must expect you to have a containing directory named <code>my-repo</code> and then
worktrees directly inside it, named corresponding to the branches they have
checked out. <code>git-prole</code> works well with this layout; <code>git prole add hotfix</code> will create a new worktree in the same directory as the main worktree.</p>
<p>Let’s set up one of our repositories like that. First, we’ll move our main
working tree from <code>/path/to/my-repo</code> to <code>/path/to/my-repo/main</code>:</p>
<pre><code>$ git worktree move . main
fatal: '.' is a main working tree
</code></pre>
<p>Oh. I guess we’ll do it the hard way: renaming <code>my-repo</code> to <code>my-repo.bak</code>,
creating a new <code>my-repo</code> directory, and then moving <code>my-repo.bak</code> to
<code>my-repo/main</code>. A three-step barrier to entry for worktrees, and that’s if you
only have one to start with. If you’ve already created some worktrees in the
parent directory, you’ll have to figure out how to reassociate them with the
main worktree once you move everything. Theoretically this is possible with
<code>git worktree repair</code>, but I found myself having to edit files in the <code>.git</code>
directory manually.</p>
<p><code>git-prole</code> will handle this conversion process for you, even with multiple
worktrees, un-committed work, or any number of other complications:</p>
<pre><code>$ git worktree list
/path/to/git-prole 719ed92 [main]
$ nix run github:9999years/git-prole -- convert
• Converting ~/git-prole to a worktree repository at ~/git-prole.
I'll move the following worktrees to new locations:
• ~/git-prole -> ~/git-prole/main
Additionally, I'll convert the repository to a bare repository.
Preparing worktree (checking out 'main')
• ~/git-prole has been converted to a worktree checkout
• You may need to `cd .` to refresh your shell
$ cd .
$ git worktree list
/path/to/git-prole (bare)
/path/to/git-prole/main 719ed92 [main]
</code></pre>
<p>Eventually, I got the repository set up the way I wanted, but there was still
one thing that bothered me: the <code>.git</code> directory. If you create a worktree
layout by moving your existing worktrees around, you end up with one special
“main” worktree, which contains the full <code>.git</code> directory shared by the other
worktrees and cannot be renamed or removed.</p>
<p>Having a main worktree is fine, but it’s smelly. I don’t <em>like</em> that all of the
worktrees are ephemeral and short-lived except for one. The solution to this is
to have the main worktree be a bare repository: a <code>.git</code> directory with no
accompanying working tree. A repository is bare if its configuration sets the
<code>core.bare</code> option to true:</p>
<pre><code>$ git status
On branch main
nothing to commit, working tree clean
$ git config set --local core.bare true
$ git status
fatal: this operation must be run in a work tree
</code></pre>
<p>So we create the <code>my-repo</code> directory and do a bare clone into it:</p>
<pre><code>$ git clone --bare [email protected]:9999years/git-prole.git
Cloning into bare repository 'git-prole.git'...
</code></pre>
<p>Great! Now let’s set up a working tree:</p>
<pre><code>$ cd git-prole.git/
$ git worktree add ../main
Preparing worktree (checking out 'main')
HEAD is now at 719ed92 Fix links to `git-worktree(1)` docs (#89)
$ cd ../main/
$ git pull
There is no tracking information for the current branch.
Please specify which branch you want to rebase against.
See git-pull(1) for details.
git pull <remote> <branch>
If you wish to set tracking information for this branch you can do so with:
git branch --set-upstream-to=origin/<branch> master
$ git branch --set-upstream-to=origin/master master
fatal: the requested upstream branch 'origin/master' does not exist
hint:
hint: If you are planning on basing your work on an upstream
hint: branch that already exists at the remote, you may need to
hint: run "git fetch" to retrieve it.
hint:
hint: If you are planning to push out a new local branch that
hint: will track its remote counterpart, you may want to use
hint: "git push -u" to set the upstream config as you push.
hint: Disable this message with "git config advice.setUpstreamFailure false"
</code></pre>
<p>…huh. Apparently, <a href="https://stackoverflow.com/a/54408181">as someone pointed out on Stack Overflow to a user with
the same problem</a>, <code>git clone --bare</code> doesn’t <em>just</em> skip
creating a working directory. According to the man page:</p>
<blockquote>
<p>Also the branch heads at the remote are copied directly to corresponding
local branch heads, without mapping them to <code>refs/remotes/origin/</code>.</p>
</blockquote>
<p>This means that instead of (e.g.) creating a remote-tracking branch
<code>origin/main</code>, Git creates a local branch <code>main</code>. Then, when you go to push or
pull, Git errors out when it can’t find a matching remote branch. You can sort
of fix this by setting a refspec for your remote (e.g., <code>git config set --local remote.origin.fetch "+refs/heads/*:refs/remotes/origin/*"</code>, good luck
remembering <em>that</em> the next time you need it), but then you have to rename all
of the refs manually, transforming <code>refs/heads/foo</code> to
<code>refs/remotes/origin/foo</code>, which is a pain to say the least!</p>
<p>Converting an existing repository to a bare one is similarly painful, although
for different reasons (it requires a song and dance to reassociate the main
worktree with the repository). Eventually, I realized that the only way to fix
these issues would be with a new tool to make setting up and working with
multiple worktrees easy.</p>
<h2 id="easier-worktrees-with-git-prole"><a class=anchor href="#easier-worktrees-with-git-prole" aria-label="Link to section easier-worktrees-with-git-prole">#</a>Easier worktrees with git-prole</h2>
<p>For a breath of fresh air, let’s clone a repository with <code>git-prole</code>, taking
advantage of the (configuration-gated) <a href="https://cli.github.com/"><code>gh</code></a> support to let us enter a
GitHub repository slug instead of a full URL:</p>
<pre><code>$ git prole clone 9999years/git-prole
Cloning into '/Users/wiggles/test-repo/git-prole'...
remote: Enumerating objects: 668, done.
remote: Counting objects: 100% (502/502), done.
remote: Compressing objects: 100% (215/215), done.
remote: Total 668 (delta 331), reused 420 (delta 286), pack-reused 166 (from 1)
Receiving objects: 100% (668/668), 277.44 KiB | 1.66 MiB/s, done.
Resolving deltas: 100% (407/407), done.
• • Move git-prole/.git to $TMPDIR/.tmpwdCaYS/.git
• In $TMPDIR/.tmpwdCaYS/.git, set core.bare=true
• Move git-prole to $TMPDIR/.tmpwdCaYS/main
• Create directory git-prole
• Move $TMPDIR/.tmpwdCaYS/.git to git-prole/.git
• In git-prole/.git, create but don't check out a worktree for main at git-prole/main
• In git-prole/main, reset the index state
• Move git-prole/main/.git to $TMPDIR/.tmpwdCaYS/main/.git
• Remove git-prole/main
• Move $TMPDIR/.tmpwdCaYS/main to git-prole/main
Preparing worktree (checking out 'main')
• git-prole has been converted to a worktree checkout
$ cd git-prole
$ git worktree list
/path/to/git-prole (bare)
/path/to/git-prole/main 719ed92 [main]
</code></pre>
<p>Nice! Let’s add a new worktree:</p>
<pre><code>$ git prole add add feature1
• Creating worktree in ~/git-prole/feature1 for feature1 tracking origin/main
Preparing worktree (new branch 'feature1')
branch 'feature1' set up to track 'origin/main'.
HEAD is now at 719ed92 Fix links to `git-worktree(1)` docs (#89)
</code></pre>
<p>New branches automatically start at the repository’s default branch, instead of
the current commit.</p>
<p>That’s the core of <code>git-prole</code>’s functionality, along with some goodies like
copying untracked files to new worktrees to keep your build caches warm and
your local configuration intact.</p>
<p>The workflow enabled by cheap checkouts is super nice and I encourage you to
give it a try! To get started with <code>git-prole</code>, check out <a href="https://9999years.github.io/git-prole/">the user
manual</a> for installation instructions.</p>
Giving My Student Loan Servicer a $560 Interest-Free Loan2024-08-20T00:00:00+00:002024-08-20T00:00:00+00:00
Rebecca Turner
https://becca.ooo/blog/an-interest-free-loan/<p>Our story begins on April 8th, 2024. I was ecstatic: I’d just finished paying
off my student loans. Well, maybe not ecstatic, given the Biden
administration’s lackluster student loan forgiveness policy, but it was a
weight off my shoulders. My joy lasted two days, until April 10th, when my
monthly autopayment got processed and Aidvantage took $560 out of my bank
account, leaving one of my loans with a balance of <em>negative</em> $560.</p>
<h2 id="wait-what"><a class=anchor href="#wait-what" aria-label="Link to section wait-what">#</a>Wait, what?</h2>
<p>Finding nothing useful in the help pages, on April 15th I submitted a support
request:</p>
<blockquote>
<p>I made a manual payment to finish paying off my loans around the time of my
monthly automatic payment. Now one of my loans has a -$560 balance. Can
you… send that back to me?</p>
</blockquote>
<p>Just two days later, I received a reply on April 17th:</p>
<blockquote>
<p>Thank you for reaching out to us regarding the overpayment amount of $560.00.
I’m happy to assist.</p>
<p>This overpayment amount will be refunded to you. Currently the estimated
timeframe for a refund to be deposited into your bank account is 8-12 weeks.</p>
</blockquote>
<p>Let’s get this out of the way: 8-12 weeks is an absurd amount of time to return
the money. But what could I do? I set a reminder on my calendar.</p>
<p>On July 10th, I called back to ask where the money was, and a support agent
informed me that a check was sent out and should have arrived by now.</p>
<h2 id="wait-a-check"><a class=anchor href="#wait-a-check" aria-label="Link to section wait-a-check">#</a>Wait, a Check?</h2>
<p>Something odd has happened at this point: the original response said a refund
would be “deposited in [my] bank account”, not that a check would be sent out!
Nobody has been able to explain this portion of the story to me. All I know is
that — despite having my bank account information, despite me explicitly opting
in to for paperless communications — someone at Aidvantage decided to send me a
check. Or, at least, they allege they sent me a check. They didn’t send
certified mail, ask for a signature, or even provision a tracking number from
the USPS, so there’s no evidence a check was ever sent.</p>
<p>Again, what could I do? I asked the support agent if they could reissue the
check, and they offered me a more appealing option: an ACH deposit. That would
be great, I said. I gave them my account and routing numbers (which they
already had) and they submitted an ACH transfer, due to be deposited into my
account in 3-4 weeks.</p>
<h2 id="wait-3-4-weeks"><a class=anchor href="#wait-3-4-weeks" aria-label="Link to section wait-3-4-weeks">#</a>Wait, 3-4 weeks?</h2>
<p>Since when does an ACH pull take 3-4 weeks? 3-5 <em>days</em> is normal, and it’s
usually shorter! I have no clue what’s going on here.</p>
<p>But what could I do? I set a reminder on my calendar. After 4 weeks, I called
back on Thursday, August 15th. The support agent informed me that they could
see the ACH was sent out and that it should, for sure, we promise, be deposited
into my account on Monday.</p>
<p>At this point the process of calling back is beginning to wear on me. I don’t
have the energy to do it on a Monday. I called back this morning, on August
20th. After hanging up on me several times, my fourth or fifth attempt of the
day gets through to a support agent. He thanks me for my patience and puts me
on hold, says he’ll look into it and might need to ask his supervisor. After a
couple minutes he thanks me for waiting and says “it says here that before we
can reissue a refund you have to either return the check to us — which you
can’t do, because it didn’t physically get to you — or wait for it to go stale,
which happens after… fourteen months? That’s ridiculous. Let me ask for
clarification.” It was a small comfort that the support agent sounded roughly
as baffled by this as I was. He re-read it a few times, possibly to make sure
he wasn’t reading the Looney Tunes version of the refund procedures. No
clarification came, and he sounded apologetic as he told me to check back in
late August, 2025.</p>
<h2 id="wait-august-2025"><a class=anchor href="#wait-august-2025" aria-label="Link to section wait-august-2025">#</a>Wait, August 2025?</h2>
<p>Astute readers may have noticed that I got the first message from Aidvantage
acknowledging the refund on April 17th, and fourteen months after April 2024 is
<em>June</em> 2025. However, as my support agent informed me, the refund check wasn’t
actually created until June 24th, 2024 — Aidvantage just twiddled their thumbs
for 68 days before bothering to print the check.</p>
<p>As for why a check was sent instead of an electronic refund in the first place?
My support agent had no clue. If we consider <a href="https://en.wikipedia.org/wiki/The_purpose_of_a_system_is_what_it_does">the purpose of a system to be
what it does</a>, however, the mechanism is clear: Aidvantage pretended
to send me a check because it’s good for their balance sheet if I give them a
$560 loan, interest-free, for a couple years. What am I going to do, stop
giving them business? <em>I’m</em> not their customer, after all.</p>
<h2 id="kafka-was-right"><a class=anchor href="#kafka-was-right" aria-label="Link to section kafka-was-right">#</a>Kafka Was Right</h2>
<p>Franz Kafka, working at an insurance company at the beginning of the 20th
century, saw these systems ossifying, and had a remarkably clear vision of
where it was all headed: a society where all human coordination of services is
replaced with bureaucratic instruments. The support agents I talked to at
Aidvantage had no power to help me. They also had no ability to contact anyone
who <em>could</em> help me: the bureaucratic machine airgaps the people I can talk to
from the people with that power.</p>
<p>A bureaucracy is an inversion of responsibility; instead of an organization
being responsible for provisioning its services to you, <em>you</em> are responsible
for filing the correct paperwork via the correct avenues at the correct time.
Will an organization make it clear to you what services they provide? What
paperwork to file? Where, when, and how to file it? That sounds expensive.</p>
<p>Kafka was powerless to stop the transformation he saw. I think of him often,
when I’m navigating an endless phone tree for the fourth or fifth time, or when
I have an issue with a website and the only way to get support is to know an
engineer working on it.</p>
<p>I’ve set a reminder on my calendar for August 25th, 2025. I’ll see you then.</p>
Announcing ghciwatch 1.02024-07-15T00:00:00+00:002024-07-15T00:00:00+00:00
Rebecca Turner
https://becca.ooo/blog/announcing-ghciwatch/<aside>
<p><em>Note: This article was originally published on <a href="https://mercury.com/blog/announcing-ghciwatch">the Mercury engineering
blog.</a> (<a href="https://web.archive.org/web/20240722213536/https://mercury.com/blog/announcing-ghciwatch">Archived on the Internet Archive Wayback Machine.</a>)</em></p>
</aside>
<p>At Mercury, we’ve developed a file-watching recompiler for Haskell projects
called <a href="https://github.com/MercuryTechnologies/ghciwatch/">ghciwatch</a>, which loads a GHCi session (a Haskell
<a href="https://en.wikipedia.org/wiki/Read%E2%80%93eval%E2%80%93print_loop">REPL</a>) and instructs it to reload or add modules when files in your
project change. Ghciwatch is very similar in design to Neil Mitchell’s
excellent <a href="https://github.com/ndmitchell/ghcid">ghcid</a>, but with a few new tricks up its sleeve:</p>
<ul>
<li>
<p>Ghciwatch is blazing fast, even for the largest Haskell projects, loading our
<a href="https://serokell.io/blog/haskell-in-production-mercury">backend server monolith</a> nearly 12 times
faster than <a href="https://github.com/haskell/haskell-language-server">haskell-language-server</a>.</p>
</li>
<li>
<p>Ghciwatch natively supports <a href="https://mercurytechnologies.github.io/ghciwatch/no-load.html">Cabal’s <code>--repl-no-load</code> option</a>
to massively speed up startup and reload times by only loading the modules
you need.</p>
</li>
<li>
<p>Ghciwatch seamlessly loads newly-created modules and can handle deleted and
removed modules without restarting the underlying GHCi session.</p>
</li>
</ul>
<p>Check out the <a href="https://mercurytechnologies.github.io/ghciwatch/introduction.html">user manual</a> to get started with ghciwatch.</p>
<h2 id="why-not-use-haskell-language-server"><a class=anchor href="#why-not-use-haskell-language-server" aria-label="Link to section why-not-use-haskell-language-server">#</a>Why not use haskell-language-server?</h2>
<p>While HLS works great for smaller projects, the Mercury backend monolith
consists of about 10,000 modules containing about 1.2M lines of Haskell code.
For projects that large, HLS’s performance starts to break down: on my 2022 Mac
Studio (M1 Ultra, 20 cores, 64GB memory), HLS takes a full 32 minutes to load
the project. After it’s loaded everything, making changes to project files will
start reloads that take at least 1.5 minutes (changes to modules nested deeper
in the dependency graph can take as long as 5 or 10 minutes to reload, if HLS
is able to reload at all). HLS is also very memory intensive, consuming 50GB of
RAM after loading the project and climbing from there.</p>
<p>In contrast, ghciwatch takes 2.7 minutes to load the entire project and about 7
seconds to reload. With <a href="https://mercurytechnologies.github.io/ghciwatch/no-load.html">Cabal’s <code>--repl-no-load</code> option</a>,
modules are only loaded when they’re changed: as a result, ghciwatch can start
up in about 6 seconds. The first change to a module will take much longer,
depending on how many modules need to be loaded (between one second and two
minutes), but subsequent reloads take between 0.2 and 7 seconds.</p>
<p>Ghciwatch is a relatively thin wrapper around GHCi, so it’s able to take
advantage of GHCi’s speed directly by interpreting modules, leveraging
<a href="https://well-typed.com/blog/2023/02/interface-files-with-core/">bytecode linking</a>, and keeping artifacts cached more
reliably between runs.</p>
<p>Before building ghciwatch, we spent a while trying to make HLS faster; as of
the time of writing, Mercury is <a href="https://opencollective.com/haskell-language-server">the largest organizational backer of HLS on
Open Collective</a>, and we’ve also <a href="https://well-typed.com/blog/2022/04/hls-performance/">hired contractors like
Well-Typed</a> to optimize HLS. While a number of incremental
improvements have been made, there’s a limit to how fast HLS can get. HLS is
built on top of GHC’s internal API and GHC is a batch compiler, which means
that HLS’s latency and responsiveness can’t really be improved without
implementing a new Haskell parser and type checker with different design goals.
In his excellent <a href="https://matklad.github.io/2022/04/25/why-lsp.html">“Why LSP?”</a> blog post, matklad explains why batch
compilers serve poorly as a basis for a language server:</p>
<blockquote>
<p>[A] batch compiler is optimized for maximum throughput, while a language
server aims to minimize latency (while not completely forgoing throughput).
Adding a latency requirement doesn’t mean that you need to optimize harder.
Rather, it means that you generally need to turn the architecture on its head
to have an acceptable latency at all.</p>
</blockquote>
<p>Practical evidence seems to support this argument:</p>
<blockquote>
<p>Language servers are a counter example to the “never rewrite” rule. [The]
majority of well regarded language servers are rewrites or alternative
implementations of batch compilers.</p>
<p>Both IntelliJ and Eclipse wrote their own compilers rather than re-using
javac inside an IDE. To provide an adequate IDE support for C#, Microsoft
rewrote their batch compiler written in C++ into an interactive self-hosted
one (project Roslyn). Dart, despite being a from-scratch, relatively modern
language, ended up with three implementations (host AOT compiler, host IDE
compiler (dart-analyzer), on-device JIT compiler). Rust tried both —
incremental evolution of rustc (RLS) and from-scratch implementation
(rust-analyzer), and rust-analyzer decisively won.</p>
</blockquote>
<p>Although we’re investigating larger changes to support our Haskell engineers in
the future, we need tools for our engineers to use in the interim. Ghciwatch is
fast enough for local development, and we’ve integrated Joseph Sumabat’s
language server <a href="https://github.com/josephsumabat/static-ls">static-ls</a> and GHC plugin
<a href="https://github.com/josephsumabat/hiedb-plugin">hiedb-plugin</a>, which leverage <a href="https://www.haskell.org/ghc/blog/20190626-HIEFiles.html">HIE files</a> to provide
features like go-to-definition, hover documentation, and even automatic imports
without the lag and memory usage of HLS.</p>
<h2 id="why-reimplement-ghcid"><a class=anchor href="#why-reimplement-ghcid" aria-label="Link to section why-reimplement-ghcid">#</a>Why reimplement ghcid?</h2>
<p>Ghcid is only about 2,000 lines of Haskell, is <a href="https://github.com/ndmitchell/ghcid/issues/346#issuecomment-974824490">no longer being
developed</a>, and is architecturally unamenable to implementing
the features we wanted. Additionally, Rust’s <a href="https://docs.rs/notify/latest/notify/">notify</a> crate is <a href="https://github.com/MercuryTechnologies/ghciwatch/pull/1#issuecomment-1583189642">much
more flexible and mature</a> than Haskell’s <a href="https://hackage.haskell.org/package/fsnotify">fsnotify</a>
package, with features like runtime reconfiguration that make watching and
responding to file changes easy to implement. As a result, we decided to go
with a fresh implementation and build our replacement from the ground up.</p>
<p>One issue we wanted to fix with ghcid is caused by a GHCi bug which renders a
GHCi session unusable after a module is moved or deleted. Ghcid doesn’t
differentiate between file modifications and deletions, so engineers must
manually restart it when modules are moved. In contrast, ghciwatch examines
<em>how</em> files change in order to determine when the underlying GHCi session needs
to be restarted. Ghciwatch also supports a number of <a href="https://mercurytechnologies.github.io/ghciwatch/lifecycle-hooks.html">lifecycle
hooks</a> so that you can use tools like <a href="https://github.com/sol/hpack">hpack</a> to
automatically regenerate your <code>.cabal</code> files before a restart.</p>
<p>Ghcid also assumes that the set of files loaded in the underlying GHCi session
will never change, and as a result it’s unable to add new files when they’re
created. This is another scenario which requires engineers to manually restart
their ghcid session. Ghciwatch tracks the set of loaded files, so it knows when
to <code>:reload</code> the session and when to <code>:add</code> a new module. Ghciwatch also tracks
if each module was loaded as a file path or as a module name to work around
<a href="https://gitlab.haskell.org/ghc/ghc/-/issues/13254">another GHCi bug</a>.</p>
<p>Additionally, we wanted to forward GHCi output to the user as soon as it’s
printed, so that engineers can see the compilation progress during longer
reloads. Ghcid, in contrast, just shows a “Reloading…” indicator during this
process. Printing output for the user requires figuring out when GHCi is
printing a prompt, which is a bit tricky because prompt lines don’t end in a
newline, so a standard line-buffered streams is insufficient here (we solved
this with <a href="https://github.com/MercuryTechnologies/ghciwatch/blob/108f5a5a5e2ce0fb509d61cf1644731281b3bab4/src/incremental_reader.rs">an abstraction we call an <code>IncrementalReader</code></a>).</p>
<p>Ghcid only reloads the session when Haskell files change, meaning it will
ignore changes to other files that impact the build, like <a href="https://hackage.haskell.org/package/persistent"><code>persistent</code>
models</a> or <a href="https://hackage.haskell.org/package/shakespeare"><code>shakespeare</code> templates.</a> While it’s
possible to pass additional filenames to the <code>--reload</code> or <code>--restart</code> options,
ghcid won’t be able to detect newly-created files that match the same patterns.
Ghciwatch <a href="https://mercurytechnologies.github.io/ghciwatch/cli.html#--reload-glob">supports matching files against globs</a> using the full
<code>gitignore</code> syntax so that it knows exactly when to reload and restart.</p>
<p>Like ghcid, ghciwatch is able to <a href="https://mercurytechnologies.github.io/ghciwatch/comment-evaluation.html">evaluate Haskell code in comments for quick
testing and debugging</a>, but ghciwatch’s eval comments are able to access
top-level bindings of the module they’re defined in, including unexported
bindings.</p>
<h2 id="what-s-next"><a class=anchor href="#what-s-next" aria-label="Link to section what-s-next">#</a>What’s next?</h2>
<p>Ghciwatch has fully replaced ghcid in our development environment, but that
doesn’t mean we’re done with it. We’re experimenting with replaying warnings
for stale files, a TUI display that could include a progress bar when modules
are compiling, and an interactive prompt for running Haskell code just like in
an unmanaged GHCi session.</p>
<p>You can try ghciwatch right now; check out the <a href="https://mercurytechnologies.github.io/ghciwatch/introduction.html">user manual</a> and
<a href="https://mercurytechnologies.github.io/ghciwatch/install.html">installation instructions</a> to get started!</p>
Why Git's diff3 merge conflicts are confusing2024-03-28T00:00:00+00:002024-03-28T00:00:00+00:00
Rebecca Turner
https://becca.ooo/blog/why-diff3-is-confusing/<p>It’s popular (and good!) advice to tell people to set <code>git config --global merge.conflictStyle diff3</code> (<a href="https://blog.nilbus.com/take-the-pain-out-of-git-conflict-resolution-use-diff3/">“Take the pain out of git conflict resolution: use
diff3”</a>), but I’ve always found <code>diff3</code>-style merge conflicts pretty
confusing. I’ve never managed to internalize if the top section is from “my
branch” or “their branch”. It turns out there’s a reason for this — <strong>the
first section in a <code>diff3</code> merge conflict is your branch in a merge, but
their branch in a rebase!</strong></p>
<p>If you have <code>feature-branch</code> checked out and you want to update it to the
current <code>main</code> branch, here’s how you’d read a <code>diff3</code> conflict:</p>
<pre data-lang="diff" class="language-diff "><code class="language-diff" data-lang="diff"><<<<<<< HEAD
The "current commit".
For `git rebase main`, this will be the code in `main`.
(Because `HEAD` is on top of `main`.)
For `git merge main`, this will be the code in `feature-branch`.
(Because `HEAD` is on top of `feature-branch`, and `main` is being
merged _into_ it.)
||||||| parent of ea45005
The conflicting lines before either branch changed them.
=======
For `git rebase main`, this will be the code in `feature-branch`.
(Because you're stacking the changes in `feature-branch` on top of
`main`.)
For `git merge main`, this will be the code in `main`.
(Because you're adding the changes in `main` to `feature-branch`.)
>>>>>>> ea45005
</code></pre>
<p>PS: Try <a href="https://github.com/git/git/commit/4496526f80b3e4952036550b279eff8d1babd60a"><code>git config --global merge.conflictStyle zdiff3</code></a>, which moves
as many lines as possible out of the merge conflict.</p>
Rust for Haskellers2023-07-07T00:00:00+00:002023-07-07T00:00:00+00:00
Rebecca Turner
https://becca.ooo/blog/rust-for-haskellers/<p>This is intended to be a brief introduction to <a href="https://www.rust-lang.org/">the Rust programming
language</a> for Haskell programmers who want to work on a Rust project
without too much fanfare. Fortunately, Rust has a lot of features we know and
love from Haskell like Hindley-Milner type inference, typeclasses, and pattern
matching, so aside from method call syntax and semicolons, Haskell programmers
should feel right at home.</p>
<h2 id="getting-started"><a class=anchor href="#getting-started" aria-label="Link to section getting-started">#</a>Getting started</h2>
<p>Rust projects are built with <code>cargo</code> (which is a bit like <code>cabal</code> or <code>stack</code>)
and <code>rustc</code>. The Rust installation is managed with a tool called <code>rustup</code> (like
<code>ghcup</code>), which can be installed at <a href="https://rustup.rs/">rustup.rs</a>, or using <code>rustup-init</code>
(e.g., <code>brew install rustup-init && rustup-init</code>).</p>
<p>Once <code>cargo</code> is installed, you’ll be able to build projects with <code>cargo build</code>.</p>
<p>Many tools that are provided separately in Haskell projects are unified into
<code>cargo</code> in Rust projects; formatting is <code>cargo fmt</code>, linting is <code>cargo clippy</code>,
documentation can be generated with <code>cargo doc</code>, tests are run with <code>cargo test</code>, and so on.</p>
<p>Using the <a href="https://rust-analyzer.github.io/">rust-analyzer</a> language server is highly recommended.</p>
<h2 id="syntax-and-structure"><a class=anchor href="#syntax-and-structure" aria-label="Link to section syntax-and-structure">#</a>Syntax and structure</h2>
<p><a href="https://cheats.rs/">cheats.rs</a> has a fantastic overview to the Rust language with
plenty of links, but here’s the basics:</p>
<p>(Run/modify this code on the <a href="https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=ae0066f38d35df9fc17d0a5e40e8fc64">Rust Playground!</a>)</p>
<pre data-lang="rust" class="language-rust "><code class="language-rust" data-lang="rust">/// A function with no return type returns `()`.
/// These triple-slashed comments are documentation comments, written in
/// Markdown and rendered with `cargo doc`.
fn main() {
// Types are inferred by default for values but mandatory for functions.
let hello = "Hello";
// `println!` is a macro, indicated by the `!` after the name.
//
// This format string is translated into a lower-level format at
// compile-time, and is able to figure out that `{hello}` is the local
// variable `hello`. Just like Template Haskell!
println!("{hello}, world!");
// Now let's create a user.
let user = User {
// Here's where Rust starts to differ from Haskell; a string literal is
// just data in the binary, so we need to copy it into a buffer before we
// can start modifying it.
//
// Rust will check that your memory accesses are safe using linear
// (affine) types.
name: "Rebecca".to_owned(),
age: None,
};
// Again, we need to `.clone()` the `user`, or else it'll be gone after this
// function call. (`how_many_bytes_in_a_name` takes a `User`, not a `&User`
// reference, so it "owns" its parameter.)
println!("My name has {} bytes", how_many_bytes_in_a_name(user.clone()));
match_demo(Some(user));
}
/// Here we're declaring a sum type. `Maybe` is called `Option` in Rust, and it's
/// in the prelude by default.
enum MyOption<T> {
None,
Some(T),
}
/// Records are called `struct`s.
/// We can “derive” instances of traits using a `#[derive()]` attribute; this
/// uses a compile-time macro to compute the requested instance.
/// Here, the `Debug` trait lets us print a representation of the object for
/// debugging, and the `Clone` trait lets us deeply copy the object.
#[derive(Debug, Clone)]
struct User {
name: String,
age: Option<u16>,
}
/// Of course, we can pattern match on values of all sorts:
fn how_many_bytes_in_a_name(User { name, .. }: User) -> usize {
// Strings are UTF-8 under the hood, so getting a count of bytes is O(1)
// and a count of codepoints is O(n).
name.len()
}
/// We can also pattern match using the `match` expression.
fn match_demo(maybe_user: Option<User>) {
match maybe_user {
Some(user) => println!("{user:?}"),
None => println!("No user found!"),
}
}
</code></pre>
<h2 id="types-galore"><a class=anchor href="#types-galore" aria-label="Link to section types-galore">#</a>Types galore!</h2>
<p>Aside from the surface syntax and the fact that Rust is a strict language,
Haskellers should find the language design familiar; Rust is based on the same
<a href="https://en.wikipedia.org/wiki/Hindley%E2%80%93Milner_type_system">Hindley-Milner type system</a> that Haskell uses, and supports sum types and
pattern matching natively.</p>
<p>Rust has a system of typeclasses and instances that’s almost directly copied
from Haskell (Rust calls them “traits”), which even supports fancy features
like associated types.</p>
<p>One of the bigger differences with Rust is its linear type system, which
enforces that values are used “once.” (To be more precise, you can have any
number of immutable references to an object, or one mutable reference, but
never both.) This gives Rust the ability to generate very efficient and
memory-safe code and also equips Rust with a first-class notion of mutability,
which is useful for all the reasons we love immutability in Haskell.</p>
<h2 id="onward"><a class=anchor href="#onward" aria-label="Link to section onward">#</a>Onward!</h2>
<p>Read more of that <a href="https://cheats.rs/">Rust cheat sheet</a>, read <a href="https://doc.rust-lang.org/book/">the Rust Book</a>,
and check out the <a href="https://doc.rust-lang.org/stable/std/">standard library documentation</a>.</p>
No Good Soap Dish2022-09-20T00:00:00+00:002022-09-20T00:00:00+00:00
Rebecca Turner
https://becca.ooo/blog/no-good-soap-dish/<p><em>note: i don’t think i’m going to finish this one.</em></p>
<p>issue with hard soap: it gets wet.</p>
<p>that’s the big thing. you get it wet, and then it’s all soft and it rubs onto
the soap dish. and the soap dish gets dirty as a result.</p>
<p>one might imagine solving this by dripping the moisture into the adjacent
sink. there’s not actually much water dripping off of the soap, though. and
another issue: soap gets very soft.</p>
So You Want to Ship a Command-Line Tool for macOS2022-09-07T00:00:00+00:002022-09-07T00:00:00+00:00
Rebecca Turner
https://becca.ooo/blog/so-you-want-to-ship-a-command-line-tool-for-macos/<p>A word of advice: <strong>don’t.</strong></p>
<p>At work, I’ve written a command-line tool which sets up the developer
environment. It installs the Nix package manager, sets up a local Postgres
instance, and handles all the complex bits of configuration. It does all of
this on Linux and macOS, and it supports <code>bash</code>, <code>zsh</code>, and <code>fish</code> for shell
configuration.</p>
<p>We build and publish releases with GitHub actions, so that engineers can
download and run the latest version of the tool when they need to set up a new
machine or repair the development environment on an existing machine.</p>
<p>On Linux, this is all fine and dandy (<a href="https://nixos.wiki/wiki/Packaging/Binaries">the usual headaches with running
anything on NixOS aside</a>), but macOS requires that
programs be code signed and notarized.</p>
<p>Code signing on macOS is a <em>nightmare.</em> Knowing a little bit about asymmetric
encryption, I expected the process to be roughly like this:</p>
<ol>
<li>Generate a public and private key (this is a blob of binary data, or roughly
“a file”).</li>
<li>Get Apple to make a certificate signed with my public key (this certificate
is also a file).</li>
<li>Run a program, pointing it to the files containing my private key and
certificate, which “signs” an executable for distribution.</li>
<li>When users go to run my program, the operating system can see that it’s
signed with a valid Apple certificate and run it without issue.</li>
</ol>
<p>This is not, in fact, how it works.</p>
<p>The first problem is that Apple provides no less than eight different sorts of
certificates, with little documentation on what they’re used for, so expect to
generate a half-dozen certificates and revoke several before you have a working
one.</p>
<p>The next issue is that the <code>codesign</code> command-line tool has a really opaque and
complex interface. There’s several options you need to actually run code that
aren’t really indicated as such (<code>-o runtime</code> and <code>-timestamp</code>, for instance,
are mandatory for notarization).</p>
<p>There’s few instructions for getting certificates into the keychain without GUI
access (like, for instance, to sign code in CI), and Apple doesn’t have much
documentation for signing anything that’s not built with Xcode.</p>
<p>I was able to solve most of the issues with actually signing code with the
<em>fantastic</em> <a href="https://gregoryszorc.com/blog/2022/04/25/expanding-apple-ecosystem-access-with-open-source,-multi-platform-code-signing/"><code>rcodesign</code> tool written by Gregory Szorc,</a> which has a
very reasonable command-line interface that actually takes all the keys and
certificates as files. I’ll excerpt a paragraph from the linked blog post that
rings true to me (emphasis my own):</p>
<blockquote>
<p>I’ve learned way too much around minutia around how Apple code signing
actually works. <strong>The mechanism is way too complex for something in the
security space.</strong> There was at least one high profile Gatekeeper bug in the
past year allowing improperly signed code to run. I suspect there will be
more: the surface area to exploit is just too large.</p>
</blockquote>
<h2 id="macos-does-not-support-running-command-line-tools-from-finder"><a class=anchor href="#macos-does-not-support-running-command-line-tools-from-finder" aria-label="Link to section macos-does-not-support-running-command-line-tools-from-finder">#</a>macOS does not support running command-line tools from Finder</h2>
<p>Once I had succeeded in signing an executable and verifying it, I immediately
ran into another issue: <a href="https://developer.apple.com/forums/thread/706379">Gatekeeper blocks command-line tools from running when
clicked.</a> Apparently this is a “known bug in macOS”, though
because it’s filed in Apple’s proprietary “Radar” bug-tracker, I can’t see any
of the details or if there are any plans to fix it.</p>
<p>This is especially unfortunate because when an engineer downloads the binary
from GitHub on macOS, the downloads pop up from the bottom of the screen, just
begging to be clicked on.</p>
<p>The solution, Quinn from the Apple Developer Forums tells us, is to “embed your
tool in an application.” This, as far as I can tell, doesn’t work either, but
let’s run through it.</p>
<p>There’s a long guide on <a href="https://developer.apple.com/documentation/xcode/embedding-a-helper-tool-in-a-sandboxed-app">Embedding a Command-Line Tool in a Sandboxed
App,</a> so I followed that, and then slowly, painfully, factored
Xcode out of it, so that I wouldn’t have to figure out how to get a 10GB Xcode
install onto the CI machine (remember, you need to be signed in to an Apple ID
to download Xcode, and there’s no way to do it from the command-line).</p>
<p>With <a href="https://www.witchoflight.com/">Cassie’s</a> help, I produced a short Swift script which (I hoped)
would do what I want: First, it finds the embedded command-line tool binary
with the <a href="https://developer.apple.com/documentation/foundation/bundle/1411412-url"><code>Bundle.url(forAuxiliaryExecutable:)</code></a> method. Then, that
URL is passed to the <a href="https://developer.apple.com/documentation/appkit/nsworkspace/3172702-open"><code>NSWorkspace.open(urls:, withApplicationAt:)</code></a> method to run the embedded command-line
tool in a new window. Finally, the <code>completionHandler</code> closes the app once the
terminal window is open.</p>
<p>(There’s <a href="https://github.com/apple/swift/issues/55127">a bug in Swift</a> that makes the <code>@main</code> attribute,
mandatory for SwiftUI apps, not work, so I needed to run <code>swiftc</code> myself,
adding a mysterious <code>-parse-as-library</code> option to fix the bug. I also needed to
<a href="https://stackoverflow.com/questions/46532610/swiftc-possible-values-for-target-command-line-option">read the Swift source code</a> to determine the possible values for
the <code>-target</code> command-line option.)</p>
<p>Now, we have to code sign:</p>
<ol>
<li>The Swift wrapper script.</li>
<li>The original command-line tool.</li>
<li>The <code>.app</code> containing both of the above.</li>
<li>The <code>.dmg</code> containing the <code>.app</code>, because an <code>.app</code> is just a directory, so
you need to zip it up to distribute it.</li>
</ol>
<p>We also need to notarize the <code>.app</code> and the <code>.dmg</code>. Interestingly, you can only
notarize <code>.pkg</code>, <code>.dmg</code>, and <code>.app</code> files (in <code>.zip</code>s) — command-line tools can
only be notarized if they’re embedded in one of the listed containers.</p>
<p>Most of the notarization docs tell you to use <code>altool</code>, which, only once you
actually get it to upload something for notarization, will tell you that it’s
deprecated and replaced by <code>notarytool</code>. <code>notarytool</code> will print <code>status: Invalid</code> if anything fails, with no additional details. (There’s a separate
<code>notarytool</code> subcommand you can use to fetch logs with more information, but
nothing in the output tells you this is an option, including turning on
verbose/debug logging.)</p>
<p>Anyways, once the app was assembled and passing all of Apple’s validation
tools, it, uh, continued to not work!</p>
<p>You can double click the app to run it, but when it tries to launch the
embedded command-line tool, we get an error that “<code>mytool-cli</code> can’t be opened
because the identity of the developer cannot be confirmed“, followed by “The
application ‘Terminal’ can’t be opened. -128”.</p>
<p>I do get one error with Apple’s tooling on these files; though the <code>.app</code>
itself passes all the checks, the embedded tool fails <code>spctl</code>’s validation:</p>
<pre><code>$ spctl -a -v --raw ./mytool
./mytool: rejected (the code is valid but does not seem to be an app)
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>assessment:authority</key>
<dict>
<key>assessment:authority:flags</key>
<integer>0</integer>
<key>assessment:authority:source</key>
<string>obsolete resource envelope</string>
<key>assessment:authority:weak</key>
<true/>
</dict>
<key>assessment:cserror</key>
<integer>-67002</integer>
<key>assessment:remote</key>
<true/>
<key>assessment:verdict</key>
<false/>
</dict>
</plist>
</code></pre>
<p>Googling for “obsolete resource envelope” gives several different bugs over a
period of years, none of which apply, and anyways <code>spctl -a -v --raw</code> shows the
same error for the <code>/bin/ls</code> that gets shipped with macOS.</p>
<p>That’s where we’re at today. I’ve <a href="https://developer.apple.com/forums/thread/713932">reported this issue on the Apple Developer
Forums,</a> but I don’t expect to get any actionable advice back,
seeing as this distribution pathway is already attempting to work around
several “known” macOS bugs.</p>
<h2 id="the-tools-are-bad"><a class=anchor href="#the-tools-are-bad" aria-label="Link to section the-tools-are-bad">#</a>The tools are bad</h2>
<p>All of these tools and all of their error messages are <em>garbage.</em> (<code>rcodesign</code>,
being not written by Apple, is a notable exception.) Here are some of the
commands you might run to check code signatures:</p>
<pre><code>spctl -a -v --raw mytool.app
codesign -verify -vvvv mytool.app
codesign -d -vvv --entitlements :- mytool.app
codesign -vvvv -R=notarized --check-notarization mytool.app
</code></pre>
<p>Just absolute spews of letters with no discernible meaning and certainly no
reasonable intuition.</p>
<p>On the Apple Developer Forums, <a href="https://developer.apple.com/forums/thread/696235">MirrorMan posts about roughly the same
issue</a> with frustration:</p>
<blockquote>
<blockquote>
<p>If the product is signed, notarised, and stapled correctly, everything
should work. If not, you’ll need to investigate why Gatekeeper is unhappy
(2), fix that, and then retest.</p>
</blockquote>
<p>“Fix that” !!!!!! No! Just no! There MUST be a way for Gatekeeper/spctl to
render an exact description of whether or not it will run something (apps,
command line tools, …) on arbitrary customer machines, immediately, right
away, from the development machine, bypassing any caching or anything else.
How many thousands of 3rd party developer days need to be lost for one Apple
developer to spend a few days to update spctl to produce 100% accurate and
useful messaging? For example, the messaging out of notarytool was very good!
It told me which of the half-dozen hoops I had to jump through next
(keychains! app-specific passwords!). It just takes too too long to do the
research to figure out what to do at all, only to be confronted with an
inexplicable error. $237billion a year, and developers have to guess? Apple
can do better. Sorry for the rant but it is necessary.</p>
</blockquote>
<p>I don’t have much more to add, except that I’ve gained a lot of empathy for my
friends who have turned their backs on the Apple ecosystem entirely for exactly
this sort of developer-hostile behavior.</p>
<h2 id="moving-on"><a class=anchor href="#moving-on" aria-label="Link to section moving-on">#</a>Moving on</h2>
<p>So, what can we do with all of this? We have a few options, but none of them
are particularly appealing.</p>
<ol>
<li>
<p>Maintain the status quo; tell users to <code>chmod +x</code> and <code>xattr -d com.apple.quarantine</code> the downloaded executable. This bypasses the code
signing mechanism but requires the user copy/paste magic commands.</p>
<p>It’s fine, ultimately — the users are engineers, so they can manage a
terminal or we can teach them to — but it’s not particularly “clean”,
especially when Apple (theoretically) offers code signing mechanisms for
this exact purpose, which my employer already pays for.</p>
</li>
<li>
<p>Download the file with <code>curl</code>. Unix command-line tools like <code>curl</code> and <code>tar</code>
don’t add the quarantine bit to the files they create on macOS, so it’s
pretty easy to run software downloaded with them.</p>
<p>Unfortunately, the repository where this tool is developed is private, so
unauthenticated <code>curl</code> downloads won’t work. I’m seriously considering
asking the CTO if we can make the repository public <em>solely</em> so we can use
<code>curl</code> as a distribution mechanism. I could even write a platform-detecting
shell script!</p>
</li>
<li>
<p>Reimplement the entire thing in Swift and distribute it as an App built with
Xcode without any embedded tools.</p>
<p>Hahahaha. Just kidding.</p>
</li>
<li>
<p>Distribute the tool as a custom Homebrew tap or something similar.</p>
<p>This tool is responsible for <em>installing</em> Homebrew, and moreover I work at a
Nix startup so plenty of engineers <em>refuse</em> to install Homebrew. I
think this is a little bit silly, but it’s my job to support them and deliver the
smoothest possible developement experience, so we can’t really distribute
this with Homebrew.</p>
</li>
</ol>
<p>I think I’ll probably distribute a code-signed executable, and macOS will say
it can’t figure out who made it even though their own tools say the signature
is fine, and I’ll cry myself to sleep at night.</p>
<h2 id="conclusions"><a class=anchor href="#conclusions" aria-label="Link to section conclusions">#</a>Conclusions</h2>
<ol>
<li>
<p>Do not attempt to “create software” for macOS. They don’t want you to. If
you want to run your own programs, install Linux and suffer like you’re
supposed to.</p>
</li>
<li>
<p>If you need to code sign software for Apple computers, use
<a href="https://gregoryszorc.com/blog/2022/04/25/expanding-apple-ecosystem-access-with-open-source,-multi-platform-code-signing/"><code>rcodesign</code></a>.</p>
</li>
</ol>
transgender surgery is way too hard to get2022-06-11T00:00:00+00:002022-06-11T00:00:00+00:00
Rebecca Turner
https://becca.ooo/blog/surgery-consult/<p>I’m trying to get an orchiectomy, or rather to decide if I want one. It’s a
procedure where a surgeon removes your testicles, and I don’t know much else
about it because I think surgery is gross and unpleasant.</p>
<p>I’ve been trying to get a consultation with one of the surgeons that does them
and other transgender bottom surgeries, so I can ask some questions about
orchiectomies and how they might interact with future surgeries. And this has
been really hard! I’m used to a hostile medical system but the level of
bureaucracy here has been really astonishing.</p>
<p>I actually did schedule a consultation with one provider very easily, and then
found out that they have had <a href="https://jezebel.com/when-surgeons-fail-their-trans-patients-1844774990">a number of malpractice suits</a> and a
history of litigating against former patients with bad experiences, as well as
a few other stories I won’t repeat here.</p>
<p>Everyone else has been virtually impossible to get ahold of. Yesterday I got an
850-word reply back from <a href="https://www.mozaiccare.net/">MoZaic Care</a>, the practice of Dr. Heidi
Wittenberg, in response to a request for a consultation.</p>
<p>Here’s what I need to talk with a doctor about a transgender surgery, even to
ask basic questions:</p>
<ol>
<li>Insurance card</li>
<li>Photo ID (front and back!)</li>
<li>Vaccination card “if vaccinated” (why ask for it then?)</li>
<li>$250 for a consult for “self-pay” patients (everyone who doesn’t have a few
Californian insurances, and of course “please be advised that Dr. Wittenberg
and Dr. Bonnington has opted out of Medicare”)</li>
<li>Prior authorization from referring physicial or PCP “for your intial
consultation” for patients with “HMO/EPO/Managed Medi-cal/Narrow Network
Plans”</li>
<li>Three letters (from your hormone therapy prescriber, psychiatrist or
psychologist (PhD), and your current therapist): required to “secure a
surgical date” and “recommended” for the consultation. “If you submit your
letters prior to your consultation, please be advised that they must be
dated within 12-months of your surgery date and may need to be revised.”</li>
<li>Ten different intake forms:
<ol>
<li>Optimizing surgical outcome agreement</li>
<li>Medical intake form</li>
<li>Financial policies</li>
<li>Notice of HIPAA privacy practices</li>
<li>Patient record sharing & medical history authorization</li>
<li>Rules of conduct agreement (“Waiting patiently for the physician’s and
staffs’ availability is expected.”)</li>
<li>Postoperative safety agreement</li>
<li>Patient registration form</li>
<li>Authorization for verbal discussion of protected health information</li>
<li>An arbitration agreement, which waives your right to a jury trial.
“Please note, the arbitration agreement must be sent back in color
(please sign with black or blue ink).”</li>
</ol>
</li>
</ol>
<p>And this is all just to get in the door! Seems like the surgeons all know we
don’t have any other choice. Part of me wonders if this process was
intentionally designed to exclude all but the most obedient and well-resourced
trans people, and the other part of me knows that <a href="https://en.wikipedia.org/wiki/The_purpose_of_a_system_is_what_it_does">the purpose of a system is
what it does.</a></p>
<p>I always wonder how cis people seem to come under the impression that
transgender surgeries are easy to come by and accessible to anyone but the
wealthiest trans people. I hope and dream that we can build a world where
transgender people are allowed authority over their own bodies.</p>
<hr />
<p>An anecdote from a friend, for contrast:</p>
<blockquote>
<p>My mom is getting a face lift in the next year, apparently it was absolutely
painless to schedule as long as they know you can pay. It’s wild how cis
people can get gender assuring surgeries no questions asked.</p>
</blockquote>
Git GUIs and why I don't like them2022-05-05T00:00:00+00:002022-05-05T00:00:00+00:00
Rebecca Turner
https://becca.ooo/blog/git-guis/<p>I’m pretty competent with the <a href="https://en.wikipedia.org/wiki/Git">Git</a> command-line interface, and I use it for
most things. But sometimes, especially when I’m navigating a particularly
thorny rebase, I like to be able to browse the commit history and view diffs as
I go. For this I like to keep a Git GUI around, but none of them are
particularly appealing to me. Here are the reasons why, largely for my own
future reference.</p>
<h2 id="gitk"><a class=anchor href="#gitk" aria-label="Link to section gitk">#</a>Gitk</h2>
<p><a href="https://git-scm.com/docs/gitk/">gitk</a> is installed by default with Git (except on, like, macOS where you have
to <code>brew install git-gui</code> separately) and is Perfectly Adequate. It is mediocre
and doesn’t do much but it does show the repo history and the diffs in a split
view, and it works fine.</p>
<p>The diff viewer can ignore whitespace and you can hide added/removed lines. You
can easily copy the commit hash, and you can search for commits with a few
options:</p>
<ul>
<li>Commits containing a string (<code>--grep</code> option to <code>git log</code>, I think)</li>
<li>Commits touching paths (extra arguments to <code>git log</code>, though not sure how
this interacts with regexp filtering)</li>
<li>Commits adding/removing a string (I think this is the <code>-S</code> option to <code>git log</code>)</li>
<li>Commits changing lines matching a string (<code>-G</code> option to <code>git log</code>)</li>
</ul>
<p>Additionally, you can use exact matches, ignore case, or use <a href="https://en.wikipedia.org/wiki/Regular_expression">regexp</a> patterns
for searching, and you can choose which fields of the commit to search (all,
headlinge, comments, author, or commiter).</p>
<p>I’m laying this out as the sort of minimum viable Git GUI feature-set. There
are other things I’d like (for example, easy file history, interactively
expanding context sections, syntax highlighting) but I don’t want anything
there to be <em>missing</em>.</p>
<h2 id="gitfiend"><a class=anchor href="#gitfiend" aria-label="Link to section gitfiend">#</a>GitFiend</h2>
<p><a href="https://gitfiend.com/">GitFiend</a> (I’m trying version 0.32.0) is a free Git GUI for macOS,
Windows, and Linux.</p>
<ul>
<li>No way to view a diff and the commit history at the same time.</li>
<li>No search feature (described as a “work in progress” on the website).</li>
<li>You can view a particular branch, or commits by a particular author. Not very general though.</li>
</ul>
<h2 id="gitup"><a class=anchor href="#gitup" aria-label="Link to section gitup">#</a>GitUp</h2>
<p><a href="https://gitup.co/">GitUp</a> (no version information on the website) is another free Git GUI
for macOS (exclusively?).</p>
<ul>
<li>No way to view a diff and the commit history at the same time.</li>
<li>Does have a search feature! Searches “branches, tags, users, commit messages,
and even commit diffs” with a setting. No way to search one of those
<em>specifically</em>, as far as I can tell.</li>
<li>Has a pane for committing changes, and a pane for viewing stashes. That’s neat.</li>
</ul>
<h2 id="fork"><a class=anchor href="#fork" aria-label="Link to section fork">#</a>Fork</h2>
<p><a href="https://git-fork.com/">Fork</a> is a Git client for macOS and Windows (not Linux?) that costs $50 with a
free evaluation period.</p>
<ul>
<li>Seems fairly polished! None of the weird graphical glitches I’m used to in
the other clients. Animations are smooth and generally unobtrusive, it looks
like an actual Mac App (not sure about Windows), and has most of the features
I want.</li>
<li>Filtering to a specific branch takes me a bit to discover, and the UI sort of
implies it’s part of a generalized commit filtering interface, but there
don’t seem to be any filters other than “the current branch”.</li>
<li>There’s a log of <code>git</code> commands Fork runs, which is an appreciated bit of
insight into what it’s doing.</li>
<li>There seems to be a branch selection widget with a nice search feature and as
part of a separate button a branch creation widget, which looks like it has a
dropdown for more options but doesn’t. Kind of weird.</li>
<li>Does let me view history and diffs at the same time! I can even pop the diff
view out into another window.</li>
<li>The diff interface is actually pretty nice; has a “view entire file” button
and a side-by-side diff mode.</li>
<li>Has a search feature that lets me search commit messages, authors, paths, and
diff content, and lets me filter results to the current branch, but no regexp
search.</li>
<li>Also lets me view the entire file tree at any commit, which is nice.</li>
<li>Doesn’t let me hide the left sidebar, though.</li>
</ul>
<h2 id="tower"><a class=anchor href="#tower" aria-label="Link to section tower">#</a>Tower</h2>
<p><a href="https://www.git-tower.com/">Tower</a> is a Git client for macOS and Windows which costs $70 per year (!). At
least from marketing, it looks like they’re building a fairly well-featured app
with that money. They advertise drag and drop, easy undos, “a unique conflict
wizard”, and file history, all features I want!</p>
<p>The <a href="https://www.git-tower.com/blog/tower-mac-8/">list of improvements in the most recent version of Tower</a> are
also pretty compelling — branch comparison and filtering, the sorts of
interactive query-type features that are really well suited to GUIs.</p>
<ul>
<li>No way to hide the left sidebar.</li>
<li>Kinda weird diff view — you can’t adjust or view context except through a
menu item, there’s not a button nearby or anything.</li>
<li>Lots of interesting features nestled away, including pull requests, but no
way to comment on pull requests even through the built-in GitHub
integration…</li>
<li>Has a “view file at commit” feature but it doesn’t seem to work for me.</li>
</ul>
<p>I was decently excited about this one but nothing really elevates it above
<code>gitk</code> or Fork for me.</p>
<h2 id="smartgit"><a class=anchor href="#smartgit" aria-label="Link to section smartgit">#</a>SmartGit</h2>
<p><a href="https://www.syntevo.com/smartgit/">SmartGit</a> is a Git GUI for macOS, Windows, and Linux that costs $60–90/year or
$264–330 for a lifetime license (with a few options in between).</p>
<ul>
<li>You can hide all the sidebars, and it looks like you can drag them around to
rearrange the panels, but you can’t, which is unfortunate. You can close and
reopen any of them, though, which is nice and more than all the other clients
do.</li>
<li>Decent diff view, lets you expand context lines which is nice but doesn’t
have great syntax highlighting. Has a side-by-side view.</li>
<li>Has an interesting blame-like tool called “investigate” which tries to figure
out the origin of certain changes, letting you go back in the history step by
step. Pretty cool feature with a pretty cool UI, and unique among all these
GUIs too!</li>
<li>Has a nice search feature which does support regexps as well as arbitrary
combinations of authors, committers, commit messages, refs, ID, name, and
content! (Not sure what all of those mean.)</li>
</ul>
<h2 id="gitkraken"><a class=anchor href="#gitkraken" aria-label="Link to section gitkraken">#</a>GitKraken</h2>
<p><a href="https://www.gitkraken.com/">GitKraken</a> is a Git GUI client for macOS, Windows, and Linux (free, $5/mo to
interact with private GitHub repos through GitHub).</p>
<ul>
<li>Gave them a lot of GitHub permissions to sign up, I guess that’s fine.</li>
<li>Has activity/audit logs, I like that!</li>
<li>Appears to have a display scaling feature but only lets you choose amounts
between 80% and 130% in 10% increments?</li>
<li>Lets me hide the left sidebar!</li>
<li>Lets me choose if I want the commit info on bottom or on the right.</li>
<li>Not a ton of UI customization… I can’t really reorganize the panels at all.</li>
<li>Weird shortcuts/keyboard support — I can view commits from a single branch,
but it seems to be a right click action with no shortcut or anything. Feels
like a common operation.</li>
<li>Doesn’t let me view history and a diff at the same time.</li>
<li>Has an interesting split view with a terminal on top and Git info on the
bottom, which has a diff viewer that doesn’t take up the whole screen. I want
that in the main view!</li>
</ul>
<h2 id="gitahead"><a class=anchor href="#gitahead" aria-label="Link to section gitahead">#</a>GitAhead</h2>
<p><a href="https://github.com/gitahead/gitahead">GitAhead</a> was a free Git GUI client for macOS that’s been discontinued as of
September 2021. RIP, it was nice when I used it (but not amazing).</p>
<h2 id="conclusion"><a class=anchor href="#conclusion" aria-label="Link to section conclusion">#</a>Conclusion</h2>
<p>None of the Git GUIs seem that good. At least SmartGit has a feature I can’t
(easily) get with the command-line. Everything else feels like it’s lagging
behind.</p>
installing nixos doesn't have to hurt2022-03-18T00:00:00+00:002022-03-18T00:00:00+00:00
Rebecca Turner
https://becca.ooo/blog/nixos-installation/<p>NixOS is well-known for being user-hostile, and installation is the worst.
Let’s look at a typical installation:</p>
<ol>
<li>
<p>We go to the <a href="https://nixos.org/manual/nixos/stable/index.html">NixOS manual</a>, not to be confused with the Nix
manual (for the Nix package manager specifically) or the Nixpkgs manual (for
the Nix packages architecture and de-facto standard library for the Nix
expression language).</p>
</li>
<li>
<p>We are greeted with a table of contents 3500px tall, which we scroll past.</p>
</li>
<li>
<p>We get to <a href="https://nixos.org/manual/nixos/stable/index.html">part 1, chapter 1, “Obtaining NixOS”</a>, which
immediately tells us that to make a bootable USB drive we need to go to
section 2.5.1 — that’s part 1, chapter 2, section 5 (“additional
installation notes”), subsection 1 (“booting from a USB drive”).</p>
<ol>
<li>
<p>Why is booting from a USB drive — how nearly every desktop or laptop
Linux installation is performed — treated by the manual as of equal
importance to PXE netboot and VirtualBox installations?</p>
</li>
<li>
<p>Why is the first step in “Obtaining NixOS” an indirection to the middle
of “installing NixOS”? Shouldn’t creating installation media come before
installation? Is a bootable USB drive really worth relegating to
“Additional installation notes”? The Fedora project offers a bootable USB
creator for Windows and MacOS, and bootable ISOs for Linux.</p>
</li>
</ol>
</li>
<li>
<p>Okay, we go through and make our bootable USB and boot from it. Now we’re
instructed to run <code>systemctl start display-manager</code> to start the graphical
environment. Shouldn’t the graphical environment start itself, if it’s
available? Again, no other distro makes you have to <em>know</em> the command to
start the display manager.</p>
</li>
<li>
<p>Next we’re told we’ll need networking to run the installer, which gives us a
whole host of conflicting commands to run:</p>
<ul>
<li>“check <code>ip a</code>” to see if networking is up, but no explanation what
successful/unsuccesful output looks like.</li>
<li>If we don’t have a DHCP server on our network, “configure networking
manually using <code>ifconfig</code>” (no mention of <code>man ifconfig</code>)</li>
<li>On the graphical installer we can configure the network through
“NetworkManager”. (Is that a command-line program to run?)</li>
<li>In non-graphical settings we are told we can use <code>nmtui</code> to configure
NetworkManager.</li>
<li>To configure the network manually (like it said to do with <code>ifconfig</code>?)
we’re told to run <code>systemctl stop NetworkManager</code>.</li>
<li>If NetworkManager is not available, configure with <code>sudo systemctl start wpa_supplicant && wpa_cli</code>, and then shows commands that are sufficient
for “most home networks”.</li>
</ul>
</li>
<li>
<p>Then we walk through manually partitioning. We can use any of <code>parted</code>,
<code>fdisk</code>, <code>gdisk</code>, <code>cfdisk</code>, and <code>cgdisk</code>! Again, the manual expects us to be
an expert and know which one is best for our use-case.</p>
</li>
<li>
<p>I could go on. The manual also expects us to decide between UEFI/GPT or
BIOS/MBR, we need to configure swap, we need to edit a text file on the
command-line…</p>
</li>
</ol>
<p>Throughout the process, the NixOS manual expects every user to be an expert and
requires a lot of decisions in response to the current environment (e.g. “in
the graphical installer, do this, otherwise, do this…”).</p>
<p>I propose writing a tool which can perform the bulk of these steps (configuring
the network, partitioning disks, generating a <code>configuration.nix</code>)
automatically or with minimal user input, like a wizard.</p>
<p>Will I do something about this myself? Perhaps, perhaps.</p>
against discord channels2021-11-02T00:00:00+00:002021-11-02T00:00:00+00:00
Rebecca Turner
https://becca.ooo/blog/against-discord-channels/<p>If you were to log on to my personal Discord server and glance at the unhideable
channel list side-panel, you would see one text channel, one voice channel, and
nothing else. Many people find that surprising, and most Discord servers I see
have an absolute plethora of channels, often neatly arranged into groups —
as an example, one server I’m in has 43 members and 21 text channels arranged
in five groups.</p>
<p>Before looking at the <em>effects</em> of using channels, and especially using a lot of
channels, let’s look at <em>why</em> channels are used. Knowing the rationales for
using channels allow us to ask if channels are serving their purpose, or what
a feature designed specifically for that purpose could look like.</p>
<ul>
<li>channels demarcate topics, so that people can choose what to look at
<ul>
<li>but do you really want to draw borders around a fixed and unmoving set of
allowed topics? second-order effect: having a #games channel and not a
#movies channel discourages discussion of movies because there’s not a
“proper place” for it</li>
<li>channels are used like this for a few reasons:
<ol>
<li>categorization is tempting! “organizing” things feels good! but we must
be careful to organize with a purpose, especially when we’re organizing
social spaces and the discussions within them.
<ul>
<li>it’s easy to start categorizing innocently, and it only takes a few
specific channels before users expect them for any discussion</li>
</ul>
</li>
<li>users see it in other discord servers and copy it without thinking</li>
<li>few people expect the structure and feature of a messaging app to have a
powerful effect on the way they socialize and communities form in it</li>
</ol>
</li>
</ul>
</li>
</ul>
<p>Are channels good at that?</p>
<ul>
<li>discord servers are usually social spaces; I’m more interested in the people
in them than the particular topics they’re talking about.
<ul>
<li>a group of people choosing specialized topics to participate in sounds more
like a <a href="https://en.wikipedia.org/wiki/Working_group">working group</a> than a social space
<ul>
<li>a topic-based channel is basically a subreddit — fine for many
circumstances, but not great for connecting with people as <em>people</em>
<ul>
<li>i like people’s personalities, the different ways they approach
situations, their experiences, etc. a lot more than i like their
engagement with a particular topic</li>
</ul>
</li>
</ul>
</li>
<li>and in that case, more channels is just more places to click before i can
find the discussion
<ul>
<li>that makes channels burdensome in small discords (with only a few dozen
active members) that tend to not have more than one or two discussions going
at a time</li>
<li>contrast with slack channels, which look similar (sidebar on the left,
names start with a <code>#</code>) but are explicitly <em>opt-in</em> — you don’t <em>want</em>
to be in every slack channel in a workspace, and you usually don’t want to
keep up with the discussions in all of them either.</li>
</ul>
</li>
<li>when is it actually important to separate discussions or hide channels?
<ul>
<li>so that people can avoid certain topics (nsfw material, medical talk, etc.)
<ul>
<li>this is needed because discord lacks tagging, content warning features,
consent screens except for nsfw channels (which, notice, require a
separate channel), etc.</li>
</ul>
</li>
<li>so that a particular discussion doesn’t overwhelm the rest of the space</li>
<li>to prevent any one channel from becoming too busy in large servers</li>
</ul>
</li>
</ul>
</li>
</ul>
<p>What are the effects of using a lot of channels — that is, what happens when a
social space has about half as discussion-slots as it does people?</p>
<ul>
<li>
<p>channels atomize discussion: most channels are rarely used</p>
<ul>
<li>eventually, users expect a channel for each discussion topic & #general is
abadoned</li>
<li>each additional channel <em>inherently</em> makes the rest of them slower and
weaker; unlike IRC channels or Slack channels, each channel in a discord has
exactly the same members, so each new channel divides the attention of all
the members one more way
<ul>
<li>you can technically make channels visible only to members with certain
roles, but it’s so much of a pain to manage i’ve only seen it done once or
twice, usually as a way to verify that only trusted users have access to the
server to protect against leaked invite likes</li>
</ul>
</li>
</ul>
</li>
<li>
<p>channels make it hard to follow discussions.</p>
<p>first, even finding out where the discussion is happening is difficult,
because channels are opaque; discord doesn’t have any sort of “this channel is
home to 50% of the server’s activity” or “10 people messaged this channel in
the last few hours” indicator, so you have to go through each one individually
and check. finding out where activity is becomes much harder when i have to
scroll up and down a partially-hidden channel list (with an auto-hiding
scroll-bar, so there’s no indication that it’s partially hidden) to see which
channel is the source of a new notification.</p>
<p>even once you’ve figured out which channel has activity, catching up becomes
harder and harder with each new channel. here’s the rough process:</p>
<ol>
<li>open channel</li>
<li>re-read a few old messages to remember context</li>
<li>read the new messages to catch up</li>
</ol>
<p>steps 1 and 2 have to be repeated for each channel. especially when lots of
channels are used with few people, there might be a half-dozen channels with a
few new messages each, so time spent remembering the previous discussion (and
context-switching, which is much harder with adhd) quickly begins to outpace
the time it takes to actually read the new messages and respond to them.</p>
<p>storing context for “where the discussion left off” is a lot harder when 40
people are having 20 discussions than it is when 40 people are having 2 or 3
discussions. it’s much easier to read a few, more-active channels.</p>
</li>
</ul>
<p>“My server is too big to not have channels — you can’t usefully stuff a
thousand people into a single channel productively. (As evidence, look at Twitch
streams!)”</p>
<ul>
<li>discord just doesn’t scale well, because it lacks good threading and
discussion-organizing features
<ul>
<li>and that’s okay, for the most part, because it’s an instant messaging app,
not a customer support forum, not a wiki, not a documentation repository,
not a zulip, etc.</li>
</ul>
</li>
</ul>
type-driven computing2021-02-01T00:00:00+00:002021-02-01T00:00:00+00:00
Rebecca Turner
https://becca.ooo/blog/type-driven-computing/<p>I really like command line user interfaces. I think they live up to a sort of
<em>conversationality</em> that computing often lacks. In the <a href="https://en.wikipedia.org/wiki/Desktop_metaphor">desktop
metaphor</a>, stacked GUI windows often feel like a clutter of
unorganized papers, whereas the command line — at its best — presents an
ordered ledger of queries and responses.</p>
<p>But the command line hasn’t really grown with us. Instead, it’s been left to
linger, forever emulating <a href="https://en.wikipedia.org/wiki/VT100">the VT100</a> — state of the art, for 1978.
I’m pretty frustrated with that, and though I’m hardly the first (Gary
Bernhardt explores some similar ideas in his 2012 talk <a href="https://www.destroyallsoftware.com/talks/a-whole-new-world">“A Whole New
World</a>), I often feel like the people I talk to,
technically-inclined or not, don’t really understand what I’m thinking about
when I talk about the ways computers fail me.</p>
<p>Therefore, here are some ideas I have for a better command line. Let’s
establish some scope and basic points first:</p>
<ul>
<li>
<p>Backwards compatibility isn’t a concern here. I’m not presenting a business
plan, I’m presenting a vision. I’m not interested in the implementation, how
to build an ecosystem around this concept, the way capitalism would hinder my
concepts, or anything else. I <em>know</em> why these systems don’t exist. Let
yourself imagine, just for a bit, that we could truly build something new.</p>
</li>
<li>
<p>The vision is inspired by command lines and type systems, but stretches
beyond a single terminal program. Ultimately, I’m talking about a different
way to <em>use</em> and <em>interact with</em> computers. As with all things in
computation, this idea works better the more tightly it’s integrated with its
surroundings.</p>
</li>
<li>
<p>I am uninterested in the grid-of-characters model for terminals. I don’t
believe the properties it offers are worth the drawbacks.</p>
</li>
</ul>
<h2 id="programs-as-types"><a class=anchor href="#programs-as-types" aria-label="Link to section programs-as-types">#</a>Programs as types</h2>
<p>When we run a command line program, we’re usually constructing an instance of a
<em>type</em> representing the invocation we want.</p>
<p>To make this concrete, let’s think of a program as a sum type, where each
subcommand corresponds to a variant. With <code>git</code>, for example, we can run the
<code>status</code> or <code>commit</code> subcommands, but we can only choose one at a time:</p>
<pre data-lang="bash" class="language-bash "><code class="language-bash" data-lang="bash">git status
git commit
</code></pre>
<p>These correspond to variants of a sum type (tagged union). These sum types can
be nested, even, for sub-subcommands and more:</p>
<pre data-lang="bash" class="language-bash "><code class="language-bash" data-lang="bash">git remote add ...
git remote set-url ...
</code></pre>
<p>For product types (records/structs), we have options/flags, which have a long
form (like <code>--long</code>, or <code>--untracked-files</code>) and sometimes a short form (like
<code>-u</code>), along with a variety of conventions for boolean switches, reading
options from files, and so on.</p>
<p>Writing the type out explicitly in Rust syntax looks kinda like this:</p>
<pre data-lang="rust" class="language-rust "><code class="language-rust" data-lang="rust">// Git is run by specifying an (optional) command and other configuration:
struct Git {
command: Option<Command>,
configuration: Configuration,
}
// We have some common options for any `git` command:
struct Configuration {
exec_path: Directory,
paginate: bool,
work_tree: Directory,
// ...
}
// Then, individual commands have their own options and subcommands:
enum Command {
Add {
verbose: bool,
dry_run: bool,
interactive: bool,
// ...
},
Remote {
Add { /* More configuration options... */ },
Rename { /* ... */ },
// ...
}
// ...
}
</code></pre>
<p>And we have ways of setting up this sort of type system, either by writing a
type directly (like Rust’s <a href="https://docs.rs/structopt/">structopt</a>) or by setting the
subcommands and constraints directly (like Python’s <a href="https://docs.python.org/3/library/argparse.html">argparse</a>).</p>
<p>And then… we throw all that information away and write completely separate
tab-completion scripts for various shells, which are usually pretty bad.</p>
<p>Although we have a pretty consistent system for understanding command line
arguments as types, both as users and as programmers, the actual user
interface, the shell, has <em>no knowledge</em> of this system whatsoever — which is
a shame because it keeps us from applying the tools we’ve built for creating
and filling out forms and other sorts of records to the problem of running
programs.</p>
<h2 id="building-usability-and-documentation-into-the-system"><a class=anchor href="#building-usability-and-documentation-into-the-system" aria-label="Link to section building-usability-and-documentation-into-the-system">#</a>Building usability and documentation into the system</h2>
<p>What could a shell with a canonical view of this type information do?</p>
<p>When you’re thinking about it, don’t imagine an extra step of processing, like
the <a href="https://fishshell.com/">fish shell</a> parsing man pages to find command line arguments
for programs that bother to distribute a man page. That’s a bolted-on solution.
A hack. It works decently in an environment filled with legacy programs, but
we’re not worrying about that right now. Instead, imagine if the command line
interface and the longer-form man page documentation came from the same source
of truth in the program, guaranteeing that they never fall out of sync, because
adding an option to the program and adding it to the documentation are the
<em>same</em> action.</p>
Slack’s threads are terrible for accessibility2020-10-23T00:00:00+00:002020-10-25T00:00:00+00:00
Rebecca Turner
https://becca.ooo/blog/slack-threads-are-terrible-for-accessibility/<p>Slack’s lackluster implementation of threading shapes the rest of its
collaborative features, largely nullifying them. Slack gives users the
<em>ability</em> to reply to messages, creating a thread. I say <em>ability</em> because
that’s all it is: a possibility that technically exists.</p>
<p>But what do Slack’s threads give us, and what do we want from them? They can’t
be organized; for one, there’s no way to move a thread from one channel to
another where it might more accurately belong, and there’s also no way to
categorize, sort, or browse threads. Slacks’ threads are branches off of a
<em>conversation,</em> not groups and bases of knowledge like they can become in a
<a href="https://www.phpbb.com/">phpbb</a> or similar. For instance, threads can’t be tagged, and it’s
difficult to add additional context to a thread — messages can’t be pinned to a
thread, not that it would really matter, because there’s no way to search a
thread. The link between a thread and its channel is tenuous as well; there’s
no notifications for new threads or messages in threads you’re not “following,”
so unless you never miss a Slack message (difficult in general, especially if
you’re attempting to enforce any sort of work-life boundaries) it’s easy to
miss even large, active threads. On the flip side, when participating in a
thread, replies are easily lost for the rest of the channel among a flurry of
other messages unless you (tediously) check “also send to channel” every time
you message — and these downfalls are compounded by the plethora of niche
single-purpose channels Slack encourages users to create; users have to check
each channel manually without the advantages of a unified feed where they can
see, sort, and filter all messages.</p>
<p>It’s not like I’m the first person to point out these flaws, either;
<a href="https://twist.com/home">Twist</a>, the Slack-replacement messaging and communication software from
Doist (the company behind <a href="https://todoist.com/home">Todoist</a>), capitalizes on Slack’s
shortcomings by prominently advertising searchability, organization, threading,
and archival features in Twist — it’s just too easy to lose information in
Slack.</p>
<p>Slack’s tools for coreferencing — that is, enabling two or more people to
“point to” the same thing — are rudimentary at best, largely because Slack has
no concept of context within the application. There’s no concept of tabs and
the “history” feature, which I discovered while writing this article — is
next-to-useless because it browses through the entire application’s state;
there’s no way to go back and forward within a chat, panel, or sidebar. Even
early browsers understood that we’d want to look at multiple webpages at once
by opening multiple windows, but modern communication applications refuse the
possibility entirely — Slack, Discord, Signal, Telegram, and even Tinder
flat-out-refuse to be viewed twice at once; the idea that a user would want to
carry on multiple conversations at once without having to switch windows or
views has seemingly never occurred to the designers or developers.</p>
<p>As a result, coreferencing is nearly-impossible, because Slack and its ilk
prevent users from talking to a person or group of people while looking at
something else — and especially while looking at multiple other things.</p>
<p>These shortcomings seem particularly ironic given that Slack is allegedly
designed to “bring teams together” and to enhance and streamline collaboration
within corporations — did Slack’s developers not notice all the ways their own
application got in the way of actually collaborating? (It wouldn’t be too
surprising, though, coming from a company which branded itself an expert on
remote work at the start of a pandemic despite having previously forbidden
remote work entirely.)</p>
<p>Slack compounds this effect by presenting notifications in a chaotic, largely
unordered interface, where there’s no way to check items off (or control how
items get checked off), where each notification is presented largely without
context (because — remember? — threads can’t be titled, renamed, or tagged),
and where notifications are scattered across the application in an
unpredictable, high-effort spread — threads, mentions, and reactions only
account for a small portion of what I’m expected to “keep up with” in a Slack
workspace; I need to check each channel with new messages (which, usually, is
all of them) manually, as well as direct messages, and perhaps I additionally
have to follow up on other tasks that people haven’t replied to (and, of
course, Slack gives me no tools for managing these).</p>
<p>It’s frustrating to be surrounded by tools and systems — particularly those,
like Slack, that I’m forced to use for school or work, and particularly those,
like Slack, which have proprietary applications and no other way to connect
through more accessible services (like the IRC bridge <a href="https://web.archive.org/web/20180307125841/https://get.slack.help/hc/en-us/articles/201727913-Connect-to-Slack-over-IRC-and-XMPP">that Slack shut down in
2018,</a> long before <a href="https://web.archive.org/web/20190325002116/https://www.reddit.com/r/Blind/comments/80ql6f/slack_taking_steps_to_improve_screen_reader/">even attempting to make the first-party
Slack applications usable for blind people with screen
readers</a>).</p>
rustconf2020-08-18T00:00:00+00:002020-08-18T00:00:00+00:00
Rebecca Turner
https://becca.ooo/rustconf/<p><a href="/rustconf/2020/">rustconf 2020: “Rust for Non-Systems Programmers”</a> (24 minutes)</p>
Open letter: Brandeis is failing its disabled students2020-02-07T00:00:00+00:002020-10-28T00:00:00+00:00
Rebecca Turner
https://becca.ooo/blog/brandeis-is-failing-its-disabled-students/<p><em>Note: This letter was originally published in <a href="https://brandeishoot.com/2020/02/07/open-letter-brandeis-is-failing-its-disabled-students/">The Brandeis Hoot</a>. The
letter itself is a near-exact copy of an email I sent to interim student
accessibility services director Scott Kalicki, explaining why I was
uninterested in arranging a closed-door meeting where my concerns would
be Heard and then promptly and completely forgotten. Unfortunately, I suspect
that these complaints will already be familiar to disabled students at most
other US universities.</em></p>
<p>Dear Brandeis Administration, Board of Trustees, Office of Budget and Financial
Planning, Student Accessibility Support and Counseling Center,</p>
<p>Thanks for asking me what I need as a disabled student to feel welcome and
succeed here. I need actual support — in the form of professionals advocating
for me and my right to participate and belong in class and executive function
counseling. It’s immensely disappointing that neither our “<a href="https://www.brandeis.edu/academic-services/accessibility/index.html">Student
Accessibility Support</a>” nor our mental health services on campus are equipped to
help with executive functioning, so that’s out, which leaves us with advocacy.
You’ve heard me out asking for accommodations, and I’m grateful for that, but I
require more. I’m still not able to participate in my classes meaningfully; I
still need more to be done. And, too much of the time, your priority during our
conversations is to lead me back toward our disappointing and lackluster set of
“pre-prepared” accommodations, rather than actually making change to construct
an environment in which I can succeed.</p>
<p>Student Accessibility Support (SAS) has an obligation to provide me the
necessary support to succeed at Brandeis — I’d love to receive that support.
But before I come to the table, I want some guarantee that SAS will be working
for me, and on my behalf, to provide me with accommodations — not on behalf
of the administration and a board that sees Brandeis as an investment vehicle
more than an educational institution existing to make as little change as
possible. I’m sick of professors not knowing what my accommodations are because
nobody bothered to tell them and/or they didn’t bother to read their letters;
I’m sick of last-minute preparations for exams planned months in advance; I’m
sick of accommodations making my life harder; I’m sick of being left behind and
left out of the classes I’m going five figures into debt to participate in.</p>
<p>Are you going to work with me and my disabled peers to give us the learning
environment we were promised? Or are you going to work with the administration
to preserve university profits and avoid making changes for the disabled
“problem students” who are too mad to shut up and accept the scraps of
accommodations they’ve been offered by an unprofessional, understaffed, and
reactionary organization?</p>
<p>I want to know what you’re doing to change SAS. It’s falling short in a lot of
catastrophic and obvious ways, as <a href="http://brandeishoot.com/2018/11/09/letter-sent-to-liebowitz-calls-for-reform-on-accessibility/">last year’s open letter to President Ron
Liebowitz</a> showed. What are you doing to enact the
institutional-level change that needs to happen in order for disabled students
at Brandeis to be given the opportunities and access we’ve been promised? Our
time here is limited. My time here is limited. What have you been doing? What
will you do to fix the enormous failures Brandeis has demonstrated to its
disabled students before I graduate in a year? And if it’s “impossible” to fix
these failures, I want to know why. I want to know why it is acceptable for me
to be collateral damage, and I want to know why we’re okay, both personally and
institutionally, with disabled students falling through the cracks, with our
education being manufactured to fail and thoughtlessly thrown away.</p>
<p>Meeting with staff and administration — deans, presidents, directors —
takes time and energy from me — precious, valuable resources that are
intensely limited due to my disability. I want some assurance that we can do
more than offer blank platitudes to each other. Spending my energy fighting for
accommodations makes it harder to attend classes and focus on my work here. I
don’t want explanations for why the system isn’t working; I want actions that
fix it. Here’s a tiny list of a few of the accommodations and changes I want,
off the top of my head. Make no mistake, this list is scattered and incoherent.
It’s scattered because that’s all I can muster. That’s all I can get out of my
brain today. In the words of Porpentine, <a href="https://thenewinquiry.com/hot-allostatic-load/">“I am too sick to write this
article.”</a> All I can squeeze out between sickness and
trauma and therapy and an unending stream of homework and essays and
obligations. It’s not a comprehensive and formal program of change to fully
reform Brandeis into a functional institution. It’s a set of specific failures
that I want addressed 30 years ago — or, if Brandeis is aiming for second
best, — today. These failures are overlapped and intertwined.</p>
<p><span id=demands></span>
Here’s what I want:</p>
<ol>
<li>
<p>Material acknowledgment that I had to and have to work harder than my abled
peers for the time at Brandeis when I didn’t have accommodations, and
during the times when my accommodations have been inadequate or a majority
of my energy has been spent attempting to navigate student accessibility
(i.e. my entire college career so far). This could be in the form of extra
credits awarded for each class (to reflect the additional effort and time I
had to put in, compared to my peers) or in the form of a flat increase of
the grades for each class (to reflect the limited time and resources I had
to work with, particularly during exams, compared to my peers). I’m open to
other options, but those two seem like the most obvious, reasonable,
inexpensive, and non-disruptive.</p>
</li>
<li>
<p>An alternative schedule that would allow me to focus intensely on one
subject, because my disability prevents me from jumping around and managing
four or five different schedules and four or five different professors who
like to communicate in four or five different ways. A schedule that lets me
dive deeply into the material and excel in a way that a scattershot choice
of four or five different areas of knowledge prevents.</p>
</li>
<li>
<p>Accommodations that focus on areas other than test-taking. Dramatically
restructured lectures, with slides that present a coherent narrative for
students to follow. Slides available online before the lectures.</p>
</li>
<li>
<p>Courses restructured for chronically ill students who need to miss class on
a nearly weekly basis or greater, allowing them to participate without
excluding them or leaving them behind.</p>
</li>
<li>
<p>A 30-student cap on every course, including introductory courses. This
would mean splitting large courses into as many sections as we require and
hiring additional faculty to teach those courses, focusing on
underrepresented and marginalized academics.</p>
</li>
<li>
<p>Failing to provide accessible and thorough course materials (including
interactions through LATTE, timely responses to emails, assignment
postings, syllabi, course texts, etc.) should be a tenure-revoking offense
— just like failing to meet any other job responsibilities would be.</p>
</li>
<li>
<p>Sign language interpreters, large print, and braille materials available
on request without needing a documented disability or any form of
means-testing.</p>
</li>
<li>
<p>Banning “no-electronics” policies that single out disabled students who use
electronics as access tools and exclude disabled students who are not rich
enough for a diagnosis. New policies which explicitly recognize that nearly
all electronics, including cell phones, can be and often are access tools.</p>
</li>
<li>
<p>Assignments to be posted to <a href="https://www.brandeis.edu/its/services/software-business-systems/latte/index.html">LATTE</a> (the name of Brandeis’
<a href="https://moodle.org/">Moodle</a> instance) in a timely fashion — in general, for LATTE to
actually be used promptly and correctly. Failing to put all the information
about my assignments and courses in one place (e.g. with hidden syllabi,
alternative course websites, critical announcements sent through emails and
not otherwise posted to the course page) makes coursework unmanageable and
leaves me (and in general, the most vulnerable students) behind.</p>
</li>
<li>
<p>I want better syllabi from all my professors, including all texts,
assignment dates, and lecture material (at least down to the nearest week)
to allow me to prepare for my courses. I want this information without
having to email back and forth or meet individually with professors (as I
did with every professor this semester).</p>
</li>
<li>
<p>I want an end to requirements for medical documentation to receive
accommodations. If Google and JP Morgan can give me ADHD assessments poorly
disguised as “personality quizzes” that they can then use to discriminate
against me in the hiring process, we can figure out some way to make
accommodations easier to access. (And yes, both of them did do that to me.
I even kept screenshots.)</p>
</li>
<li>
<p>Increased student wages. My summer internships pay more than three times
what I make at Brandeis. (An immense privilege!) Why doesn’t Brandeis value
my talents? Increased pay would let me devote more of my time to keeping up
with the classes I struggle with.</p>
</li>
<li>
<p>An independent council of disabled students and faculty members with a
budget and the powers to enact changes to accommodations and university
policy and to terminate professors with a history of misconduct.</p>
</li>
<li>
<p>A safety net for all students, but in particular new students. My
first-year advisor didn’t meet with me or reply to my emails for an entire
semester, leaving me to figure out everything on my own. That’s water under
the bridge for me, but I don’t want it to happen to anyone else.</p>
</li>
<li>
<p>I want professors who want to teach. Professors who are hostile to students
who ask questions or express interest should not be tolerated. It doesn’t
serve students for the administration to force professors who only want to
do research to teach courses. Similarly, I want Brandeis to hire professors
based partially on their skill as educators, not exclusively on their
expertise in their field. Given that the vast majority of courses are
taught at the undergraduate level, it doesn’t make sense to prioritize
incredibly specialized knowledge over the ability to share the basics
within each field.</p>
</li>
<li>
<p>Mandatory accessibility training for every staff and faculty member on
campus, so that I never get a look of surprise (or get scolded in front of
a class) when I use an accommodation again. I should not need to explain my
disability or my accommodations to professors who are capable of talking to
SAS directly, because my time and energy is precious and limited. Having
SAS “encourage that I self-advocate” is infuriating and counterproductive
and has resulted in accommodations nearly falling through multiple times.</p>
</li>
<li>
<p>I want at least half of the SAS staff to be disabled, and for a dramatic
increase in SAS staffing. Roughly <a href="https://www.cdc.gov/media/releases/2018/p0816-disability.html">1 in 4 American adults live with a
disability</a>. Brandeis has <a href="https://www.brandeis.edu/about/facts/index.html">a population of
5,800</a> students, and let’s say that a single employee
can effectively manage accommodations for about 50 students (probably
stretching it). That works out to 29 SAS employees. Why do we not have that
currently?</p>
</li>
<li>
<p>Similarly dramatic increases to <a href="https://www.brandeis.edu/counseling/">Brandeis Counseling Center (BCC)</a>
staffing, reflecting the inability of mental health workers to adequately
care for caseloads larger than 15 or 20 people at a time. An end to the
policies that use group therapy to triage “low-priority” students. A
dramatically increased budget. More than one trans therapist. Disabled
therapists and an actually accessible building. An end to waiting lists for
intake appointments in their entirety. Therapists specializing in learning
disorders, ADHD, psychosis, OCD, PTSD and complex post-traumatic stress
disorder (CPTSD), personality disorders, dissociative disorders, and
<a href="https://www.pluralpride.com/playbook">plurality</a>. Therapists working in politically aware frameworks who
are able to adequately address external stressors in a way that cognitive
behavioral therapy (CBT) does not. Therapists who don’t reproduce trauma
within their sessions, in particular with disabled students, trans students
and students of color.</p>
</li>
<li>
<p>Admissions quotas for physically and mentally disabled students
commensurate with the general population. Admissions quotas for students of
color (and in particular for Black and Native students) and poor students
commensurate with the general population. I want the unbalanced student
population corrected in four years and the staff and faculty demographics
corrected in six years.</p>
</li>
<li>
<p>Increased transportation options on campus as well as to Waltham and
Boston/Cambridge, running at least twice as frequently. The ability to
reserve space on a bus at any time (arbitrarily long or short) in advance
of the bus’s journey with decreased barriers to entry and support for
mobile devices in order to make driving a car no longer a necessity for
students who live off campus.</p>
</li>
<li>
<p>Wheelchair accessibility for every floor of every building on campus —
including <a href="https://www.brandeis.edu/skyline/">Skyline</a>, which, despite the tour guides’ brags that
it’s <a href="https://www.brandeis.edu/skyline/about.html">the most accessible building on campus,</a> has heavy,
difficult-to-open, manual interior doors. I want tour guides to announce
every building that isn’t wheelchair accessible when showing the campus to
prospective students. And I want real accessibility, not just “well
technically you can get to <a href="https://www.brandeis.edu/about/visiting/map.html?bldgid=0067">Goldsmith</a> 300 by entering <a href="https://www.brandeis.edu/volen/">Volen</a> across the quad,
going down a long hallway, taking an elevator, crossing the skybridge, and
passing through several heavy and non-automatic (i.e. not wheelchair
accessible) doors”-style accessibility. I want the wheelchair-accessible
routes through campus to be direct and convenient, not coiled through back
hallways like they were designed by Wile E. Coyote.</p>
</li>
</ol>
<p>I’m tired of the avoidant non-responses I’ve gotten from SAS in the past year,
and I’m tired of wasting my breath in closed-door meetings. I hope we can make
some broader changes to allow me to succeed here, as well as my disabled peers
and future students, many of whom are not as eloquent or as aware of their poor
treatment as I am.</p>
<p>The last time I met with SAS, I was told that “there’s not a brick that doesn’t
move” when it comes to accessibility. Here are some bricks. Let’s get moving.</p>
Introduction to functional programming in Java2020-01-10T00:00:00+00:002020-01-10T00:00:00+00:00
Rebecca Turner
https://becca.ooo/blog/functional-programming-in-java/<p><strong>Note:</strong> This article was originally written for an undergraduate course on the
Scheme textbook <a href="https://en.wikipedia.org/wiki/Structure_and_Interpretation_of_Computer_Programs"><em>Structure and Interpretation of Computer Programs</em></a>
(SICP). Students entering this course typically had 1–2 semesters of experience
programming in Java and we found that many students could solve the course
excercises in Java, but failed to translate the code to Scheme. This article
helped bridge the gap between the courses to show students how to write programs
that accomplish the same tasks using different techniques and languages, as well
as introducing them to some basic functional programming concepts.</p>
<p>This is a new piece of documentation, written just for this course, so let me
know if parts of it are unclear to you — or if you like it! Expect to read it
fairly slowly (like any technical writing), and consult the <a href="https://docs.oracle.com/en/java/javase/12/docs/api/">Java
documentation</a> if there’s methods you don’t recognize — though the most
important ones should be linked.</p>
<p>A significant portion of this course is about learning functional programming
techniques; functional programming solves problems very differently from
object-oriented programming (which is what we use when we’re writing Java),
and the difference in tactics that functional programming requires combined
with the difference in syntax that Scheme in specific requires can be
challenging to Java programmers.</p>
<p>This reading is optional but recommended — if you find yourself thinking that
you know how to solve a problem in Java but unsure how to solve it in Scheme,
make sure to give it a read!</p>
<h2 id="object-oriented-programming"><a class=anchor href="#object-oriented-programming" aria-label="Link to section object-oriented-programming">#</a>Object-oriented programming</h2>
<blockquote>
<p>Quick: off the top of your head, what is object-oriented programming about?</p>
<p>Got an idea yet?</p>
<p>If you thought any of the words “<a href="https://en.wikipedia.org/wiki/Encapsulation_(computer_programming)">encapsulation</a>”, “<a href="https://en.wikipedia.org/wiki/Inheritance_(object-oriented_programming)">inheritance</a>”,
“<a href="https://en.wikipedia.org/wiki/Polymorphism_(computer_science)">polymorphism</a>”, “<a href="https://en.wikipedia.org/wiki/Information_hiding">information hiding</a>”, “<a href="https://en.wikipedia.org/wiki/Abstraction_(computer_science)">abstraction</a>”, or “<a href="https://en.wikipedia.org/wiki/Virtual_method_table">vtables</a>”,
you are <em>wrong.</em></p>
<p>If you thought any of the words “<a href="https://en.wikipedia.org/wiki/Class_(computer_programming)">class</a>”, “<a href="https://en.wikipedia.org/wiki/Prototype-based_programming">prototype</a>”, or “<a href="https://en.wikipedia.org/wiki/Data_type">type</a>”, you
are <em>still wrong.</em></p>
<p><strong>Object-oriented programming is about <em>objects</em>: bundles of state and
behavior.</strong> The rest is optional fluff. And object-oriented <em>languages</em> are
defined only by having built-in support for bundling state and behavior,
<em>not</em> by having built-in support for classes. You may notice we don’t call it
“class-oriented programming”.</p>
<p>— <a href="https://eev.ee/blog/2013/03/03/the-controller-pattern-is-awful-and-other-oo-heresy/">“The controller pattern is awful (and other OO heresy)” by Eevee</a></p>
</blockquote>
<p>Let’s make this clear with some Java code. Here’s a simple <code>Point</code> class, which
represents a point in 2-dimensional space with <code>double</code> coordinates.</p>
<pre data-lang="java" class="language-java "><code class="language-java" data-lang="java">public class Point {
public double x;
public double y;
}
</code></pre>
<p>This <a href="https://en.wikipedia.org/wiki/Class_(computer_programming)">class</a> is a bundle of <em>state</em>; the state of an object is what the <em>is</em>;
here, the state is its coordinates. We can make <code>Point</code>s, modify them, and pass
them around, but they don’t <em>do</em> anything.</p>
<pre data-lang="java" class="language-java "><code class="language-java" data-lang="java">Point p = new Point();
p.x = 1;
p.y = 3;
System.out.println("p.x = " + p.x);
System.out.println("p.y = " + p.y);
</code></pre>
<p>Let’s give our <code>Point</code> some behavior, with a <code>rotate</code> method, which will
calculate the new coordinates using the 2-dimensional <a href="https://en.wikipedia.org/wiki/Rotation_matrix">rotation matrix</a>:</p>
<pre data-lang="java" class="language-java "><code class="language-java" data-lang="java">import java.lang.Math;
public class Point {
public double x;
public double y;
Point(double x, double y) {
this.x = x;
this.y = y;
}
void rotate(double radians) {
double newX = x * Math.cos(radians) - y * Math.sin(radians);
double newY = x * Math.sin(radians) + y * Math.cos(radians);
this.x = newX;
this.y = newY;
}
@Override
public String toString() {
return "Point(" + x + ", " + y +")";
}
}
</code></pre>
<p>Now we can make and modify <code>Point</code>s (just like before), but <code>Point</code>s also
inherently <em>know how to do something</em>: rotate!</p>
<pre data-lang="java" class="language-java "><code class="language-java" data-lang="java">Point p = new Point(1, 3);
System.out.println("p = " + p);
p.rotate(Math.PI / 2); // 90° CCW
System.out.println("p = " + p);
</code></pre>
<p>Prints <code>p = Point(1.0, 3.0)</code>, and then <code>p = Point(-3.0, 1.0000000000000002)</code>
(because <a href="https://0.30000000000000004.com/"><code>double</code>s round in strange ways</a>). Now we
have an <em>object</em>, with <em>state</em> (what it is; the <code>x</code> and <code>y</code> coordinates) and
<em>behavior</em> (what it can <em>do</em>; rotate).</p>
<h2 id="mutation"><a class=anchor href="#mutation" aria-label="Link to section mutation">#</a>Mutation</h2>
<p>The <code>rotate</code> method we made is particularly interesting, because it returns
<code>void</code>, storing its result in its own data. We call it <strong>mutation</strong> when the
state of an object is changed.</p>
<p>This can either be problematic (now, if we pass a <code>Point</code> to some method, we
don’t have any guarantee that the <code>Point</code> won’t be changed by the method, which
bites each new programmer <em>at least</em> once) or advantageous (we can easily give
other methods the power to change the objects we care about by mutating them).</p>
<p>One thing, however, is certain: when we add mutability to our objects, our
program becomes more complex. What if methods never mutated objects? Functional
programming advocates say that objects which <em>can’t</em> be mutated (<em>immutable</em>
objects) are simpler to reason about and program with, because we never have to
account for data changing without us knowing about it.</p>
<p>Functions which don’t mutate data or cause other side-effects (like modifying
files, printing data, or making network requests) are called <em>pure</em>. Let’s
rewrite our <code>rotate</code> method to be pure:</p>
<pre data-lang="java" class="language-java "><code class="language-java" data-lang="java">Point rotate(double radians) {
return new Point(
x * Math.cos(radians) - y * Math.sin(radians),
x * Math.sin(radians) + y * Math.cos(radians)
);
}
</code></pre>
<p>The change is pretty minimal. Now, we can use it like this:</p>
<pre data-lang="java" class="language-java "><code class="language-java" data-lang="java">Point p = new Point(1, 3);
System.out.println("p = " + p);
System.out.println("rotated = " + p.rotate(Math.PI / 2)); // 90° CCW
System.out.println("p = " + p);
</code></pre>
<p>which prints</p>
<pre data-lang="java" class="language-java "><code class="language-java" data-lang="java">p = Point(1.0, 3.0)
rotated = Point(-3.0, 1.0000000000000002)
p = Point(1.0, 3.0)
</code></pre>
<h2 id="composing-functions-together"><a class=anchor href="#composing-functions-together" aria-label="Link to section composing-functions-together">#</a>Composing functions together</h2>
<p>When a program is made up entirely of pure functions, it can be easier to read,
write, and understand the code.</p>
<p>Suppose we have a list of points represented as a string, like this:</p>
<!-- Generated with this Mathematica code:
4 {N[Sin[#]], N[Cos[#]]} & /@ Range[1, 8, Pi/8]
-->
<pre><code>3.36588,2.16121 3.93673,0.708629 3.90825,-0.851834 3.28476,-2.28261
2.16121,-3.36588 0.708629,-3.93673 -0.851834,-3.90825 -2.28261,-3.28476
-3.36588,-2.16121 -3.93673,-0.708629 -3.90825,0.851834 -3.28476,2.28261
-2.16121,3.36588 -0.708629,3.93673 0.851834,3.90825 2.28261,3.28476
3.36588,2.16121 3.93673,0.708629
</code></pre>
<p>That is, points are separated by space characters, and coordinates within a
point are separated by a comma.</p>
<p>Here’s how an introductory Java programmer might try to parse the string into a
list of <code>Point</code> objects:</p>
<pre data-lang="java" class="language-java "><code class="language-java" data-lang="java">public static List<Point> parsePoints(String str) {
try (Scanner scanner = new Scanner(str)) {
ArrayList<String> pointStrings = new ArrayList<>();
while (scanner.hasNext()) {
pointStrings.add(scanner.next());
}
ArrayList<Point> points = new ArrayList<>();
for (int i = 0; i < pointStrings.size(); i++) {
String[] numbers = pointStrings.get(i).split(",");
double x = Double.parseDouble(numbers[0]);
double y = Double.parseDouble(numbers[1]);
points.add(new Point(x, y));
}
return points;
}
}
</code></pre>
<p>This implementation of <code>parsePoints</code> <em>works</em> fine, but it’s got some problems.</p>
<ol>
<li>We use too much data; first, we split <code>str</code> by whitespace with the <code>Scanner</code>
and store it in <code>pointStrings</code>, which uses almost as much data as storing
<code>str</code> originally did. Then, the very next thing we do is throw it all out
after we make <code>points</code>.</li>
<li>The code isn’t very clear; we build up <code>pointStrings</code> from the <code>Scanner</code>, we
go through <code>pointStrings</code> again, then there’s… indexes into the <code>numbers</code>
array? It’s weird, and if someone passed bad data to this method, they’d
probably be confused to get an <code>ArrayIndexOutOfBoundsException</code>.</li>
</ol>
<p>But, under the hood, we’re only really <em>doing</em> a few things:</p>
<ol>
<li>Splitting <code>str</code> by whitespace.</li>
<li>Splitting the resulting strings by commas.</li>
<li>Parsing all the elements as <code>double</code>s.</li>
<li>Turning the parsed <code>double</code>s into <code>Point</code>s.</li>
</ol>
<p>Note that each operation here goes directly to the next operation. With that in
mind, let’s split up our <code>parsePoints</code> method a bit:</p>
<pre data-lang="java" class="language-java "><code class="language-java" data-lang="java">static String[] splitWhitespace(String str) {
return str.split("\\s+");
}
static String[] splitCommas(String str) {
return str.split(",");
}
static List<Double> parseDoubles(String[] strs) {
ArrayList<Double> doubles = new ArrayList<>();
for (String str : strs) {
doubles.add(Double.parseDouble(str));
}
return doubles;
}
static Point toPoint(List<Double> doubles) {
return new Point(doubles.get(0), doubles.get(1));
}
public static List<Point> parsePoints(String str) {
String[] coords = splitWhitespace(str);
ArrayList<String[]> splitByCommas = new ArrayList<>();
for (String coord : coords) {
splitByCommas.add(splitCommas(coord));
}
ArrayList<List<Double>> doubles = new ArrayList<>();
for (String[] doubleStrings : splitByCommas) {
doubles.add(parseDoubles(doubleStrings));
}
ArrayList<Point> points = new ArrayList<>();
for (List<Double> rawPoint : doubles) {
points.add(toPoint(rawPoint));
}
return points;
}
</code></pre>
<p>This code is a bit clearer, but it’s much longer. Further, astute readers might
have noticed that this pattern is repeated several times:</p>
<pre data-lang="java" class="language-java "><code class="language-java" data-lang="java">ArrayList<NewType> newList = new ArrayList<>();
for (OldType element : oldList) {
newList.add(oldTypeToNewType(element));
}
</code></pre>
<p>That is, building up a list by calling a method on every element of another
list. Functional programmers have a name for this operation, and it’s called
<em>mapping</em> a list, because it shows you a <em>map</em> from a list of one type to a
list of another type.</p>
<p>What do we know about the method <code>oldTypeToNewType</code>? It must look something
like this, because it has one parameter of type <code>OldType</code> and returns a value
of type <code>NewType</code>:</p>
<pre data-lang="java" class="language-java "><code class="language-java" data-lang="java">NewType oldTypeToNewType(OldType o) {
// ...
}
</code></pre>
<p>Java actually already includes this as an interface called <a href="https://docs.oracle.com/en/java/javase/12/docs/api/java.base/java/util/function/Function.html"><code>Funtion<T, R></code></a>, where <code>T</code> is <code>OldType</code> (the function’s input) and <code>R</code> is
<code>NewType</code> (the function’s return-type). With that in mind, let’s write a
<code>mapList</code> function that turns a list of <code>T</code> into a list of <code>R</code>:</p>
<pre data-lang="java" class="language-java "><code class="language-java" data-lang="java">static <T, R> List<R> mapList(List<T> input, Function<T, R> mappingFunction) {
ArrayList<R> ret = new ArrayList<>();
for (T t : input) {
ret.add(mappingFunction.apply(t));
}
return ret;
}
</code></pre>
<p>Then, we can create classes to call <code>mapList</code> with. For example, here’s
<code>parseDoubles</code> using <code>mapList</code>:</p>
<pre data-lang="java" class="language-java "><code class="language-java" data-lang="java">static List<Double> parseDoubles(String[] strs) {
return mapList(Arrays.asList(strs), new Function<String, Double>() {
@Override
public Double apply(String s) {
return Double.parseDouble(s);
}
});
}
</code></pre>
<p>The syntax <a href="https://docs.oracle.com/javase/tutorial/java/javaOO/anonymousclasses.html"><code>new Function<...>() { ... }</code></a> is just like
declaring <code>class MyClass<...> implements Function<...> { ... }</code> and then
constructing it exactly once. Fortunately, Java gives us shorter syntax for any
<a href="https://docs.oracle.com/en/java/javase/12/docs/api/java.base/java/lang/FunctionalInterface.html">functional interface</a>, which represents the exact same thing:</p>
<pre data-lang="java" class="language-java "><code class="language-java" data-lang="java">static List<Double> parseDoubles(String[] strs) {
return mapList(Arrays.asList(strs), s -> Double.parseDouble(s));
}
</code></pre>
<p>This shortcut is called a <em><a href="https://docs.oracle.com/javase/tutorial/java/javaOO/lambdaexpressions.html">lambda expression</a></em>. Further, it turns out that the
pattern of writing lambdas of the form <code>var -> MyClass.method(var)</code> is so
common there’s another special syntax for it, called <em><a href="https://docs.oracle.com/javase/tutorial/java/javaOO/methodreferences.html">method references</a></em>,
which look like <code>MyClass::method</code> (in this case, <code>Double::parseDouble</code>):</p>
<pre data-lang="java" class="language-java "><code class="language-java" data-lang="java">static List<Double> parseDoubles(String[] strs) {
return mapList(Arrays.asList(strs), Double::parseDouble);
}
</code></pre>
<p>Now, we can rewrite our <code>Point</code> parser using <code>mapList</code>:</p>
<pre data-lang="java" class="language-java "><code class="language-java" data-lang="java">public static List<Point> parsePoints(String str) {
String[] coords = splitWhitespace(str);
List<String[]> splitByCommas = mapList(Arrays.asList(coords), PointParser::splitCommas);
List<List<Double>> doubles = mapList(splitByCommas, PointParser::parseDoubles);
return mapList(doubles, PointParser::toPoint);
}
</code></pre>
<p>Much better! This code is very declarative: we write what we mean. First, we
split by whitespace. Then, we split by commas. Then, we parse as doubles. Then,
we map to points.</p>
<p>This is still fairly wasteful in terms of memory, though — we create not just
one but <em>three</em> intermediate lists, each of which gets discarded after being
used only once.</p>
<p>We can solve this problem with a <a href="https://docs.oracle.com/en/java/javase/12/docs/api/java.base/java/util/stream/Stream.html"><strong>stream</strong></a> (sometimes called a <em>lazy
iterator</em>). Streams are specifically designed to be composable with functions,
so that we can generate a stream from some source (like reading from a file,
finding matches in a string, or more traditionally from some data structure
like an array or tree) and then specify that it be mapped over in various ways.</p>
<p>Unlike our <code>mapList</code> function, mapping over a stream doesn’t read any data from
the underlying source; it just writes down a note saying the stream is also
being mapped over by a particular value. Therefore, we don’t need to generate
any intermediate lists — we can just add our mapping functions one by one, and
the stream remembers how to transform its data before using it when we start
reading values from the stream.</p>
<p>And how do we read values from a stream? Unlike arrays or other simple data
structures, streams are designed to let us ask more questions than just “what
element is at index <em>i</em> in this structure?” Streams have methods that let us ask:</p>
<ul>
<li>Does this function return <code>true</code> when applied to every element in the stream?</li>
<li>Does the stream have at least <em>n</em> items?</li>
<li>Find the first element in the stream that this function returns <code>true</code> for.</li>
<li>What’s the maximum value in the stream?</li>
<li>Collect the values in this stream (to an <code>ArrayList</code>, a <code>HashMap</code>, a
<code>String</code>, etc.).</li>
</ul>
<p>We can generate streams a few different ways in Java, but for the most part the
<code>Arrays.stream</code> method, the <code>Collection.stream</code> method, and the <code>Collectors</code>
methods are suitable for converting most data to and from streams.</p>
<p>Rewriting the point parser with <code>Stream</code>s, we get:</p>
<pre data-lang="java" class="language-java "><code class="language-java" data-lang="java">public class PointParser {
static final Pattern COMMA = Pattern.compile(",");
static final Pattern WHITESPACE = Pattern.compile("\\s+");
static Point toPoint(Iterator<Double> doubles) {
return new Point(doubles.next(), doubles.next());
}
public static List<Point> parsePoints(String str) {
return WHITESPACE.splitAsStream(str) // (1)
.map(s -> COMMA.splitAsStream(s) // (2)
.map(Double::parseDouble)
.iterator()) // (3)
.map(PointParser::toPoint)
.collect(Collectors.toList()); // (4)
}
}
</code></pre>
<p>Notes:</p>
<ol>
<li>
<p>The <a href="https://docs.oracle.com/en/java/javase/12/docs/api/java.base/java/util/regex/Pattern.html#splitAsStream(java.lang.CharSequence)"><code>Pattern.splitAsStream</code></a> method is exactly like
<a href="https://docs.oracle.com/en/java/javase/12/docs/api/java.base/java/lang/String.html#split(java.lang.String)"><code>String.split</code></a> except it returns a <code>Stream</code> instead of an
array.</p>
</li>
<li>
<p>Inside one of our calls to <code>map</code>, we create <em>another</em> <code>Stream</code>; streams work
best together!</p>
<p>These calls to <code>map</code> don’t actually get items out of the stream; they’re
<em>lazy</em>, and all they do is make the stream remember that it’s mapping by
that function.</p>
</li>
<li>
<p>The call to <code>iterator</code> turns a <code>Stream</code> into an <code>Iterator</code>; it’s a
<em>finalizing</em> operation, which means that it consumes the elements of the
stream and returns a result. Once we have an <code>Iterator</code>, we can’t <code>map</code> on
it any more.</p>
</li>
<li>
<p>Finally, we <em>collect</em> the elements of the stream into a <code>List</code> with the
<code>collect</code> method and the <code>Collectors.toList()</code> utility method.</p>
</li>
</ol>
<p>This code is (I think) much clearer than our first attempt, and a bit shorter
as an extra bonus. I’ll admit that code writtten with streams can look a bit
alien at first, and it takes some practice to get used to!</p>
<p>In this code, we don’t use a single index or <code>for</code> loop — and why should we?
After all, none of the operations need to know anything about the elements
before or after them; the indexes were only to overcome limitations in the ways
we can refer to data structures.</p>
<p>Additionally, no intermediate lists are created; when the stream is being
collected, each element from the pattern-splitter is applied to all our mapping
functions in a row before being returned. The <em>processing</em> steps aren’t
separated from each other, which saves memory, even though we were able to
<em>declare</em> the processing steps separately. In APIs focused on streams, it’s
often easier to return (and take as parameters) streams themselves, rather than
frequently collecting data to structures like lists at method boundaries.</p>
<p>There’s another advantage of this style of programming, in addition to saving
memory; because we never reassign values, the functions are immutable and pure.
In Java, we’re not inherently motivated to write pure functions. But in other
languages (notably pure functional programming languages like Haskell, or other
functional programming languages like Scheme), we’re <em>forced</em> to write pure
functions; learning to construct pure functions and immutable data structures
will help you reason about and solve problems in a functional programming style.</p>
who i am not2019-04-22T00:00:00+00:002019-04-22T00:00:00+00:00
Rebecca Turner
https://becca.ooo/not/<p>there are many rebecca turners. i am not these ones</p>
<ul>
<li><a href="https://kolektiva.social/@iarna">software engineer rebecca turner, aka iarna</a>, currently employed by
microsoft but perhaps best-known for her work at npm, where she was
responsible for <a href="https://github.com/npm/cli">the npm cli</a>, among other things</li>
<li><a href="https://en.m.wikipedia.org/wiki/Rebecca_Turner">british olympian rebecca turner</a></li>
<li><a href="https://www.supertalk.fm/shows/goodthings/">host of mississippi talk show “good things with rebecca turner”</a></li>
<li><a href="http://www.world-of-lucid-dreaming.com/lucid-dreamer.html">founder of “world of lucid dreaming” rebecca turner</a></li>
<li><a href="http://rebeccaturner.net/">folk singer/songwriter rebecca turner</a></li>
<li><a href="https://www.imdb.com/name/nm4576641/">actress rebecca turner</a></li>
<li><a href="http://www.wealthstrategyadvisors.net/">“wealth strategy advisor” rebecca turner</a></li>
<li><a href="https://twitter.com/rebeccasturner">abc news journalist rebecca turner</a></li>
<li><a href="https://iris.ucl.ac.uk/iris/browse/profile?upi=RTURN24">senior research associate at the mrc clinical trials unit at ucl rebecca
turner, phd</a></li>
<li><a href="http://rebeccaturnerconsulting.com/about.html">executive coach / consultant dr. rebecca turner</a></li>
<li><a href="http://www.iaapa.org/news/iaapa-press-office/rebecca-turner-joins-iaapa-as-vice-president-of-education-professional-development-and-training-services">vice president of education and professional development at the international association of amusement parks and attractions rebecca turner</a></li>
<li><a href="http://sjeds.org/private-christian-elementary-school-jacksonville-fl/faculty-staff/">kindergarten teacher at the san jose episcopal day school rebecca turner</a></li>
<li><a href="https://www.fairsentencingofyouth.org/team/turner/">lawyer and montgomery implementation coordinator at the campaign for the
fair sentencing of youth rebecca turner</a></li>
<li><a href="https://health.usnews.com/doctors/rebecca-turner-568245">“family medicine doctor in missouri” rebecca turner</a></li>
<li><a href="https://www.littlerealestate.com.au/our-team/rebecca-turner">australian property manager rebecca turner</a></li>
<li><a href="https://rebeccaturnernutrition.com/">“MS, RD, LD, author, registered dietitian, radio host, television presenter, and girl mom” rebecca turner</a></li>
</ul>
Unicode Resources2018-04-06T00:00:00+00:002018-04-06T00:00:00+00:00
Rebecca Turner
https://becca.ooo/unicode/<p>Created by <a href="/">rebecca</a> with major help from <a href="https://twitter.com/FakeUnicode">@FakeUnicode</a></p>
<hr />
<p><a href="https://en.wikipedia.org/wiki/Unicode">Unicode</a> is a standard “registry” of characters designed to catalogue all the
world’s writing systems and serve as a superset of all previously-existing
character sets used and supported by <a href="https://unicode.org/consortium/members.html">just about everyone</a> and <a href="https://unicode.org/resources/">all their
software</a>. <a href="https://stackoverflow.com/questions/2241348/what-is-unicode-utf-8-utf-16">Get into the details on Stack Overflow</a> or see some of the <a href="https://en.wikipedia.org/wiki/Unicode#Mapping_and_encodings">Unicode
encodings</a> on Wikipedia.</p>
<h1 id="charts-and-character-data"><a class=anchor href="#charts-and-character-data" aria-label="Link to section charts-and-character-data">#</a>Charts and character data</h1>
<p><span class=bullet></span><a href="https://unicode.org/charts/">PDF charts on unicode.org</a> — The only
complete and perpetually up-to-date reference.</p>
<p><span class=bullet></span><a href="https://en.wikipedia.org/wiki/Unicode_block">List of Unicode blocks on
Wikipedia</a> — Also available as a <a href="https://unicode.org/Public/UNIDATA/Blocks.txt">TXT file on unicode.org</a></p>
<p><a href="https://unicode.org/Public/UNIDATA/">UNIDATA on unicode.org</a> — “This directory contains the final data
files for the Unicode Character Database, for Version 8.0.0 of the
Unicode Standard.”</p>
<p><a href="https://www.copypastecharacter.com/all-characters">CopyPasteCharacter</a> — Contains a bunch of common characters for easy
copying.</p>
<p><a href="https://unicode.org/alloc/Pipeline.html">Proposed new characters on unicode.org</a></p>
<p><a href="https://en.wikipedia.org/wiki/Plane_(Unicode)">Unicode planes reference</a> — Confused about what “BMP” means? Check
here.</p>
<h1 id="string-analysis"><a class=anchor href="#string-analysis" aria-label="Link to section string-analysis">#</a>String analysis</h1>
<p><span class=bullet></span><a href="https://apps.timwhitlock.info/unicode/inspect?s=%F0%9F%92%81u">Unicode Inspector</a> — By
<a href="https://twitter.com/timwhitlock">@timwhitlock</a>, displays the codepoint, byte breakdown, block, symbol, name,
and surrogates for each character in a string.</p>
<p><a href="http://unicode.scarfboy.com/?s=4a1e">Scarfboy search and string analysis</a></p>
<p><a href="https://jsfiddle.net/vrog8Lkf/">Understanding UTF-8 on jsfiddle</a> — By <a href="https://twitter.com/FakeUnicode">@FakeUnicode</a> on Twitter.
Displays binary/hex breakdown of strings of text.</p>
<p><a href="https://babelstone.co.uk/Unicode/whatisit.html">What Unicode character is this?</a> — By <a href="https://babelstone.co.uk/">BabelStone</a>.</p>
<h1 id="search"><a class=anchor href="#search" aria-label="Link to section search">#</a>Search</h1>
<p><a href="http://unicode.scarfboy.com/?s=4a1e">Scarfboy search and string analysis</a></p>
<p><a href="https://codepoints.net/search?q=punctuation">codepoints.net</a> — Powerful search engine with previews in Unifont</p>
<p><a href="https://amp-what.com/unicode/search/">&what;</a></p>
<p><a href="https://fileformat.info/info/unicode/char/search.htm">FileFormat.info</a></p>
<p><a href="https://charcod.es/">charcod.es</a></p>
<h1 id="shape-recognition"><a class=anchor href="#shape-recognition" aria-label="Link to section shape-recognition">#</a>Shape recognition</h1>
<p><span class=bullet></span><a href="https://shapecatcher.com/">shapecatcher</a> — Recognizes Unicode
characters through a drawing field.</p>
<p><a href="https://jisho.org/#radical">Kanji search on jisho.org</a> — Find kanji characters by their parts.</p>
<p><a href="https://kanji.sljfaq.org/draw-canvas.html">Handwritten kanji recognition</a> — Like <a href="https://shapecatcher.com/">shapecatcher</a> for kanji.</p>
<p><a href="https://translate.google.com/">Google Translate</a> — Click the pencil icon in the input area to enable
shape recognition. Great for writing short bits of text in a language you don’t
know.</p>
<h1 id="conversion-programming"><a class=anchor href="#conversion-programming" aria-label="Link to section conversion-programming">#</a>Conversion (programming)</h1>
<p><span class=bullet></span><a href="https://xem.github.io/escape/">Xem’s EscApe utility on Github</a> — By <a href="https://xem.github.io/">Maxime
Euzière</a>. Converts any Unicode string to a bunch of different escape sequence
syntaxes.</p>
<p><a href="https://paulschou.com/tools/xlate/">ASCII Xlate</a> — Converts plain ASCII, binary, octal, hex, base32,
base64, ASCII85, and decimal ASCII. Also calculates various hashes!</p>
<p><a href="https://unicodetools.com/">Guide to converting to UTF-8 in various programming languages</a></p>
<h1 id="conversion-decorative-or-linguistic"><a class=anchor href="#conversion-decorative-or-linguistic" aria-label="Link to section conversion-decorative-or-linguistic">#</a>Conversion (decorative or linguistic)</h1>
<p><span class=bullet></span><a href="http://mar.cx/unicate/">Unicate</a> — Converts to various Latin
Unicode “alphabets” (e.g. full-width, math scripts, etc.).</p>
<p><a href="https://qaz.wtf/u/convert.cgi?text=unicode.9999yea.rs">Text converter on qaz.wtf</a> — Similar to the above.</p>
<p><span class=bullet></span><a href="https://eeemo.net/">Zalgo generator on eeemo.net</a> — generates
Zalgo text.</p>
<p><a href="https://chrome.google.com/webstore/detail/convert-text/mcpglhjaahelnpjalcaeecgkjhkpokdn">Convert Text on the Chrome web store</a> — Converts the case of text as
a Chrome extension. Includes Zalgo generator and fullwidth transform.</p>
<p><a href="https://adamvarga.com/strike/">Strikethrough converter</a> — By <a href="https://adamvarga.com/">Adam Varga</a>.</p>
<p><a href="https://jsfiddle.net/xHrxM/13/">Emojify text on jsfiddle</a> — By <a href="https://twitter.com/FakeUnicode">@FakeUnicode</a> on Twitter. Transforms
text to emoji.</p>
<p><a href="/unicode/grid/">Acrostic generator</a> — Dubiously useful. Also converts
to Unicode Math Monospace.</p>
<p><a href="/unicode/abbreviate/">Abbreviator</a> — Saves characters in tweets by using
precomposed characters.</p>
<p><a href="https://www.unicod.es/">Unitools</a> — A good compilation of a bunch of other tools. Honorary
mention.</p>
<p><a href="https://brailletranslator.org/">Braille converter</a></p>
<h1 id="coverage-fonts-support"><a class=anchor href="#coverage-fonts-support" aria-label="Link to section coverage-fonts-support">#</a>Coverage: fonts & support</h1>
<p><span class=bullet></span><a href="http://alanwood.net/unicode/fonts.html">Alan Wood’s Unicode font list</a> — Probably the most
complete list of high-coverage fonts.</p>
<p><span class=bullet></span><a href="https://fsd.it/shop/fonts/pragmatapro/">PragmataPro</a> — A monospaced programming font with
6,000 glyphs (and rising).</p>
<p><span class=bullet></span><a href="https://mathew-kurian.github.io/CharacterMap/">CharacterMap</a> — Analyzes glyphs from font files.</p>
<p><a href="https://google.com/get/noto/">Google Noto Fonts</a> — A set of sans and serif fonts supporting 581
languages (as of April 2016), with about 50% glyph coverage.</p>
<p><a href="http://cheat-sheets.org/sites/font.su/">Unicode fonts by writing system</a></p>
<p><a href="http://visibone.com/htmlref/char/cer.htm">Preview of all codepoints in the BMP</a> — Useful for testing coverage.</p>
<p><a href="https://unifoundry.com/unifont/">GNU Unifont</a></p>
<p><a href="https://babelstone.co.uk/Fonts/Han.html">BabelStone’s Han font</a></p>
<p><a href="https://evertype.com/emono/">Everson Mono font</a></p>
<p><a href="https://fonts.jp/hanazono/">Hanazono font</a></p>
<p><a href="https://en.wikipedia.org/wiki/Code2000">Code2000 font</a></p>
<h1 id="emoji"><a class=anchor href="#emoji" aria-label="Link to section emoji">#</a>Emoji</h1>
<p><span class=bullet></span><a href="https://unicode.org/emoji/charts/full-emoji-list.html">Full Emoji Charts</a> — all emoji with comparison pictures of
implementations on various platforms.</p>
<p><a href="https://unicode.org/L2/L2009/09027-emoji-backgrnd.pdf">Emoji Symbols: Background Data on unicode.org</a>, a.k.a. (L2/09-027) — Japanese
carrier background data for the original emoji import (mostly historic value).</p>
<p><a href="https://en.wikipedia.org/wiki/Miscellaneous_Symbols_and_Pictographs#History">List of proposals and their associated codepoints on Wikipedia</a></p>
<p><a href="https://unicode.org/emoji/future/emoji-candidates.html">Possible upcoming emoji on unicode.org</a></p>
<p><a href="https://unicode.org/emoji/charts/emoji-zwj-sequences.html">List of emoji ZWJ ligatures on unicode.org</a> — Note: These are <em>not</em>
actually emoji or part of the Unicode standard. Implementation, support,
and blame lies entirely with the third parties involved.</p>
<p><a href="https://unicode.org/emoji/charts/text-style.html">Text vs Emoji reference on unicode.org</a> — Shows which characters
(should) render as emoji or text.</p>
<h1 id="ascii-unicode-art-kaomoji"><a class=anchor href="#ascii-unicode-art-kaomoji" aria-label="Link to section ascii-unicode-art-kaomoji">#</a>ASCII/Unicode art & Kaomoji</h1>
<p><span class=bullet></span><a href="https://cutekaomoji.com/">cutekaomoji.com</a></p>
<p><a href="https://japaneseemoticons.me/all-japanese-emoticons/">List of 10,000+ kaomoji</a></p>
<p><a href="https://textfac.es/">textfac.es</a></p>
<p><a href="https://asciiart.website/">ASCII art collection on asciiart.website</a>, formerly chris.com</p>
<p><a href="https://asciiworld.com/">ASCII art on asciiworld.com</a></p>
<p><a href="https://patorjk.com/software/taag/#p=display&f=Graffiti&t=unicode.9999yea.rs">ASCII text art generator</a></p>
<h1 id="inserting-characters"><a class=anchor href="#inserting-characters" aria-label="Link to section inserting-characters">#</a>Inserting characters</h1>
<p><a href="https://vim.wikia.com/wiki/Entering_special_characters">Vim</a> — Insert mode: <code><C-v>uxxxx</code></p>
<p><a href="https://superuser.com/questions/394405/how-to-type-a-unicode-character-by-its-number-in-emacs">Emacs</a> — <code><C-X> 8 <CR> xxxx <CR></code></p>
<p><a href="https://poynton.com/notes/misc/mac-unicode-hex-input.html">Mac OS X</a> — (☑︎ Unicode Hex Input) <code><⌥-xxxx></code></p>
<p><a href="https://en.wikipedia.org/wiki/Unicode_input#In_Microsoft_Windows">Windows</a> — (☑︎ Registry Key) <code><A-xxxx></code></p>
<p><a href="https://en.wikipedia.org/wiki/Unicode_input#In_X11_.28Linux_and_other_Unix_variants.29">Unix in GTK applications</a> — <code><C-S-uxxxx></code></p>
<h1 id="misc"><a class=anchor href="#misc" aria-label="Link to section misc">#</a>Misc.</h1>
<p><span class=bullet></span><a href="https://jsfiddle.net/SaqVU/4/">Random Unicode character generator on jsfiddle</a> —
By <a href="https://twitter.com/FakeUnicode">@FakeUnicode</a> on Twitter.</p>
<p><a href="/arrows">List of Unicode arrows</a> — Courtesy of <a href="https://twitter.com/fabrizioschiavi">@fabrizioschiavi</a> of
<a href="https://fsd.it/shop/fonts/pragmatapro/">Pragmata Pro</a> fame.</p>
<p><a href="https://getemoji.com/#twitter">Emoji allowed in Twitter usernames</a></p>
<p><a href="https://mothereff.in/byte-counter">Character/byte counter</a></p>
ASCII table2018-04-04T00:00:00+00:002018-04-04T00:00:00+00:00
Rebecca Turner
https://becca.ooo/ascii/<p><a href="/ascii.txt">Available as a terminal-friendly .txt!</a> (<a href="/ascii-long.txt">long version</a>), or with <a href="https://becca.ooo/ascii/simple">less details</a>.</p>
<table><thead><tr><th><a href="https://en.wikipedia.org/wiki/ASCII#Character_groups">Char</a></th><th><a href="https://en.wikipedia.org/wiki/Decimal">Dec</a></th><th><a href="https://en.wikipedia.org/wiki/Octal">Oct</a></th><th><a href="https://en.wikipedia.org/wiki/Hexadecimal">Hex</a></th><th><a href="https://en.wikipedia.org/wiki/Binary_number#Representation">Bin</a></th><th><a href="https://en.wikipedia.org/wiki/C0_and_C1_control_codes#C0_.28ASCII_and_derivatives.29">C0</a></th><th><a href="https://en.wikipedia.org/wiki/Escape_sequences_in_C#Table_of_escape_sequences">C</a></th><th><a href="https://en.wikipedia.org/wiki/Control_Pictures">U</a></th><th><a href="https://en.wikipedia.org/wiki/Basic_Latin_(Unicode_block)">Name</a></th></tr></thead><tbody>
<tr><td>(nul)</td><td>0</td><td>0000</td><td>0x00</td><td>00000000</td><td>^@</td><td>\0</td><td>␀</td><td>Null</td></tr>
<tr><td>(soh)</td><td>1</td><td>0001</td><td>0x01</td><td>00000001</td><td>^A</td><td></td><td>␁</td><td>Start of Heading</td></tr>
<tr><td>(stx)</td><td>2</td><td>0002</td><td>0x02</td><td>00000010</td><td>^B</td><td></td><td>␂</td><td>Start of Text</td></tr>
<tr><td>(etx)</td><td>3</td><td>0003</td><td>0x03</td><td>00000011</td><td>^C</td><td></td><td>␃</td><td>End of Text</td></tr>
<tr><td>(eot)</td><td>4</td><td>0004</td><td>0x04</td><td>00000100</td><td>^D</td><td></td><td>␄</td><td>End of Transmission</td></tr>
<tr><td>(enq)</td><td>5</td><td>0005</td><td>0x05</td><td>00000101</td><td>^E</td><td></td><td>␅</td><td>Enquiry</td></tr>
<tr><td>(ack)</td><td>6</td><td>0006</td><td>0x06</td><td>00000110</td><td>^F</td><td></td><td>␆</td><td>Acknowledgement</td></tr>
<tr><td>(bel)</td><td>7</td><td>0007</td><td>0x07</td><td>00000111</td><td>^G</td><td>\a</td><td>␇</td><td>Bell</td></tr>
<tr><td>(bs)</td><td>8</td><td>0010</td><td>0x08</td><td>00001000</td><td>^H</td><td>\b</td><td>␈</td><td>Backspace</td></tr>
<tr><td>(ht)</td><td>9</td><td>0011</td><td>0x09</td><td>00001001</td><td>^I</td><td>\t</td><td>␉</td><td>Horizontal Tab</td></tr>
<tr><td>(nl)</td><td>10</td><td>0012</td><td>0x0a</td><td>00001010</td><td>^J</td><td>\n</td><td>␊</td><td>Line Feed</td></tr>
<tr><td>(vt)</td><td>11</td><td>0013</td><td>0x0b</td><td>00001011</td><td>^K</td><td>\v</td><td>␋</td><td>Vertical Tab</td></tr>
<tr><td>(np)</td><td>12</td><td>0014</td><td>0x0c</td><td>00001100</td><td>^L</td><td>\f</td><td>␌</td><td>Form Feed</td></tr>
<tr><td>(cr)</td><td>13</td><td>0015</td><td>0x0d</td><td>00001101</td><td>^M</td><td>\r</td><td>␍</td><td>Carriage Return</td></tr>
<tr><td>(so)</td><td>14</td><td>0016</td><td>0x0e</td><td>00001110</td><td>^N</td><td></td><td>␎</td><td>Shift Out</td></tr>
<tr><td>(si)</td><td>15</td><td>0017</td><td>0x0f</td><td>00001111</td><td>^O</td><td></td><td>␏</td><td>Shift In</td></tr>
<tr><td>(dle)</td><td>16</td><td>0020</td><td>0x10</td><td>00010000</td><td>^P</td><td></td><td>␐</td><td>Data Link Escape</td></tr>
<tr><td>(dc1)</td><td>17</td><td>0021</td><td>0x11</td><td>00010001</td><td>^Q</td><td></td><td>␑</td><td>Device Control 1 (often XON)</td></tr>
<tr><td>(dc2)</td><td>18</td><td>0022</td><td>0x12</td><td>00010010</td><td>^R</td><td></td><td>␒</td><td>Device Control 2</td></tr>
<tr><td>(dc3)</td><td>19</td><td>0023</td><td>0x13</td><td>00010011</td><td>^S</td><td></td><td>␓</td><td>Device Control 3 (often XOFF)</td></tr>
<tr><td>(dc4)</td><td>20</td><td>0024</td><td>0x14</td><td>00010100</td><td>^T</td><td></td><td>␔</td><td>Device Control 4</td></tr>
<tr><td>(nak)</td><td>21</td><td>0025</td><td>0x15</td><td>00010101</td><td>^U</td><td></td><td>␕</td><td>Negative Acknowledgement</td></tr>
<tr><td>(syn)</td><td>22</td><td>0026</td><td>0x16</td><td>00010110</td><td>^V</td><td></td><td>␖</td><td>Synchronous Idle</td></tr>
<tr><td>(etb)</td><td>23</td><td>0027</td><td>0x17</td><td>00010111</td><td>^W</td><td></td><td>␗</td><td>End of Transmission Block</td></tr>
<tr><td>(can)</td><td>24</td><td>0030</td><td>0x18</td><td>00011000</td><td>^X</td><td></td><td>␘</td><td>Cancel</td></tr>
<tr><td>(em)</td><td>25</td><td>0031</td><td>0x19</td><td>00011001</td><td>^Y</td><td></td><td>␙</td><td>End of Medium</td></tr>
<tr><td>(sub)</td><td>26</td><td>0032</td><td>0x1a</td><td>00011010</td><td>^Z</td><td></td><td>␚</td><td>Substitute</td></tr>
<tr><td>(esc)</td><td>27</td><td>0033</td><td>0x1b</td><td>00011011</td><td>^[</td><td>\e</td><td>␛</td><td>Escape</td></tr>
<tr><td>(fs)</td><td>28</td><td>0034</td><td>0x1c</td><td>00011100</td><td>^\</td><td></td><td>␜</td><td>File Separator</td></tr>
<tr><td>(gs)</td><td>29</td><td>0035</td><td>0x1d</td><td>00011101</td><td>^]</td><td></td><td>␝</td><td>Group Separator</td></tr>
<tr><td>(rs)</td><td>30</td><td>0036</td><td>0x1e</td><td>00011110</td><td>^^</td><td></td><td>␞</td><td>Record Separator</td></tr>
<tr><td>(us)</td><td>31</td><td>0037</td><td>0x1f</td><td>00011111</td><td>^\</td><td></td><td>␟</td><td>Unit Separator</td></tr>
<tr><td>(sp)</td><td>32</td><td>0040</td><td>0x20</td><td>00100000</td><td></td><td></td><td>␠</td><td>Space</td></tr>
<tr><td>!</td><td>33</td><td>0041</td><td>0x21</td><td>00100001</td><td></td><td></td><td></td><td>Exclamation mark</td></tr>
<tr><td>"</td><td>34</td><td>0042</td><td>0x22</td><td>00100010</td><td></td><td>\"</td><td></td><td>Quotation mark</td></tr>
<tr><td>#</td><td>35</td><td>0043</td><td>0x23</td><td>00100011</td><td></td><td></td><td></td><td>Number sign</td></tr>
<tr><td>$</td><td>36</td><td>0044</td><td>0x24</td><td>00100100</td><td></td><td></td><td></td><td>Dollar sign</td></tr>
<tr><td>%</td><td>37</td><td>0045</td><td>0x25</td><td>00100101</td><td></td><td></td><td></td><td>Percent sign</td></tr>
<tr><td>&</td><td>38</td><td>0046</td><td>0x26</td><td>00100110</td><td></td><td></td><td></td><td>Ampersand</td></tr>
<tr><td>'</td><td>39</td><td>0047</td><td>0x27</td><td>00100111</td><td></td><td>\'</td><td></td><td>Apostrophe</td></tr>
<tr><td>(</td><td>40</td><td>0050</td><td>0x28</td><td>00101000</td><td></td><td></td><td></td><td>Left parenthesis</td></tr>
<tr><td>)</td><td>41</td><td>0051</td><td>0x29</td><td>00101001</td><td></td><td></td><td></td><td>Right parenthesis</td></tr>
<tr><td>*</td><td>42</td><td>0052</td><td>0x2a</td><td>00101010</td><td></td><td></td><td></td><td>Asterisk</td></tr>
<tr><td>+</td><td>43</td><td>0053</td><td>0x2b</td><td>00101011</td><td></td><td></td><td></td><td>Plus sign</td></tr>
<tr><td>,</td><td>44</td><td>0054</td><td>0x2c</td><td>00101100</td><td></td><td></td><td></td><td>Comma</td></tr>
<tr><td>-</td><td>45</td><td>0055</td><td>0x2d</td><td>00101101</td><td></td><td></td><td></td><td>Hyphen-minus</td></tr>
<tr><td>.</td><td>46</td><td>0056</td><td>0x2e</td><td>00101110</td><td></td><td></td><td></td><td>Full stop</td></tr>
<tr><td>/</td><td>47</td><td>0057</td><td>0x2f</td><td>00101111</td><td></td><td></td><td></td><td>Solidus</td></tr>
<tr><td>0</td><td>48</td><td>0060</td><td>0x30</td><td>00110000</td><td></td><td></td><td></td><td>Digit zero</td></tr>
<tr><td>1</td><td>49</td><td>0061</td><td>0x31</td><td>00110001</td><td></td><td></td><td></td><td>Digit one</td></tr>
<tr><td>2</td><td>50</td><td>0062</td><td>0x32</td><td>00110010</td><td></td><td></td><td></td><td>Digit two</td></tr>
<tr><td>3</td><td>51</td><td>0063</td><td>0x33</td><td>00110011</td><td></td><td></td><td></td><td>Digit three</td></tr>
<tr><td>4</td><td>52</td><td>0064</td><td>0x34</td><td>00110100</td><td></td><td></td><td></td><td>Digit four</td></tr>
<tr><td>5</td><td>53</td><td>0065</td><td>0x35</td><td>00110101</td><td></td><td></td><td></td><td>Digit five</td></tr>
<tr><td>6</td><td>54</td><td>0066</td><td>0x36</td><td>00110110</td><td></td><td></td><td></td><td>Digit six</td></tr>
<tr><td>7</td><td>55</td><td>0067</td><td>0x37</td><td>00110111</td><td></td><td></td><td></td><td>Digit seven</td></tr>
<tr><td>8</td><td>56</td><td>0070</td><td>0x38</td><td>00111000</td><td></td><td></td><td></td><td>Digit eight</td></tr>
<tr><td>9</td><td>57</td><td>0071</td><td>0x39</td><td>00111001</td><td></td><td></td><td></td><td>Digit nine</td></tr>
<tr><td>:</td><td>58</td><td>0072</td><td>0x3a</td><td>00111010</td><td></td><td></td><td></td><td>Colon</td></tr>
<tr><td>;</td><td>59</td><td>0073</td><td>0x3b</td><td>00111011</td><td></td><td></td><td></td><td>Semicolon</td></tr>
<tr><td><</td><td>60</td><td>0074</td><td>0x3c</td><td>00111100</td><td></td><td></td><td></td><td>Less-than sign</td></tr>
<tr><td>=</td><td>61</td><td>0075</td><td>0x3d</td><td>00111101</td><td></td><td></td><td></td><td>Equals sign</td></tr>
<tr><td>></td><td>62</td><td>0076</td><td>0x3e</td><td>00111110</td><td></td><td></td><td></td><td>Greater-than sign</td></tr>
<tr><td>?</td><td>63</td><td>0077</td><td>0x3f</td><td>00111111</td><td></td><td>\?</td><td></td><td>Question mark</td></tr>
<tr><td>@</td><td>64</td><td>0100</td><td>0x40</td><td>01000000</td><td></td><td></td><td></td><td>Commercial at</td></tr>
<tr><td>A</td><td>65</td><td>0101</td><td>0x41</td><td>01000001</td><td></td><td></td><td></td><td>Latin capital letter A</td></tr>
<tr><td>B</td><td>66</td><td>0102</td><td>0x42</td><td>01000010</td><td></td><td></td><td></td><td>Latin capital letter B</td></tr>
<tr><td>C</td><td>67</td><td>0103</td><td>0x43</td><td>01000011</td><td></td><td></td><td></td><td>Latin capital letter C</td></tr>
<tr><td>D</td><td>68</td><td>0104</td><td>0x44</td><td>01000100</td><td></td><td></td><td></td><td>Latin capital letter D</td></tr>
<tr><td>E</td><td>69</td><td>0105</td><td>0x45</td><td>01000101</td><td></td><td></td><td></td><td>Latin capital letter E</td></tr>
<tr><td>F</td><td>70</td><td>0106</td><td>0x46</td><td>01000110</td><td></td><td></td><td></td><td>Latin capital letter F</td></tr>
<tr><td>G</td><td>71</td><td>0107</td><td>0x47</td><td>01000111</td><td></td><td></td><td></td><td>Latin capital letter G</td></tr>
<tr><td>H</td><td>72</td><td>0110</td><td>0x48</td><td>01001000</td><td></td><td></td><td></td><td>Latin capital letter H</td></tr>
<tr><td>I</td><td>73</td><td>0111</td><td>0x49</td><td>01001001</td><td></td><td></td><td></td><td>Latin capital letter I</td></tr>
<tr><td>J</td><td>74</td><td>0112</td><td>0x4a</td><td>01001010</td><td></td><td></td><td></td><td>Latin capital letter J</td></tr>
<tr><td>K</td><td>75</td><td>0113</td><td>0x4b</td><td>01001011</td><td></td><td></td><td></td><td>Latin capital letter K</td></tr>
<tr><td>L</td><td>76</td><td>0114</td><td>0x4c</td><td>01001100</td><td></td><td></td><td></td><td>Latin capital letter L</td></tr>
<tr><td>M</td><td>77</td><td>0115</td><td>0x4d</td><td>01001101</td><td></td><td></td><td></td><td>Latin capital letter M</td></tr>
<tr><td>N</td><td>78</td><td>0116</td><td>0x4e</td><td>01001110</td><td></td><td></td><td></td><td>Latin capital letter N</td></tr>
<tr><td>O</td><td>79</td><td>0117</td><td>0x4f</td><td>01001111</td><td></td><td></td><td></td><td>Latin capital letter O</td></tr>
<tr><td>P</td><td>80</td><td>0120</td><td>0x50</td><td>01010000</td><td></td><td></td><td></td><td>Latin capital letter P</td></tr>
<tr><td>Q</td><td>81</td><td>0121</td><td>0x51</td><td>01010001</td><td></td><td></td><td></td><td>Latin capital letter Q</td></tr>
<tr><td>R</td><td>82</td><td>0122</td><td>0x52</td><td>01010010</td><td></td><td></td><td></td><td>Latin capital letter R</td></tr>
<tr><td>S</td><td>83</td><td>0123</td><td>0x53</td><td>01010011</td><td></td><td></td><td></td><td>Latin capital letter S</td></tr>
<tr><td>T</td><td>84</td><td>0124</td><td>0x54</td><td>01010100</td><td></td><td></td><td></td><td>Latin capital letter T</td></tr>
<tr><td>U</td><td>85</td><td>0125</td><td>0x55</td><td>01010101</td><td></td><td></td><td></td><td>Latin capital letter U</td></tr>
<tr><td>V</td><td>86</td><td>0126</td><td>0x56</td><td>01010110</td><td></td><td></td><td></td><td>Latin capital letter V</td></tr>
<tr><td>W</td><td>87</td><td>0127</td><td>0x57</td><td>01010111</td><td></td><td></td><td></td><td>Latin capital letter W</td></tr>
<tr><td>X</td><td>88</td><td>0130</td><td>0x58</td><td>01011000</td><td></td><td></td><td></td><td>Latin capital letter X</td></tr>
<tr><td>Y</td><td>89</td><td>0131</td><td>0x59</td><td>01011001</td><td></td><td></td><td></td><td>Latin capital letter Y</td></tr>
<tr><td>Z</td><td>90</td><td>0132</td><td>0x5a</td><td>01011010</td><td></td><td></td><td></td><td>Latin capital letter Z</td></tr>
<tr><td>[</td><td>91</td><td>0133</td><td>0x5b</td><td>01011011</td><td></td><td></td><td></td><td>Left square bracket</td></tr>
<tr><td>\</td><td>92</td><td>0134</td><td>0x5c</td><td>01011100</td><td></td><td>\\</td><td></td><td>Reverse solidus (backslash)</td></tr>
<tr><td>]</td><td>93</td><td>0135</td><td>0x5d</td><td>01011101</td><td></td><td></td><td></td><td>Right square bracket</td></tr>
<tr><td>^</td><td>94</td><td>0136</td><td>0x5e</td><td>01011110</td><td></td><td></td><td></td><td>Circumflex accent (caret)</td></tr>
<tr><td>_</td><td>95</td><td>0137</td><td>0x5f</td><td>01011111</td><td></td><td></td><td></td><td>Low line (underscore)</td></tr>
<tr><td>`</td><td>96</td><td>0140</td><td>0x60</td><td>01100000</td><td></td><td></td><td></td><td>Grave accent (backtick)</td></tr>
<tr><td>a</td><td>97</td><td>0141</td><td>0x61</td><td>01100001</td><td></td><td></td><td></td><td>Latin small letter a</td></tr>
<tr><td>b</td><td>98</td><td>0142</td><td>0x62</td><td>01100010</td><td></td><td></td><td></td><td>Latin small letter b</td></tr>
<tr><td>c</td><td>99</td><td>0143</td><td>0x63</td><td>01100011</td><td></td><td></td><td></td><td>Latin small letter c</td></tr>
<tr><td>d</td><td>100</td><td>0144</td><td>0x64</td><td>01100100</td><td></td><td></td><td></td><td>Latin small letter d</td></tr>
<tr><td>e</td><td>101</td><td>0145</td><td>0x65</td><td>01100101</td><td></td><td></td><td></td><td>Latin small letter e</td></tr>
<tr><td>f</td><td>102</td><td>0146</td><td>0x66</td><td>01100110</td><td></td><td></td><td></td><td>Latin small letter f</td></tr>
<tr><td>g</td><td>103</td><td>0147</td><td>0x67</td><td>01100111</td><td></td><td></td><td></td><td>Latin small letter g</td></tr>
<tr><td>h</td><td>104</td><td>0150</td><td>0x68</td><td>01101000</td><td></td><td></td><td></td><td>Latin small letter h</td></tr>
<tr><td>i</td><td>105</td><td>0151</td><td>0x69</td><td>01101001</td><td></td><td></td><td></td><td>Latin small letter i</td></tr>
<tr><td>j</td><td>106</td><td>0152</td><td>0x6a</td><td>01101010</td><td></td><td></td><td></td><td>Latin small letter j</td></tr>
<tr><td>k</td><td>107</td><td>0153</td><td>0x6b</td><td>01101011</td><td></td><td></td><td></td><td>Latin small letter k</td></tr>
<tr><td>l</td><td>108</td><td>0154</td><td>0x6c</td><td>01101100</td><td></td><td></td><td></td><td>Latin small letter l</td></tr>
<tr><td>m</td><td>109</td><td>0155</td><td>0x6d</td><td>01101101</td><td></td><td></td><td></td><td>Latin small letter m</td></tr>
<tr><td>n</td><td>110</td><td>0156</td><td>0x6e</td><td>01101110</td><td></td><td></td><td></td><td>Latin small letter n</td></tr>
<tr><td>o</td><td>111</td><td>0157</td><td>0x6f</td><td>01101111</td><td></td><td></td><td></td><td>Latin small letter o</td></tr>
<tr><td>p</td><td>112</td><td>0160</td><td>0x70</td><td>01110000</td><td></td><td></td><td></td><td>Latin small letter p</td></tr>
<tr><td>q</td><td>113</td><td>0161</td><td>0x71</td><td>01110001</td><td></td><td></td><td></td><td>Latin small letter q</td></tr>
<tr><td>r</td><td>114</td><td>0162</td><td>0x72</td><td>01110010</td><td></td><td></td><td></td><td>Latin small letter r</td></tr>
<tr><td>s</td><td>115</td><td>0163</td><td>0x73</td><td>01110011</td><td></td><td></td><td></td><td>Latin small letter s</td></tr>
<tr><td>t</td><td>116</td><td>0164</td><td>0x74</td><td>01110100</td><td></td><td></td><td></td><td>Latin small letter t</td></tr>
<tr><td>u</td><td>117</td><td>0165</td><td>0x75</td><td>01110101</td><td></td><td></td><td></td><td>Latin small letter u</td></tr>
<tr><td>v</td><td>118</td><td>0166</td><td>0x76</td><td>01110110</td><td></td><td></td><td></td><td>Latin small letter v</td></tr>
<tr><td>w</td><td>119</td><td>0167</td><td>0x77</td><td>01110111</td><td></td><td></td><td></td><td>Latin small letter w</td></tr>
<tr><td>x</td><td>120</td><td>0170</td><td>0x78</td><td>01111000</td><td></td><td></td><td></td><td>Latin small letter x</td></tr>
<tr><td>y</td><td>121</td><td>0171</td><td>0x79</td><td>01111001</td><td></td><td></td><td></td><td>Latin small letter y</td></tr>
<tr><td>z</td><td>122</td><td>0172</td><td>0x7a</td><td>01111010</td><td></td><td></td><td></td><td>Latin small letter z</td></tr>
<tr><td>{</td><td>123</td><td>0173</td><td>0x7b</td><td>01111011</td><td></td><td></td><td></td><td>Left curly bracket</td></tr>
<tr><td>|</td><td>124</td><td>0174</td><td>0x7c</td><td>01111100</td><td></td><td></td><td></td><td>Vertical line</td></tr>
<tr><td>}</td><td>125</td><td>0175</td><td>0x7d</td><td>01111101</td><td></td><td></td><td></td><td>Right curly bracket</td></tr>
<tr><td>~</td><td>126</td><td>0176</td><td>0x7e</td><td>01111110</td><td></td><td></td><td></td><td>Tilde</td></tr>
<tr><td>(del)</td><td>127</td><td>0177</td><td>0x7f</td><td>01111111</td><td>^?</td><td></td><td>␡</td><td>Delete</td></tr>
</tbody></table>
<p><a href="http://www.bahaistudies.net/asma/ascii.pdf">PDF of the 1963 ASCII specification</a></p>
<p><a href="https://en.wikipedia.org/wiki/Control_character">Control characters on Wikipedia</a></p>
<p><a href="http://www.aivosto.com/vbtips/control-characters.html">Detailed historical
information about ASCII and control characters</a></p>
i c the light2017-05-25T00:00:00+00:002017-05-25T00:00:00+00:00
Rebecca Turner
https://becca.ooo/i-c-the-light/<p>A distance-estimating ray marcher</p>
<p>From <a href="http://blog.hvidtfeldts.net/index.php/2011/06/distance-estimated-3d-fractals-part-i/">Syntopia</a>:</p>
<blockquote>
<p>Classic raytracing shoots one (or more) rays per pixel and calculates where the
rays intersect the geometry in the scene. Normally the geometry is described by
a set of primitives, like triangles or spheres, and some kind of spatial
acceleration structure is used to quickly identify which primitives intersect
the rays.</p>
<p>Distance Estimation, on the other hand, is a ray marching technique.</p>
<p>Instead of calculating the exact intersection between the camera ray and the
geometry, you proceed in small steps along the ray and check how close you are
to the object you are rendering. When you are closer than a certain threshold,
you stop. In order to do this, you must have a function that tells you how
close you are to the object: a Distance Estimator. The value of the distance
estimator tells you how large a step you are allowed to march along the ray,
since you are guaranteed not to hit anything within this radius.</p>
</blockquote>
<ul>
<li><a href="/i-c-the-light.pdf">My paper on the ray marcher I wrote</a></li>
<li><a href="https://github.com/9999years/i-c-the-light">github.com/9999years/i-c-the-light</a></li>
</ul>
<p><img src="/img/i-c-the-light/julia.jpg" alt="A distance-estimated rendering of a pink 3D slice of a 4D quaternion Julia set fractal" /></p>