rebecca® https://becca.ooo the digital home of rebecca turner, software engineer & mathemagician Zola en Mon, 10 Nov 2025 00:00:00 +0000 Vertical Integration is the Only Thing That Matters Mon, 10 Nov 2025 00:00:00 +0000 Rebecca Turner https://becca.ooo/blog/vertical-integration/ https://becca.ooo/blog/vertical-integration/ <p>On the subject of developer tooling, or perhaps computer programs more broadly, I have become increasingly convinced that vertical integration is the only thing that matters. I also think that the inability of developer productivity startups to vertically integrate their offerings has hindered their adoption and utility. I’d like to talk about what I mean by “vertical integration” and why we don’t have it today.</p> <h2 id="what-is-vertical-integration"><a class=anchor href="#what-is-vertical-integration" aria-label="Link to section what-is-vertical-integration">#</a>What is vertical integration?</h2> <p>When I say “vertical integration”, I am referring to tight integration between different tools in a stack.</p> <p>Here are some workflows that are possible with a vertically-integrated stack:</p> <ol> <li> <p>You create a new branch and build the project you’re working on. You haven’t made any changes yet, so the build finishes in under a second because your build system shares artifacts with builds from CI and deploys. This build system also caches artifacts from builds you initiate locally, so a coworker who checks out your branch (also a single click from the PR) will be able to run your code immediately.</p> </li> <li> <p>A test fails in CI. You click a button to open the failing test in your editor, which gives you an interactive call-stack (potentially spanning multiple binaries and programming languages) to explore.</p> </li> <li> <p>A check on your PR fails because your code wasn’t formatted correctly. You click a button to apply the formatting changes directly to your code, and they’re immediately reflected in your editor.</p> </li> <li> <p>You merge a PR which breaks a service. Your PR is initially deployed as a canary to a small portion of clients. The deploy system detects an elevated failure rate, and rolls back your PR automatically. This adds a note to your PR and messages you on your Slack-equivalent. The note indicates which checks failed, and you can click them to see a history of those checks.</p> </li> <li> <p>You click on a function in your editor and are presented with a list of usages of that function. The list includes entries from other projects, written in other languages, and across RPC boundaries.</p> </li> <li> <p>A service crashes in production. A ticket is automatically opened and routed to your team (or a note is added to an existing ticket). You click a button to open the line of code that crashed in your editor, and again you’re shown an interactive call-stack for the failure.</p> </li> </ol> <p>All of these workflows span multiple parts of the stack: local builds and CI builds, the test runner and the editor, the code review system and CI, the deploy system and version control. These features are often derisively referred to as “glue code”. <strong>The glue code <em>is</em> the vertical integration work.</strong></p> <p>I’d like to draw your attention to a couple different themes here:</p> <ol> <li> <p>None of the features here are particularly shocking, but they all require cooperation between tools that aren’t used to cooperating. Your test runner knows the call stack of a failing test, but it can’t make that information available in a format your editor or terminal is able to consume. Your deploy system runs an optimized build and then throws away all the artifacts, so if you want to build the same commit you need to start from scratch. The compilation was already run, but your build system isn’t able to grab artifacts from CI because your build system doesn’t know that you <em>have</em> CI.</p> </li> <li> <p>IDEs make some of these workflows possible, albeit scoped to a single project. This is because IDEs implement glue integrations between a bunch of different test runners, build systems, languages, and so on.</p> </li> <li> <p>Many of these workflows require infrastructure to be running independently of your CI, deploy system, and developer workstations. If you want CI to tell you if a test has failed recently in other builds, you need a system that knows when that test was run and what the results were. If you want to share build artifacts between CI and developer builds, you need a system to cache those artifacts and accurately identify when they can be reused.</p> </li> </ol> <h2 id="why-isn-t-a-vertically-integrated-stack-the-default"><a class=anchor href="#why-isn-t-a-vertically-integrated-stack-the-default" aria-label="Link to section why-isn-t-a-vertically-integrated-stack-the-default">#</a>Why isn’t a vertically-integrated stack the default?</h2> <p>The reasons why vertically-integrated stacks aren’t common are a bit different for open source and industrial users, but industry funds open source so there’s a fair amount of overlap.</p> <h3 id="vertical-integration-in-open-source"><a class=anchor href="#vertical-integration-in-open-source" aria-label="Link to section vertical-integration-in-open-source">#</a>Vertical integration in open source</h3> <p>Open source projects tend to be miniscule. Even projects with hundreds of thousands of lines of code are uncommon, while codebases of that size seem quaint to industrial engineers.</p> <p>While I’m sure that many engineers would enjoy having access to the workflows I described at the start of the post, the motivation is a lot less pressing. Who needs shared build artifacts when a clean optimized build takes a couple minutes? Who needs integration between the test suite and the code editor when the entire codebase fits in the working memory of the only engineer who seriously works on it? And gradual rollouts don’t even make sense when the project doesn’t publicly provide a hosted instance of the software.</p> <p>Integrated stacks also present a coordination issue for open source projects. Does the integration between the test runner and the code editor live in the test runner or the code editor? How is it tested? Open source is full of petty kings and overgrown forum moderators, so this sort of collaboration is rarely feasible in practice.</p> <p>Open source is also full of software freedom acolytes who insist that each tool must “do one thing well.” To these engineers, project A maintaining an integration with project B is a threat to the ability of users to swap out project B for a different tool; the best approach, to them, is for every tool to behave as if no other tool exists. The fact that this results in strictly less-capable tools seems to be lost on these engineers.</p> <p>At the same time, many open source projects are owned or funded largely by a single corporation with no motivation (or ability) to make their internal stack available externally. Any integrations in the project must therefore be compatible with the stack used by those corporations internally. For similar reasons, it is also common to see projects with test suites or build systems that cannot be used outside of the organization that funds them.</p> <h3 id="vertical-integration-in-industry"><a class=anchor href="#vertical-integration-in-industry" aria-label="Link to section vertical-integration-in-industry">#</a>Vertical integration in industry</h3> <p>Industrial users aren’t interested in a smooth developer experience because it makes engineers’ lives easier. Industrial users want a vertically-integrated stack because engineer time is <em>expensive.</em> This means that a vertically-integrated stack must save time in an absolute sense; that is, <em>including</em> the time it takes to implement and adopt the system.</p> <p>These migration costs push industrial users away from tools that span the stack. When startups are small, the overhead of implementing such a system increases the likelihood that the startup will fail. An organization must be stable enough in order for spending time on integration work to be justifiable. But once an organization <em>is</em> stable enough to want to build out a smoother developer experience, they’ve already developed their own bespoke build and deploy system, often spanning multiple repos with complex dependencies and subtle quirks. At this point, the costs of switching to an off-the-shelf vertically-integrated stack are prohibitively large.</p> <p>Organizations that investigate e.g. switching to Bazel will hear horror stories of migration efforts that went on for years before being scrapped. This is another issue with implementing tight vertical integrations across the stack: deep investment is needed, over an extended period of time, in order for benefits to materialize. Many younger organizations used to bringing features from planning to production in one or two quarters find this sort of investment very challenging to justify.</p> <h3 id="vertical-integration-as-a-product"><a class=anchor href="#vertical-integration-as-a-product" aria-label="Link to section vertical-integration-as-a-product">#</a>Vertical integration as a product</h3> <p>Could we build a vertically-integrated development environment as a product and sell it to companies who would otherwise build a similar but less-capable system internally? On the surface, this sounds like a great idea: lots of organizations are devoting a lot of effort to building out pretty similar systems for building, deploying, and editing code. It would be fantastic if we could centralize that effort, saving each organization valuable engineering time while delivering a better experience.</p> <p>But remember what we’re talking about: integrations between a code forge, build system, deploy system, CI, and code editor. Large organizations are either unwilling or unable to adopt radically different tools. Your developer tools startup will then need to build and maintain integrations between <em>N</em> products for each of your <em>M</em> large clients. At the same time, you’ll have to compromise on features; if you can’t ship a patch to GitHub, you can’t build a link to open a failing test case locally into their UI.</p> <p>Selling integrations is also a risky position for a company to be in. There’s a natural incentive for the companies being integrated to write replacement integrations in-house and claw back some of the profits that are going to the external competitor. Vendors selling integrations may also find themselves in an uncomfortable position when the services they’re integrating with raise their prices or restrict API access. Cursor, for example, had to <a href="https://techcrunch.com/2025/07/07/cursor-apologizes-for-unclear-pricing-changes-that-upset-users/">hastily change their pricing</a> after Anthropic changed their prices. Cursor also had to <a href="https://devclass.com/2025/04/08/vs-code-extension-marketplace-wars-cursor-users-hit-roadblocks/">beg Microsoft to unban Cursor</a> from the VS Code extension marketplace. Anthropic and Microsoft both view Cursor as a competitor and would rather they did not exist at all. This is not a foundation for a dynamic of productive collaboration between these companies on integrations between their services!</p> <p>For smaller organizations, adopting an integrated developer environment requires a huge leap of faith. GitHub is a very well-known and mature product. Every enterprise developer tool provides a GitHub integration (not as good as first-party support, but much better than nothing). What if the organization providing your developer tooling shuts down? It will be extremely challenging to convince early adopters that the costs are worth it. The migration costs we discussed earlier also apply to <em>leaving</em> such a system, except that users may find themselves <em>forced</em> to leave such a system with little notice.</p> <p><a href="https://www.scottkennedy.us/developer-cloud.html">Scott Kennedy notes in “It’s not just an IDE: building the developer cloud is hard”</a> that it’s possible to build a <em>piece</em> of the vision successfully, but benefits remain limited without cooperation from the rest of the stack:</p> <blockquote> <p>Startups or open source historically focused on one piece of the puzzle. Sourcegraph has been working hard for a decade just to get the CodeSearch piece right. Bazel might be open source, but it has a “batteries not included” feel. The full experience requires putting many pieces together. It’s hard.</p> </blockquote> <p>When (e.g.) code search and the build system are produced by different corporations, the profit motive discourages corporations from building tight integrations with other products in the same space.</p> <p>The vertical integration, the glue code linking these different products together, really is the thing that creates the value here. And the glue code is inherently tied to the quirks of the infrastructure of the corporation that built it:</p> <blockquote> <p>We’ve also seen dozens of companies successfully take Google/Facebook’s internal tooling ideas and build billion dollar B2B SaaS companies. So where is my developer cloud?</p> <p>It doesn’t exist because it’s really fucking hard.</p> <p>Google can’t make theirs available because they can’t disentangle from their internal architecture without damaging their internal productivity. I know because I was there when they tried. It’s hard.</p> </blockquote> <p>The value comes from the whole stack being tightly integrated. If you can’t ship the whole stack, you’re very limited in the features you can provide. Glue code is very challenging to sell. Vertical integration is the only thing that matters.</p> (Dark Souls Font) WiFi Restored Fri, 26 Sep 2025 00:00:00 +0000 Rebecca Turner https://becca.ooo/blog/wifi-restored/ https://becca.ooo/blog/wifi-restored/ <p><em>Previously:</em> <a href="https://becca.ooo/blog/my-iphone-hates-my-wifi/">My iPhone Hates My WiFi</a>.</p> <p>For about a year, iPhones in my household would frequently disconnect from the wifi, with no clear explanation or cause. After a month of frustration I got desperate enough to spend $300 on a Ruckus R350 access point that didn’t help (NB: an access point is “the thing that makes the wifi”). And then I just… gave up for almost a year, I guess. Well, my blog post a couple weeks ago got me interested in doing a little more debugging. The next time my phone claimed to be connected-but-not to the wifi (with a connection listed in the system settings but not in the status bar at the top of the screen), I went into the connection details to see… nothing in the DHCP section!</p> <p>I confirmed that DHCP seemed messed up the next couple times the connection faltered. Sometimes the whole DHCP section was blank, and sometimes I had a garbage address in 169.254/16. My good friend <a href="https://selectric.space/">Liz</a>, who knows much more about networking than I, informed me that the 169.254/16 block is reserved for <a href="https://en.wikipedia.org/wiki/Link-local_address">link-local addresses</a> and generally indicates a DHCP failure.</p> <p>My landlord has graciously included a $50 surcharge on my monthly rent for mandatory Xfinity internet service, so I’ve been using the Xfinity-provided modem up until now. I’ve never really liked this (there is something very unnerving about Xfinity having my wifi password) and the DHCP logs available to me in the modem’s web interface were nonexistent, so at this point I was forced to accept the conclusion I’d been trying desperately not to reach for almost a year: after spending $300 on a new access point, I needed to pony up even more cash for a new modem. For reasons I don’t really understand, your ISP needs to approve of the specific modem you’re using; you can’t just get a random DOCSIS 3.1 modem. So I went to dig up the list of <a href="https://www.xfinity.com/support/articles/list-of-approved-cable-modems">Xfinity-approved modems</a> and saw the <a href="https://store.ui.com/us/en/category/internet-solutions/collections/pro-internet-solutions/products/uci">Unifi UCI</a> listed as an option. Well, the dolls seem to like the Ubiquiti hardware, so I went for it. Who can resist the allure of bead-blasted galvanized steel and those adorable little touch screens?</p> <p>A couple days later, my shiny new modem arrived, so I unboxed it and plugged it in:</p> <blockquote> <p><time aria-hidden=true>5:22pm</time> <strong>Me:</strong> might be time to set up a modem<br> <time aria-hidden=true>5:22pm</time> <strong>Me:</strong> how hard could it possibly be<br> <time aria-hidden=true>5:26pm</time> <strong>Me:</strong> wait do i need a gateway -_-<br> <time aria-hidden=true>6:10pm</time> <strong>Liz:</strong> …yes.<br> <time aria-hidden=true>6:10pm</time> <strong>Me:</strong> Fuck<br> <time aria-hidden=true>6:11pm</time> <strong>Me:</strong> liz…<br> <time aria-hidden=true>6:11pm</time> <strong>Me:</strong> what is a gateway<br> <time aria-hidden=true>6:11pm</time> <strong>Liz:</strong> some kind of main router, y’know<br> <time aria-hidden=true>6:11pm</time> <strong>Liz:</strong> idk if the uniif <em>[sic]</em> cable modem does that! or if it’s just a modem!<br> <time aria-hidden=true>6:11pm</time> <strong>Liz:</strong> but knowing unifi it’s probably just a modem.<br> <time aria-hidden=true>6:11pm</time> <strong>Me:</strong> i have switches. i have an access point.<br> <time aria-hidden=true>6:11pm</time> <strong>Liz:</strong> but you need a ROUTER<br> <time aria-hidden=true>6:11pm</time> <strong>Me:</strong> <span class=visually-hidden>fuck</span><span aria-hidden=true class=imessage-bloom><span>f</span><span>u</span><span>u</span><span>u</span><span>u</span><span>u</span><span>u</span><span>u</span><span>u</span><span>u</span><span>u</span><span>c</span><span>k</span></span><br> <time aria-hidden=true>6:12pm</time> <strong>Me:</strong> g-d dammit<br> <time aria-hidden=true>6:12pm</time> <strong>Liz:</strong> lmao</p> </blockquote> <p>So I bought <a href="https://store.ui.com/us/en/category/all-cloud-gateways/products/ux7">a Unifi gateway</a>. In for a penny, in for a pound, right? At this point in my adventure a $200 gateway-cum-access-point seemed downright thrifty to me. I got the gateway set up and… everything has worked perfectly. Nobody in my home has found themselves unexpectedly disconnected from the internet. I can finally stop thinking about it!! The Ubiquiti management interface and app are extremely slick, surfacing tons of useful information: latency and throughput graphs, alerting when the internet goes down, and, of course, an (empty!!!) log of DHCP failures.</p> <p>Thus ends the 2024–2025 Bad WiFi Saga!</p> My iPhone Hates My WiFi Thu, 11 Sep 2025 00:00:00 +0000 Rebecca Turner https://becca.ooo/blog/my-iphone-hates-my-wifi/ https://becca.ooo/blog/my-iphone-hates-my-wifi/ <p>Sometimes, my iPhone disconnects from my wifi. The little icon disappears from the status bar. Then, when I go to the settings app to reconnect, it claims that it’s still connected to the wifi. Maddening. What follows is an infuriating ritual of flicking the airplane mode and wifi switches on and off until it will stay connected. Sometimes it will connect, showing a strong signal in the status bar, only for it to immediately disappear. It probably takes 3-5 iterations on average to get it to stick. I am not aware of any way to reproduce the issue, which has greatly hindered my ability to resolve it. I have not been taking notes as thoroughly as I should have. The bug is intermittent. On a ‘bad day’ I will be disconnected from the wifi a handful of times.</p> <p>It started, I think, with iOS 18 (released September 2024). This is one of the things I really wish I’d written down. The bug impacts my partner’s iPhone, too.</p> <p>The network itself isn’t going down. My partner will tell me their phone has disconnected from the wifi while my phone remains perfectly connected. Some Android phones have issues with the network. My MacBook seems largely immune to its issues. Ethernet, of course, remains reliable as ever.</p> <p>I bought a new Ruckus R350 access point in November 2024 to see if that would help. It didn’t. I have not tried replacing my modem, but my ISP did it anyways unprompted. I’m not sure if I’m allowed to <abbr title="Bring Your Own Device">BYOD</abbr>; it’s a big box in a frankly inconvenient spot in my living room with one of those big “it’s a felony to look at this” stickers on it.</p> <p>Do you, reader, have any ideas for debugging this issue, or even reproducing it? Are you trapped in this strange predicament with me? <a href="/contact">Drop me a line.</a></p> My Student Loan Servicer took $560 and kept it for a year Mon, 08 Sep 2025 00:00:00 +0000 Rebecca Turner https://becca.ooo/blog/an-interest-free-loan-2/ https://becca.ooo/blog/an-interest-free-loan-2/ <p>A quick update on last year’s <a href="/blog/an-interest-free-loan">“Giving My Student Loan Servicer a $560 Interest-Free Loan”</a>: they gave me my money back!</p> <h2 id="timeline-recap"><a class=anchor href="#timeline-recap" aria-label="Link to section timeline-recap">#</a>Timeline recap</h2> <p>Here’s the story I covered in my post last year:</p> <ul> <li>2024-04-08: I finish paying off my student loans.</li> <li>2024-04-08: Aidvantage processes my monthly autopay despite my loans having a balance of $0, taking an extra $560 from my bank account and leaving me with a negative balance on my loan.</li> <li>2024-04-15: I submit a support request asking for my money back.</li> <li>2024-04-17: Aidvantage responds and says they’ll return the money in 8-12 weeks.</li> <li>2024-06-24: A refund check is created 68 days after my request (I only find out about this after the fact).</li> <li>2024-07-10: I call Aidvantage to ask where my money is. The agent says that a check was sent out and should have arrived. They do not have any proof that a check was sent out. The agent offers to ACH the refund instead, and says it should take 3-4 weeks.</li> <li>2024-08-15: I call Aidvantage to ask where my money is. The agent says that the ACH was never a possibility, because they have to wait for the check they “sent out” to expire. I am informed it takes 14 months for a check to expire and told to call back on August 25th, 2025.</li> </ul> <h2 id="new-developments"><a class=anchor href="#new-developments" aria-label="Link to section new-developments">#</a>New developments</h2> <p>At some point after all these interactions, the negative balance on my loan disappeared from the website. Luckily, this didn’t turn out to be a problem, but I wasn’t sure I would be able to convince them that they owed me money if it wasn’t visible in their database.</p> <p>I recently received a letter from Aidvantage dated July 3rd, 2025 titled “<strong>ACTION REQUIRED—YOU MAY BE OWED A REFUND</strong>”, asking me to verify my address before a refund check could be created. The letter did not explain <em>how</em> much of a refund I was owed. I called up Aidvantage, verified my address, and they sent me a check. Thankfully, the check actually arrived this time.</p> <p>On July 25th, 2025, the check for $560 cleared and my student loans are finally, <em>truly</em> complete. If I’d been able to invest that money in the S&amp;P 500 over that period, I would have $714 instead of $560. Unfortunately for me, if you have someone’s routing and account numbers, you can just take $560 from their account and hold on to it interest-free for a year. What are they going to do, consult with a lawyer for $600 an hour?</p> How to turn an iPhone's ringer on Fri, 15 Aug 2025 00:00:00 +0000 Rebecca Turner https://becca.ooo/blog/how-to-turn-an-iphones-ringer-on/ https://becca.ooo/blog/how-to-turn-an-iphones-ringer-on/ <h2 id="foreword-missed-calls"><a class=anchor href="#foreword-missed-calls" aria-label="Link to section foreword-missed-calls">#</a>Foreword: Missed calls</h2> <p>Today I had a delivery scheduled for a piece of furniture, so I had my iPhone’s ringer on. This is pretty unusual for me; my policy is that if you want to get on the phone with me, you need to either tell me in advance when you’ll call or be in my contacts. Otherwise, unknown numbers are silenced.</p> <p>I don’t think this pattern of phone usage is that unusual. A <a href="https://www.pewresearch.org/internet/2025/07/31/online-scams-and-attacks-in-america-today/">2025 Pew survey</a> survey found that 68% of adult Americans get scam calls at least weekly and 31% of Americans get scam calls daily. Here’s a Redditor who <a href="https://www.reddit.com/r/AskOldPeople/comments/1dhlytr/how_many_scam_callers_do_you_get_in_a_day/">received 47 scam calls in a single day</a>, and lots of the replies are from people who have settled on my strategy as well:</p> <blockquote> <p>i do not have a landline and my cellphone only rings for numbers in my contact list.</p> </blockquote> <blockquote> <p>If I know the caller, they’re on my contacts list. If they’re calling from an unfamiliar number for some reason, they’ll leave a message and I’ll call right back.</p> </blockquote> <blockquote> <p>I don’t answer unless I know who’s calling.</p> </blockquote> <blockquote> <p>I don’t answer my phone, so it’s not an issue.</p> </blockquote> <p>Sometimes, though, you <em>do</em> need to answer a phone call from a stranger. Maybe you have a bulky delivery scheduled, like me. Perhaps you’re waiting on a call from a potential employer. Or you could be waiting to hear the results of a medical test. Something all of these situations have in common is that missing the phone call has consequences.</p> <p>So you turn your ringer on. How hard could it be?</p> <h2 id="settings-that-control-iphone-call-notifications"><a class=anchor href="#settings-that-control-iphone-call-notifications" aria-label="Link to section settings-that-control-iphone-call-notifications">#</a>Settings that control iPhone call notifications</h2> <p>I’d been burned before, so after I turned my ringer on, I went to Settings → Apps → Phone and disabled Silence Unknown Callers. Do you know what happened next? After work, I noticed a notification for a missed call. It turns out there’s way more settings that can silence calls than I thought. Here’s a list of as many as I can find. If you learn about more settings, please <a href="/contact">let me know</a> and I can add them here.</p> <p>Please note that this article does not cover turning an iPhone’s ringer <em>off</em>, which irritatingly involves a slightly different list of settings. For example, in the Contacts app you can add individual contacts to an “Emergency Bypass” list which causes calls from those contacts to <em>always</em> ring audibly.</p> <h3 id="the-physical-silence-switch"><a class=anchor href="#the-physical-silence-switch" aria-label="Link to section the-physical-silence-switch">#</a>The physical silence switch</h3> <p>On the side of the phone is a physical switch to silence notifications and calls (and also, like, sound effects in games? I don’t really know why it does double-duty like this). Make sure that’s not set to “silent”.</p> <h3 id="settings-sounds-haptics"><a class=anchor href="#settings-sounds-haptics" aria-label="Link to section settings-sounds-haptics">#</a>Settings → Sounds &amp; Haptics</h3> <p>Set the <strong>Ringtone and Alerts</strong> volume to the <strong>maximum</strong> (or whatever you want). If vibration is important as well, set <strong>Haptics</strong> to <strong>Always Play</strong>.</p> <p>If you have <strong>Change with Buttons</strong> enabled, you can use the physical volume up button to change the ringtone volume. Make sure that’s not silent either!</p> <p>If <strong>Change with Buttons</strong> is <em>not</em> enabled, then the physical volume buttons will have <em>no effect</em> on the ringtone volume. Many users find this unintuitive, to say the least.</p> <p>I saw someone say to make sure you have a sound assigned to <strong>Ringtone</strong>, but I couldn’t see a way to <em>not</em> have a sound assigned, so that might be a relic from a previous iOS version.</p> <h3 id="settings-apps-phone"><a class=anchor href="#settings-apps-phone" aria-label="Link to section settings-apps-phone">#</a>Settings → Apps → Phone</h3> <p>Set <strong>Silence Unknown Callers</strong> to <strong>Off</strong>.</p> <p>Under <strong>Call Blocking &amp; Identification</strong>, set <strong>Silence Junk Callers</strong> to <strong>Off</strong>.</p> <p>Under <strong>Blocked Contacts</strong>, make sure the phone number you’re expecting a call from (lol, lmao, they never tell you ahead of time) isn’t present.</p> <p>Also, make sure that <strong>Call Forwarding</strong> is set to <strong>Off</strong>, although that setting doesn’t exist on my phone.</p> <p>I saw someone online say that they needed to set <strong>Announce Calls</strong> to <strong>Always</strong>, but I think that setting just makes Siri read information about the incoming caller out loud and it’s not actually related to the ringer (I tested this).</p> <h3 id="do-not-disturb"><a class=anchor href="#do-not-disturb" aria-label="Link to section do-not-disturb">#</a>Do Not Disturb</h3> <p>Open the <strong>Control Center</strong> and make sure <strong>Do Not Disturb</strong> is not enabled. You should see a gray button that says “Focus” with a gray moon icon next to it. If <strong>Do Not Disturb</strong> is enabled, the button will say “Do Not Disturb” and have a blue moon icon next to it.</p> <p>If it’s in a different focus mode, you may want to disable that. Focus modes can be customized under <strong>Settings → Focus</strong>:</p> <ul> <li>For a given focus mode, go to <strong>Notifications → Apps</strong> and make sure that if it’s in “Silence Notifications From” mode that the “Phone” app is <em>not</em> present, and if it’s in “Allow Notifications From” mode that the “Phone” app <em>is</em> present.</li> <li>Under <strong>Notifications → People</strong>, make sure that <strong>Allow Calls from Silenced People</strong> is enabled.</li> <li>Under <strong>Focus Filters</strong>, make sure that <strong>Silent Mode</strong> is either not present or set to <strong>Turn Off</strong>.</li> </ul> <p>Note that focus modes (including Do Not Disturb) can be activated automatically on a schedule, so make sure that no focus modes will be enabled while you’re waiting for a call. In particular, the <strong>Sleep</strong> focus mode is automatically activated if you have a sleep schedule set in the <strong>Clock</strong> app under the <strong>Alarms</strong> tab. The <strong>Sleep</strong> focus mode schedule will show up in the settings like the other focus modes, though, so you don’t need to go to the <strong>Clock</strong> app specifically to turn it off.</p> <h3 id="settings-screen-time-communication-limits"><a class=anchor href="#settings-screen-time-communication-limits" aria-label="Link to section settings-screen-time-communication-limits">#</a>Settings → Screen Time → Communication Limits</h3> <p>Set <strong>During Screen Time</strong> to <strong>Everyone</strong>. Set <strong>During Downtime</strong> to <strong>Everyone</strong>.</p> <h3 id="settings-face-id-passcode-attention-aware-features"><a class=anchor href="#settings-face-id-passcode-attention-aware-features" aria-label="Link to section settings-face-id-passcode-attention-aware-features">#</a>Settings → Face ID &amp; Passcode → Attention Aware Features</h3> <p>Make sure that <strong>Attention Aware Features</strong> is <strong>disabled</strong>. If these are enabled, your iPhone may silence calls if it notices you looking at it (!!!!!).</p> <h3 id="settings-accessibility-touch-call-audio-routing"><a class=anchor href="#settings-accessibility-touch-call-audio-routing" aria-label="Link to section settings-accessibility-touch-call-audio-routing">#</a>Settings → Accessibility → Touch → Call Audio Routing</h3> <p>Under the <strong>Touch</strong> section, for some fucking reason, you can set <strong>Call Audio Routing</strong> to be “Automatic”, “Bluetooth Headset”, or “Speaker”. If you will not hear your phone’s speaker, or if you will not hear a connected Bluetooth headset, pick the other option.</p> <p>Also under <strong>Call Audio Routing</strong>, set <strong>Auto-Answer Calls</strong> to <strong>Off</strong>. (Why is this under “Call Audio Routing”??)</p> <h3 id="settings-accessibility-hearing-devices"><a class=anchor href="#settings-accessibility-hearing-devices" aria-label="Link to section settings-accessibility-hearing-devices">#</a>Settings → Accessibility → Hearing Devices</h3> <p>I think there’s some relevant settings for <strong>Hearing Devices</strong>, but I don’t have any of those, so I can’t help you here. Here’s <a href="https://support.apple.com/guide/iphone/use-hearing-devices-iph470b1833/ios">a page on using hearing devices from the iPhone User Guide.</a></p> <h3 id="headset-mode"><a class=anchor href="#headset-mode" aria-label="Link to section headset-mode">#</a>Headset mode</h3> <p>Maybe some phones can get stuck in “headphone mode” or “headset mode”? Unclear. In the <strong>Control Center</strong>, tap the <strong>AirPlay icon</strong> and make sure you’re not playing audio to headphones unless you want to be doing that.</p> <h3 id="notification-settings-on-your-mac"><a class=anchor href="#notification-settings-on-your-mac" aria-label="Link to section notification-settings-on-your-mac">#</a>Notification settings on your Mac</h3> <p>In the Settings app on your Mac, under <strong>Focus</strong>, if you have <strong>Share across devices</strong> enabled (NB: setting names are lowercase on macOS?), then enabling (e.g.) Do Not Disturb on your Mac will also enable it on your phone.</p> <h2 id="afterword-build-predictable-systems"><a class=anchor href="#afterword-build-predictable-systems" aria-label="Link to section afterword-build-predictable-systems">#</a>Afterword: Build predictable systems</h2> <p>I could say a lot more about this, but I need to feed my cats, so I’ll keep it brief. One of the biggest problems with the status quo here is that it’s not easy to <strong>predict</strong> if a given call will be silenced or not. You’ve disabled “Silence Unknown Callers”. Will your phone ring when the deliver person calls? Until it happens, there’s no way to know. You should be able to type in a phone number and have your phone tell you if it will ring or not. Nobody should ever be <em>surprised</em> by the behavior of their phone’s <em>ringer</em>.</p> <p>Also, give me a big button that says “if the ringer does not go off for my next phone call I will do something drastic and unreasonable.” It could even be one of those fancy new Control Center widgets they added in iOS 18. Call me, Apple, let’s link and build this.</p> macOS dotfiles should not go in ~/Library/Application Support Mon, 17 Mar 2025 00:00:00 +0000 Rebecca Turner https://becca.ooo/blog/macos-dotfiles/ https://becca.ooo/blog/macos-dotfiles/ <p>One of my pet peeves is when command-line tools look for user configuration files in <code>~/Library/Application Support</code> when running on macOS. In addition to offering poor ergonomics for users, I believe this behavior is <em>incorrect</em> according to the documentation which is cited to justify it. Instead, command-line tools should implement the <a href="https://specifications.freedesktop.org/basedir-spec/latest/">XDG Base Directory Specification</a> and look for configuration files in <code>$XDG_CONFIG_HOME</code>, which defaults to <code>~/.config</code>.</p> <p>Usually, when a program looks for configuration files in <code>~/Library/Application Support</code>, it’s not because of an intentional design decision. Instead, the author delegated that decision to a library, and several popular libraries for determining platform-specific configuration directories use <code>~/Library/Application Support</code> as the default configuration directory on macOS, including <a href="https://pypi.org/project/platformdirs/">Python’s <code>platformdirs</code> package</a> (<a href="https://pypistats.org/packages/platformdirs">242 million downloads / month</a>), <a href="https://www.npmjs.com/package/env-paths">JavaScript’s <code>env-paths</code></a> (95 million downloads / month), <a href="https://lib.rs/crates/dirs">Rust’s <code>dirs</code> crate</a> (4.8 million downloads / month), and <a href="https://pkg.go.dev/github.com/adrg/xdg">Go’s <code>adrg/xdg</code> package</a> (used in 913 packages). While <code>~/Library/Application Support</code> may be the correct configuration directory for <em>GUI apps</em> which manage their configuration files for the user automatically, the vast majority of applications using these libraries are command-line utilities whose configuration belongs in <code>~/.config</code>.</p> <p>The main reason I dislike programs looking for dotfiles in <code>~/Library/Application Support</code> is that it’s <em>unexpected.</em> As <a href="https://github.com/nushell/nushell/issues/893#issuecomment-1736736555">Ashlin Eldridge explained on GitHub</a>:</p> <blockquote> <p>As a new user, it’s extremely surprising to me to that a modern tool like <code>nu</code> would put config files under Application Support on macOS. It’s even more surprising to see that some people are <a href="https://github.com/nushell/nushell/discussions/5279">actively</a> <a href="https://news.ycombinator.com/item?id=33425312">against</a> adopting the XDG standard.</p> <p>The origins of XDG no longer matter at this point. No one really cares what the X, D, and G stand for. The fact is that <a href="https://wiki.archlinux.jp/index.php/XDG_Base_Directory">hundreds</a> of tools support it (including Git, Emacs, Neovim, Tmux) and users have come to expect support for it. It’s the closest thing we have to a standard that allows users to control the placement of their config files (which facilitates users’ ability to source control their files) and it also includes support for files that shouldn’t be source controlled such as cache and data files.</p> </blockquote> <p>If a user explains that they found a program’s behavior surprising, many engineers will respond by telling the user that their expectations were wrong. However, I find that the reason users form an incorrect mental model of a program’s behavior is usually because the program’s design violated an established convention from some broader context. In accordance with <a href="https://en.wikipedia.org/wiki/Principle_of_least_astonishment">the Principle of Least Astonishment</a>, I prefer to change the program to behave how users expect rather than futilely attempt to change what users expect.</p> <p>Users expect macOS CLI tools to look for configuration files in <code>~/.config</code> is because <strong>that’s what almost every other program does.</strong> Your program should probably not disagree with Bash and Vim and Git about where configuration files are supposed to go!</p> <p>These conventions help us explore unfamiliar systems by letting us apply knowledge about one part of the system to other parts of the system. Consistency is predictability. When every tool works the same except for a small handful, users get justifiably annoyed at the exceptions for wasting their time.</p> <h2 id="what-do-dotfile-managers-do"><a class=anchor href="#what-do-dotfile-managers-do" aria-label="Link to section what-do-dotfile-managers-do">#</a>What do dotfile managers do?</h2> <p>Many engineers use (or maintain) some sort of dotfile manager to handle tracking these configuration files. If configuration files are really supposed to go in <code>~/Library/Application Support</code>, we would expect dotfile managers to put configuration files there by default when running on macOS. After all, the whole point of a dotfile manager is to simplify managing configuration files across multiple machines and platforms. Let’s take a quick survey to see how some popular dotfile managers behave on macOS:</p> <ul> <li><a href="https://www.chezmoi.io/">chezmoi</a> (14.3k stars) makes no effort to link configuration to <code>~/Library/Application Support</code>, despite including <a href="https://www.chezmoi.io/user-guide/machines/macos/">documentation for using chezmoi on macOS specifically</a>.</li> <li><a href="https://github.com/anishathalye/dotbot">dotbot</a> (7.3k stars) ignores <code>~/Library/Application Support</code>. Despite including macOS-specific configuration files in the examples, the README shows a configuration for VS Code that will be completely ignored on macOS.</li> <li><a href="https://github.com/yadm-dev/yadm">yadm</a> (5.5k stars) makes no effort to support <code>~/Library/Application Support</code>. The example configurations include macOS-specific portions but also do not reference <code>~/Library/Application Support</code>.</li> <li><a href="https://github.com/thoughtbot/rcm">rcm</a> (3.2k stars) makes no effort to support <code>~/Library/Application Support</code>.</li> <li><a href="https://www.gnu.org/software/stow/">GNU Stow</a> makes (unsurprisingly) no effort to support <code>~/Library/Application Support</code>.</li> </ul> <p>Most of these tools are <em>able</em> to link configuration files to <code>~/Library/Application Support</code> if you ask them to explicitly (although macOS will regularly replace your symlinks with copies of the destination files), but the fact that they don’t bother to do so by default is revealing. Users only want this behavior for specific misbehaving programs.</p> <h2 id="an-appeal-to-logic"><a class=anchor href="#an-appeal-to-logic" aria-label="Link to section an-appeal-to-logic">#</a>An appeal to logic</h2> <p>Perhaps “consistency” isn’t a good enough reason for you to place your program’s configuration files in <code>~/.config</code> like everybody else. Maybe you think that programs should follow the platform guidelines, even if they don’t match developer expectations. <a href="https://github.com/dirs-dev/directories-rs/issues/47#issuecomment-478337412">@soc on GitHub</a> phrases it like this:</p> <blockquote> <p>As a general advice for macOS devs: If you are developing on their platform, do what the platform owners tell you. They are the lords, you are the share-croppers.</p> </blockquote> <p>Elsewhere, <a href="https://github.com/nushell/nushell/issues/12103#issuecomment-1982144513">Reilly Wood rebukes</a> a user asking if there would “be any downside to just not using <code>Application Support</code> ever” by saying that “[Nushell would] no longer be following the <a href="https://developer.apple.com/library/archive/documentation/FileManagement/Conceptual/FileSystemProgrammingGuide/FileSystemOverview/FileSystemOverview.html#//apple_ref/doc/uid/TP40010672-CH2-SW6">macOS Standard Directories guidelines</a>.”</p> <p>We’ll read through those guidelines in a minute, but it’s not entirely clear to me that they’re relevant in the first place. The <a href="https://specifications.freedesktop.org/basedir-spec/latest/">XDG Base Directory Specification</a> mentions Unix a few times but lists no carveouts for macOS or any other operating system. If <code>~/.config</code> is accepted as the standard location for configuration files on Unix-like operating systems, then surely it would be the standard location on macOS as well, given that macOS is a Unix by way of BSD.</p> <p>But suppose we accept that the XDG specification only applies to <em>some</em> Unix operating systems, despite making no mention of this. The <a href="https://developer.apple.com/library/archive/documentation/FileManagement/Conceptual/FileSystemProgrammingGuide/FileSystemOverview/FileSystemOverview.html#//apple_ref/doc/uid/TP40010672-CH2-SW6">macOS Standard Directories</a> documentation starts by stating that “[w]hether provided by the system or created by your app, every file has its place in macOS”, and honestly we could stop reading right there, because <strong>a command-line tool is not the system or an app.</strong></p> <p>While this might seem like a pedantic nitpick, in reality it’s <em>critical</em> to correctly interpreting this documentation! I suspect that many of the developers who cite these guidelines do not know very much about macOS and think that “app” is just what macOS calls executables, but that’s not true. A couple screens down, the same page says that <code>/bin</code> “[c]ontains essential command-line binaries”, so we can be sure the authors aren’t conflating apps and command-line tools. Apps on macOS are installed in <code>/Applications</code> and are subject to <a href="https://developer.apple.com/documentation/xcode/preparing-your-app-for-distribution">a number of additional requirements</a>, from a bundle ID (a unique reverse-DNS identifier for an app, like <code>com.apple.Preview</code> for <code>Preview.app</code>) to application icons, launch screens, code signing, notarization, app sandboxing, and runtime hardening. Needless to say, none of the developers shipping command-line utilities are following any of <em>these</em> guidelines.</p> <p>Moving on, <a href="https://developer.apple.com/library/archive/documentation/FileManagement/Conceptual/FileSystemProgrammingGuide/MacOSXDirectories/MacOSXDirectories.html#//apple_ref/doc/uid/TP40010672-CH10-SW1">the documentation for <code>~/Library/Application Support</code></a> says that it:</p> <blockquote> <p>Contains all app-specific data and support files. These are the files that your app creates and manages on behalf of the user and can include files that contain user data.</p> </blockquote> <p>Again, command-line tools do not have <em>any</em> app-specific data because they are not apps. Further, dotfiles are usually written by hand and not managed <em>on behalf of</em> the user.</p> <p>The next paragraph makes it very clear that <code>~/Library/Application Support</code> is for apps only:</p> <blockquote> <p>By convention, all of these items should be put in a subdirectory whose name matches the bundle identifier of the app. For example, if your app is named <code>MyApp</code> and has the bundle identifier <code>com.example.MyApp</code>, you would put your app’s user-specific data files and resources in the <code>~/Library/Application Support/com.example.MyApp/</code> directory. Your app is responsible for creating this directory as needed.</p> </blockquote> <p>Command-line tools do not have a bundle identifier, including many command-line tools shipped with macOS by Apple (such as <code>ls</code> or <code>vim</code>). And even though <a href="https://git.lix.systems/lix-project/lix/issues/515">Apple feels very comfortable shipping command-line tools that behave in a subtly different manner from their equivalents on other modern Unix-descendent operating systems,</a> the macOS variants of <code>bash</code>, <code>zsh</code>, <code>git</code>, and <code>vim</code> all look for their configuration files in the same place: <code>~/.config</code>. Unless your tool is installed in <code>/Applications</code>, there’s no reason for its configuration to live in <code>~/Library/Application Support</code>.</p> <h2 id="when-should-you-use-library-application-support"><a class=anchor href="#when-should-you-use-library-application-support" aria-label="Link to section when-should-you-use-library-application-support">#</a>When <em>should</em> you use <code>~/Library/Application Support</code></h2> <p>With the guidelines and conventions listed above in mind, your application should store its configuration files in <code>~/Library/Application Support</code> instead of <code>$XDG_CONFIG_HOME</code> if both of the following conditions apply:</p> <ol> <li>It is a GUI application installed in <code>/Applications</code> or <code>~/Applications</code>.</li> <li>It manages its configuration files automatically on behalf of the user, rather than expecting the user to write their configuration in a text file themself.</li> </ol> <h2 id="tl-dr"><a class=anchor href="#tl-dr" aria-label="Link to section tl-dr">#</a>TL;DR</h2> <p>Users do not expect command-line tools to look for configuration files in <code>~/Library/Application Support</code> on macOS. Dotfile managers do not place configuration files in <code>~/Library/Application Support</code> on macOS. The most commonly-cited justifications for placing configuration files in <code>~/Library/Application Support</code> are not meant to apply to command-line tools at all. Even command-line tools like <code>bash</code> and <code>git</code> shipped by Apple look for their configuration files in <code>~/.config</code>.</p> <p>Please, please, <em>please</em> just use the <a href="https://specifications.freedesktop.org/basedir-spec/latest/">XDG Base Directory Specification</a>.</p> Global State in Haskell with IORef Thu, 02 Jan 2025 00:00:00 +0000 Rebecca Turner https://becca.ooo/blog/global-state-in-haskell-with-ioref/ https://becca.ooo/blog/global-state-in-haskell-with-ioref/ <p>AKA “The <a href="https://hackage.haskell.org/package/base/docs/Data-IORef.html"><code>IORef</code></a> trick”.</p> <p>In Haskell, functions are pure and there’s no global state. But sometimes global state is convenient! If you want to add some extra bit of configuration that you would need to thread through some large portion of code in a fairly mechanical fashion (for example, “are we running in verbose mode” or “is this a dry run”) and your program is written in <code>IO</code> or some other context that’s not very amenable to being extended with additional data, or if you’re prototyping a solution before implementing it “the right way”, then a little bit of mutable global state can be a very useful thing indeed. To be clear, I’m not saying you should add global state to your programs. In fact, I will caution you against it: this is the sort of thing that gets your PRs rejected.</p> <p>This technique, which I will call “the <a href="https://hackage.haskell.org/package/base/docs/Data-IORef.html"><code>IORef</code></a> trick”, allows us to implement a global variable that can be read or written in any <code>IO</code> action. I believe the <code>IORef</code> trick is fairly well-known, but not talked about much — I couldn’t find it documented anywhere when I needed it (and of course I didn’t <em>really</em> need it). I suspect that this is because your coworkers are worried you will start littering global mutable state everywhere you go, if only you knew how, and they would rather you do things “the right way” and. Admittedly, they’re probably correct, so be responsible with this technique!</p> <h2 id="the-ioref-trick"><a class=anchor href="#the-ioref-trick" aria-label="Link to section the-ioref-trick">#</a>The <code>IORef</code> trick</h2> <p>Here’s how the trick works:</p> <pre data-lang="haskell" class="language-haskell "><code class="language-haskell" data-lang="haskell">module MyParameter (getMyParameter, setMyParameter) where import Data.IORef (IORef, atomicModifyIORef&#x27;, newIORef) import System.IO.Unsafe (unsafePerformIO) {-# NOINLINE myParameterRef #-} myParameterRef :: IORef Int myParameterRef = unsafePerformIO (newIORef 0) setMyParameter :: Int -&gt; IO () setMyParameter newValue = atomicModifyIORef&#x27; myParameterRef (\_ -&gt; (newValue, ())) getMyParameter :: IO Int getMyParameter = atomicModifyIORef&#x27; myParameterRef (\value -&gt; (value, value)) </code></pre> <p>First, we create a new <a href="https://hackage.haskell.org/package/base/docs/Data-IORef.html"><code>IORef</code></a> called <code>myParameterRef</code> to store our global variable and initialize it with its default value. This is where the naughtiness lives: we have to use <a href="https://hackage.haskell.org/package/base/docs/System-IO-Unsafe.html#v:unsafePerformIO"><code>unsafePerformIO</code></a> to create the <code>IORef</code>. It’s very important to mark this value as <a href="https://www.haskellforall.com/2024/11/the-haskell-inlining-and-specialization.html#noinline"><code>NOINLINE</code></a> or GHC may end up running <code>newIORef</code> multiple times, causing you an enormous headache. If your global variable has no sensible default value, you may want to adapt this code to work with an <a href="https://hackage.haskell.org/package/base/docs/Control-Concurrent-MVar.html"><code>MVar</code></a> instead.</p> <p>Next, we write a setter for our variable using <code>atomicModifyIORef'</code>. Evaluating <code>atomicModifyIORef' ref (\old -&gt; (new, result))</code> will set <code>ref</code> to <code>new</code> and return <code>result</code>. Note that the supplied lambda has access to the previous value as well. (This is explained in the documentation for <code>atomicModifyIORef</code>, but only in the <em>seventh</em> paragraph, so I thought it might be worth calling attention to.)</p> <p>We could simplify <code>setMyParameter</code> by writing it in terms of <code>atomicWriteIORef</code>, but there’s no corresponding <code>atomicReadIORef</code> function so we’ll need to import <code>atomicModifyIORef'</code> to write the getter.</p> <p>Finally, we write a getter in terms of <code>atomicModifyIORef'</code>, which re-stores the old value in the <code>IORef</code> again and then returns it.</p> <h2 id="strictness"><a class=anchor href="#strictness" aria-label="Link to section strictness">#</a>Strictness</h2> <p>The single quote at the end of <code>atomicModifyIORef'</code> means (in this context) that it’s the strict version of <code>atomicModifyIORef</code>. (You may hear this symbol referred to as a tick mark or a <a href="https://en.wikipedia.org/wiki/Prime_(symbol)">prime</a>.) Strictness here means that running <code>setMyParameter newValue</code> will evaluate <code>newValue</code> to <a href="https://stackoverflow.com/a/6889335">weak head normal form</a>. One alternative is to leave the value unevaluated, which may cause <a href="https://stackoverflow.com/a/5892699">lazy IO</a> problems, <a href="https://stackoverflow.com/a/61670171">memory leaks</a>, or <a href="http://blog.ezyang.com/2011/05/anatomy-of-a-thunk-leak/">thunk buildup</a>. Another alternative is to use <a href="https://hackage.haskell.org/package/deepseq/docs/Control-DeepSeq.html"><code>deepseq</code></a> to fully force the value. I think that <code>atomicModifyIORef'</code> is a good default but the correct choice will depend on the semantics of your particular application.</p> <h2 id="thread-safety"><a class=anchor href="#thread-safety" aria-label="Link to section thread-safety">#</a>Thread safety</h2> <p>The “atomic” in <code>atomicModifyIORef'</code> means that a <a href="https://stackoverflow.com/a/286705">memory fence</a> is added when executing this function, which will make calls to the getter and setter return consistent values even with <a href="https://mortoray.com/cpu-reordering-what-is-actually-being-reordered/">instruction reordering</a> on multiple threads at the cost of some performance. If you don’t require thread safety, you can rewrite the getter and setter in terms of <code>readIORef</code> and <code>writeIORef</code>.</p> Reading git range-diff output Wed, 11 Dec 2024 00:00:00 +0000 Rebecca Turner https://becca.ooo/blog/git-range-diff/ https://becca.ooo/blog/git-range-diff/ <p>The <a href="https://git-scm.com/docs/git-range-diff"><code>git range-diff</code></a> command compares two commit ranges. You can use this to check how a branch changed before and after a rebase, or to find differences between a commit on your <code>main</code> branch and a corresponding backport on a long-term-support branch.</p> <p>If you’ve ever tried to use <code>git diff</code> to figure out the changes introduced by resolving merge conflicts, you might understand the pain point this helps address — <code>git diff</code> shows you <em>all</em> the changes between two commits, so you get all the changes introduced by the merge as well as all the conflict resolutions. <code>git range-diff</code> factors these changes out and makes it a lot easier to see how a branch changed over time. (It is an endless pet peeve of mine that <a href="https://mitchellh.com/writing/github-changesets">GitHub does not give any visibility into how a PR changes when you push commits to it</a>. As a result, many developers who have only used version control through GitHub have no idea that this sort of tool is even possible, despite being trivial to construct.)</p> <p>I’ve started to use <code>git range-diff</code> more lately, but I’ve been somewhat baffled by the output format, which the man page considers unworthy of explanation. Last night I read some of the Git source code to figure out how <code>git range-diff</code> works — here’s what I found.</p> <h2 id="diffs-of-diffs"><a class=anchor href="#diffs-of-diffs" aria-label="Link to section diffs-of-diffs">#</a>Diffs of diffs</h2> <p><strong><code>git range-diff</code> runs two <code>git diff</code> operations and then <code>diff</code>s the results.</strong> The output syntax is not very special — it’s <em>literally</em> just a diff of diffs. The first column of diff markers shows you how the diffs changed, and the second column shows you the diff itself.</p> <p>Let’s walk through a set of changes to learn how to read diffs of diffs. Suppose we have a shell script which finds YAML files:</p> <pre data-lang="bash" class="language-bash "><code class="language-bash" data-lang="bash">find \ -path &#x27;*.yaml&#x27; </code></pre> <p>You notice that it’s not picking up files with the <code>.yml</code> extension, so you add another clause:</p> <pre data-lang="bash" class="language-bash "><code class="language-bash" data-lang="bash">find \ -path &#x27;*.yaml&#x27; \ -o -path &#x27;*.yml&#x27; </code></pre> <p>Meanwhile, your colleague runs <a href="https://www.shellcheck.net/">ShellCheck</a> on the script, notices that <a href="https://www.shellcheck.net/wiki/SC2185">some implementations of <code>find</code> have no default path</a> and adds one to fix the script on macOS and other BSDs:</p> <pre data-lang="bash" class="language-bash "><code class="language-bash" data-lang="bash">find \ . \ -path &#x27;*.yaml&#x27; </code></pre> <p>You merge in your colleague’s changes, producing this result:</p> <pre data-lang="bash" class="language-bash "><code class="language-bash" data-lang="bash">find \ . \ -path &#x27;*.yaml&#x27; \ -o -path &#x27;*.yml&#x27; </code></pre> <p>Then, you run <code>git range-diff main BEFORE_MERGE AFTER_MERGE</code>, perhaps using <a href="https://git-scm.com/docs/git-reflog"><code>git reflog</code></a> to find the commit hashes before and after merging in the changes from <code>main</code>. Here’s the diff output:</p> <pre data-lang="diff" class="language-diff "><code class="language-diff" data-lang="diff"> ## find-yaml.sh ## @@ - #!&#x2F;usr&#x2F;bin&#x2F;env bash find \ + . \ - -path &#x27;*.yaml&#x27; + -path &#x27;*.yaml&#x27; \ + -o -path &#x27;*.yml&#x27; </code></pre> <p>First, note the <code>- #!/usr/bin/env bash</code> line. Due to the extra argument to <code>find</code> added in <code>main</code>, the changes in the <code>AFTER_MERGE</code> commit start one line later in the file, so the diff’s <em>context</em> starts one line later as well. This shows up as the context line being <em>removed</em>, even though it wasn’t changed on either side!</p> <p>This is a rather confusing result of <code>git range-diff</code>’s extremely naïve implementation — it makes no effort to distinguish between context lines and actual meaningful elements of the diff. This is the sort of obvious UX pitfall that made me assume that <em>surely</em> <code>git range-diff</code> would do something <em>slightly</em> smarter than <em>literally</em> diffing two diffs, but I don’t know why I expected that. <a href="https://stevelosh.com/blog/2013/04/git-koans/">It’s Git, after all.</a></p> <p>Similarly, the next <code>+</code> line shows us that the <code>.</code> argument is present in the <code>AFTER_MERGE</code> commit, but that it wasn’t added by the <code>AFTER_MERGE</code> commit. Unlike <code>git diff</code>, which shows you all changes between two commits, <code>git range-diff</code> will only show you changes from previous commits when they’re present as context in the diffs of the commits you actually care about.</p> <p>The rest of the changes are present in both versions of the diff.</p> <p>To help make it easier to distinguish between a diff marker in the first column and one in the second column, <code>git range-diff</code> applies <a href="https://git-scm.com/docs/git-range-diff#Documentation/git-range-diff.txt---no-dual-color">“dual coloring”</a>, which colors the background (instead of the text) of the first column of diff markers.</p> <p>Suppose you want to make your intent clearer, so you go and add a comment to explain your change:</p> <pre data-lang="bash" class="language-bash "><code class="language-bash" data-lang="bash"># Make sure to find `.yaml` and `.yml` files! find \ . \ -path &#x27;*.yaml&#x27; \ -o -path &#x27;*.yml&#x27; </code></pre> <p>You amend your commit and run the range-diff again. Now, we see this line in the output:</p> <pre data-lang="diff" class="language-diff "><code class="language-diff" data-lang="diff">++# Make sure to find `.yaml` and `.yml` files! </code></pre> <p>This is showing us that we’ve added a line in the <code>AFTER_MERGE</code> commit only. If you’re comparing a commit before and after fixing merge conflicts, this could indicate a new change that was added unintentionally, so pay attention to these!</p> <p>Exercise for the reader: What do <code>--</code>, <code>-+</code>, and <code>+-</code> lines indicate?</p> <h2 id="diffs-of-logs"><a class=anchor href="#diffs-of-logs" aria-label="Link to section diffs-of-logs">#</a>Diffs of logs</h2> <p><code>git diff</code> only deals with the contents of files, but <code>git range-diff</code> deals with ranges of commits, so those need to be compared as well!</p> <p>Here’s how Git shows a series of commits being compared:</p> <pre><code> 1: d17004e2f4 = 1: 92954fddce rts: Tighten up invariants of PACK 2: 42f1801df4 &lt; -: ---------- testsuite: Fix badly escaped literals -: ---------- &gt; 2: cdfd86e951 testsuite: Fix badly escaped literals 3: 4d7afaaa7f = 3: 2dbf88daed rts&#x2F;Interpreter: Improve documentation of TEST*_P instructions 4: 5f7c2d3e99 ! 4: e646db18c4 rts: Annotate BCOs with their Name </code></pre> <p>From left to right, we have:</p> <ol> <li> <p>The position of the left-hand side’s commit in the sequence being compared. For example, <code>1:</code> indicates this is the first commit in the range of commits on the LHS, and <code>-:</code> indicates that no corresponding commit was found on the left-hand side.</p> </li> <li> <p>The (abbreviated) commit hash of the LHS commit, like <code>d17004e2f4</code>.</p> </li> <li> <p>A marker indicating if the commits are identical (<code>=</code>), only present on the LHS (<code>&lt;</code>), only present on the RHS (<code>&gt;</code>), or different (<code>!</code>).</p> </li> <li> <p>The position of the RHS’s commit in the sequence being compared.</p> </li> <li> <p>The (abbreviated) commit hash of the RHS commit, like <code>92954fddce</code>. Note that this can be different than the LHS commit hash even if the diffs are identical, because the parent commits can be different.</p> </li> <li> <p>The subject line of the commit message. If the message differs between the LHS and RHS, the RHS’s commit message is used here and a diff between the two commit messages is shown.</p> </li> </ol> <p>Note that in the middle of the log, we have two commits that Git can’t find a match for in the other side of the sequence:</p> <pre><code> 2: 42f1801df4 &lt; -: ---------- testsuite: Fix badly escaped literals -: ---------- &gt; 2: cdfd86e951 testsuite: Fix badly escaped literals </code></pre> <p>Similar to <a href="https://git-scm.com/docs/git-diff#Documentation/git-diff.txt---find-renamesltngt"><code>git diff</code>’s rename detection</a>, <code>git range-diff</code> sets a threshold beyond which large changes between diffs will be considered entirely different commits. We can adjust these costs by setting <code>--creation-factor=90</code> (a percentage where higher numbers are more forgiving of large changes) to force the comparison of the two diffs to be shown.</p> <h2 id="parting-thoughts"><a class=anchor href="#parting-thoughts" aria-label="Link to section parting-thoughts">#</a>Parting thoughts</h2> <p>I felt pretty silly once I realized that the <code>git range-diff</code> output is <em>literally</em> just a diff of diffs. In fact, when reading the Git source code to figure out how it was formatting the output, <a href="https://jade.fyi/">Jade</a> discovered that <a href="https://github.com/git/git/blob/caacdb5dfd60540ecec30ec479f147f3c8167e11/range-diff.c#L52-L66"><code>git range-diff</code> actually runs <code>git log --patch</code></a> to get the diffs it feeds into the meta-diff. Perhaps the authors assumed the implementation was so obvious that the output needed no explanation!</p> <p>Even if you don’t know quite how the <code>git range-diff</code> output works, it’s still a useful tool for comparing ranges because it filters out a lot of the noise, and if you know the before/after of both ranges it’s usually pretty easy to reconstruct the context.</p> <p>If you haven’t used <code>git range-diff</code> before, give it a try! I’ve been finding it very handy to have in my toolkit lately and wanted to help make it more accessible for more people. Happy rebasing!</p> Bisecting GHC is easier than I thought Thu, 05 Dec 2024 00:00:00 +0000 Rebecca Turner https://becca.ooo/blog/bisecting-ghc/ https://becca.ooo/blog/bisecting-ghc/ <h2 id="introduction"><a class=anchor href="#introduction" aria-label="Link to section introduction">#</a>Introduction</h2> <p>At work, I’m busy upgrading our compiler from <a href="https://www.haskell.org/ghc/" title="GHC, the Glasgow Haskell Compiler">GHC</a> 9.6 to GHC 9.10. A couple days ago I found some failing tests which eventually turned out to be <a href="https://gitlab.haskell.org/ghc/ghc/-/issues/25529">a compiler bug</a>! Simon Peyton Jones took the time to reply with <a href="https://gitlab.haskell.org/ghc/ghc/-/issues/25529#note_597713" title="Simon Peyton Jones left a detailed comment explaining the bug">a very detailed comment</a> explaining how GHC’s constraint solving logic led to the bug, so be sure to check that out if you’re curious about how the bug works.</p> <p>To find the commit that introduced the bug I used <a href="https://git-scm.com/docs/git-bisect"><code>git bisect</code></a> to perform a binary search over the commits. The principle behind <code>git bisect</code> is simple, but building a compiler is a thorny problem in practice.</p> <p><strong>It turns out that building GHC only takes about 12 minutes.</strong> This means that you can search through 1000 commits in about two hours or one million commits in four hours, which is much faster feedback than I assumed before I started! It happens that there’s just about 1000 commits between GHC 9.6 and 9.8, so I was able to get this done in about a day, including the time it took me to write and iron out the bisection script. (Note that the 12 minute figure is for a <a href="https://en.wikipedia.org/wiki/Bootstrapping_(compilers)">stage 1</a> build; it’s probably not optimized but it was good enough to find my bug! I also tried building GHC on my laptop and it took 19 minutes, so your mileage may vary.)</p> <p>Here’s how I performed the bisection, from start to finish. This is a wandering account featuring a lot of errors that are more-or-less irrelevant to the final product, so feel free to <a href="https://becca.ooo/blog/bisecting-ghc/#putting-it-all-together">skip to the end if you want to see the finished bisection test script</a>. However, if you’re planning on performing a similar bisection yourself, you may find the errors and the methods I used to work around them helpful.</p> <h2 id="key-takeaways"><a class=anchor href="#key-takeaways" aria-label="Link to section key-takeaways">#</a>Key takeaways</h2> <p>Build systems and dependencies for compilers are constantly changing. Bisecting a large range of commits means you have to be able to build a large range of commits. Choosing a set of tools that can build every commit in the range is not always possible.</p> <p>Relatedly, the GHC build system is <em>very</em> sensitive to the particular set of tools in use. Attempting to build and run GHC with a boot GHC from <a href="https://nixos.org/manual/nixpkgs/stable/#preface">Nixpkgs</a>, I found a wide variety of different errors depending on which revision of Nixpkgs I used.</p> <p>Building old revisions of software with old versions of tools can be quite frustrating. You will encounter bugs that have been fixed in newer revisions of the software, or revisions of the software which only work with old versions of the tools. You may have to cherry-pick or conditionally apply patches to work around these issues.</p> <p>Tools like Nixpkgs are indispensable for large-scale integration work like this. Being able to easily build, patch, and cache a wide range of versions of software makes it so much easier to find a set of tools that can build the relevant versions of GHC.</p> <h2 id="what-to-bisect"><a class=anchor href="#what-to-bisect" aria-label="Link to section what-to-bisect">#</a>What to bisect?</h2> <p>First, we want to determine the range of commits to bisect on. I used <a href="https://www.haskell.org/ghcup/">ghcup</a> to download different versions of GHC and run my reproducer with them, which allowed me to determine that the error was introduced somewhere between GHC 9.6 and GHC 9.8. I checked out <a href="https://gitlab.haskell.org/ghc/ghc">the GHC repository</a> and listed the tags, eventually determining that I wanted the range of commits from the <code>ghc-9.7-start</code> tag to the <code>ghc-9.8.1-release</code> tag. (Note that there is no publicly released GHC 9.7, so this is rather unintuitive unless you already follow the GHC development process!)</p> <h2 id="submodules-to-whatever-the-opposite-of-a-rescue-is"><a class=anchor href="#submodules-to-whatever-the-opposite-of-a-rescue-is" aria-label="Link to section submodules-to-whatever-the-opposite-of-a-rescue-is">#</a>Submodules to whatever the opposite of a rescue is</h2> <p>If you’ve touched the GHC repository at all, you might have modified files, untracked files which exist in the release you want to check out, or — worst of all — modified submodules. To clean out your working tree, you need to run the following commands:</p> <pre data-lang="bash" class="language-bash "><code class="language-bash" data-lang="bash"># Remove untracked files: # -d is for &quot;recurse into directories&quot;, -x is for &quot;remove ignored files&quot;. git clean --force -dx # Undo changes to tracked files: git reset --hard # Unregister submodules and remove their worktrees: git submodule deinit --all --force </code></pre> <p>The submodules make working in the GHC repo quite frustrating, but I recently read <a href="https://www.cyberdemon.org/2024/03/20/submodules.html">“Demystifying git submodules”</a> by Dmitry Mazin, which helped me build out my mental model. Except… <code>git submodule update</code>, which <a href="https://git-scm.com/docs/git-submodule#Documentation/git-submodule.txt-update--init--remote-N--no-fetch--no-recommend-shallow-f--force--checkout--rebase--merge--referenceltrepositorygt--ref-formatltformatgt--depthltdepthgt--recursive--jobsltngt--no-single-branch--filterltfilter-specgt--ltpathgt82308203">the manual page promises</a> will “update the registered submodules to match what the superproject expects by […] updating the working tree of the submodules”, doesn’t seem to work like I expect it to.</p> <p>Here’s what happens. After running a build on <code>ghc-9.7-start</code>, Git tells me that content has been modified in the <code>libraries/unix</code> submodule:</p> <pre><code>$ git status HEAD detached at ghc-9.7-start Changes not staged for commit: (use &quot;git add &lt;file&gt;...&quot; to update what will be committed) (use &quot;git restore &lt;file&gt;...&quot; to discard changes in working directory) (commit or discard the untracked or modified content in submodules) modified: libraries&#x2F;unix (modified content) </code></pre> <p>I update the submodules:</p> <pre><code>$ git submodule update --init --recursive --force ... Submodule path &#x27;utils&#x2F;haddock&#x27;: checked out &#x27;261a7c8ac5b5ff29e6e0380690cbb6ee9730f985&#x27; ... $ git status HEAD detached at ghc-9.7-start nothing to commit, working tree clean </code></pre> <p>But then, when I go to checkout another branch, Git tells me that untracked files in <code>utils/haddock</code> are getting in the way:</p> <pre><code>$ git switch master error: The following untracked working tree files would be overwritten by checkout: utils&#x2F;haddock&#x2F;.github&#x2F;mergify.yml ... Aborting </code></pre> <p><strong>UPDATE:</strong> I sent this article to my friend who works as a firmware engineer — and, therefore, uses submodules gleefully — and she immediately suggested that a submodule might have been <em>removed</em>. I checked and that’s what’s going on here. It had never occurred to me because I was checking out a later commit starting at an earlier one, and I had assumed that a repository with 31 submodules didn’t get there by <em>removing</em> them. What follows are my flailing attempts to fix this without removing all the submodules. Skip ahead to <a href="https://becca.ooo/blog/bisecting-ghc/#getting-ready">“Getting ready”</a> if you don’t want to read that.</p> <p>I try to remove the untracked files, but it doesn’t help:</p> <pre><code>$ git clean -fdx $ git submodule foreach git clean -fdx ... Entering &#x27;utils&#x2F;haddock&#x27; ... $ git switch master error: The following untracked working tree files would be overwritten by checkout: utils&#x2F;haddock&#x2F;.github&#x2F;mergify.yml ... Aborting </code></pre> <p>In fact, Git thinks that the working tree for <code>utils/haddock</code> is clean and that <em>all</em> of those files are tracked:</p> <pre><code>$ pushd utils&#x2F;haddock $ git status HEAD detached at 261a7c8a nothing to commit, working tree clean $ git ls-files .github&#x2F;mergify.yml .github&#x2F;mergify.yml $ popd </code></pre> <p>Removing the checkouts for the submodules entirely seems to work, but I can’t figure out why:</p> <pre><code>$ git submodule deinit --all --force ... Cleared directory &#x27;utils&#x2F;haddock&#x27; Submodule &#x27;utils&#x2F;haddock&#x27; (https:&#x2F;&#x2F;gitlab.haskell.org&#x2F;ghc&#x2F;haddock.git) unregistered for path &#x27;utils&#x2F;haddock&#x27; ... $ git switch master Previous HEAD position was 261a7c8a Bump GHC version to 9.7 Switched to branch &#x27;master&#x27; Your branch is up to date with &#x27;origin&#x2F;master&#x27;. </code></pre> <h2 id="getting-ready"><a class=anchor href="#getting-ready" aria-label="Link to section getting-ready">#</a>Getting ready</h2> <p>Now that we know the range of commits we want to bisect, let’s get GHC building. If you haven’t built GHC before (I hadn’t!), <a href="https://gitlab.haskell.org/ghc/ghc/-/wikis/building/preparation">the GHC Wiki has a good article on setting up your system for building GHC</a>.</p> <p>After getting everything set up, the build process looks roughly like this:</p> <pre data-lang="bash" class="language-bash "><code class="language-bash" data-lang="bash"># Clone and update the submodules. git submodule update --init --recursive # Run `autoreconf`, copy `config.sub` into submodules. .&#x2F;boot # Configure for building; `$CONFIGURE_ARGS` is set by `ghc.nix` and contains # paths to the `gmp` and `ncurses` libraries. .&#x2F;configure $CONFIGURE_ARGS # Build GHC with the bespoke Hadrian build system: .&#x2F;hadrian&#x2F;build -j --flavour=Quick </code></pre> <p>I chose to use <a href="https://gitlab.haskell.org/ghc/ghc.nix">ghc.nix</a> to install the dependencies. This worked really well, but when I attempted to reproduce my builds for this writeup everything fell apart! It turned out that I had been using <a href="https://github.com/alpmestan/ghc.nix">an old version of ghc.nix</a> from a GitHub repository that hadn’t been updated since October 2023. By pure coincidence, this was perfectly suited for building the revisions of GHC I was interested in, which were authored between December 2022 and October 2023.</p> <p>I also made a couple tweaks to the <code>userSettings</code> attribute-set in ghc.nix’s <code>flake.nix</code>:</p> <ul> <li>I set <code>withIde = false;</code> to disable building the huge and expensive <code>haskell-language-server</code>, which I wouldn’t be using.</li> <li>I set <code>bootghc = "ghc96";</code> because I’m building older versions of GHC, which isn’t necessarily supported with a newer GHC. For bisecting larger ranges or historical versions, it might be nice to build some tooling to dynamically chose a boot GHC based on the commit being built. (Also, I would learn later that GHC 9.4 would have been a better choice.)</li> </ul> <h2 id="fixing-up-ghc-nix-a-potpourri-of-build-errors"><a class=anchor href="#fixing-up-ghc-nix-a-potpourri-of-build-errors" aria-label="Link to section fixing-up-ghc-nix-a-potpourri-of-build-errors">#</a>Fixing up ghc.nix &amp; a potpourri of build errors</h2> <p>I suspect that the smarter move here would have been to just point at an old commit of ghc.nix and call it a day, but instead I’ve spent my downtime for the last couple days fixing up the latest revisions of ghc.nix to make it easier to perform these builds in the future.</p> <p>With <a href="https://github.com/alpmestan/ghc.nix/commit/c31bcab7a74569e6bd37dbe7de94c97cad1e35e1">the old revision of ghc.nix</a>, I got a hash mismatch error:</p> <pre><code>$ nix develop error: NAR hash mismatch in input &#x27;github:commercialhaskell&#x2F;all-cabal-hashes&#x2F;bd2d976d126b7730d82c772a207cf34e927aa69d&#x27; (&#x2F;nix&#x2F;store&#x2F;i19cc0hifz3f7iayz477v6s2aajyl1ii-source), expected &#x27;sha256-c6R3PkzqDCeAIqB+aygnjIMOmnkAmepyakOqtb8oQrg=&#x27;, got &#x27;sha256-7wwhcWxDLTKHghemMh30Le7D5pi7+eSUCg4jj7yS+jM=&#x27; </code></pre> <p>I fixed this by updating the relevant input with <code>nix flake update all-cabal-hashes</code>.</p> <p><code>./boot</code> and <code>./configure</code> worked fine, but I ran into trouble when I attempted to run the actual build process:</p> <pre><code>$ .&#x2F;hadrian&#x2F;build -j --flavour=Quick Resolving dependencies... Error: cabal: Could not resolve dependencies: [__0] trying: hadrian-0.1.0.0 (user goal) [__1] trying: base-4.18.0.0&#x2F;installed-4.18.0.0 (dependency of hadrian) [__2] next goal: Cabal (dependency of hadrian) [__2] rejecting: Cabal-3.10.1.0&#x2F;installed-3.10.1.0 (conflict: hadrian =&gt; Cabal&gt;=3.2 &amp;&amp; &lt;3.9) ... </code></pre> <p>In retrospect, this should have been a signal to find a better version of GHC and Cabal to build with. According to the (<em>very</em> well-hidden) <a href="https://web.archive.org/web/20241203234539/https://gitlab.haskell.org/ghc/ghc/-/wikis/commentary/libraries/version-history">GHC Boot Library Version History</a> page on the GHC wiki, GHC 9.4 was distributed with Cabal 3.8, so that’s what we likely want.</p> <p>I decided to charge on, though. Here’s what I needed to do to get GHC to build with “close enough” versions of the build tools!</p> <p>First, you need to tell <a href="https://gitlab.haskell.org/ghc/ghc/-/blob/fc3a2232da89ed4442b52a99ba1826d04362a7e8/hadrian/build-cabal#L5">the Cabal that’s building Hadrian</a> to ignore the dependency version bounds with <a href="https://cabal.readthedocs.io/en/latest/cabal-project-description-file.html#cfg-flag---allow-newer"><code>--allow-newer</code></a>:</p> <pre><code>CABFLAGS=--allow-newer .&#x2F;hadrian&#x2F;build -j --flavour=Quick </code></pre> <p>This will build Hadrian and then GHC. However, it won’t get very far:</p> <pre><code># cabal-configure (for _build&#x2F;stageBoot&#x2F;linters&#x2F;lint-whitespace&#x2F;setup-config) Error: hadrian: Encountered missing or private dependencies: Error: hadrian: Encountered missing or private dependencies: mtl &gt;=2.1 &amp;&amp; &lt;2.3 mtl &gt;=2.1 &amp;&amp; &lt;2.3 </code></pre> <p>For most command-line options, we can use the <a href="https://gitlab.haskell.org/ghc/ghc/-/blob/c3fc9b861fd00a85a4fcbd9960b8242d9fabe04b/hadrian/doc/user-settings.md#key--value-and-key--value-style-settings">Hadrian user settings</a> to add Cabal options (e.g., <code>echo ".*.cabal.configure.opts += ..." &gt;&gt; _build/hadrian.settings</code>), but the Hadrian build system works with <a href="https://cabal.readthedocs.io/en/stable/setup-commands.html">Cabal’s lower-level <code>Setup.hs</code> interface</a>, which doesn’t support the <code>--allow-newer</code> flag, so we have to patch the <code>.cabal</code> files manually.</p> <p>I used a tool called <a href="https://github.com/NixOS/jailbreak-cabal"><code>jailbreak-cabal</code></a> to remove version bounds from dependency specifications in <code>.cabal</code> files. Here’s the command to run, which skips the hundreds of <code>.cabal</code> files in integration tests:</p> <pre data-lang="bash" class="language-bash "><code class="language-bash" data-lang="bash">find . \ ! &#x27;(&#x27; \ -path .&#x2F;testsuite -prune \ -o -path .&#x2F;libraries&#x2F;Cabal&#x2F;Cabal-tests&#x2F;tests -prune \ -o -path .&#x2F;libraries&#x2F;Cabal&#x2F;cabal-install&#x2F;tests -prune \ -o -path .&#x2F;libraries&#x2F;Cabal&#x2F;cabal-testsuite&#x2F;PackageTests -prune \ &#x27;)&#x27; \ -path &#x27;*.cabal&#x27; \ -exec jailbreak-cabal &#x27;{}&#x27; &#x27;;&#x27; </code></pre> <p>With that in place, my builds completed successfully, but I still needed to make a few other tweaks to get the build working with the new canonical ghc.nix repository.</p> <p>First, I needed to <a href="https://gitlab.haskell.org/ghc/ghc.nix/-/merge_requests/24">remove <code>gdb</code> from the included tools</a> because <code>gdb</code> isn’t available on macOS.</p> <p>Then, I needed to <a href="https://gitlab.haskell.org/ghc/ghc.nix/-/merge_requests/25">export <code>CXX</code> with an absolute path</a> so that the <code>./configure</code> script wouldn’t fail due to a quoting issue. This bug is fixed on recent versions of GHC, but it’s present on <code>ghc-9.7-start</code>, which we want to build!</p> <p>After fixing the <code>./configure</code> step, I could attempt the build, but it didn’t get very far:</p> <pre><code>| Run GhcPkg Update (Stage0 InTreeLibs): _build&#x2F;stage0&#x2F;libraries&#x2F;ghc-boot-th&#x2F;inplace-pkg-config =&gt; none dieVerbatim: user error (Error: hadrian: &#x27;&#x2F;nix&#x2F;store&#x2F;3dahn98q1m46ickndsg797zp8gv801b2-ghc-9.6.6-with-packages&#x2F;bin&#x2F;ghc&#x27; exited with an error: ghc-9.6.6: can&#x27;t find a package database at _build&#x2F;stage0&#x2F;libraries&#x2F;ghc-boot-th&#x2F;build&#x2F;package.conf.inplace ) Build failed. </code></pre> <p>At this point, I suspected my GHC / Cabal versions were to blame, so I started to look for more historically-accurate versions of the tools. First I found the date of the commit I was trying to build:</p> <pre><code>$ git show commit fc3a2232da89ed4442b52a99ba1826d04362a7e8 (HEAD, tag: ghc-9.7-start) Author: Ben Gamari &lt;[email protected]&gt; Date: Thu Dec 22 13:45:06 2022 -0500 Bump GHC version to 9.7 </code></pre> <p>Then I asked for <a href="https://github.com/nixos/nixpkgs/commit/012700eae502f6054a056cf7b94f78ff549e278d">a commit around that time in Nixpkgs</a>:</p> <pre><code>$ git log --before=&#x27;Thu Dec 22 13:45:06 2022&#x27; nixos-unstable commit 012700eae502f6054a056cf7b94f78ff549e278d Merge: 0550dfc0228a 5f1760cb902c Author: Fabian Affolter &lt;[email protected]&gt; Date: Thu Dec 22 22:38:20 2022 +0100 Merge pull request #207299 from r-ryantm&#x2F;auto-update&#x2F;python3.10-python-crontab python310Packages.python-crontab: 2.6.0 -&gt; 2.7.1 </code></pre> <p>If you’ve read <a href="https://git-scm.com/docs/git-rev-parse#_specifying_revisions">the <code>git rev-parse</code> documentation</a>, you might be tempted to try <code>git log 'nixos-unstable@{Thu Dec 22 13:45:06 2022}'</code>, but that will attempt to find what <em>your</em> checkout had <code>nixos-unstable</code> pointing to on that date.</p> <p>Attempting to build GHC from that revision gave me a compiler error:</p> <pre><code>ghc: panic! (the &#x27;impossible&#x27; happened) GHC version 9.4.2: Template variable unbound in rewrite rule Variable: sg_shgu :: WasmTypeTag &#x27;I32 ~R# WasmTypeTag w_sgB4 Rule &quot;SC:$j0&quot; Rule bndrs: [sg_shgu] LHS args: [ty_word_X16] Actual args: [ty_word_X16] Call stack: CallStack (from HasCallStack): callStackDoc, called at compiler&#x2F;GHC&#x2F;Utils&#x2F;Panic.hs:182:37 in ghc:GHC.Utils.Panic pprPanic, called at compiler&#x2F;GHC&#x2F;Core&#x2F;Rules.hs:619:10 in ghc:GHC.Core.Rules </code></pre> <p>I jumped ahead and tried <a href="https://github.com/nixos/nixpkgs/commit/33650d3ae02cac67b6e8f75f2b997b1e0556b676">a commit from a year later</a>. That built successfully, but running any of the executables it produced failed:</p> <pre><code>$ _build&#x2F;stage1&#x2F;bin&#x2F;ghc dyld[12621]: symbol not found in flat namespace &#x27;_unixzm2zi8zi3zi0zminplace_SystemziPosixziFiles_getFileStatus_closure&#x27; fish: Job 1, &#x27;_build&#x2F;stage1&#x2F;bin&#x2F;ghc&#x27; terminated by signal SIGABRT (Abort) </code></pre> <p>It was around this time that I finally looked at the old ghc.nix repository again and noticed it was pointing to <a href="https://github.com/nixos/nixpkgs/commit/5cfafa12d57374f48bcc36fda3274ada276cf69e">a commit on the <code>nixos-23.05</code> branch of Nixpkgs.</a> A stable release branch! Why hadn’t I thought of that earlier? I only needed one more tweak to <a href="https://gitlab.haskell.org/ghc/ghc.nix/-/merge_requests/26">fix ghc.nix with old versions of Nixpkgs that don’t include a top-level <code>pkgs.happy</code> attribute</a> and my builds started completing successfully.</p> <p>Now we’re ready to start bisecting!</p> <h2 id="starting-the-bisection"><a class=anchor href="#starting-the-bisection" aria-label="Link to section starting-the-bisection">#</a>Starting the bisection</h2> <p>First, mark a commit you know is bad and a commit you know is good:</p> <pre data-lang="bash" class="language-bash "><code class="language-bash" data-lang="bash">git bisect start ghc-9.8.1-release ghc-9.7-start </code></pre> <p>Then run the bisection using a script to determine which commits are good:</p> <pre data-lang="bash" class="language-bash "><code class="language-bash" data-lang="bash">git bisect run ..&#x2F;test.sh </code></pre> <p>Note that the <code>test.sh</code> script is in the <em>parent</em> directory so it doesn’t get deleted when we run <code>git clean</code>.</p> <h2 id="the-bisection-script"><a class=anchor href="#the-bisection-script" aria-label="Link to section the-bisection-script">#</a>The bisection script</h2> <p>We’ll need to build GHC repeatedly across a variety of different versions, so we’ll need some tweaks to get things working smoothly.</p> <p>First, we’ll want to remove any previous build artifacts and get the submodules set up correctly:</p> <pre data-lang="bash" class="language-bash "><code class="language-bash" data-lang="bash"># Log commands as we run them. set -x # Reset the submodules in case of changes (e.g., to generated files). # This takes about 5 seconds, or a minute for a fresh clone. git submodule update --init --force --recursive || exit 128 # Remove existing build products. # This takes roughly no time. time rm -rf _build || exit 128 </code></pre> <p>Why exit with code 128 if these commands fail? According to <a href="https://git-scm.com/docs/git-bisect#_bisect_run">the <code>git bisect</code> man page</a>:</p> <blockquote> <p>Note that the script […] should exit with code 0 if the current source code is good/old, and exit with a code between 1 and 127 (inclusive), except 125, if the current source code is bad/new.</p> <p>Any other exit code will abort the bisect process. It should be noted that a program that terminates via <code>exit(-1)</code> leaves <code>$? = 255</code>, (see the <code>exit(3)</code> manual page), as the value is chopped with <code>&amp; 0377</code>.</p> <p>The special exit code 125 should be used when the current source code cannot be tested. If the script exits with this code, the current revision will be skipped (see <code>git bisect skip</code> above). 125 was chosen as the highest sensible value to use for this purpose, because 126 and 127 are used by POSIX shells to signal specific error status (127 is for command not found, 126 is for command found but not executable—these details do not matter, as they are normal errors in the script, as far as <code>bisect run</code> is concerned).</p> </blockquote> <p>The first script I tried looked like this:</p> <pre data-lang="bash" class="language-bash "><code class="language-bash" data-lang="bash">function build { git submodule update --init --force --recursive rm -rf _build .&#x2F;boot # ... } set -ex build || exit 128 </code></pre> <p>I hoped that this would run the commands in <code>build</code>, returning 1 from <code>build</code> if any of the commands failed, and then exiting with <code>128</code> as a result. It turns out this is not the case! According to <a href="https://www.gnu.org/software/bash/manual/html_node/The-Set-Builtin.html">the Bash manual</a>, using <code>set -e</code> will cause the script to:</p> <blockquote> <p>Exit immediately if a pipeline, which may consist of a single simple command, a list, or a compound command returns a non-zero status. The shell does not exit if the command that fails is […] part of any command executed in a <code>&amp;&amp;</code> or <code>||</code> list except the command following the final <code>&amp;&amp;</code> or <code>||</code></p> <p>If a compound command or shell function executes in a context where <code>-e</code> is being ignored, none of the commands executed within the compound command or function body will be affected by the <code>-e</code> setting, even if <code>-e</code> is set and a command returns a failure status.</p> </blockquote> <p>Therefore, because we run <code>build || exit 128</code>, <code>build</code> will <em>keep going</em> if one of the commands in it fails! I googled “bash errexit site:gnu.org” to find the documentation I’ve linked above, but the first result was <a href="https://lists.gnu.org/archive/html/bug-bash/2012-12/msg00093.html">a message sent to the Bash mailing list titled “why does errexit exist in its current utterly useless form?”</a>, complaining about this exact problem.</p> <p>To work around this, we have to append <code>|| exit 128</code> to each command in the build process. To state the obvious, this is tedious and error-prone.</p> <p>Anyways, now we’re ready to run the build! Booting the repository takes about two seconds:</p> <pre><code>.&#x2F;boot || exit 128 </code></pre> <p>Configuring it takes about 30 seconds:</p> <pre><code>.&#x2F;configure $CONFIGURE_ARGS || exit 128 </code></pre> <p>And building Hadrian and then GHC takes about 12 minutes on my desktop (M1 Ultra with 20 cores and 64GB RAM) or about 19 minutes on my laptop (M2 Max with 12 cores and 32GB RAM):</p> <pre><code>CABFLAGS=--allow-newer .&#x2F;hadrian&#x2F;build -j --flavour=Quick || exit 128 </code></pre> <p>When we’re done, make sure to reset any changes we made to files in the working tree so that Git can check out the next revision:</p> <pre><code>git reset --hard || exit 128 </code></pre> <p>Double-check that we’ve actually produced a <code>runghc</code> executable, and then we can use the built compiler to run our tests:</p> <pre><code># Make sure we actually built a compiler: if [[ ! -e _build&#x2F;stage1&#x2F;bin&#x2F;runghc ]] then exit 128 fi tmp=$(mktemp) _build&#x2F;stage1&#x2F;bin&#x2F;runghc ..&#x2F;Main.hs &gt; &quot;$tmp&quot; || exit 128 # If the output doesn&#x27;t contain &#x27;action&#x27;, the bug is present in this commit. grep --quiet action &quot;$tmp&quot; </code></pre> <p>And that’s it! The finished product isn’t <em>too</em> complex, but the road to get there involved a lot of subtle pitfalls I wanted to write down.</p> <h2 id="putting-it-all-together"><a class=anchor href="#putting-it-all-together" aria-label="Link to section putting-it-all-together">#</a>Putting it all together</h2> <p>Here’s what the bisection test script looks like when it’s all assembled together! <a href="https://gist.github.com/9999years/9ad5e0929c44b1908d66473cf66ddfc4">The script is also available as a GitHub Gist.</a></p> <pre><code>#!&#x2F;usr&#x2F;bin&#x2F;env bash # Log commands as we run them, and exit if any command produces an error. set -x # Attempt to build GHC. # # If we can&#x27;t build GHC, exit with 128. This will abort the entire `git bisect` # instead of erroneously marking the commit as &#x27;bad&#x27;. # # From `man git-bisect`: # &gt; Note that the script [...] should exit with code # &gt; 0 if the current source code is good&#x2F;old, and exit with a code between 1 # &gt; and 127 (inclusive), except 125, if the current source code is bad&#x2F;new. # &gt; # &gt; Any other exit code will abort the bisect process. It should be noted that # &gt; a program that terminates via exit(-1) leaves $? = 255, (see the exit(3) # &gt; manual page), as the value is chopped with &amp; 0377. # 5 seconds: # Reset the submodules if there&#x27;s changes like generated files. time git submodule update --init --force --recursive || exit 128 # Remove existing build products. time rm -rf _build || exit 128 # Now we run the actual build process. # See: https:&#x2F;&#x2F;gitlab.haskell.org&#x2F;ghc&#x2F;ghc&#x2F;-&#x2F;wikis&#x2F;building&#x2F;preparation # 2 seconds: time .&#x2F;boot || exit 128 # 30 seconds: time .&#x2F;configure $CONFIGURE_ARGS || exit 128 # ~12 minutes (M1 Ultra, 20 cores), ~19 minutes (???, 12 cores): time CABFLAGS=--allow-newer .&#x2F;hadrian&#x2F;build -j --flavour=Quick || exit 128 # Reset any modified files or checking out the next commit will fail: time git reset --hard || exit 128 # Make sure we actually built a compiler: if [[ ! -e _build&#x2F;stage1&#x2F;bin&#x2F;runghc ]] then exit 128 fi tmp=$(mktemp) _build&#x2F;stage1&#x2F;bin&#x2F;runghc &quot;..&#x2F;Main.hs&quot; &gt; &quot;$tmp&quot; || exit 128 # If the output contains &#x27;action&#x27;, we&#x27;re all OK. # If it just says &#x27;run&#x27;, we have a problem! grep --quiet action &quot;$tmp&quot; </code></pre> Announcing git-prole Tue, 22 Oct 2024 00:00:00 +0000 Rebecca Turner https://becca.ooo/blog/announcing-git-prole/ https://becca.ooo/blog/announcing-git-prole/ <p><strong>TL;DR:</strong> I’ve built <a href="https://github.com/9999years/git-prole"><code>git-prole</code></a>, an ergonomic <a href="https://git-scm.com/docs/git-worktree"><code>git-worktree</code></a> manager. Check out the <a href="https://9999years.github.io/git-prole/">user manual</a> to get started! If you don’t know what a Git worktree is or why you might want one, read on.</p> <p>If you’re anything like me, you have a couple repositories cloned that are cumbersome enough to clone more than once:</p> <pre><code>$ ls -l drwxr-xr-x - rebeccat 15 Oct 11:13 mwb drwxr-xr-x - rebeccat 11 Oct 14:19 mwb2 drwxr-xr-x - rebeccat 30 Sep 10:06 mwb3 drwxr-xr-x - rebeccat 19 Oct 13:31 mwb4 drwxr-xr-x - rebeccat 2 Aug 16:45 mwb5 drwxr-xr-x - rebeccat 3 Oct 12:10 mwb6 </code></pre> <p>This feels a little bit silly, but there’s some good reasons for it:</p> <ul> <li>If builds are expensive, then switching branches can incur a significant time cost.</li> <li>You might want to check out a colleague’s branch, but you have uncommitted work you’re hesitant to commit or stash.</li> <li>It’s often convenient to compare your code against the repository’s main branch. <code>git diff</code> can do this, of course, but chances are it’s not very well integrated into your editor, or you need to be able to run the code in addition to reading it.</li> </ul> <p>At the same time, there’s some costs to this approach:</p> <ul> <li>The repositories have to be cloned separately, which is annoying. You can clone from a local repository, but then you have to be careful to add the correct remote, remove the old remote, and set up a bunch of remote-tracking branches again.</li> <li>You can’t easily share work between the repositories. I often want to split a couple of small fixes off from a current branch, but there’s not a great way to do this without generating a patch and applying it manually or pushing your work to a remote.</li> <li>The clones are separate repositories, so it’s not easy to add new checkouts or remove unused ones. You effectively have to decide how many branches you want to be able to check out at the same time in advance.</li> </ul> <p>Git worktrees, then, are a feature that allows you to associate multiple checkouts with a single repository, like this:</p> <pre><code>my-repo&#x2F; main&#x2F; # A checkout for the main branch .git&#x2F; # The main .git directory. README.md feature1&#x2F; # A checkout for work on a feature .git # A file containing `gitdir: &#x2F;Path&#x2F;to&#x2F;my-repo&#x2F;main&#x2F;.git` README.md </code></pre> <p>And, as it turns out, with a little fiddling, you can also use a bare checkout for the <code>.git</code> directory; this means that none of the worktrees are “special” and they can all be deleted (if you try to set up a repository like this yourself, you will have problems; more on this later).</p> <pre><code>my-repo&#x2F; .git&#x2F; # A bare repository main&#x2F; # A checkout for the main branch .git # A file with the path to the bare repository README.md feature1&#x2F; # A checkout for work on a feature .git README.md </code></pre> <h2 id="features"><a class=anchor href="#features" aria-label="Link to section features">#</a>Features</h2> <p><a href="https://9999years.github.io/git-prole/"><code>git-prole</code></a> includes a number of tools to make using multiple worktrees as ergonomic as possible:</p> <ul> <li> <p><code>git prole clone URL [DESTINATION]</code> will clone a repository into the multiple-worktrees layout described above.</p> </li> <li> <p><code>git prole convert</code> will convert the current repository into a worktree layout. (Use <code>--dry-run</code> to see what will happen.)</p> </li> <li> <p><code>git prole add</code> will add a new worktree:</p> <ul> <li> <p><code>git prole add feature1</code> will create a <code>feature1</code> directory next to the rest of your worktrees; <code>git worktree add feature1</code>, in contrast, will create a <code>feature1</code> subdirectory nested under the current worktree.</p> </li> <li> <p>Branches created with <code>git prole add</code> will start at and track the repository’s main branch by default.</p> </li> <li> <p><code>git prole add</code> will copy untracked files to the new worktree by default, making it easy to start a new worktree with a warm build cache.</p> </li> <li> <p><code>git prole add</code> can run commands when a new worktree is created, so that you can warm up caches by running a command like <code>direnv allow</code>.</p> </li> <li> <p><code>git prole add</code> can perform regex substitutions on branch names to compute a directory name, so that you can run <code>git prole add -b myname/team-1234-my-ticket-with-a-very-long-title</code> and get a directory name like <code>my-ticket</code>.</p> </li> <li> <p><code>git prole add</code> respects the <code>-c</code>/<code>--create</code> option (to match <code>git switch</code>); <code>git worktree add</code> only allows <code>-b</code> (with no long-form option available).</p> </li> </ul> </li> </ul> <h2 id="why-hadn-t-i-heard-of-worktrees-before"><a class=anchor href="#why-hadn-t-i-heard-of-worktrees-before" aria-label="Link to section why-hadn-t-i-heard-of-worktrees-before">#</a>Why hadn’t I heard of worktrees before?</h2> <p>This is a pretty cool feature, and once we have some tooling like <code>git-prole</code> to paper over the ergonomic challenges it makes a great workflow! So why haven’t you heard of worktrees before?</p> <p>For reasons I can’t fully determine, worktrees have received basically zero fanfare except from your coworker who runs <code>nix-darwin</code>. The <code>git-worktree</code> command was <a href="https://github.com/git/git/commit/7783eb2e59684492e75068443e1f77f64fe37cc9">first present</a> in Git 2.5.0, released in 2015. However, <a href="https://github.com/git/git/blob/master/Documentation/RelNotes/2.5.0.txt">the release notes for Git 2.5.0</a>, which include dozens of updates like “Clarify in the <code>Makefile</code> a guideline to decide use of <code>USE_NSEC</code>”, have nothing to say about worktrees. GitHub actually published a blog post titled <a href="https://github.blog/open-source/git/git-2-5-including-multiple-worktrees-and-triangular-workflows/">“Git 2.5, including multiple worktrees and triangular workflows”</a>, which received, as far as I can tell, <a href="https://www.reddit.com/r/git/comments/3f5shh/git_25_including_multiple_worktrees_and/">one Reddit post</a> with 35 upvotes and two top-level comments, neither of which were excited for the new feature.</p> <p>Recently, worktrees have started to garner a little more attention (see, for example, matklad’s <a href="https://matklad.github.io/2024/07/25/git-worktrees.html">“How I Use Git Worktrees”</a>), but the second search result for “git worktrees” is a Stack Overflow question titled <a href="https://stackoverflow.com/questions/31935776/what-would-i-use-git-worktree-for">“What would I use <code>git-worktree</code> for?”</a></p> <h2 id="trying-and-failing-with-worktrees"><a class=anchor href="#trying-and-failing-with-worktrees" aria-label="Link to section trying-and-failing-with-worktrees">#</a>Trying and failing with worktrees</h2> <p>A couple weeks ago, I finally decided to take a couple hours and figure out how to use worktrees. Let’s create a worktree for a new branch, which I’ll call <code>feature1</code>:</p> <pre><code>$ git worktree add feature1 Preparing worktree (new branch &#x27;feature1&#x27;) branch &#x27;feature1&#x27; set up to track &#x27;main&#x27;. HEAD is now at b1b2888 Fix static binary builds in releases (#86) </code></pre> <p>Great, that was easy! Now, let’s just type <code>git status</code>, which I enter reflexively after nearly any Git command:</p> <pre><code>$ git status On branch main Your branch is up to date with &#x27;origin&#x2F;main&#x27;. Untracked files: (use &quot;git add &lt;file&gt;...&quot; to include in what will be committed) feature1&#x2F; nothing added to commit but untracked files present (use &quot;git add&quot; to track) </code></pre> <p>…huh. The first argument to <code>git worktree add</code> is used as a branch name, but it’s actually a path, and it will happily create a worktree nested in another worktree, despite (presumably) nobody ever wanting to do this.</p> <p>The actual invocation you want is something like this, where Git uses the last <em>component</em> of the path as the branch name:</p> <pre><code>$ git worktree add ..&#x2F;feature1 </code></pre> <p>And then we get to the next ergonomic issue with Git worktrees: what do you name them? If you’re not a worktree user already, you probably have your repository cloned in some “projects” directory, containing a whole bunch of repositories. So when you start working on <code>feature1</code>, you should really disambiguate it:</p> <pre><code>$ git worktree add ..&#x2F;my-repo-feature1 </code></pre> <p>This is a bad experience! You have to type a path manually, prefix the path with the repository name, and then you have to type the repository name again when you <code>cd</code> to the checkout later. Can we do better? Of course!</p> <h2 id="a-better-layout-for-a-worktree-repository"><a class=anchor href="#a-better-layout-for-a-worktree-repository" aria-label="Link to section a-better-layout-for-a-worktree-repository">#</a>A better layout for a worktree repository</h2> <p>If we look at the example workflow from the GitHub blog post above, we can see that they expect you to have the workflow layout I showed you at the start of the post:</p> <pre><code>$ cd ..&#x2F;main $ rm -rf ..&#x2F;hotfix </code></pre> <p>GitHub must expect you to have a containing directory named <code>my-repo</code> and then worktrees directly inside it, named corresponding to the branches they have checked out. <code>git-prole</code> works well with this layout; <code>git prole add hotfix</code> will create a new worktree in the same directory as the main worktree.</p> <p>Let’s set up one of our repositories like that. First, we’ll move our main working tree from <code>/path/to/my-repo</code> to <code>/path/to/my-repo/main</code>:</p> <pre><code>$ git worktree move . main fatal: &#x27;.&#x27; is a main working tree </code></pre> <p>Oh. I guess we’ll do it the hard way: renaming <code>my-repo</code> to <code>my-repo.bak</code>, creating a new <code>my-repo</code> directory, and then moving <code>my-repo.bak</code> to <code>my-repo/main</code>. A three-step barrier to entry for worktrees, and that’s if you only have one to start with. If you’ve already created some worktrees in the parent directory, you’ll have to figure out how to reassociate them with the main worktree once you move everything. Theoretically this is possible with <code>git worktree repair</code>, but I found myself having to edit files in the <code>.git</code> directory manually.</p> <p><code>git-prole</code> will handle this conversion process for you, even with multiple worktrees, un-committed work, or any number of other complications:</p> <pre><code>$ git worktree list &#x2F;path&#x2F;to&#x2F;git-prole 719ed92 [main] $ nix run github:9999years&#x2F;git-prole -- convert • Converting ~&#x2F;git-prole to a worktree repository at ~&#x2F;git-prole. I&#x27;ll move the following worktrees to new locations: • ~&#x2F;git-prole -&gt; ~&#x2F;git-prole&#x2F;main Additionally, I&#x27;ll convert the repository to a bare repository. Preparing worktree (checking out &#x27;main&#x27;) • ~&#x2F;git-prole has been converted to a worktree checkout • You may need to `cd .` to refresh your shell $ cd . $ git worktree list &#x2F;path&#x2F;to&#x2F;git-prole (bare) &#x2F;path&#x2F;to&#x2F;git-prole&#x2F;main 719ed92 [main] </code></pre> <p>Eventually, I got the repository set up the way I wanted, but there was still one thing that bothered me: the <code>.git</code> directory. If you create a worktree layout by moving your existing worktrees around, you end up with one special “main” worktree, which contains the full <code>.git</code> directory shared by the other worktrees and cannot be renamed or removed.</p> <p>Having a main worktree is fine, but it’s smelly. I don’t <em>like</em> that all of the worktrees are ephemeral and short-lived except for one. The solution to this is to have the main worktree be a bare repository: a <code>.git</code> directory with no accompanying working tree. A repository is bare if its configuration sets the <code>core.bare</code> option to true:</p> <pre><code>$ git status On branch main nothing to commit, working tree clean $ git config set --local core.bare true $ git status fatal: this operation must be run in a work tree </code></pre> <p>So we create the <code>my-repo</code> directory and do a bare clone into it:</p> <pre><code>$ git clone --bare [email protected]:9999years&#x2F;git-prole.git Cloning into bare repository &#x27;git-prole.git&#x27;... </code></pre> <p>Great! Now let’s set up a working tree:</p> <pre><code>$ cd git-prole.git&#x2F; $ git worktree add ..&#x2F;main Preparing worktree (checking out &#x27;main&#x27;) HEAD is now at 719ed92 Fix links to `git-worktree(1)` docs (#89) $ cd ..&#x2F;main&#x2F; $ git pull There is no tracking information for the current branch. Please specify which branch you want to rebase against. See git-pull(1) for details. git pull &lt;remote&gt; &lt;branch&gt; If you wish to set tracking information for this branch you can do so with: git branch --set-upstream-to=origin&#x2F;&lt;branch&gt; master $ git branch --set-upstream-to=origin&#x2F;master master fatal: the requested upstream branch &#x27;origin&#x2F;master&#x27; does not exist hint: hint: If you are planning on basing your work on an upstream hint: branch that already exists at the remote, you may need to hint: run &quot;git fetch&quot; to retrieve it. hint: hint: If you are planning to push out a new local branch that hint: will track its remote counterpart, you may want to use hint: &quot;git push -u&quot; to set the upstream config as you push. hint: Disable this message with &quot;git config advice.setUpstreamFailure false&quot; </code></pre> <p>…huh. Apparently, <a href="https://stackoverflow.com/a/54408181">as someone pointed out on Stack Overflow to a user with the same problem</a>, <code>git clone --bare</code> doesn’t <em>just</em> skip creating a working directory. According to the man page:</p> <blockquote> <p>Also the branch heads at the remote are copied directly to corresponding local branch heads, without mapping them to <code>refs/remotes/origin/</code>.</p> </blockquote> <p>This means that instead of (e.g.) creating a remote-tracking branch <code>origin/main</code>, Git creates a local branch <code>main</code>. Then, when you go to push or pull, Git errors out when it can’t find a matching remote branch. You can sort of fix this by setting a refspec for your remote (e.g., <code>git config set --local remote.origin.fetch "+refs/heads/*:refs/remotes/origin/*"</code>, good luck remembering <em>that</em> the next time you need it), but then you have to rename all of the refs manually, transforming <code>refs/heads/foo</code> to <code>refs/remotes/origin/foo</code>, which is a pain to say the least!</p> <p>Converting an existing repository to a bare one is similarly painful, although for different reasons (it requires a song and dance to reassociate the main worktree with the repository). Eventually, I realized that the only way to fix these issues would be with a new tool to make setting up and working with multiple worktrees easy.</p> <h2 id="easier-worktrees-with-git-prole"><a class=anchor href="#easier-worktrees-with-git-prole" aria-label="Link to section easier-worktrees-with-git-prole">#</a>Easier worktrees with git-prole</h2> <p>For a breath of fresh air, let’s clone a repository with <code>git-prole</code>, taking advantage of the (configuration-gated) <a href="https://cli.github.com/"><code>gh</code></a> support to let us enter a GitHub repository slug instead of a full URL:</p> <pre><code>$ git prole clone 9999years&#x2F;git-prole Cloning into &#x27;&#x2F;Users&#x2F;wiggles&#x2F;test-repo&#x2F;git-prole&#x27;... remote: Enumerating objects: 668, done. remote: Counting objects: 100% (502&#x2F;502), done. remote: Compressing objects: 100% (215&#x2F;215), done. remote: Total 668 (delta 331), reused 420 (delta 286), pack-reused 166 (from 1) Receiving objects: 100% (668&#x2F;668), 277.44 KiB | 1.66 MiB&#x2F;s, done. Resolving deltas: 100% (407&#x2F;407), done. • • Move git-prole&#x2F;.git to $TMPDIR&#x2F;.tmpwdCaYS&#x2F;.git • In $TMPDIR&#x2F;.tmpwdCaYS&#x2F;.git, set core.bare=true • Move git-prole to $TMPDIR&#x2F;.tmpwdCaYS&#x2F;main • Create directory git-prole • Move $TMPDIR&#x2F;.tmpwdCaYS&#x2F;.git to git-prole&#x2F;.git • In git-prole&#x2F;.git, create but don&#x27;t check out a worktree for main at git-prole&#x2F;main • In git-prole&#x2F;main, reset the index state • Move git-prole&#x2F;main&#x2F;.git to $TMPDIR&#x2F;.tmpwdCaYS&#x2F;main&#x2F;.git • Remove git-prole&#x2F;main • Move $TMPDIR&#x2F;.tmpwdCaYS&#x2F;main to git-prole&#x2F;main Preparing worktree (checking out &#x27;main&#x27;) • git-prole has been converted to a worktree checkout $ cd git-prole $ git worktree list &#x2F;path&#x2F;to&#x2F;git-prole (bare) &#x2F;path&#x2F;to&#x2F;git-prole&#x2F;main 719ed92 [main] </code></pre> <p>Nice! Let’s add a new worktree:</p> <pre><code>$ git prole add add feature1 • Creating worktree in ~&#x2F;git-prole&#x2F;feature1 for feature1 tracking origin&#x2F;main Preparing worktree (new branch &#x27;feature1&#x27;) branch &#x27;feature1&#x27; set up to track &#x27;origin&#x2F;main&#x27;. HEAD is now at 719ed92 Fix links to `git-worktree(1)` docs (#89) </code></pre> <p>New branches automatically start at the repository’s default branch, instead of the current commit.</p> <p>That’s the core of <code>git-prole</code>’s functionality, along with some goodies like copying untracked files to new worktrees to keep your build caches warm and your local configuration intact.</p> <p>The workflow enabled by cheap checkouts is super nice and I encourage you to give it a try! To get started with <code>git-prole</code>, check out <a href="https://9999years.github.io/git-prole/">the user manual</a> for installation instructions.</p> Giving My Student Loan Servicer a $560 Interest-Free Loan Tue, 20 Aug 2024 00:00:00 +0000 Rebecca Turner https://becca.ooo/blog/an-interest-free-loan/ https://becca.ooo/blog/an-interest-free-loan/ <p>Our story begins on April 8th, 2024. I was ecstatic: I’d just finished paying off my student loans. Well, maybe not ecstatic, given the Biden administration’s lackluster student loan forgiveness policy, but it was a weight off my shoulders. My joy lasted two days, until April 10th, when my monthly autopayment got processed and Aidvantage took $560 out of my bank account, leaving one of my loans with a balance of <em>negative</em> $560.</p> <h2 id="wait-what"><a class=anchor href="#wait-what" aria-label="Link to section wait-what">#</a>Wait, what?</h2> <p>Finding nothing useful in the help pages, on April 15th I submitted a support request:</p> <blockquote> <p>I made a manual payment to finish paying off my loans around the time of my monthly automatic payment. Now one of my loans has a -$560 balance. Can you… send that back to me?</p> </blockquote> <p>Just two days later, I received a reply on April 17th:</p> <blockquote> <p>Thank you for reaching out to us regarding the overpayment amount of $560.00. I’m happy to assist.</p> <p>This overpayment amount will be refunded to you. Currently the estimated timeframe for a refund to be deposited into your bank account is 8-12 weeks.</p> </blockquote> <p>Let’s get this out of the way: 8-12 weeks is an absurd amount of time to return the money. But what could I do? I set a reminder on my calendar.</p> <p>On July 10th, I called back to ask where the money was, and a support agent informed me that a check was sent out and should have arrived by now.</p> <h2 id="wait-a-check"><a class=anchor href="#wait-a-check" aria-label="Link to section wait-a-check">#</a>Wait, a Check?</h2> <p>Something odd has happened at this point: the original response said a refund would be “deposited in [my] bank account”, not that a check would be sent out! Nobody has been able to explain this portion of the story to me. All I know is that — despite having my bank account information, despite me explicitly opting in to for paperless communications — someone at Aidvantage decided to send me a check. Or, at least, they allege they sent me a check. They didn’t send certified mail, ask for a signature, or even provision a tracking number from the USPS, so there’s no evidence a check was ever sent.</p> <p>Again, what could I do? I asked the support agent if they could reissue the check, and they offered me a more appealing option: an ACH deposit. That would be great, I said. I gave them my account and routing numbers (which they already had) and they submitted an ACH transfer, due to be deposited into my account in 3-4 weeks.</p> <h2 id="wait-3-4-weeks"><a class=anchor href="#wait-3-4-weeks" aria-label="Link to section wait-3-4-weeks">#</a>Wait, 3-4 weeks?</h2> <p>Since when does an ACH pull take 3-4 weeks? 3-5 <em>days</em> is normal, and it’s usually shorter! I have no clue what’s going on here.</p> <p>But what could I do? I set a reminder on my calendar. After 4 weeks, I called back on Thursday, August 15th. The support agent informed me that they could see the ACH was sent out and that it should, for sure, we promise, be deposited into my account on Monday.</p> <p>At this point the process of calling back is beginning to wear on me. I don’t have the energy to do it on a Monday. I called back this morning, on August 20th. After hanging up on me several times, my fourth or fifth attempt of the day gets through to a support agent. He thanks me for my patience and puts me on hold, says he’ll look into it and might need to ask his supervisor. After a couple minutes he thanks me for waiting and says “it says here that before we can reissue a refund you have to either return the check to us — which you can’t do, because it didn’t physically get to you — or wait for it to go stale, which happens after… fourteen months? That’s ridiculous. Let me ask for clarification.” It was a small comfort that the support agent sounded roughly as baffled by this as I was. He re-read it a few times, possibly to make sure he wasn’t reading the Looney Tunes version of the refund procedures. No clarification came, and he sounded apologetic as he told me to check back in late August, 2025.</p> <h2 id="wait-august-2025"><a class=anchor href="#wait-august-2025" aria-label="Link to section wait-august-2025">#</a>Wait, August 2025?</h2> <p>Astute readers may have noticed that I got the first message from Aidvantage acknowledging the refund on April 17th, and fourteen months after April 2024 is <em>June</em> 2025. However, as my support agent informed me, the refund check wasn’t actually created until June 24th, 2024 — Aidvantage just twiddled their thumbs for 68 days before bothering to print the check.</p> <p>As for why a check was sent instead of an electronic refund in the first place? My support agent had no clue. If we consider <a href="https://en.wikipedia.org/wiki/The_purpose_of_a_system_is_what_it_does">the purpose of a system to be what it does</a>, however, the mechanism is clear: Aidvantage pretended to send me a check because it’s good for their balance sheet if I give them a $560 loan, interest-free, for a couple years. What am I going to do, stop giving them business? <em>I’m</em> not their customer, after all.</p> <h2 id="kafka-was-right"><a class=anchor href="#kafka-was-right" aria-label="Link to section kafka-was-right">#</a>Kafka Was Right</h2> <p>Franz Kafka, working at an insurance company at the beginning of the 20th century, saw these systems ossifying, and had a remarkably clear vision of where it was all headed: a society where all human coordination of services is replaced with bureaucratic instruments. The support agents I talked to at Aidvantage had no power to help me. They also had no ability to contact anyone who <em>could</em> help me: the bureaucratic machine airgaps the people I can talk to from the people with that power.</p> <p>A bureaucracy is an inversion of responsibility; instead of an organization being responsible for provisioning its services to you, <em>you</em> are responsible for filing the correct paperwork via the correct avenues at the correct time. Will an organization make it clear to you what services they provide? What paperwork to file? Where, when, and how to file it? That sounds expensive.</p> <p>Kafka was powerless to stop the transformation he saw. I think of him often, when I’m navigating an endless phone tree for the fourth or fifth time, or when I have an issue with a website and the only way to get support is to know an engineer working on it.</p> <p>I’ve set a reminder on my calendar for August 25th, 2025. I’ll see you then.</p> Announcing ghciwatch 1.0 Mon, 15 Jul 2024 00:00:00 +0000 Rebecca Turner https://becca.ooo/blog/announcing-ghciwatch/ https://becca.ooo/blog/announcing-ghciwatch/ <aside> <p><em>Note: This article was originally published on <a href="https://mercury.com/blog/announcing-ghciwatch">the Mercury engineering blog.</a> (<a href="https://web.archive.org/web/20240722213536/https://mercury.com/blog/announcing-ghciwatch">Archived on the Internet Archive Wayback Machine.</a>)</em></p> </aside> <p>At Mercury, we’ve developed a file-watching recompiler for Haskell projects called <a href="https://github.com/MercuryTechnologies/ghciwatch/">ghciwatch</a>, which loads a GHCi session (a Haskell <a href="https://en.wikipedia.org/wiki/Read%E2%80%93eval%E2%80%93print_loop">REPL</a>) and instructs it to reload or add modules when files in your project change. Ghciwatch is very similar in design to Neil Mitchell’s excellent <a href="https://github.com/ndmitchell/ghcid">ghcid</a>, but with a few new tricks up its sleeve:</p> <ul> <li> <p>Ghciwatch is blazing fast, even for the largest Haskell projects, loading our <a href="https://serokell.io/blog/haskell-in-production-mercury">backend server monolith</a> nearly 12 times faster than <a href="https://github.com/haskell/haskell-language-server">haskell-language-server</a>.</p> </li> <li> <p>Ghciwatch natively supports <a href="https://mercurytechnologies.github.io/ghciwatch/no-load.html">Cabal’s <code>--repl-no-load</code> option</a> to massively speed up startup and reload times by only loading the modules you need.</p> </li> <li> <p>Ghciwatch seamlessly loads newly-created modules and can handle deleted and removed modules without restarting the underlying GHCi session.</p> </li> </ul> <p>Check out the <a href="https://mercurytechnologies.github.io/ghciwatch/introduction.html">user manual</a> to get started with ghciwatch.</p> <h2 id="why-not-use-haskell-language-server"><a class=anchor href="#why-not-use-haskell-language-server" aria-label="Link to section why-not-use-haskell-language-server">#</a>Why not use haskell-language-server?</h2> <p>While HLS works great for smaller projects, the Mercury backend monolith consists of about 10,000 modules containing about 1.2M lines of Haskell code. For projects that large, HLS’s performance starts to break down: on my 2022 Mac Studio (M1 Ultra, 20 cores, 64GB memory), HLS takes a full 32 minutes to load the project. After it’s loaded everything, making changes to project files will start reloads that take at least 1.5 minutes (changes to modules nested deeper in the dependency graph can take as long as 5 or 10 minutes to reload, if HLS is able to reload at all). HLS is also very memory intensive, consuming 50GB of RAM after loading the project and climbing from there.</p> <p>In contrast, ghciwatch takes 2.7 minutes to load the entire project and about 7 seconds to reload. With <a href="https://mercurytechnologies.github.io/ghciwatch/no-load.html">Cabal’s <code>--repl-no-load</code> option</a>, modules are only loaded when they’re changed: as a result, ghciwatch can start up in about 6 seconds. The first change to a module will take much longer, depending on how many modules need to be loaded (between one second and two minutes), but subsequent reloads take between 0.2 and 7 seconds.</p> <p>Ghciwatch is a relatively thin wrapper around GHCi, so it’s able to take advantage of GHCi’s speed directly by interpreting modules, leveraging <a href="https://well-typed.com/blog/2023/02/interface-files-with-core/">bytecode linking</a>, and keeping artifacts cached more reliably between runs.</p> <p>Before building ghciwatch, we spent a while trying to make HLS faster; as of the time of writing, Mercury is <a href="https://opencollective.com/haskell-language-server">the largest organizational backer of HLS on Open Collective</a>, and we’ve also <a href="https://well-typed.com/blog/2022/04/hls-performance/">hired contractors like Well-Typed</a> to optimize HLS. While a number of incremental improvements have been made, there’s a limit to how fast HLS can get. HLS is built on top of GHC’s internal API and GHC is a batch compiler, which means that HLS’s latency and responsiveness can’t really be improved without implementing a new Haskell parser and type checker with different design goals. In his excellent <a href="https://matklad.github.io/2022/04/25/why-lsp.html">“Why LSP?”</a> blog post, matklad explains why batch compilers serve poorly as a basis for a language server:</p> <blockquote> <p>[A] batch compiler is optimized for maximum throughput, while a language server aims to minimize latency (while not completely forgoing throughput). Adding a latency requirement doesn’t mean that you need to optimize harder. Rather, it means that you generally need to turn the architecture on its head to have an acceptable latency at all.</p> </blockquote> <p>Practical evidence seems to support this argument:</p> <blockquote> <p>Language servers are a counter example to the “never rewrite” rule. [The] majority of well regarded language servers are rewrites or alternative implementations of batch compilers.</p> <p>Both IntelliJ and Eclipse wrote their own compilers rather than re-using javac inside an IDE. To provide an adequate IDE support for C#, Microsoft rewrote their batch compiler written in C++ into an interactive self-hosted one (project Roslyn). Dart, despite being a from-scratch, relatively modern language, ended up with three implementations (host AOT compiler, host IDE compiler (dart-analyzer), on-device JIT compiler). Rust tried both — incremental evolution of rustc (RLS) and from-scratch implementation (rust-analyzer), and rust-analyzer decisively won.</p> </blockquote> <p>Although we’re investigating larger changes to support our Haskell engineers in the future, we need tools for our engineers to use in the interim. Ghciwatch is fast enough for local development, and we’ve integrated Joseph Sumabat’s language server <a href="https://github.com/josephsumabat/static-ls">static-ls</a> and GHC plugin <a href="https://github.com/josephsumabat/hiedb-plugin">hiedb-plugin</a>, which leverage <a href="https://www.haskell.org/ghc/blog/20190626-HIEFiles.html">HIE files</a> to provide features like go-to-definition, hover documentation, and even automatic imports without the lag and memory usage of HLS.</p> <h2 id="why-reimplement-ghcid"><a class=anchor href="#why-reimplement-ghcid" aria-label="Link to section why-reimplement-ghcid">#</a>Why reimplement ghcid?</h2> <p>Ghcid is only about 2,000 lines of Haskell, is <a href="https://github.com/ndmitchell/ghcid/issues/346#issuecomment-974824490">no longer being developed</a>, and is architecturally unamenable to implementing the features we wanted. Additionally, Rust’s <a href="https://docs.rs/notify/latest/notify/">notify</a> crate is <a href="https://github.com/MercuryTechnologies/ghciwatch/pull/1#issuecomment-1583189642">much more flexible and mature</a> than Haskell’s <a href="https://hackage.haskell.org/package/fsnotify">fsnotify</a> package, with features like runtime reconfiguration that make watching and responding to file changes easy to implement. As a result, we decided to go with a fresh implementation and build our replacement from the ground up.</p> <p>One issue we wanted to fix with ghcid is caused by a GHCi bug which renders a GHCi session unusable after a module is moved or deleted. Ghcid doesn’t differentiate between file modifications and deletions, so engineers must manually restart it when modules are moved. In contrast, ghciwatch examines <em>how</em> files change in order to determine when the underlying GHCi session needs to be restarted. Ghciwatch also supports a number of <a href="https://mercurytechnologies.github.io/ghciwatch/lifecycle-hooks.html">lifecycle hooks</a> so that you can use tools like <a href="https://github.com/sol/hpack">hpack</a> to automatically regenerate your <code>.cabal</code> files before a restart.</p> <p>Ghcid also assumes that the set of files loaded in the underlying GHCi session will never change, and as a result it’s unable to add new files when they’re created. This is another scenario which requires engineers to manually restart their ghcid session. Ghciwatch tracks the set of loaded files, so it knows when to <code>:reload</code> the session and when to <code>:add</code> a new module. Ghciwatch also tracks if each module was loaded as a file path or as a module name to work around <a href="https://gitlab.haskell.org/ghc/ghc/-/issues/13254">another GHCi bug</a>.</p> <p>Additionally, we wanted to forward GHCi output to the user as soon as it’s printed, so that engineers can see the compilation progress during longer reloads. Ghcid, in contrast, just shows a “Reloading…” indicator during this process. Printing output for the user requires figuring out when GHCi is printing a prompt, which is a bit tricky because prompt lines don’t end in a newline, so a standard line-buffered streams is insufficient here (we solved this with <a href="https://github.com/MercuryTechnologies/ghciwatch/blob/108f5a5a5e2ce0fb509d61cf1644731281b3bab4/src/incremental_reader.rs">an abstraction we call an <code>IncrementalReader</code></a>).</p> <p>Ghcid only reloads the session when Haskell files change, meaning it will ignore changes to other files that impact the build, like <a href="https://hackage.haskell.org/package/persistent"><code>persistent</code> models</a> or <a href="https://hackage.haskell.org/package/shakespeare"><code>shakespeare</code> templates.</a> While it’s possible to pass additional filenames to the <code>--reload</code> or <code>--restart</code> options, ghcid won’t be able to detect newly-created files that match the same patterns. Ghciwatch <a href="https://mercurytechnologies.github.io/ghciwatch/cli.html#--reload-glob">supports matching files against globs</a> using the full <code>gitignore</code> syntax so that it knows exactly when to reload and restart.</p> <p>Like ghcid, ghciwatch is able to <a href="https://mercurytechnologies.github.io/ghciwatch/comment-evaluation.html">evaluate Haskell code in comments for quick testing and debugging</a>, but ghciwatch’s eval comments are able to access top-level bindings of the module they’re defined in, including unexported bindings.</p> <h2 id="what-s-next"><a class=anchor href="#what-s-next" aria-label="Link to section what-s-next">#</a>What’s next?</h2> <p>Ghciwatch has fully replaced ghcid in our development environment, but that doesn’t mean we’re done with it. We’re experimenting with replaying warnings for stale files, a TUI display that could include a progress bar when modules are compiling, and an interactive prompt for running Haskell code just like in an unmanaged GHCi session.</p> <p>You can try ghciwatch right now; check out the <a href="https://mercurytechnologies.github.io/ghciwatch/introduction.html">user manual</a> and <a href="https://mercurytechnologies.github.io/ghciwatch/install.html">installation instructions</a> to get started!</p> Why Git's diff3 merge conflicts are confusing Thu, 28 Mar 2024 00:00:00 +0000 Rebecca Turner https://becca.ooo/blog/why-diff3-is-confusing/ https://becca.ooo/blog/why-diff3-is-confusing/ <p>It’s popular (and good!) advice to tell people to set <code>git config --global merge.conflictStyle diff3</code> (<a href="https://blog.nilbus.com/take-the-pain-out-of-git-conflict-resolution-use-diff3/">“Take the pain out of git conflict resolution: use diff3”</a>), but I’ve always found <code>diff3</code>-style merge conflicts pretty confusing. I’ve never managed to internalize if the top section is from “my branch” or “their branch”. It turns out there’s a reason for this — <strong>the first section in a <code>diff3</code> merge conflict is your branch in a merge, but their branch in a rebase!</strong></p> <p>If you have <code>feature-branch</code> checked out and you want to update it to the current <code>main</code> branch, here’s how you’d read a <code>diff3</code> conflict:</p> <pre data-lang="diff" class="language-diff "><code class="language-diff" data-lang="diff">&lt;&lt;&lt;&lt;&lt;&lt;&lt; HEAD The &quot;current commit&quot;. For `git rebase main`, this will be the code in `main`. (Because `HEAD` is on top of `main`.) For `git merge main`, this will be the code in `feature-branch`. (Because `HEAD` is on top of `feature-branch`, and `main` is being merged _into_ it.) ||||||| parent of ea45005 The conflicting lines before either branch changed them. ======= For `git rebase main`, this will be the code in `feature-branch`. (Because you&#x27;re stacking the changes in `feature-branch` on top of `main`.) For `git merge main`, this will be the code in `main`. (Because you&#x27;re adding the changes in `main` to `feature-branch`.) &gt;&gt;&gt;&gt;&gt;&gt;&gt; ea45005 </code></pre> <p>PS: Try <a href="https://github.com/git/git/commit/4496526f80b3e4952036550b279eff8d1babd60a"><code>git config --global merge.conflictStyle zdiff3</code></a>, which moves as many lines as possible out of the merge conflict.</p> Rust for Haskellers Fri, 07 Jul 2023 00:00:00 +0000 Rebecca Turner https://becca.ooo/blog/rust-for-haskellers/ https://becca.ooo/blog/rust-for-haskellers/ <p>This is intended to be a brief introduction to <a href="https://www.rust-lang.org/">the Rust programming language</a> for Haskell programmers who want to work on a Rust project without too much fanfare. Fortunately, Rust has a lot of features we know and love from Haskell like Hindley-Milner type inference, typeclasses, and pattern matching, so aside from method call syntax and semicolons, Haskell programmers should feel right at home.</p> <h2 id="getting-started"><a class=anchor href="#getting-started" aria-label="Link to section getting-started">#</a>Getting started</h2> <p>Rust projects are built with <code>cargo</code> (which is a bit like <code>cabal</code> or <code>stack</code>) and <code>rustc</code>. The Rust installation is managed with a tool called <code>rustup</code> (like <code>ghcup</code>), which can be installed at <a href="https://rustup.rs/">rustup.rs</a>, or using <code>rustup-init</code> (e.g., <code>brew install rustup-init &amp;&amp; rustup-init</code>).</p> <p>Once <code>cargo</code> is installed, you’ll be able to build projects with <code>cargo build</code>.</p> <p>Many tools that are provided separately in Haskell projects are unified into <code>cargo</code> in Rust projects; formatting is <code>cargo fmt</code>, linting is <code>cargo clippy</code>, documentation can be generated with <code>cargo doc</code>, tests are run with <code>cargo test</code>, and so on.</p> <p>Using the <a href="https://rust-analyzer.github.io/">rust-analyzer</a> language server is highly recommended.</p> <h2 id="syntax-and-structure"><a class=anchor href="#syntax-and-structure" aria-label="Link to section syntax-and-structure">#</a>Syntax and structure</h2> <p><a href="https://cheats.rs/">cheats.rs</a> has a fantastic overview to the Rust language with plenty of links, but here’s the basics:</p> <p>(Run/modify this code on the <a href="https://play.rust-lang.org/?version=stable&amp;mode=debug&amp;edition=2021&amp;gist=ae0066f38d35df9fc17d0a5e40e8fc64">Rust Playground!</a>)</p> <pre data-lang="rust" class="language-rust "><code class="language-rust" data-lang="rust">&#x2F;&#x2F;&#x2F; A function with no return type returns `()`. &#x2F;&#x2F;&#x2F; These triple-slashed comments are documentation comments, written in &#x2F;&#x2F;&#x2F; Markdown and rendered with `cargo doc`. fn main() { &#x2F;&#x2F; Types are inferred by default for values but mandatory for functions. let hello = &quot;Hello&quot;; &#x2F;&#x2F; `println!` is a macro, indicated by the `!` after the name. &#x2F;&#x2F; &#x2F;&#x2F; This format string is translated into a lower-level format at &#x2F;&#x2F; compile-time, and is able to figure out that `{hello}` is the local &#x2F;&#x2F; variable `hello`. Just like Template Haskell! println!(&quot;{hello}, world!&quot;); &#x2F;&#x2F; Now let&#x27;s create a user. let user = User { &#x2F;&#x2F; Here&#x27;s where Rust starts to differ from Haskell; a string literal is &#x2F;&#x2F; just data in the binary, so we need to copy it into a buffer before we &#x2F;&#x2F; can start modifying it. &#x2F;&#x2F; &#x2F;&#x2F; Rust will check that your memory accesses are safe using linear &#x2F;&#x2F; (affine) types. name: &quot;Rebecca&quot;.to_owned(), age: None, }; &#x2F;&#x2F; Again, we need to `.clone()` the `user`, or else it&#x27;ll be gone after this &#x2F;&#x2F; function call. (`how_many_bytes_in_a_name` takes a `User`, not a `&amp;User` &#x2F;&#x2F; reference, so it &quot;owns&quot; its parameter.) println!(&quot;My name has {} bytes&quot;, how_many_bytes_in_a_name(user.clone())); match_demo(Some(user)); } &#x2F;&#x2F;&#x2F; Here we&#x27;re declaring a sum type. `Maybe` is called `Option` in Rust, and it&#x27;s &#x2F;&#x2F;&#x2F; in the prelude by default. enum MyOption&lt;T&gt; { None, Some(T), } &#x2F;&#x2F;&#x2F; Records are called `struct`s. &#x2F;&#x2F;&#x2F; We can “derive” instances of traits using a `#[derive()]` attribute; this &#x2F;&#x2F;&#x2F; uses a compile-time macro to compute the requested instance. &#x2F;&#x2F;&#x2F; Here, the `Debug` trait lets us print a representation of the object for &#x2F;&#x2F;&#x2F; debugging, and the `Clone` trait lets us deeply copy the object. #[derive(Debug, Clone)] struct User { name: String, age: Option&lt;u16&gt;, } &#x2F;&#x2F;&#x2F; Of course, we can pattern match on values of all sorts: fn how_many_bytes_in_a_name(User { name, .. }: User) -&gt; usize { &#x2F;&#x2F; Strings are UTF-8 under the hood, so getting a count of bytes is O(1) &#x2F;&#x2F; and a count of codepoints is O(n). name.len() } &#x2F;&#x2F;&#x2F; We can also pattern match using the `match` expression. fn match_demo(maybe_user: Option&lt;User&gt;) { match maybe_user { Some(user) =&gt; println!(&quot;{user:?}&quot;), None =&gt; println!(&quot;No user found!&quot;), } } </code></pre> <h2 id="types-galore"><a class=anchor href="#types-galore" aria-label="Link to section types-galore">#</a>Types galore!</h2> <p>Aside from the surface syntax and the fact that Rust is a strict language, Haskellers should find the language design familiar; Rust is based on the same <a href="https://en.wikipedia.org/wiki/Hindley%E2%80%93Milner_type_system">Hindley-Milner type system</a> that Haskell uses, and supports sum types and pattern matching natively.</p> <p>Rust has a system of typeclasses and instances that’s almost directly copied from Haskell (Rust calls them “traits”), which even supports fancy features like associated types.</p> <p>One of the bigger differences with Rust is its linear type system, which enforces that values are used “once.” (To be more precise, you can have any number of immutable references to an object, or one mutable reference, but never both.) This gives Rust the ability to generate very efficient and memory-safe code and also equips Rust with a first-class notion of mutability, which is useful for all the reasons we love immutability in Haskell.</p> <h2 id="onward"><a class=anchor href="#onward" aria-label="Link to section onward">#</a>Onward!</h2> <p>Read more of that <a href="https://cheats.rs/">Rust cheat sheet</a>, read <a href="https://doc.rust-lang.org/book/">the Rust Book</a>, and check out the <a href="https://doc.rust-lang.org/stable/std/">standard library documentation</a>.</p> No Good Soap Dish Tue, 20 Sep 2022 00:00:00 +0000 Rebecca Turner https://becca.ooo/blog/no-good-soap-dish/ https://becca.ooo/blog/no-good-soap-dish/ <p><em>note: i don’t think i’m going to finish this one.</em></p> <p>issue with hard soap: it gets wet.</p> <p>that’s the big thing. you get it wet, and then it’s all soft and it rubs onto the soap dish. and the soap dish gets dirty as a result.</p> <p>one might imagine solving this by dripping the moisture into the adjacent sink. there’s not actually much water dripping off of the soap, though. and another issue: soap gets very soft.</p> So You Want to Ship a Command-Line Tool for macOS Wed, 07 Sep 2022 00:00:00 +0000 Rebecca Turner https://becca.ooo/blog/so-you-want-to-ship-a-command-line-tool-for-macos/ https://becca.ooo/blog/so-you-want-to-ship-a-command-line-tool-for-macos/ <p>A word of advice: <strong>don’t.</strong></p> <p>At work, I’ve written a command-line tool which sets up the developer environment. It installs the Nix package manager, sets up a local Postgres instance, and handles all the complex bits of configuration. It does all of this on Linux and macOS, and it supports <code>bash</code>, <code>zsh</code>, and <code>fish</code> for shell configuration.</p> <p>We build and publish releases with GitHub actions, so that engineers can download and run the latest version of the tool when they need to set up a new machine or repair the development environment on an existing machine.</p> <p>On Linux, this is all fine and dandy (<a href="https://nixos.wiki/wiki/Packaging/Binaries">the usual headaches with running anything on NixOS aside</a>), but macOS requires that programs be code signed and notarized.</p> <p>Code signing on macOS is a <em>nightmare.</em> Knowing a little bit about asymmetric encryption, I expected the process to be roughly like this:</p> <ol> <li>Generate a public and private key (this is a blob of binary data, or roughly “a file”).</li> <li>Get Apple to make a certificate signed with my public key (this certificate is also a file).</li> <li>Run a program, pointing it to the files containing my private key and certificate, which “signs” an executable for distribution.</li> <li>When users go to run my program, the operating system can see that it’s signed with a valid Apple certificate and run it without issue.</li> </ol> <p>This is not, in fact, how it works.</p> <p>The first problem is that Apple provides no less than eight different sorts of certificates, with little documentation on what they’re used for, so expect to generate a half-dozen certificates and revoke several before you have a working one.</p> <p>The next issue is that the <code>codesign</code> command-line tool has a really opaque and complex interface. There’s several options you need to actually run code that aren’t really indicated as such (<code>-o runtime</code> and <code>-timestamp</code>, for instance, are mandatory for notarization).</p> <p>There’s few instructions for getting certificates into the keychain without GUI access (like, for instance, to sign code in CI), and Apple doesn’t have much documentation for signing anything that’s not built with Xcode.</p> <p>I was able to solve most of the issues with actually signing code with the <em>fantastic</em> <a href="https://gregoryszorc.com/blog/2022/04/25/expanding-apple-ecosystem-access-with-open-source,-multi-platform-code-signing/"><code>rcodesign</code> tool written by Gregory Szorc,</a> which has a very reasonable command-line interface that actually takes all the keys and certificates as files. I’ll excerpt a paragraph from the linked blog post that rings true to me (emphasis my own):</p> <blockquote> <p>I’ve learned way too much around minutia around how Apple code signing actually works. <strong>The mechanism is way too complex for something in the security space.</strong> There was at least one high profile Gatekeeper bug in the past year allowing improperly signed code to run. I suspect there will be more: the surface area to exploit is just too large.</p> </blockquote> <h2 id="macos-does-not-support-running-command-line-tools-from-finder"><a class=anchor href="#macos-does-not-support-running-command-line-tools-from-finder" aria-label="Link to section macos-does-not-support-running-command-line-tools-from-finder">#</a>macOS does not support running command-line tools from Finder</h2> <p>Once I had succeeded in signing an executable and verifying it, I immediately ran into another issue: <a href="https://developer.apple.com/forums/thread/706379">Gatekeeper blocks command-line tools from running when clicked.</a> Apparently this is a “known bug in macOS”, though because it’s filed in Apple’s proprietary “Radar” bug-tracker, I can’t see any of the details or if there are any plans to fix it.</p> <p>This is especially unfortunate because when an engineer downloads the binary from GitHub on macOS, the downloads pop up from the bottom of the screen, just begging to be clicked on.</p> <p>The solution, Quinn from the Apple Developer Forums tells us, is to “embed your tool in an application.” This, as far as I can tell, doesn’t work either, but let’s run through it.</p> <p>There’s a long guide on <a href="https://developer.apple.com/documentation/xcode/embedding-a-helper-tool-in-a-sandboxed-app">Embedding a Command-Line Tool in a Sandboxed App,</a> so I followed that, and then slowly, painfully, factored Xcode out of it, so that I wouldn’t have to figure out how to get a 10GB Xcode install onto the CI machine (remember, you need to be signed in to an Apple ID to download Xcode, and there’s no way to do it from the command-line).</p> <p>With <a href="https://www.witchoflight.com/">Cassie’s</a> help, I produced a short Swift script which (I hoped) would do what I want: First, it finds the embedded command-line tool binary with the <a href="https://developer.apple.com/documentation/foundation/bundle/1411412-url"><code>Bundle.url(forAuxiliaryExecutable:)</code></a> method. Then, that URL is passed to the <a href="https://developer.apple.com/documentation/appkit/nsworkspace/3172702-open"><code>NSWorkspace.open(urls:, withApplicationAt:)</code></a> method to run the embedded command-line tool in a new window. Finally, the <code>completionHandler</code> closes the app once the terminal window is open.</p> <p>(There’s <a href="https://github.com/apple/swift/issues/55127">a bug in Swift</a> that makes the <code>@main</code> attribute, mandatory for SwiftUI apps, not work, so I needed to run <code>swiftc</code> myself, adding a mysterious <code>-parse-as-library</code> option to fix the bug. I also needed to <a href="https://stackoverflow.com/questions/46532610/swiftc-possible-values-for-target-command-line-option">read the Swift source code</a> to determine the possible values for the <code>-target</code> command-line option.)</p> <p>Now, we have to code sign:</p> <ol> <li>The Swift wrapper script.</li> <li>The original command-line tool.</li> <li>The <code>.app</code> containing both of the above.</li> <li>The <code>.dmg</code> containing the <code>.app</code>, because an <code>.app</code> is just a directory, so you need to zip it up to distribute it.</li> </ol> <p>We also need to notarize the <code>.app</code> and the <code>.dmg</code>. Interestingly, you can only notarize <code>.pkg</code>, <code>.dmg</code>, and <code>.app</code> files (in <code>.zip</code>s) — command-line tools can only be notarized if they’re embedded in one of the listed containers.</p> <p>Most of the notarization docs tell you to use <code>altool</code>, which, only once you actually get it to upload something for notarization, will tell you that it’s deprecated and replaced by <code>notarytool</code>. <code>notarytool</code> will print <code>status: Invalid</code> if anything fails, with no additional details. (There’s a separate <code>notarytool</code> subcommand you can use to fetch logs with more information, but nothing in the output tells you this is an option, including turning on verbose/debug logging.)</p> <p>Anyways, once the app was assembled and passing all of Apple’s validation tools, it, uh, continued to not work!</p> <p>You can double click the app to run it, but when it tries to launch the embedded command-line tool, we get an error that “<code>mytool-cli</code> can’t be opened because the identity of the developer cannot be confirmed“, followed by “The application ‘Terminal’ can’t be opened. -128”.</p> <p>I do get one error with Apple’s tooling on these files; though the <code>.app</code> itself passes all the checks, the embedded tool fails <code>spctl</code>’s validation:</p> <pre><code>$ spctl -a -v --raw .&#x2F;mytool .&#x2F;mytool: rejected (the code is valid but does not seem to be an app) &lt;?xml version=&quot;1.0&quot; encoding=&quot;UTF-8&quot;?&gt; &lt;!DOCTYPE plist PUBLIC &quot;-&#x2F;&#x2F;Apple&#x2F;&#x2F;DTD PLIST 1.0&#x2F;&#x2F;EN&quot; &quot;http:&#x2F;&#x2F;www.apple.com&#x2F;DTDs&#x2F;PropertyList-1.0.dtd&quot;&gt; &lt;plist version=&quot;1.0&quot;&gt; &lt;dict&gt; &lt;key&gt;assessment:authority&lt;&#x2F;key&gt; &lt;dict&gt; &lt;key&gt;assessment:authority:flags&lt;&#x2F;key&gt; &lt;integer&gt;0&lt;&#x2F;integer&gt; &lt;key&gt;assessment:authority:source&lt;&#x2F;key&gt; &lt;string&gt;obsolete resource envelope&lt;&#x2F;string&gt; &lt;key&gt;assessment:authority:weak&lt;&#x2F;key&gt; &lt;true&#x2F;&gt; &lt;&#x2F;dict&gt; &lt;key&gt;assessment:cserror&lt;&#x2F;key&gt; &lt;integer&gt;-67002&lt;&#x2F;integer&gt; &lt;key&gt;assessment:remote&lt;&#x2F;key&gt; &lt;true&#x2F;&gt; &lt;key&gt;assessment:verdict&lt;&#x2F;key&gt; &lt;false&#x2F;&gt; &lt;&#x2F;dict&gt; &lt;&#x2F;plist&gt; </code></pre> <p>Googling for “obsolete resource envelope” gives several different bugs over a period of years, none of which apply, and anyways <code>spctl -a -v --raw</code> shows the same error for the <code>/bin/ls</code> that gets shipped with macOS.</p> <p>That’s where we’re at today. I’ve <a href="https://developer.apple.com/forums/thread/713932">reported this issue on the Apple Developer Forums,</a> but I don’t expect to get any actionable advice back, seeing as this distribution pathway is already attempting to work around several “known” macOS bugs.</p> <h2 id="the-tools-are-bad"><a class=anchor href="#the-tools-are-bad" aria-label="Link to section the-tools-are-bad">#</a>The tools are bad</h2> <p>All of these tools and all of their error messages are <em>garbage.</em> (<code>rcodesign</code>, being not written by Apple, is a notable exception.) Here are some of the commands you might run to check code signatures:</p> <pre><code>spctl -a -v --raw mytool.app codesign -verify -vvvv mytool.app codesign -d -vvv --entitlements :- mytool.app codesign -vvvv -R=notarized --check-notarization mytool.app </code></pre> <p>Just absolute spews of letters with no discernible meaning and certainly no reasonable intuition.</p> <p>On the Apple Developer Forums, <a href="https://developer.apple.com/forums/thread/696235">MirrorMan posts about roughly the same issue</a> with frustration:</p> <blockquote> <blockquote> <p>If the product is signed, notarised, and stapled correctly, everything should work. If not, you’ll need to investigate why Gatekeeper is unhappy (2), fix that, and then retest.</p> </blockquote> <p>“Fix that” !!!!!! No! Just no! There MUST be a way for Gatekeeper/spctl to render an exact description of whether or not it will run something (apps, command line tools, …) on arbitrary customer machines, immediately, right away, from the development machine, bypassing any caching or anything else. How many thousands of 3rd party developer days need to be lost for one Apple developer to spend a few days to update spctl to produce 100% accurate and useful messaging? For example, the messaging out of notarytool was very good! It told me which of the half-dozen hoops I had to jump through next (keychains! app-specific passwords!). It just takes too too long to do the research to figure out what to do at all, only to be confronted with an inexplicable error. $237billion a year, and developers have to guess? Apple can do better. Sorry for the rant but it is necessary.</p> </blockquote> <p>I don’t have much more to add, except that I’ve gained a lot of empathy for my friends who have turned their backs on the Apple ecosystem entirely for exactly this sort of developer-hostile behavior.</p> <h2 id="moving-on"><a class=anchor href="#moving-on" aria-label="Link to section moving-on">#</a>Moving on</h2> <p>So, what can we do with all of this? We have a few options, but none of them are particularly appealing.</p> <ol> <li> <p>Maintain the status quo; tell users to <code>chmod +x</code> and <code>xattr -d com.apple.quarantine</code> the downloaded executable. This bypasses the code signing mechanism but requires the user copy/paste magic commands.</p> <p>It’s fine, ultimately — the users are engineers, so they can manage a terminal or we can teach them to — but it’s not particularly “clean”, especially when Apple (theoretically) offers code signing mechanisms for this exact purpose, which my employer already pays for.</p> </li> <li> <p>Download the file with <code>curl</code>. Unix command-line tools like <code>curl</code> and <code>tar</code> don’t add the quarantine bit to the files they create on macOS, so it’s pretty easy to run software downloaded with them.</p> <p>Unfortunately, the repository where this tool is developed is private, so unauthenticated <code>curl</code> downloads won’t work. I’m seriously considering asking the CTO if we can make the repository public <em>solely</em> so we can use <code>curl</code> as a distribution mechanism. I could even write a platform-detecting shell script!</p> </li> <li> <p>Reimplement the entire thing in Swift and distribute it as an App built with Xcode without any embedded tools.</p> <p>Hahahaha. Just kidding.</p> </li> <li> <p>Distribute the tool as a custom Homebrew tap or something similar.</p> <p>This tool is responsible for <em>installing</em> Homebrew, and moreover I work at a Nix startup so plenty of engineers <em>refuse</em> to install Homebrew. I think this is a little bit silly, but it’s my job to support them and deliver the smoothest possible developement experience, so we can’t really distribute this with Homebrew.</p> </li> </ol> <p>I think I’ll probably distribute a code-signed executable, and macOS will say it can’t figure out who made it even though their own tools say the signature is fine, and I’ll cry myself to sleep at night.</p> <h2 id="conclusions"><a class=anchor href="#conclusions" aria-label="Link to section conclusions">#</a>Conclusions</h2> <ol> <li> <p>Do not attempt to “create software” for macOS. They don’t want you to. If you want to run your own programs, install Linux and suffer like you’re supposed to.</p> </li> <li> <p>If you need to code sign software for Apple computers, use <a href="https://gregoryszorc.com/blog/2022/04/25/expanding-apple-ecosystem-access-with-open-source,-multi-platform-code-signing/"><code>rcodesign</code></a>.</p> </li> </ol> transgender surgery is way too hard to get Sat, 11 Jun 2022 00:00:00 +0000 Rebecca Turner https://becca.ooo/blog/surgery-consult/ https://becca.ooo/blog/surgery-consult/ <p>I’m trying to get an orchiectomy, or rather to decide if I want one. It’s a procedure where a surgeon removes your testicles, and I don’t know much else about it because I think surgery is gross and unpleasant.</p> <p>I’ve been trying to get a consultation with one of the surgeons that does them and other transgender bottom surgeries, so I can ask some questions about orchiectomies and how they might interact with future surgeries. And this has been really hard! I’m used to a hostile medical system but the level of bureaucracy here has been really astonishing.</p> <p>I actually did schedule a consultation with one provider very easily, and then found out that they have had <a href="https://jezebel.com/when-surgeons-fail-their-trans-patients-1844774990">a number of malpractice suits</a> and a history of litigating against former patients with bad experiences, as well as a few other stories I won’t repeat here.</p> <p>Everyone else has been virtually impossible to get ahold of. Yesterday I got an 850-word reply back from <a href="https://www.mozaiccare.net/">MoZaic Care</a>, the practice of Dr. Heidi Wittenberg, in response to a request for a consultation.</p> <p>Here’s what I need to talk with a doctor about a transgender surgery, even to ask basic questions:</p> <ol> <li>Insurance card</li> <li>Photo ID (front and back!)</li> <li>Vaccination card “if vaccinated” (why ask for it then?)</li> <li>$250 for a consult for “self-pay” patients (everyone who doesn’t have a few Californian insurances, and of course “please be advised that Dr. Wittenberg and Dr. Bonnington has opted out of Medicare”)</li> <li>Prior authorization from referring physicial or PCP “for your intial consultation” for patients with “HMO/EPO/Managed Medi-cal/Narrow Network Plans”</li> <li>Three letters (from your hormone therapy prescriber, psychiatrist or psychologist (PhD), and your current therapist): required to “secure a surgical date” and “recommended” for the consultation. “If you submit your letters prior to your consultation, please be advised that they must be dated within 12-months of your surgery date and may need to be revised.”</li> <li>Ten different intake forms: <ol> <li>Optimizing surgical outcome agreement</li> <li>Medical intake form</li> <li>Financial policies</li> <li>Notice of HIPAA privacy practices</li> <li>Patient record sharing &amp; medical history authorization</li> <li>Rules of conduct agreement (“Waiting patiently for the physician’s and staffs’ availability is expected.”)</li> <li>Postoperative safety agreement</li> <li>Patient registration form</li> <li>Authorization for verbal discussion of protected health information</li> <li>An arbitration agreement, which waives your right to a jury trial. “Please note, the arbitration agreement must be sent back in color (please sign with black or blue ink).”</li> </ol> </li> </ol> <p>And this is all just to get in the door! Seems like the surgeons all know we don’t have any other choice. Part of me wonders if this process was intentionally designed to exclude all but the most obedient and well-resourced trans people, and the other part of me knows that <a href="https://en.wikipedia.org/wiki/The_purpose_of_a_system_is_what_it_does">the purpose of a system is what it does.</a></p> <p>I always wonder how cis people seem to come under the impression that transgender surgeries are easy to come by and accessible to anyone but the wealthiest trans people. I hope and dream that we can build a world where transgender people are allowed authority over their own bodies.</p> <hr /> <p>An anecdote from a friend, for contrast:</p> <blockquote> <p>My mom is getting a face lift in the next year, apparently it was absolutely painless to schedule as long as they know you can pay. It’s wild how cis people can get gender assuring surgeries no questions asked.</p> </blockquote> Git GUIs and why I don't like them Thu, 05 May 2022 00:00:00 +0000 Rebecca Turner https://becca.ooo/blog/git-guis/ https://becca.ooo/blog/git-guis/ <p>I’m pretty competent with the <a href="https://en.wikipedia.org/wiki/Git">Git</a> command-line interface, and I use it for most things. But sometimes, especially when I’m navigating a particularly thorny rebase, I like to be able to browse the commit history and view diffs as I go. For this I like to keep a Git GUI around, but none of them are particularly appealing to me. Here are the reasons why, largely for my own future reference.</p> <h2 id="gitk"><a class=anchor href="#gitk" aria-label="Link to section gitk">#</a>Gitk</h2> <p><a href="https://git-scm.com/docs/gitk/">gitk</a> is installed by default with Git (except on, like, macOS where you have to <code>brew install git-gui</code> separately) and is Perfectly Adequate. It is mediocre and doesn’t do much but it does show the repo history and the diffs in a split view, and it works fine.</p> <p>The diff viewer can ignore whitespace and you can hide added/removed lines. You can easily copy the commit hash, and you can search for commits with a few options:</p> <ul> <li>Commits containing a string (<code>--grep</code> option to <code>git log</code>, I think)</li> <li>Commits touching paths (extra arguments to <code>git log</code>, though not sure how this interacts with regexp filtering)</li> <li>Commits adding/removing a string (I think this is the <code>-S</code> option to <code>git log</code>)</li> <li>Commits changing lines matching a string (<code>-G</code> option to <code>git log</code>)</li> </ul> <p>Additionally, you can use exact matches, ignore case, or use <a href="https://en.wikipedia.org/wiki/Regular_expression">regexp</a> patterns for searching, and you can choose which fields of the commit to search (all, headlinge, comments, author, or commiter).</p> <p>I’m laying this out as the sort of minimum viable Git GUI feature-set. There are other things I’d like (for example, easy file history, interactively expanding context sections, syntax highlighting) but I don’t want anything there to be <em>missing</em>.</p> <h2 id="gitfiend"><a class=anchor href="#gitfiend" aria-label="Link to section gitfiend">#</a>GitFiend</h2> <p><a href="https://gitfiend.com/">GitFiend</a> (I’m trying version 0.32.0) is a free Git GUI for macOS, Windows, and Linux.</p> <ul> <li>No way to view a diff and the commit history at the same time.</li> <li>No search feature (described as a “work in progress” on the website).</li> <li>You can view a particular branch, or commits by a particular author. Not very general though.</li> </ul> <h2 id="gitup"><a class=anchor href="#gitup" aria-label="Link to section gitup">#</a>GitUp</h2> <p><a href="https://gitup.co/">GitUp</a> (no version information on the website) is another free Git GUI for macOS (exclusively?).</p> <ul> <li>No way to view a diff and the commit history at the same time.</li> <li>Does have a search feature! Searches “branches, tags, users, commit messages, and even commit diffs” with a setting. No way to search one of those <em>specifically</em>, as far as I can tell.</li> <li>Has a pane for committing changes, and a pane for viewing stashes. That’s neat.</li> </ul> <h2 id="fork"><a class=anchor href="#fork" aria-label="Link to section fork">#</a>Fork</h2> <p><a href="https://git-fork.com/">Fork</a> is a Git client for macOS and Windows (not Linux?) that costs $50 with a free evaluation period.</p> <ul> <li>Seems fairly polished! None of the weird graphical glitches I’m used to in the other clients. Animations are smooth and generally unobtrusive, it looks like an actual Mac App (not sure about Windows), and has most of the features I want.</li> <li>Filtering to a specific branch takes me a bit to discover, and the UI sort of implies it’s part of a generalized commit filtering interface, but there don’t seem to be any filters other than “the current branch”.</li> <li>There’s a log of <code>git</code> commands Fork runs, which is an appreciated bit of insight into what it’s doing.</li> <li>There seems to be a branch selection widget with a nice search feature and as part of a separate button a branch creation widget, which looks like it has a dropdown for more options but doesn’t. Kind of weird.</li> <li>Does let me view history and diffs at the same time! I can even pop the diff view out into another window.</li> <li>The diff interface is actually pretty nice; has a “view entire file” button and a side-by-side diff mode.</li> <li>Has a search feature that lets me search commit messages, authors, paths, and diff content, and lets me filter results to the current branch, but no regexp search.</li> <li>Also lets me view the entire file tree at any commit, which is nice.</li> <li>Doesn’t let me hide the left sidebar, though.</li> </ul> <h2 id="tower"><a class=anchor href="#tower" aria-label="Link to section tower">#</a>Tower</h2> <p><a href="https://www.git-tower.com/">Tower</a> is a Git client for macOS and Windows which costs $70 per year (!). At least from marketing, it looks like they’re building a fairly well-featured app with that money. They advertise drag and drop, easy undos, “a unique conflict wizard”, and file history, all features I want!</p> <p>The <a href="https://www.git-tower.com/blog/tower-mac-8/">list of improvements in the most recent version of Tower</a> are also pretty compelling — branch comparison and filtering, the sorts of interactive query-type features that are really well suited to GUIs.</p> <ul> <li>No way to hide the left sidebar.</li> <li>Kinda weird diff view — you can’t adjust or view context except through a menu item, there’s not a button nearby or anything.</li> <li>Lots of interesting features nestled away, including pull requests, but no way to comment on pull requests even through the built-in GitHub integration…</li> <li>Has a “view file at commit” feature but it doesn’t seem to work for me.</li> </ul> <p>I was decently excited about this one but nothing really elevates it above <code>gitk</code> or Fork for me.</p> <h2 id="smartgit"><a class=anchor href="#smartgit" aria-label="Link to section smartgit">#</a>SmartGit</h2> <p><a href="https://www.syntevo.com/smartgit/">SmartGit</a> is a Git GUI for macOS, Windows, and Linux that costs $60–90/year or $264–330 for a lifetime license (with a few options in between).</p> <ul> <li>You can hide all the sidebars, and it looks like you can drag them around to rearrange the panels, but you can’t, which is unfortunate. You can close and reopen any of them, though, which is nice and more than all the other clients do.</li> <li>Decent diff view, lets you expand context lines which is nice but doesn’t have great syntax highlighting. Has a side-by-side view.</li> <li>Has an interesting blame-like tool called “investigate” which tries to figure out the origin of certain changes, letting you go back in the history step by step. Pretty cool feature with a pretty cool UI, and unique among all these GUIs too!</li> <li>Has a nice search feature which does support regexps as well as arbitrary combinations of authors, committers, commit messages, refs, ID, name, and content! (Not sure what all of those mean.)</li> </ul> <h2 id="gitkraken"><a class=anchor href="#gitkraken" aria-label="Link to section gitkraken">#</a>GitKraken</h2> <p><a href="https://www.gitkraken.com/">GitKraken</a> is a Git GUI client for macOS, Windows, and Linux (free, $5/mo to interact with private GitHub repos through GitHub).</p> <ul> <li>Gave them a lot of GitHub permissions to sign up, I guess that’s fine.</li> <li>Has activity/audit logs, I like that!</li> <li>Appears to have a display scaling feature but only lets you choose amounts between 80% and 130% in 10% increments?</li> <li>Lets me hide the left sidebar!</li> <li>Lets me choose if I want the commit info on bottom or on the right.</li> <li>Not a ton of UI customization… I can’t really reorganize the panels at all.</li> <li>Weird shortcuts/keyboard support — I can view commits from a single branch, but it seems to be a right click action with no shortcut or anything. Feels like a common operation.</li> <li>Doesn’t let me view history and a diff at the same time.</li> <li>Has an interesting split view with a terminal on top and Git info on the bottom, which has a diff viewer that doesn’t take up the whole screen. I want that in the main view!</li> </ul> <h2 id="gitahead"><a class=anchor href="#gitahead" aria-label="Link to section gitahead">#</a>GitAhead</h2> <p><a href="https://github.com/gitahead/gitahead">GitAhead</a> was a free Git GUI client for macOS that’s been discontinued as of September 2021. RIP, it was nice when I used it (but not amazing).</p> <h2 id="conclusion"><a class=anchor href="#conclusion" aria-label="Link to section conclusion">#</a>Conclusion</h2> <p>None of the Git GUIs seem that good. At least SmartGit has a feature I can’t (easily) get with the command-line. Everything else feels like it’s lagging behind.</p> installing nixos doesn't have to hurt Fri, 18 Mar 2022 00:00:00 +0000 Rebecca Turner https://becca.ooo/blog/nixos-installation/ https://becca.ooo/blog/nixos-installation/ <p>NixOS is well-known for being user-hostile, and installation is the worst. Let’s look at a typical installation:</p> <ol> <li> <p>We go to the <a href="https://nixos.org/manual/nixos/stable/index.html">NixOS manual</a>, not to be confused with the Nix manual (for the Nix package manager specifically) or the Nixpkgs manual (for the Nix packages architecture and de-facto standard library for the Nix expression language).</p> </li> <li> <p>We are greeted with a table of contents 3500px tall, which we scroll past.</p> </li> <li> <p>We get to <a href="https://nixos.org/manual/nixos/stable/index.html">part 1, chapter 1, “Obtaining NixOS”</a>, which immediately tells us that to make a bootable USB drive we need to go to section 2.5.1 — that’s part 1, chapter 2, section 5 (“additional installation notes”), subsection 1 (“booting from a USB drive”).</p> <ol> <li> <p>Why is booting from a USB drive — how nearly every desktop or laptop Linux installation is performed — treated by the manual as of equal importance to PXE netboot and VirtualBox installations?</p> </li> <li> <p>Why is the first step in “Obtaining NixOS” an indirection to the middle of “installing NixOS”? Shouldn’t creating installation media come before installation? Is a bootable USB drive really worth relegating to “Additional installation notes”? The Fedora project offers a bootable USB creator for Windows and MacOS, and bootable ISOs for Linux.</p> </li> </ol> </li> <li> <p>Okay, we go through and make our bootable USB and boot from it. Now we’re instructed to run <code>systemctl start display-manager</code> to start the graphical environment. Shouldn’t the graphical environment start itself, if it’s available? Again, no other distro makes you have to <em>know</em> the command to start the display manager.</p> </li> <li> <p>Next we’re told we’ll need networking to run the installer, which gives us a whole host of conflicting commands to run:</p> <ul> <li>“check <code>ip a</code>” to see if networking is up, but no explanation what successful/unsuccesful output looks like.</li> <li>If we don’t have a DHCP server on our network, “configure networking manually using <code>ifconfig</code>” (no mention of <code>man ifconfig</code>)</li> <li>On the graphical installer we can configure the network through “NetworkManager”. (Is that a command-line program to run?)</li> <li>In non-graphical settings we are told we can use <code>nmtui</code> to configure NetworkManager.</li> <li>To configure the network manually (like it said to do with <code>ifconfig</code>?) we’re told to run <code>systemctl stop NetworkManager</code>.</li> <li>If NetworkManager is not available, configure with <code>sudo systemctl start wpa_supplicant &amp;&amp; wpa_cli</code>, and then shows commands that are sufficient for “most home networks”.</li> </ul> </li> <li> <p>Then we walk through manually partitioning. We can use any of <code>parted</code>, <code>fdisk</code>, <code>gdisk</code>, <code>cfdisk</code>, and <code>cgdisk</code>! Again, the manual expects us to be an expert and know which one is best for our use-case.</p> </li> <li> <p>I could go on. The manual also expects us to decide between UEFI/GPT or BIOS/MBR, we need to configure swap, we need to edit a text file on the command-line…</p> </li> </ol> <p>Throughout the process, the NixOS manual expects every user to be an expert and requires a lot of decisions in response to the current environment (e.g. “in the graphical installer, do this, otherwise, do this…”).</p> <p>I propose writing a tool which can perform the bulk of these steps (configuring the network, partitioning disks, generating a <code>configuration.nix</code>) automatically or with minimal user input, like a wizard.</p> <p>Will I do something about this myself? Perhaps, perhaps.</p> against discord channels Tue, 02 Nov 2021 00:00:00 +0000 Rebecca Turner https://becca.ooo/blog/against-discord-channels/ https://becca.ooo/blog/against-discord-channels/ <p>If you were to log on to my personal Discord server and glance at the unhideable channel list side-panel, you would see one text channel, one voice channel, and nothing else. Many people find that surprising, and most Discord servers I see have an absolute plethora of channels, often neatly arranged into groups — as an example, one server I’m in has 43 members and 21 text channels arranged in five groups.</p> <p>Before looking at the <em>effects</em> of using channels, and especially using a lot of channels, let’s look at <em>why</em> channels are used. Knowing the rationales for using channels allow us to ask if channels are serving their purpose, or what a feature designed specifically for that purpose could look like.</p> <ul> <li>channels demarcate topics, so that people can choose what to look at <ul> <li>but do you really want to draw borders around a fixed and unmoving set of allowed topics? second-order effect: having a #games channel and not a #movies channel discourages discussion of movies because there’s not a “proper place” for it</li> <li>channels are used like this for a few reasons: <ol> <li>categorization is tempting! “organizing” things feels good! but we must be careful to organize with a purpose, especially when we’re organizing social spaces and the discussions within them. <ul> <li>it’s easy to start categorizing innocently, and it only takes a few specific channels before users expect them for any discussion</li> </ul> </li> <li>users see it in other discord servers and copy it without thinking</li> <li>few people expect the structure and feature of a messaging app to have a powerful effect on the way they socialize and communities form in it</li> </ol> </li> </ul> </li> </ul> <p>Are channels good at that?</p> <ul> <li>discord servers are usually social spaces; I’m more interested in the people in them than the particular topics they’re talking about. <ul> <li>a group of people choosing specialized topics to participate in sounds more like a <a href="https://en.wikipedia.org/wiki/Working_group">working group</a> than a social space <ul> <li>a topic-based channel is basically a subreddit — fine for many circumstances, but not great for connecting with people as <em>people</em> <ul> <li>i like people’s personalities, the different ways they approach situations, their experiences, etc. a lot more than i like their engagement with a particular topic</li> </ul> </li> </ul> </li> <li>and in that case, more channels is just more places to click before i can find the discussion <ul> <li>that makes channels burdensome in small discords (with only a few dozen active members) that tend to not have more than one or two discussions going at a time</li> <li>contrast with slack channels, which look similar (sidebar on the left, names start with a <code>#</code>) but are explicitly <em>opt-in</em> — you don’t <em>want</em> to be in every slack channel in a workspace, and you usually don’t want to keep up with the discussions in all of them either.</li> </ul> </li> <li>when is it actually important to separate discussions or hide channels? <ul> <li>so that people can avoid certain topics (nsfw material, medical talk, etc.) <ul> <li>this is needed because discord lacks tagging, content warning features, consent screens except for nsfw channels (which, notice, require a separate channel), etc.</li> </ul> </li> <li>so that a particular discussion doesn’t overwhelm the rest of the space</li> <li>to prevent any one channel from becoming too busy in large servers</li> </ul> </li> </ul> </li> </ul> <p>What are the effects of using a lot of channels — that is, what happens when a social space has about half as discussion-slots as it does people?</p> <ul> <li> <p>channels atomize discussion: most channels are rarely used</p> <ul> <li>eventually, users expect a channel for each discussion topic &amp; #general is abadoned</li> <li>each additional channel <em>inherently</em> makes the rest of them slower and weaker; unlike IRC channels or Slack channels, each channel in a discord has exactly the same members, so each new channel divides the attention of all the members one more way <ul> <li>you can technically make channels visible only to members with certain roles, but it’s so much of a pain to manage i’ve only seen it done once or twice, usually as a way to verify that only trusted users have access to the server to protect against leaked invite likes</li> </ul> </li> </ul> </li> <li> <p>channels make it hard to follow discussions.</p> <p>first, even finding out where the discussion is happening is difficult, because channels are opaque; discord doesn’t have any sort of “this channel is home to 50% of the server’s activity” or “10 people messaged this channel in the last few hours” indicator, so you have to go through each one individually and check. finding out where activity is becomes much harder when i have to scroll up and down a partially-hidden channel list (with an auto-hiding scroll-bar, so there’s no indication that it’s partially hidden) to see which channel is the source of a new notification.</p> <p>even once you’ve figured out which channel has activity, catching up becomes harder and harder with each new channel. here’s the rough process:</p> <ol> <li>open channel</li> <li>re-read a few old messages to remember context</li> <li>read the new messages to catch up</li> </ol> <p>steps 1 and 2 have to be repeated for each channel. especially when lots of channels are used with few people, there might be a half-dozen channels with a few new messages each, so time spent remembering the previous discussion (and context-switching, which is much harder with adhd) quickly begins to outpace the time it takes to actually read the new messages and respond to them.</p> <p>storing context for “where the discussion left off” is a lot harder when 40 people are having 20 discussions than it is when 40 people are having 2 or 3 discussions. it’s much easier to read a few, more-active channels.</p> </li> </ul> <p>“My server is too big to not have channels — you can’t usefully stuff a thousand people into a single channel productively. (As evidence, look at Twitch streams!)”</p> <ul> <li>discord just doesn’t scale well, because it lacks good threading and discussion-organizing features <ul> <li>and that’s okay, for the most part, because it’s an instant messaging app, not a customer support forum, not a wiki, not a documentation repository, not a zulip, etc.</li> </ul> </li> </ul> type-driven computing Mon, 01 Feb 2021 00:00:00 +0000 Rebecca Turner https://becca.ooo/blog/type-driven-computing/ https://becca.ooo/blog/type-driven-computing/ <p>I really like command line user interfaces. I think they live up to a sort of <em>conversationality</em> that computing often lacks. In the <a href="https://en.wikipedia.org/wiki/Desktop_metaphor">desktop metaphor</a>, stacked GUI windows often feel like a clutter of unorganized papers, whereas the command line — at its best — presents an ordered ledger of queries and responses.</p> <p>But the command line hasn’t really grown with us. Instead, it’s been left to linger, forever emulating <a href="https://en.wikipedia.org/wiki/VT100">the VT100</a> — state of the art, for 1978. I’m pretty frustrated with that, and though I’m hardly the first (Gary Bernhardt explores some similar ideas in his 2012 talk <a href="https://www.destroyallsoftware.com/talks/a-whole-new-world">“A Whole New World</a>), I often feel like the people I talk to, technically-inclined or not, don’t really understand what I’m thinking about when I talk about the ways computers fail me.</p> <p>Therefore, here are some ideas I have for a better command line. Let’s establish some scope and basic points first:</p> <ul> <li> <p>Backwards compatibility isn’t a concern here. I’m not presenting a business plan, I’m presenting a vision. I’m not interested in the implementation, how to build an ecosystem around this concept, the way capitalism would hinder my concepts, or anything else. I <em>know</em> why these systems don’t exist. Let yourself imagine, just for a bit, that we could truly build something new.</p> </li> <li> <p>The vision is inspired by command lines and type systems, but stretches beyond a single terminal program. Ultimately, I’m talking about a different way to <em>use</em> and <em>interact with</em> computers. As with all things in computation, this idea works better the more tightly it’s integrated with its surroundings.</p> </li> <li> <p>I am uninterested in the grid-of-characters model for terminals. I don’t believe the properties it offers are worth the drawbacks.</p> </li> </ul> <h2 id="programs-as-types"><a class=anchor href="#programs-as-types" aria-label="Link to section programs-as-types">#</a>Programs as types</h2> <p>When we run a command line program, we’re usually constructing an instance of a <em>type</em> representing the invocation we want.</p> <p>To make this concrete, let’s think of a program as a sum type, where each subcommand corresponds to a variant. With <code>git</code>, for example, we can run the <code>status</code> or <code>commit</code> subcommands, but we can only choose one at a time:</p> <pre data-lang="bash" class="language-bash "><code class="language-bash" data-lang="bash">git status git commit </code></pre> <p>These correspond to variants of a sum type (tagged union). These sum types can be nested, even, for sub-subcommands and more:</p> <pre data-lang="bash" class="language-bash "><code class="language-bash" data-lang="bash">git remote add ... git remote set-url ... </code></pre> <p>For product types (records/structs), we have options/flags, which have a long form (like <code>--long</code>, or <code>--untracked-files</code>) and sometimes a short form (like <code>-u</code>), along with a variety of conventions for boolean switches, reading options from files, and so on.</p> <p>Writing the type out explicitly in Rust syntax looks kinda like this:</p> <pre data-lang="rust" class="language-rust "><code class="language-rust" data-lang="rust">&#x2F;&#x2F; Git is run by specifying an (optional) command and other configuration: struct Git { command: Option&lt;Command&gt;, configuration: Configuration, } &#x2F;&#x2F; We have some common options for any `git` command: struct Configuration { exec_path: Directory, paginate: bool, work_tree: Directory, &#x2F;&#x2F; ... } &#x2F;&#x2F; Then, individual commands have their own options and subcommands: enum Command { Add { verbose: bool, dry_run: bool, interactive: bool, &#x2F;&#x2F; ... }, Remote { Add { &#x2F;* More configuration options... *&#x2F; }, Rename { &#x2F;* ... *&#x2F; }, &#x2F;&#x2F; ... } &#x2F;&#x2F; ... } </code></pre> <p>And we have ways of setting up this sort of type system, either by writing a type directly (like Rust’s <a href="https://docs.rs/structopt/">structopt</a>) or by setting the subcommands and constraints directly (like Python’s <a href="https://docs.python.org/3/library/argparse.html">argparse</a>).</p> <p>And then… we throw all that information away and write completely separate tab-completion scripts for various shells, which are usually pretty bad.</p> <p>Although we have a pretty consistent system for understanding command line arguments as types, both as users and as programmers, the actual user interface, the shell, has <em>no knowledge</em> of this system whatsoever — which is a shame because it keeps us from applying the tools we’ve built for creating and filling out forms and other sorts of records to the problem of running programs.</p> <h2 id="building-usability-and-documentation-into-the-system"><a class=anchor href="#building-usability-and-documentation-into-the-system" aria-label="Link to section building-usability-and-documentation-into-the-system">#</a>Building usability and documentation into the system</h2> <p>What could a shell with a canonical view of this type information do?</p> <p>When you’re thinking about it, don’t imagine an extra step of processing, like the <a href="https://fishshell.com/">fish shell</a> parsing man pages to find command line arguments for programs that bother to distribute a man page. That’s a bolted-on solution. A hack. It works decently in an environment filled with legacy programs, but we’re not worrying about that right now. Instead, imagine if the command line interface and the longer-form man page documentation came from the same source of truth in the program, guaranteeing that they never fall out of sync, because adding an option to the program and adding it to the documentation are the <em>same</em> action.</p> Slack’s threads are terrible for accessibility Fri, 23 Oct 2020 00:00:00 +0000 Rebecca Turner https://becca.ooo/blog/slack-threads-are-terrible-for-accessibility/ https://becca.ooo/blog/slack-threads-are-terrible-for-accessibility/ <p>Slack’s lackluster implementation of threading shapes the rest of its collaborative features, largely nullifying them. Slack gives users the <em>ability</em> to reply to messages, creating a thread. I say <em>ability</em> because that’s all it is: a possibility that technically exists.</p> <p>But what do Slack’s threads give us, and what do we want from them? They can’t be organized; for one, there’s no way to move a thread from one channel to another where it might more accurately belong, and there’s also no way to categorize, sort, or browse threads. Slacks’ threads are branches off of a <em>conversation,</em> not groups and bases of knowledge like they can become in a <a href="https://www.phpbb.com/">phpbb</a> or similar. For instance, threads can’t be tagged, and it’s difficult to add additional context to a thread — messages can’t be pinned to a thread, not that it would really matter, because there’s no way to search a thread. The link between a thread and its channel is tenuous as well; there’s no notifications for new threads or messages in threads you’re not “following,” so unless you never miss a Slack message (difficult in general, especially if you’re attempting to enforce any sort of work-life boundaries) it’s easy to miss even large, active threads. On the flip side, when participating in a thread, replies are easily lost for the rest of the channel among a flurry of other messages unless you (tediously) check “also send to channel” every time you message — and these downfalls are compounded by the plethora of niche single-purpose channels Slack encourages users to create; users have to check each channel manually without the advantages of a unified feed where they can see, sort, and filter all messages.</p> <p>It’s not like I’m the first person to point out these flaws, either; <a href="https://twist.com/home">Twist</a>, the Slack-replacement messaging and communication software from Doist (the company behind <a href="https://todoist.com/home">Todoist</a>), capitalizes on Slack’s shortcomings by prominently advertising searchability, organization, threading, and archival features in Twist — it’s just too easy to lose information in Slack.</p> <p>Slack’s tools for coreferencing — that is, enabling two or more people to “point to” the same thing — are rudimentary at best, largely because Slack has no concept of context within the application. There’s no concept of tabs and the “history” feature, which I discovered while writing this article — is next-to-useless because it browses through the entire application’s state; there’s no way to go back and forward within a chat, panel, or sidebar. Even early browsers understood that we’d want to look at multiple webpages at once by opening multiple windows, but modern communication applications refuse the possibility entirely — Slack, Discord, Signal, Telegram, and even Tinder flat-out-refuse to be viewed twice at once; the idea that a user would want to carry on multiple conversations at once without having to switch windows or views has seemingly never occurred to the designers or developers.</p> <p>As a result, coreferencing is nearly-impossible, because Slack and its ilk prevent users from talking to a person or group of people while looking at something else — and especially while looking at multiple other things.</p> <p>These shortcomings seem particularly ironic given that Slack is allegedly designed to “bring teams together” and to enhance and streamline collaboration within corporations — did Slack’s developers not notice all the ways their own application got in the way of actually collaborating? (It wouldn’t be too surprising, though, coming from a company which branded itself an expert on remote work at the start of a pandemic despite having previously forbidden remote work entirely.)</p> <p>Slack compounds this effect by presenting notifications in a chaotic, largely unordered interface, where there’s no way to check items off (or control how items get checked off), where each notification is presented largely without context (because — remember? — threads can’t be titled, renamed, or tagged), and where notifications are scattered across the application in an unpredictable, high-effort spread — threads, mentions, and reactions only account for a small portion of what I’m expected to “keep up with” in a Slack workspace; I need to check each channel with new messages (which, usually, is all of them) manually, as well as direct messages, and perhaps I additionally have to follow up on other tasks that people haven’t replied to (and, of course, Slack gives me no tools for managing these).</p> <p>It’s frustrating to be surrounded by tools and systems — particularly those, like Slack, that I’m forced to use for school or work, and particularly those, like Slack, which have proprietary applications and no other way to connect through more accessible services (like the IRC bridge <a href="https://web.archive.org/web/20180307125841/https://get.slack.help/hc/en-us/articles/201727913-Connect-to-Slack-over-IRC-and-XMPP">that Slack shut down in 2018,</a> long before <a href="https://web.archive.org/web/20190325002116/https://www.reddit.com/r/Blind/comments/80ql6f/slack_taking_steps_to_improve_screen_reader/">even attempting to make the first-party Slack applications usable for blind people with screen readers</a>).</p> rustconf Tue, 18 Aug 2020 00:00:00 +0000 Rebecca Turner https://becca.ooo/rustconf/ https://becca.ooo/rustconf/ <p><a href="/rustconf/2020/">rustconf 2020: “Rust for Non-Systems Programmers”</a> (24 minutes)</p> Open letter: Brandeis is failing its disabled students Fri, 07 Feb 2020 00:00:00 +0000 Rebecca Turner https://becca.ooo/blog/brandeis-is-failing-its-disabled-students/ https://becca.ooo/blog/brandeis-is-failing-its-disabled-students/ <p><em>Note: This letter was originally published in <a href="https://brandeishoot.com/2020/02/07/open-letter-brandeis-is-failing-its-disabled-students/">The Brandeis Hoot</a>. The letter itself is a near-exact copy of an email I sent to interim student accessibility services director Scott Kalicki, explaining why I was uninterested in arranging a closed-door meeting where my concerns would be Heard and then promptly and completely forgotten. Unfortunately, I suspect that these complaints will already be familiar to disabled students at most other US universities.</em></p> <p>Dear Brandeis Administration, Board of Trustees, Office of Budget and Financial Planning, Student Accessibility Support and Counseling Center,</p> <p>Thanks for asking me what I need as a disabled student to feel welcome and succeed here. I need actual support — in the form of professionals advocating for me and my right to participate and belong in class and executive function counseling. It’s immensely disappointing that neither our “<a href="https://www.brandeis.edu/academic-services/accessibility/index.html">Student Accessibility Support</a>” nor our mental health services on campus are equipped to help with executive functioning, so that’s out, which leaves us with advocacy. You’ve heard me out asking for accommodations, and I’m grateful for that, but I require more. I’m still not able to participate in my classes meaningfully; I still need more to be done. And, too much of the time, your priority during our conversations is to lead me back toward our disappointing and lackluster set of “pre-prepared” accommodations, rather than actually making change to construct an environment in which I can succeed.</p> <p>Student Accessibility Support (SAS) has an obligation to provide me the necessary support to succeed at Brandeis — I’d love to receive that support. But before I come to the table, I want some guarantee that SAS will be working for me, and on my behalf, to provide me with accommodations — not on behalf of the administration and a board that sees Brandeis as an investment vehicle more than an educational institution existing to make as little change as possible. I’m sick of professors not knowing what my accommodations are because nobody bothered to tell them and/or they didn’t bother to read their letters; I’m sick of last-minute preparations for exams planned months in advance; I’m sick of accommodations making my life harder; I’m sick of being left behind and left out of the classes I’m going five figures into debt to participate in.</p> <p>Are you going to work with me and my disabled peers to give us the learning environment we were promised? Or are you going to work with the administration to preserve university profits and avoid making changes for the disabled “problem students” who are too mad to shut up and accept the scraps of accommodations they’ve been offered by an unprofessional, understaffed, and reactionary organization?</p> <p>I want to know what you’re doing to change SAS. It’s falling short in a lot of catastrophic and obvious ways, as <a href="http://brandeishoot.com/2018/11/09/letter-sent-to-liebowitz-calls-for-reform-on-accessibility/">last year’s open letter to President Ron Liebowitz</a> showed. What are you doing to enact the institutional-level change that needs to happen in order for disabled students at Brandeis to be given the opportunities and access we’ve been promised? Our time here is limited. My time here is limited. What have you been doing? What will you do to fix the enormous failures Brandeis has demonstrated to its disabled students before I graduate in a year? And if it’s “impossible” to fix these failures, I want to know why. I want to know why it is acceptable for me to be collateral damage, and I want to know why we’re okay, both personally and institutionally, with disabled students falling through the cracks, with our education being manufactured to fail and thoughtlessly thrown away.</p> <p>Meeting with staff and administration — deans, presidents, directors — takes time and energy from me — precious, valuable resources that are intensely limited due to my disability. I want some assurance that we can do more than offer blank platitudes to each other. Spending my energy fighting for accommodations makes it harder to attend classes and focus on my work here. I don’t want explanations for why the system isn’t working; I want actions that fix it. Here’s a tiny list of a few of the accommodations and changes I want, off the top of my head. Make no mistake, this list is scattered and incoherent. It’s scattered because that’s all I can muster. That’s all I can get out of my brain today. In the words of Porpentine, <a href="https://thenewinquiry.com/hot-allostatic-load/">“I am too sick to write this article.”</a> All I can squeeze out between sickness and trauma and therapy and an unending stream of homework and essays and obligations. It’s not a comprehensive and formal program of change to fully reform Brandeis into a functional institution. It’s a set of specific failures that I want addressed 30 years ago — or, if Brandeis is aiming for second best, — today. These failures are overlapped and intertwined.</p> <p><span id=demands></span> Here’s what I want:</p> <ol> <li> <p>Material acknowledgment that I had to and have to work harder than my abled peers for the time at Brandeis when I didn’t have accommodations, and during the times when my accommodations have been inadequate or a majority of my energy has been spent attempting to navigate student accessibility (i.e. my entire college career so far). This could be in the form of extra credits awarded for each class (to reflect the additional effort and time I had to put in, compared to my peers) or in the form of a flat increase of the grades for each class (to reflect the limited time and resources I had to work with, particularly during exams, compared to my peers). I’m open to other options, but those two seem like the most obvious, reasonable, inexpensive, and non-disruptive.</p> </li> <li> <p>An alternative schedule that would allow me to focus intensely on one subject, because my disability prevents me from jumping around and managing four or five different schedules and four or five different professors who like to communicate in four or five different ways. A schedule that lets me dive deeply into the material and excel in a way that a scattershot choice of four or five different areas of knowledge prevents.</p> </li> <li> <p>Accommodations that focus on areas other than test-taking. Dramatically restructured lectures, with slides that present a coherent narrative for students to follow. Slides available online before the lectures.</p> </li> <li> <p>Courses restructured for chronically ill students who need to miss class on a nearly weekly basis or greater, allowing them to participate without excluding them or leaving them behind.</p> </li> <li> <p>A 30-student cap on every course, including introductory courses. This would mean splitting large courses into as many sections as we require and hiring additional faculty to teach those courses, focusing on underrepresented and marginalized academics.</p> </li> <li> <p>Failing to provide accessible and thorough course materials (including interactions through LATTE, timely responses to emails, assignment postings, syllabi, course texts, etc.) should be a tenure-revoking offense — just like failing to meet any other job responsibilities would be.</p> </li> <li> <p>Sign language interpreters, large print, and braille materials available on request without needing a documented disability or any form of means-testing.</p> </li> <li> <p>Banning “no-electronics” policies that single out disabled students who use electronics as access tools and exclude disabled students who are not rich enough for a diagnosis. New policies which explicitly recognize that nearly all electronics, including cell phones, can be and often are access tools.</p> </li> <li> <p>Assignments to be posted to <a href="https://www.brandeis.edu/its/services/software-business-systems/latte/index.html">LATTE</a> (the name of Brandeis’ <a href="https://moodle.org/">Moodle</a> instance) in a timely fashion — in general, for LATTE to actually be used promptly and correctly. Failing to put all the information about my assignments and courses in one place (e.g. with hidden syllabi, alternative course websites, critical announcements sent through emails and not otherwise posted to the course page) makes coursework unmanageable and leaves me (and in general, the most vulnerable students) behind.</p> </li> <li> <p>I want better syllabi from all my professors, including all texts, assignment dates, and lecture material (at least down to the nearest week) to allow me to prepare for my courses. I want this information without having to email back and forth or meet individually with professors (as I did with every professor this semester).</p> </li> <li> <p>I want an end to requirements for medical documentation to receive accommodations. If Google and JP Morgan can give me ADHD assessments poorly disguised as “personality quizzes” that they can then use to discriminate against me in the hiring process, we can figure out some way to make accommodations easier to access. (And yes, both of them did do that to me. I even kept screenshots.)</p> </li> <li> <p>Increased student wages. My summer internships pay more than three times what I make at Brandeis. (An immense privilege!) Why doesn’t Brandeis value my talents? Increased pay would let me devote more of my time to keeping up with the classes I struggle with.</p> </li> <li> <p>An independent council of disabled students and faculty members with a budget and the powers to enact changes to accommodations and university policy and to terminate professors with a history of misconduct.</p> </li> <li> <p>A safety net for all students, but in particular new students. My first-year advisor didn’t meet with me or reply to my emails for an entire semester, leaving me to figure out everything on my own. That’s water under the bridge for me, but I don’t want it to happen to anyone else.</p> </li> <li> <p>I want professors who want to teach. Professors who are hostile to students who ask questions or express interest should not be tolerated. It doesn’t serve students for the administration to force professors who only want to do research to teach courses. Similarly, I want Brandeis to hire professors based partially on their skill as educators, not exclusively on their expertise in their field. Given that the vast majority of courses are taught at the undergraduate level, it doesn’t make sense to prioritize incredibly specialized knowledge over the ability to share the basics within each field.</p> </li> <li> <p>Mandatory accessibility training for every staff and faculty member on campus, so that I never get a look of surprise (or get scolded in front of a class) when I use an accommodation again. I should not need to explain my disability or my accommodations to professors who are capable of talking to SAS directly, because my time and energy is precious and limited. Having SAS “encourage that I self-advocate” is infuriating and counterproductive and has resulted in accommodations nearly falling through multiple times.</p> </li> <li> <p>I want at least half of the SAS staff to be disabled, and for a dramatic increase in SAS staffing. Roughly <a href="https://www.cdc.gov/media/releases/2018/p0816-disability.html">1 in 4 American adults live with a disability</a>. Brandeis has <a href="https://www.brandeis.edu/about/facts/index.html">a population of 5,800</a> students, and let’s say that a single employee can effectively manage accommodations for about 50 students (probably stretching it). That works out to 29 SAS employees. Why do we not have that currently?</p> </li> <li> <p>Similarly dramatic increases to <a href="https://www.brandeis.edu/counseling/">Brandeis Counseling Center (BCC)</a> staffing, reflecting the inability of mental health workers to adequately care for caseloads larger than 15 or 20 people at a time. An end to the policies that use group therapy to triage “low-priority” students. A dramatically increased budget. More than one trans therapist. Disabled therapists and an actually accessible building. An end to waiting lists for intake appointments in their entirety. Therapists specializing in learning disorders, ADHD, psychosis, OCD, PTSD and complex post-traumatic stress disorder (CPTSD), personality disorders, dissociative disorders, and <a href="https://www.pluralpride.com/playbook">plurality</a>. Therapists working in politically aware frameworks who are able to adequately address external stressors in a way that cognitive behavioral therapy (CBT) does not. Therapists who don’t reproduce trauma within their sessions, in particular with disabled students, trans students and students of color.</p> </li> <li> <p>Admissions quotas for physically and mentally disabled students commensurate with the general population. Admissions quotas for students of color (and in particular for Black and Native students) and poor students commensurate with the general population. I want the unbalanced student population corrected in four years and the staff and faculty demographics corrected in six years.</p> </li> <li> <p>Increased transportation options on campus as well as to Waltham and Boston/Cambridge, running at least twice as frequently. The ability to reserve space on a bus at any time (arbitrarily long or short) in advance of the bus’s journey with decreased barriers to entry and support for mobile devices in order to make driving a car no longer a necessity for students who live off campus.</p> </li> <li> <p>Wheelchair accessibility for every floor of every building on campus — including <a href="https://www.brandeis.edu/skyline/">Skyline</a>, which, despite the tour guides’ brags that it’s <a href="https://www.brandeis.edu/skyline/about.html">the most accessible building on campus,</a> has heavy, difficult-to-open, manual interior doors. I want tour guides to announce every building that isn’t wheelchair accessible when showing the campus to prospective students. And I want real accessibility, not just “well technically you can get to <a href="https://www.brandeis.edu/about/visiting/map.html?bldgid=0067">Goldsmith</a> 300 by entering <a href="https://www.brandeis.edu/volen/">Volen</a> across the quad, going down a long hallway, taking an elevator, crossing the skybridge, and passing through several heavy and non-automatic (i.e. not wheelchair accessible) doors”-style accessibility. I want the wheelchair-accessible routes through campus to be direct and convenient, not coiled through back hallways like they were designed by Wile E. Coyote.</p> </li> </ol> <p>I’m tired of the avoidant non-responses I’ve gotten from SAS in the past year, and I’m tired of wasting my breath in closed-door meetings. I hope we can make some broader changes to allow me to succeed here, as well as my disabled peers and future students, many of whom are not as eloquent or as aware of their poor treatment as I am.</p> <p>The last time I met with SAS, I was told that “there’s not a brick that doesn’t move” when it comes to accessibility. Here are some bricks. Let’s get moving.</p> Introduction to functional programming in Java Fri, 10 Jan 2020 00:00:00 +0000 Rebecca Turner https://becca.ooo/blog/functional-programming-in-java/ https://becca.ooo/blog/functional-programming-in-java/ <p><strong>Note:</strong> This article was originally written for an undergraduate course on the Scheme textbook <a href="https://en.wikipedia.org/wiki/Structure_and_Interpretation_of_Computer_Programs"><em>Structure and Interpretation of Computer Programs</em></a> (SICP). Students entering this course typically had 1–2 semesters of experience programming in Java and we found that many students could solve the course excercises in Java, but failed to translate the code to Scheme. This article helped bridge the gap between the courses to show students how to write programs that accomplish the same tasks using different techniques and languages, as well as introducing them to some basic functional programming concepts.</p> <p>This is a new piece of documentation, written just for this course, so let me know if parts of it are unclear to you — or if you like it! Expect to read it fairly slowly (like any technical writing), and consult the <a href="https://docs.oracle.com/en/java/javase/12/docs/api/">Java documentation</a> if there’s methods you don’t recognize — though the most important ones should be linked.</p> <p>A significant portion of this course is about learning functional programming techniques; functional programming solves problems very differently from object-oriented programming (which is what we use when we’re writing Java), and the difference in tactics that functional programming requires combined with the difference in syntax that Scheme in specific requires can be challenging to Java programmers.</p> <p>This reading is optional but recommended — if you find yourself thinking that you know how to solve a problem in Java but unsure how to solve it in Scheme, make sure to give it a read!</p> <h2 id="object-oriented-programming"><a class=anchor href="#object-oriented-programming" aria-label="Link to section object-oriented-programming">#</a>Object-oriented programming</h2> <blockquote> <p>Quick: off the top of your head, what is object-oriented programming about?</p> <p>Got an idea yet?</p> <p>If you thought any of the words “<a href="https://en.wikipedia.org/wiki/Encapsulation_(computer_programming)">encapsulation</a>”, “<a href="https://en.wikipedia.org/wiki/Inheritance_(object-oriented_programming)">inheritance</a>”, “<a href="https://en.wikipedia.org/wiki/Polymorphism_(computer_science)">polymorphism</a>”, “<a href="https://en.wikipedia.org/wiki/Information_hiding">information hiding</a>”, “<a href="https://en.wikipedia.org/wiki/Abstraction_(computer_science)">abstraction</a>”, or “<a href="https://en.wikipedia.org/wiki/Virtual_method_table">vtables</a>”, you are <em>wrong.</em></p> <p>If you thought any of the words “<a href="https://en.wikipedia.org/wiki/Class_(computer_programming)">class</a>”, “<a href="https://en.wikipedia.org/wiki/Prototype-based_programming">prototype</a>”, or “<a href="https://en.wikipedia.org/wiki/Data_type">type</a>”, you are <em>still wrong.</em></p> <p><strong>Object-oriented programming is about <em>objects</em>: bundles of state and behavior.</strong> The rest is optional fluff. And object-oriented <em>languages</em> are defined only by having built-in support for bundling state and behavior, <em>not</em> by having built-in support for classes. You may notice we don’t call it “class-oriented programming”.</p> <p>— <a href="https://eev.ee/blog/2013/03/03/the-controller-pattern-is-awful-and-other-oo-heresy/">“The controller pattern is awful (and other OO heresy)” by Eevee</a></p> </blockquote> <p>Let’s make this clear with some Java code. Here’s a simple <code>Point</code> class, which represents a point in 2-dimensional space with <code>double</code> coordinates.</p> <pre data-lang="java" class="language-java "><code class="language-java" data-lang="java">public class Point { public double x; public double y; } </code></pre> <p>This <a href="https://en.wikipedia.org/wiki/Class_(computer_programming)">class</a> is a bundle of <em>state</em>; the state of an object is what the <em>is</em>; here, the state is its coordinates. We can make <code>Point</code>s, modify them, and pass them around, but they don’t <em>do</em> anything.</p> <pre data-lang="java" class="language-java "><code class="language-java" data-lang="java">Point p = new Point(); p.x = 1; p.y = 3; System.out.println(&quot;p.x = &quot; + p.x); System.out.println(&quot;p.y = &quot; + p.y); </code></pre> <p>Let’s give our <code>Point</code> some behavior, with a <code>rotate</code> method, which will calculate the new coordinates using the 2-dimensional <a href="https://en.wikipedia.org/wiki/Rotation_matrix">rotation matrix</a>:</p> <pre data-lang="java" class="language-java "><code class="language-java" data-lang="java">import java.lang.Math; public class Point { public double x; public double y; Point(double x, double y) { this.x = x; this.y = y; } void rotate(double radians) { double newX = x * Math.cos(radians) - y * Math.sin(radians); double newY = x * Math.sin(radians) + y * Math.cos(radians); this.x = newX; this.y = newY; } @Override public String toString() { return &quot;Point(&quot; + x + &quot;, &quot; + y +&quot;)&quot;; } } </code></pre> <p>Now we can make and modify <code>Point</code>s (just like before), but <code>Point</code>s also inherently <em>know how to do something</em>: rotate!</p> <pre data-lang="java" class="language-java "><code class="language-java" data-lang="java">Point p = new Point(1, 3); System.out.println(&quot;p = &quot; + p); p.rotate(Math.PI &#x2F; 2); &#x2F;&#x2F; 90° CCW System.out.println(&quot;p = &quot; + p); </code></pre> <p>Prints <code>p = Point(1.0, 3.0)</code>, and then <code>p = Point(-3.0, 1.0000000000000002)</code> (because <a href="https://0.30000000000000004.com/"><code>double</code>s round in strange ways</a>). Now we have an <em>object</em>, with <em>state</em> (what it is; the <code>x</code> and <code>y</code> coordinates) and <em>behavior</em> (what it can <em>do</em>; rotate).</p> <h2 id="mutation"><a class=anchor href="#mutation" aria-label="Link to section mutation">#</a>Mutation</h2> <p>The <code>rotate</code> method we made is particularly interesting, because it returns <code>void</code>, storing its result in its own data. We call it <strong>mutation</strong> when the state of an object is changed.</p> <p>This can either be problematic (now, if we pass a <code>Point</code> to some method, we don’t have any guarantee that the <code>Point</code> won’t be changed by the method, which bites each new programmer <em>at least</em> once) or advantageous (we can easily give other methods the power to change the objects we care about by mutating them).</p> <p>One thing, however, is certain: when we add mutability to our objects, our program becomes more complex. What if methods never mutated objects? Functional programming advocates say that objects which <em>can’t</em> be mutated (<em>immutable</em> objects) are simpler to reason about and program with, because we never have to account for data changing without us knowing about it.</p> <p>Functions which don’t mutate data or cause other side-effects (like modifying files, printing data, or making network requests) are called <em>pure</em>. Let’s rewrite our <code>rotate</code> method to be pure:</p> <pre data-lang="java" class="language-java "><code class="language-java" data-lang="java">Point rotate(double radians) { return new Point( x * Math.cos(radians) - y * Math.sin(radians), x * Math.sin(radians) + y * Math.cos(radians) ); } </code></pre> <p>The change is pretty minimal. Now, we can use it like this:</p> <pre data-lang="java" class="language-java "><code class="language-java" data-lang="java">Point p = new Point(1, 3); System.out.println(&quot;p = &quot; + p); System.out.println(&quot;rotated = &quot; + p.rotate(Math.PI &#x2F; 2)); &#x2F;&#x2F; 90° CCW System.out.println(&quot;p = &quot; + p); </code></pre> <p>which prints</p> <pre data-lang="java" class="language-java "><code class="language-java" data-lang="java">p = Point(1.0, 3.0) rotated = Point(-3.0, 1.0000000000000002) p = Point(1.0, 3.0) </code></pre> <h2 id="composing-functions-together"><a class=anchor href="#composing-functions-together" aria-label="Link to section composing-functions-together">#</a>Composing functions together</h2> <p>When a program is made up entirely of pure functions, it can be easier to read, write, and understand the code.</p> <p>Suppose we have a list of points represented as a string, like this:</p> <!-- Generated with this Mathematica code: 4 {N[Sin[#]], N[Cos[#]]} & /@ Range[1, 8, Pi/8] --> <pre><code>3.36588,2.16121 3.93673,0.708629 3.90825,-0.851834 3.28476,-2.28261 2.16121,-3.36588 0.708629,-3.93673 -0.851834,-3.90825 -2.28261,-3.28476 -3.36588,-2.16121 -3.93673,-0.708629 -3.90825,0.851834 -3.28476,2.28261 -2.16121,3.36588 -0.708629,3.93673 0.851834,3.90825 2.28261,3.28476 3.36588,2.16121 3.93673,0.708629 </code></pre> <p>That is, points are separated by space characters, and coordinates within a point are separated by a comma.</p> <p>Here’s how an introductory Java programmer might try to parse the string into a list of <code>Point</code> objects:</p> <pre data-lang="java" class="language-java "><code class="language-java" data-lang="java">public static List&lt;Point&gt; parsePoints(String str) { try (Scanner scanner = new Scanner(str)) { ArrayList&lt;String&gt; pointStrings = new ArrayList&lt;&gt;(); while (scanner.hasNext()) { pointStrings.add(scanner.next()); } ArrayList&lt;Point&gt; points = new ArrayList&lt;&gt;(); for (int i = 0; i &lt; pointStrings.size(); i++) { String[] numbers = pointStrings.get(i).split(&quot;,&quot;); double x = Double.parseDouble(numbers[0]); double y = Double.parseDouble(numbers[1]); points.add(new Point(x, y)); } return points; } } </code></pre> <p>This implementation of <code>parsePoints</code> <em>works</em> fine, but it’s got some problems.</p> <ol> <li>We use too much data; first, we split <code>str</code> by whitespace with the <code>Scanner</code> and store it in <code>pointStrings</code>, which uses almost as much data as storing <code>str</code> originally did. Then, the very next thing we do is throw it all out after we make <code>points</code>.</li> <li>The code isn’t very clear; we build up <code>pointStrings</code> from the <code>Scanner</code>, we go through <code>pointStrings</code> again, then there’s… indexes into the <code>numbers</code> array? It’s weird, and if someone passed bad data to this method, they’d probably be confused to get an <code>ArrayIndexOutOfBoundsException</code>.</li> </ol> <p>But, under the hood, we’re only really <em>doing</em> a few things:</p> <ol> <li>Splitting <code>str</code> by whitespace.</li> <li>Splitting the resulting strings by commas.</li> <li>Parsing all the elements as <code>double</code>s.</li> <li>Turning the parsed <code>double</code>s into <code>Point</code>s.</li> </ol> <p>Note that each operation here goes directly to the next operation. With that in mind, let’s split up our <code>parsePoints</code> method a bit:</p> <pre data-lang="java" class="language-java "><code class="language-java" data-lang="java">static String[] splitWhitespace(String str) { return str.split(&quot;\\s+&quot;); } static String[] splitCommas(String str) { return str.split(&quot;,&quot;); } static List&lt;Double&gt; parseDoubles(String[] strs) { ArrayList&lt;Double&gt; doubles = new ArrayList&lt;&gt;(); for (String str : strs) { doubles.add(Double.parseDouble(str)); } return doubles; } static Point toPoint(List&lt;Double&gt; doubles) { return new Point(doubles.get(0), doubles.get(1)); } public static List&lt;Point&gt; parsePoints(String str) { String[] coords = splitWhitespace(str); ArrayList&lt;String[]&gt; splitByCommas = new ArrayList&lt;&gt;(); for (String coord : coords) { splitByCommas.add(splitCommas(coord)); } ArrayList&lt;List&lt;Double&gt;&gt; doubles = new ArrayList&lt;&gt;(); for (String[] doubleStrings : splitByCommas) { doubles.add(parseDoubles(doubleStrings)); } ArrayList&lt;Point&gt; points = new ArrayList&lt;&gt;(); for (List&lt;Double&gt; rawPoint : doubles) { points.add(toPoint(rawPoint)); } return points; } </code></pre> <p>This code is a bit clearer, but it’s much longer. Further, astute readers might have noticed that this pattern is repeated several times:</p> <pre data-lang="java" class="language-java "><code class="language-java" data-lang="java">ArrayList&lt;NewType&gt; newList = new ArrayList&lt;&gt;(); for (OldType element : oldList) { newList.add(oldTypeToNewType(element)); } </code></pre> <p>That is, building up a list by calling a method on every element of another list. Functional programmers have a name for this operation, and it’s called <em>mapping</em> a list, because it shows you a <em>map</em> from a list of one type to a list of another type.</p> <p>What do we know about the method <code>oldTypeToNewType</code>? It must look something like this, because it has one parameter of type <code>OldType</code> and returns a value of type <code>NewType</code>:</p> <pre data-lang="java" class="language-java "><code class="language-java" data-lang="java">NewType oldTypeToNewType(OldType o) { &#x2F;&#x2F; ... } </code></pre> <p>Java actually already includes this as an interface called <a href="https://docs.oracle.com/en/java/javase/12/docs/api/java.base/java/util/function/Function.html"><code>Funtion&lt;T, R&gt;</code></a>, where <code>T</code> is <code>OldType</code> (the function’s input) and <code>R</code> is <code>NewType</code> (the function’s return-type). With that in mind, let’s write a <code>mapList</code> function that turns a list of <code>T</code> into a list of <code>R</code>:</p> <pre data-lang="java" class="language-java "><code class="language-java" data-lang="java">static &lt;T, R&gt; List&lt;R&gt; mapList(List&lt;T&gt; input, Function&lt;T, R&gt; mappingFunction) { ArrayList&lt;R&gt; ret = new ArrayList&lt;&gt;(); for (T t : input) { ret.add(mappingFunction.apply(t)); } return ret; } </code></pre> <p>Then, we can create classes to call <code>mapList</code> with. For example, here’s <code>parseDoubles</code> using <code>mapList</code>:</p> <pre data-lang="java" class="language-java "><code class="language-java" data-lang="java">static List&lt;Double&gt; parseDoubles(String[] strs) { return mapList(Arrays.asList(strs), new Function&lt;String, Double&gt;() { @Override public Double apply(String s) { return Double.parseDouble(s); } }); } </code></pre> <p>The syntax <a href="https://docs.oracle.com/javase/tutorial/java/javaOO/anonymousclasses.html"><code>new Function&lt;...&gt;() { ... }</code></a> is just like declaring <code>class MyClass&lt;...&gt; implements Function&lt;...&gt; { ... }</code> and then constructing it exactly once. Fortunately, Java gives us shorter syntax for any <a href="https://docs.oracle.com/en/java/javase/12/docs/api/java.base/java/lang/FunctionalInterface.html">functional interface</a>, which represents the exact same thing:</p> <pre data-lang="java" class="language-java "><code class="language-java" data-lang="java">static List&lt;Double&gt; parseDoubles(String[] strs) { return mapList(Arrays.asList(strs), s -&gt; Double.parseDouble(s)); } </code></pre> <p>This shortcut is called a <em><a href="https://docs.oracle.com/javase/tutorial/java/javaOO/lambdaexpressions.html">lambda expression</a></em>. Further, it turns out that the pattern of writing lambdas of the form <code>var -&gt; MyClass.method(var)</code> is so common there’s another special syntax for it, called <em><a href="https://docs.oracle.com/javase/tutorial/java/javaOO/methodreferences.html">method references</a></em>, which look like <code>MyClass::method</code> (in this case, <code>Double::parseDouble</code>):</p> <pre data-lang="java" class="language-java "><code class="language-java" data-lang="java">static List&lt;Double&gt; parseDoubles(String[] strs) { return mapList(Arrays.asList(strs), Double::parseDouble); } </code></pre> <p>Now, we can rewrite our <code>Point</code> parser using <code>mapList</code>:</p> <pre data-lang="java" class="language-java "><code class="language-java" data-lang="java">public static List&lt;Point&gt; parsePoints(String str) { String[] coords = splitWhitespace(str); List&lt;String[]&gt; splitByCommas = mapList(Arrays.asList(coords), PointParser::splitCommas); List&lt;List&lt;Double&gt;&gt; doubles = mapList(splitByCommas, PointParser::parseDoubles); return mapList(doubles, PointParser::toPoint); } </code></pre> <p>Much better! This code is very declarative: we write what we mean. First, we split by whitespace. Then, we split by commas. Then, we parse as doubles. Then, we map to points.</p> <p>This is still fairly wasteful in terms of memory, though — we create not just one but <em>three</em> intermediate lists, each of which gets discarded after being used only once.</p> <p>We can solve this problem with a <a href="https://docs.oracle.com/en/java/javase/12/docs/api/java.base/java/util/stream/Stream.html"><strong>stream</strong></a> (sometimes called a <em>lazy iterator</em>). Streams are specifically designed to be composable with functions, so that we can generate a stream from some source (like reading from a file, finding matches in a string, or more traditionally from some data structure like an array or tree) and then specify that it be mapped over in various ways.</p> <p>Unlike our <code>mapList</code> function, mapping over a stream doesn’t read any data from the underlying source; it just writes down a note saying the stream is also being mapped over by a particular value. Therefore, we don’t need to generate any intermediate lists — we can just add our mapping functions one by one, and the stream remembers how to transform its data before using it when we start reading values from the stream.</p> <p>And how do we read values from a stream? Unlike arrays or other simple data structures, streams are designed to let us ask more questions than just “what element is at index <em>i</em> in this structure?” Streams have methods that let us ask:</p> <ul> <li>Does this function return <code>true</code> when applied to every element in the stream?</li> <li>Does the stream have at least <em>n</em> items?</li> <li>Find the first element in the stream that this function returns <code>true</code> for.</li> <li>What’s the maximum value in the stream?</li> <li>Collect the values in this stream (to an <code>ArrayList</code>, a <code>HashMap</code>, a <code>String</code>, etc.).</li> </ul> <p>We can generate streams a few different ways in Java, but for the most part the <code>Arrays.stream</code> method, the <code>Collection.stream</code> method, and the <code>Collectors</code> methods are suitable for converting most data to and from streams.</p> <p>Rewriting the point parser with <code>Stream</code>s, we get:</p> <pre data-lang="java" class="language-java "><code class="language-java" data-lang="java">public class PointParser { static final Pattern COMMA = Pattern.compile(&quot;,&quot;); static final Pattern WHITESPACE = Pattern.compile(&quot;\\s+&quot;); static Point toPoint(Iterator&lt;Double&gt; doubles) { return new Point(doubles.next(), doubles.next()); } public static List&lt;Point&gt; parsePoints(String str) { return WHITESPACE.splitAsStream(str) &#x2F;&#x2F; (1) .map(s -&gt; COMMA.splitAsStream(s) &#x2F;&#x2F; (2) .map(Double::parseDouble) .iterator()) &#x2F;&#x2F; (3) .map(PointParser::toPoint) .collect(Collectors.toList()); &#x2F;&#x2F; (4) } } </code></pre> <p>Notes:</p> <ol> <li> <p>The <a href="https://docs.oracle.com/en/java/javase/12/docs/api/java.base/java/util/regex/Pattern.html#splitAsStream(java.lang.CharSequence)"><code>Pattern.splitAsStream</code></a> method is exactly like <a href="https://docs.oracle.com/en/java/javase/12/docs/api/java.base/java/lang/String.html#split(java.lang.String)"><code>String.split</code></a> except it returns a <code>Stream</code> instead of an array.</p> </li> <li> <p>Inside one of our calls to <code>map</code>, we create <em>another</em> <code>Stream</code>; streams work best together!</p> <p>These calls to <code>map</code> don’t actually get items out of the stream; they’re <em>lazy</em>, and all they do is make the stream remember that it’s mapping by that function.</p> </li> <li> <p>The call to <code>iterator</code> turns a <code>Stream</code> into an <code>Iterator</code>; it’s a <em>finalizing</em> operation, which means that it consumes the elements of the stream and returns a result. Once we have an <code>Iterator</code>, we can’t <code>map</code> on it any more.</p> </li> <li> <p>Finally, we <em>collect</em> the elements of the stream into a <code>List</code> with the <code>collect</code> method and the <code>Collectors.toList()</code> utility method.</p> </li> </ol> <p>This code is (I think) much clearer than our first attempt, and a bit shorter as an extra bonus. I’ll admit that code writtten with streams can look a bit alien at first, and it takes some practice to get used to!</p> <p>In this code, we don’t use a single index or <code>for</code> loop — and why should we? After all, none of the operations need to know anything about the elements before or after them; the indexes were only to overcome limitations in the ways we can refer to data structures.</p> <p>Additionally, no intermediate lists are created; when the stream is being collected, each element from the pattern-splitter is applied to all our mapping functions in a row before being returned. The <em>processing</em> steps aren’t separated from each other, which saves memory, even though we were able to <em>declare</em> the processing steps separately. In APIs focused on streams, it’s often easier to return (and take as parameters) streams themselves, rather than frequently collecting data to structures like lists at method boundaries.</p> <p>There’s another advantage of this style of programming, in addition to saving memory; because we never reassign values, the functions are immutable and pure. In Java, we’re not inherently motivated to write pure functions. But in other languages (notably pure functional programming languages like Haskell, or other functional programming languages like Scheme), we’re <em>forced</em> to write pure functions; learning to construct pure functions and immutable data structures will help you reason about and solve problems in a functional programming style.</p> who i am not Mon, 22 Apr 2019 00:00:00 +0000 Rebecca Turner https://becca.ooo/not/ https://becca.ooo/not/ <p>there are many rebecca turners. i am not these ones</p> <ul> <li><a href="https://kolektiva.social/@iarna">software engineer rebecca turner, aka iarna</a>, currently employed by microsoft but perhaps best-known for her work at npm, where she was responsible for <a href="https://github.com/npm/cli">the npm cli</a>, among other things</li> <li><a href="https://en.m.wikipedia.org/wiki/Rebecca_Turner">british olympian rebecca turner</a></li> <li><a href="https://www.supertalk.fm/shows/goodthings/">host of mississippi talk show “good things with rebecca turner”</a></li> <li><a href="http://www.world-of-lucid-dreaming.com/lucid-dreamer.html">founder of “world of lucid dreaming” rebecca turner</a></li> <li><a href="http://rebeccaturner.net/">folk singer/songwriter rebecca turner</a></li> <li><a href="https://www.imdb.com/name/nm4576641/">actress rebecca turner</a></li> <li><a href="http://www.wealthstrategyadvisors.net/">“wealth strategy advisor” rebecca turner</a></li> <li><a href="https://twitter.com/rebeccasturner">abc news journalist rebecca turner</a></li> <li><a href="https://iris.ucl.ac.uk/iris/browse/profile?upi=RTURN24">senior research associate at the mrc clinical trials unit at ucl rebecca turner, phd</a></li> <li><a href="http://rebeccaturnerconsulting.com/about.html">executive coach / consultant dr. rebecca turner</a></li> <li><a href="http://www.iaapa.org/news/iaapa-press-office/rebecca-turner-joins-iaapa-as-vice-president-of-education-professional-development-and-training-services">vice president of education and professional development at the international association of amusement parks and attractions rebecca turner</a></li> <li><a href="http://sjeds.org/private-christian-elementary-school-jacksonville-fl/faculty-staff/">kindergarten teacher at the san jose episcopal day school rebecca turner</a></li> <li><a href="https://www.fairsentencingofyouth.org/team/turner/">lawyer and montgomery implementation coordinator at the campaign for the fair sentencing of youth rebecca turner</a></li> <li><a href="https://health.usnews.com/doctors/rebecca-turner-568245">“family medicine doctor in missouri” rebecca turner</a></li> <li><a href="https://www.littlerealestate.com.au/our-team/rebecca-turner">australian property manager rebecca turner</a></li> <li><a href="https://rebeccaturnernutrition.com/">“MS, RD, LD, author, registered dietitian, radio host, television presenter, and girl mom” rebecca turner</a></li> </ul> Unicode Resources Fri, 06 Apr 2018 00:00:00 +0000 Rebecca Turner https://becca.ooo/unicode/ https://becca.ooo/unicode/ <p>Created by <a href="/">rebecca</a> with major help from <a href="https://twitter.com/FakeUnicode">@FakeUnicode</a></p> <hr /> <p><a href="https://en.wikipedia.org/wiki/Unicode">Unicode</a> is a standard “registry” of characters designed to catalogue all the world’s writing systems and serve as a superset of all previously-existing character sets used and supported by <a href="https://unicode.org/consortium/members.html">just about everyone</a> and <a href="https://unicode.org/resources/">all their software</a>. <a href="https://stackoverflow.com/questions/2241348/what-is-unicode-utf-8-utf-16">Get into the details on Stack Overflow</a> or see some of the <a href="https://en.wikipedia.org/wiki/Unicode#Mapping_and_encodings">Unicode encodings</a> on Wikipedia.</p> <h1 id="charts-and-character-data"><a class=anchor href="#charts-and-character-data" aria-label="Link to section charts-and-character-data">#</a>Charts and character data</h1> <p><span class=bullet></span><a href="https://unicode.org/charts/">PDF charts on unicode.org</a> — The only complete and perpetually up-to-date reference.</p> <p><span class=bullet></span><a href="https://en.wikipedia.org/wiki/Unicode_block">List of Unicode blocks on Wikipedia</a> — Also available as a <a href="https://unicode.org/Public/UNIDATA/Blocks.txt">TXT file on unicode.org</a></p> <p><a href="https://unicode.org/Public/UNIDATA/">UNIDATA on unicode.org</a> — “This directory contains the final data files for the Unicode Character Database, for Version 8.0.0 of the Unicode Standard.”</p> <p><a href="https://www.copypastecharacter.com/all-characters">CopyPasteCharacter</a> — Contains a bunch of common characters for easy copying.</p> <p><a href="https://unicode.org/alloc/Pipeline.html">Proposed new characters on unicode.org</a></p> <p><a href="https://en.wikipedia.org/wiki/Plane_(Unicode)">Unicode planes reference</a> — Confused about what “BMP” means? Check here.</p> <h1 id="string-analysis"><a class=anchor href="#string-analysis" aria-label="Link to section string-analysis">#</a>String analysis</h1> <p><span class=bullet></span><a href="https://apps.timwhitlock.info/unicode/inspect?s=%F0%9F%92%81u">Unicode Inspector</a> — By <a href="https://twitter.com/timwhitlock">@timwhitlock</a>, displays the codepoint, byte breakdown, block, symbol, name, and surrogates for each character in a string.</p> <p><a href="http://unicode.scarfboy.com/?s=4a1e">Scarfboy search and string analysis</a></p> <p><a href="https://jsfiddle.net/vrog8Lkf/">Understanding UTF-8 on jsfiddle</a> — By <a href="https://twitter.com/FakeUnicode">@FakeUnicode</a> on Twitter. Displays binary/hex breakdown of strings of text.</p> <p><a href="https://babelstone.co.uk/Unicode/whatisit.html">What Unicode character is this?</a> — By <a href="https://babelstone.co.uk/">BabelStone</a>.</p> <h1 id="search"><a class=anchor href="#search" aria-label="Link to section search">#</a>Search</h1> <p><a href="http://unicode.scarfboy.com/?s=4a1e">Scarfboy search and string analysis</a></p> <p><a href="https://codepoints.net/search?q=punctuation">codepoints.net</a> — Powerful search engine with previews in Unifont</p> <p><a href="https://amp-what.com/unicode/search/">&amp;what;</a></p> <p><a href="https://fileformat.info/info/unicode/char/search.htm">FileFormat.info</a></p> <p><a href="https://charcod.es/">charcod.es</a></p> <h1 id="shape-recognition"><a class=anchor href="#shape-recognition" aria-label="Link to section shape-recognition">#</a>Shape recognition</h1> <p><span class=bullet></span><a href="https://shapecatcher.com/">shapecatcher</a> — Recognizes Unicode characters through a drawing field.</p> <p><a href="https://jisho.org/#radical">Kanji search on jisho.org</a> — Find kanji characters by their parts.</p> <p><a href="https://kanji.sljfaq.org/draw-canvas.html">Handwritten kanji recognition</a> — Like <a href="https://shapecatcher.com/">shapecatcher</a> for kanji.</p> <p><a href="https://translate.google.com/">Google Translate</a> — Click the pencil icon in the input area to enable shape recognition. Great for writing short bits of text in a language you don’t know.</p> <h1 id="conversion-programming"><a class=anchor href="#conversion-programming" aria-label="Link to section conversion-programming">#</a>Conversion (programming)</h1> <p><span class=bullet></span><a href="https://xem.github.io/escape/">Xem’s EscApe utility on Github</a> — By <a href="https://xem.github.io/">Maxime Euzière</a>. Converts any Unicode string to a bunch of different escape sequence syntaxes.</p> <p><a href="https://paulschou.com/tools/xlate/">ASCII Xlate</a> — Converts plain ASCII, binary, octal, hex, base32, base64, ASCII85, and decimal ASCII. Also calculates various hashes!</p> <p><a href="https://unicodetools.com/">Guide to converting to UTF-8 in various programming languages</a></p> <h1 id="conversion-decorative-or-linguistic"><a class=anchor href="#conversion-decorative-or-linguistic" aria-label="Link to section conversion-decorative-or-linguistic">#</a>Conversion (decorative or linguistic)</h1> <p><span class=bullet></span><a href="http://mar.cx/unicate/">Unicate</a> — Converts to various Latin Unicode “alphabets” (e.g. full-width, math scripts, etc.).</p> <p><a href="https://qaz.wtf/u/convert.cgi?text=unicode.9999yea.rs">Text converter on qaz.wtf</a> — Similar to the above.</p> <p><span class=bullet></span><a href="https://eeemo.net/">Zalgo generator on eeemo.net</a> — generates Zalgo text.</p> <p><a href="https://chrome.google.com/webstore/detail/convert-text/mcpglhjaahelnpjalcaeecgkjhkpokdn">Convert Text on the Chrome web store</a> — Converts the case of text as a Chrome extension. Includes Zalgo generator and fullwidth transform.</p> <p><a href="https://adamvarga.com/strike/">Strikethrough converter</a> — By <a href="https://adamvarga.com/">Adam Varga</a>.</p> <p><a href="https://jsfiddle.net/xHrxM/13/">Emojify text on jsfiddle</a> — By <a href="https://twitter.com/FakeUnicode">@FakeUnicode</a> on Twitter. Transforms text to emoji.</p> <p><a href="/unicode/grid/">Acrostic generator</a> — Dubiously useful. Also converts to Unicode Math Monospace.</p> <p><a href="/unicode/abbreviate/">Abbreviator</a> — Saves characters in tweets by using precomposed characters.</p> <p><a href="https://www.unicod.es/">Unitools</a> — A good compilation of a bunch of other tools. Honorary mention.</p> <p><a href="https://brailletranslator.org/">Braille converter</a></p> <h1 id="coverage-fonts-support"><a class=anchor href="#coverage-fonts-support" aria-label="Link to section coverage-fonts-support">#</a>Coverage: fonts &amp; support</h1> <p><span class=bullet></span><a href="http://alanwood.net/unicode/fonts.html">Alan Wood’s Unicode font list</a> — Probably the most complete list of high-coverage fonts.</p> <p><span class=bullet></span><a href="https://fsd.it/shop/fonts/pragmatapro/">PragmataPro</a> — A monospaced programming font with 6,000 glyphs (and rising).</p> <p><span class=bullet></span><a href="https://mathew-kurian.github.io/CharacterMap/">CharacterMap</a> — Analyzes glyphs from font files.</p> <p><a href="https://google.com/get/noto/">Google Noto Fonts</a> — A set of sans and serif fonts supporting 581 languages (as of April 2016), with about 50% glyph coverage.</p> <p><a href="http://cheat-sheets.org/sites/font.su/">Unicode fonts by writing system</a></p> <p><a href="http://visibone.com/htmlref/char/cer.htm">Preview of all codepoints in the BMP</a> — Useful for testing coverage.</p> <p><a href="https://unifoundry.com/unifont/">GNU Unifont</a></p> <p><a href="https://babelstone.co.uk/Fonts/Han.html">BabelStone’s Han font</a></p> <p><a href="https://evertype.com/emono/">Everson Mono font</a></p> <p><a href="https://fonts.jp/hanazono/">Hanazono font</a></p> <p><a href="https://en.wikipedia.org/wiki/Code2000">Code2000 font</a></p> <h1 id="emoji"><a class=anchor href="#emoji" aria-label="Link to section emoji">#</a>Emoji</h1> <p><span class=bullet></span><a href="https://unicode.org/emoji/charts/full-emoji-list.html">Full Emoji Charts</a> — all emoji with comparison pictures of implementations on various platforms.</p> <p><a href="https://unicode.org/L2/L2009/09027-emoji-backgrnd.pdf">Emoji Symbols: Background Data on unicode.org</a>, a.k.a. (L2/09-027) — Japanese carrier background data for the original emoji import (mostly historic value).</p> <p><a href="https://en.wikipedia.org/wiki/Miscellaneous_Symbols_and_Pictographs#History">List of proposals and their associated codepoints on Wikipedia</a></p> <p><a href="https://unicode.org/emoji/future/emoji-candidates.html">Possible upcoming emoji on unicode.org</a></p> <p><a href="https://unicode.org/emoji/charts/emoji-zwj-sequences.html">List of emoji ZWJ ligatures on unicode.org</a> — Note: These are <em>not</em> actually emoji or part of the Unicode standard. Implementation, support, and blame lies entirely with the third parties involved.</p> <p><a href="https://unicode.org/emoji/charts/text-style.html">Text vs Emoji reference on unicode.org</a> — Shows which characters (should) render as emoji or text.</p> <h1 id="ascii-unicode-art-kaomoji"><a class=anchor href="#ascii-unicode-art-kaomoji" aria-label="Link to section ascii-unicode-art-kaomoji">#</a>ASCII/Unicode art &amp; Kaomoji</h1> <p><span class=bullet></span><a href="https://cutekaomoji.com/">cutekaomoji.com</a></p> <p><a href="https://japaneseemoticons.me/all-japanese-emoticons/">List of 10,000+ kaomoji</a></p> <p><a href="https://textfac.es/">textfac.es</a></p> <p><a href="https://asciiart.website/">ASCII art collection on asciiart.website</a>, formerly chris.com</p> <p><a href="https://asciiworld.com/">ASCII art on asciiworld.com</a></p> <p><a href="https://patorjk.com/software/taag/#p=display&amp;f=Graffiti&amp;t=unicode.9999yea.rs">ASCII text art generator</a></p> <h1 id="inserting-characters"><a class=anchor href="#inserting-characters" aria-label="Link to section inserting-characters">#</a>Inserting characters</h1> <p><a href="https://vim.wikia.com/wiki/Entering_special_characters">Vim</a> — Insert mode: <code>&lt;C-v&gt;uxxxx</code></p> <p><a href="https://superuser.com/questions/394405/how-to-type-a-unicode-character-by-its-number-in-emacs">Emacs</a> — <code>&lt;C-X&gt; 8 &lt;CR&gt; xxxx &lt;CR&gt;</code></p> <p><a href="https://poynton.com/notes/misc/mac-unicode-hex-input.html">Mac OS X</a> — (☑︎ Unicode Hex Input) <code>&lt;⌥-xxxx&gt;</code></p> <p><a href="https://en.wikipedia.org/wiki/Unicode_input#In_Microsoft_Windows">Windows</a> — (☑︎ Registry Key) <code>&lt;A-xxxx&gt;</code></p> <p><a href="https://en.wikipedia.org/wiki/Unicode_input#In_X11_.28Linux_and_other_Unix_variants.29">Unix in GTK applications</a> — <code>&lt;C-S-uxxxx&gt;</code></p> <h1 id="misc"><a class=anchor href="#misc" aria-label="Link to section misc">#</a>Misc.</h1> <p><span class=bullet></span><a href="https://jsfiddle.net/SaqVU/4/">Random Unicode character generator on jsfiddle</a> — By <a href="https://twitter.com/FakeUnicode">@FakeUnicode</a> on Twitter.</p> <p><a href="/arrows">List of Unicode arrows</a> — Courtesy of <a href="https://twitter.com/fabrizioschiavi">@fabrizioschiavi</a> of <a href="https://fsd.it/shop/fonts/pragmatapro/">Pragmata Pro</a> fame.</p> <p><a href="https://getemoji.com/#twitter">Emoji allowed in Twitter usernames</a></p> <p><a href="https://mothereff.in/byte-counter">Character/byte counter</a></p> ASCII table Wed, 04 Apr 2018 00:00:00 +0000 Rebecca Turner https://becca.ooo/ascii/ https://becca.ooo/ascii/ <p><a href="/ascii.txt">Available as a terminal-friendly .txt!</a> (<a href="/ascii-long.txt">long version</a>), or with <a href="https://becca.ooo/ascii/simple">less details</a>.</p> <table><thead><tr><th><a href="https://en.wikipedia.org/wiki/ASCII#Character_groups">Char</a></th><th><a href="https://en.wikipedia.org/wiki/Decimal">Dec</a></th><th><a href="https://en.wikipedia.org/wiki/Octal">Oct</a></th><th><a href="https://en.wikipedia.org/wiki/Hexadecimal">Hex</a></th><th><a href="https://en.wikipedia.org/wiki/Binary_number#Representation">Bin</a></th><th><a href="https://en.wikipedia.org/wiki/C0_and_C1_control_codes#C0_.28ASCII_and_derivatives.29">C0</a></th><th><a href="https://en.wikipedia.org/wiki/Escape_sequences_in_C#Table_of_escape_sequences">C</a></th><th><a href="https://en.wikipedia.org/wiki/Control_Pictures">U</a></th><th><a href="https://en.wikipedia.org/wiki/Basic_Latin_(Unicode_block)">Name</a></th></tr></thead><tbody> <tr><td>(nul)</td><td>0</td><td>0000</td><td>0x00</td><td>00000000</td><td>^@</td><td>\0</td><td>␀</td><td>Null</td></tr> <tr><td>(soh)</td><td>1</td><td>0001</td><td>0x01</td><td>00000001</td><td>^A</td><td></td><td>␁</td><td>Start of Heading</td></tr> <tr><td>(stx)</td><td>2</td><td>0002</td><td>0x02</td><td>00000010</td><td>^B</td><td></td><td>␂</td><td>Start of Text</td></tr> <tr><td>(etx)</td><td>3</td><td>0003</td><td>0x03</td><td>00000011</td><td>^C</td><td></td><td>␃</td><td>End of Text</td></tr> <tr><td>(eot)</td><td>4</td><td>0004</td><td>0x04</td><td>00000100</td><td>^D</td><td></td><td>␄</td><td>End of Transmission</td></tr> <tr><td>(enq)</td><td>5</td><td>0005</td><td>0x05</td><td>00000101</td><td>^E</td><td></td><td>␅</td><td>Enquiry</td></tr> <tr><td>(ack)</td><td>6</td><td>0006</td><td>0x06</td><td>00000110</td><td>^F</td><td></td><td>␆</td><td>Acknowledgement</td></tr> <tr><td>(bel)</td><td>7</td><td>0007</td><td>0x07</td><td>00000111</td><td>^G</td><td>\a</td><td>␇</td><td>Bell</td></tr> <tr><td>(bs)</td><td>8</td><td>0010</td><td>0x08</td><td>00001000</td><td>^H</td><td>\b</td><td>␈</td><td>Backspace</td></tr> <tr><td>(ht)</td><td>9</td><td>0011</td><td>0x09</td><td>00001001</td><td>^I</td><td>\t</td><td>␉</td><td>Horizontal Tab</td></tr> <tr><td>(nl)</td><td>10</td><td>0012</td><td>0x0a</td><td>00001010</td><td>^J</td><td>\n</td><td>␊</td><td>Line Feed</td></tr> <tr><td>(vt)</td><td>11</td><td>0013</td><td>0x0b</td><td>00001011</td><td>^K</td><td>\v</td><td>␋</td><td>Vertical Tab</td></tr> <tr><td>(np)</td><td>12</td><td>0014</td><td>0x0c</td><td>00001100</td><td>^L</td><td>\f</td><td>␌</td><td>Form Feed</td></tr> <tr><td>(cr)</td><td>13</td><td>0015</td><td>0x0d</td><td>00001101</td><td>^M</td><td>\r</td><td>␍</td><td>Carriage Return</td></tr> <tr><td>(so)</td><td>14</td><td>0016</td><td>0x0e</td><td>00001110</td><td>^N</td><td></td><td>␎</td><td>Shift Out</td></tr> <tr><td>(si)</td><td>15</td><td>0017</td><td>0x0f</td><td>00001111</td><td>^O</td><td></td><td>␏</td><td>Shift In</td></tr> <tr><td>(dle)</td><td>16</td><td>0020</td><td>0x10</td><td>00010000</td><td>^P</td><td></td><td>␐</td><td>Data Link Escape</td></tr> <tr><td>(dc1)</td><td>17</td><td>0021</td><td>0x11</td><td>00010001</td><td>^Q</td><td></td><td>␑</td><td>Device Control 1 (often XON)</td></tr> <tr><td>(dc2)</td><td>18</td><td>0022</td><td>0x12</td><td>00010010</td><td>^R</td><td></td><td>␒</td><td>Device Control 2</td></tr> <tr><td>(dc3)</td><td>19</td><td>0023</td><td>0x13</td><td>00010011</td><td>^S</td><td></td><td>␓</td><td>Device Control 3 (often XOFF)</td></tr> <tr><td>(dc4)</td><td>20</td><td>0024</td><td>0x14</td><td>00010100</td><td>^T</td><td></td><td>␔</td><td>Device Control 4</td></tr> <tr><td>(nak)</td><td>21</td><td>0025</td><td>0x15</td><td>00010101</td><td>^U</td><td></td><td>␕</td><td>Negative Acknowledgement</td></tr> <tr><td>(syn)</td><td>22</td><td>0026</td><td>0x16</td><td>00010110</td><td>^V</td><td></td><td>␖</td><td>Synchronous Idle</td></tr> <tr><td>(etb)</td><td>23</td><td>0027</td><td>0x17</td><td>00010111</td><td>^W</td><td></td><td>␗</td><td>End of Transmission Block</td></tr> <tr><td>(can)</td><td>24</td><td>0030</td><td>0x18</td><td>00011000</td><td>^X</td><td></td><td>␘</td><td>Cancel</td></tr> <tr><td>(em)</td><td>25</td><td>0031</td><td>0x19</td><td>00011001</td><td>^Y</td><td></td><td>␙</td><td>End of Medium</td></tr> <tr><td>(sub)</td><td>26</td><td>0032</td><td>0x1a</td><td>00011010</td><td>^Z</td><td></td><td>␚</td><td>Substitute</td></tr> <tr><td>(esc)</td><td>27</td><td>0033</td><td>0x1b</td><td>00011011</td><td>^[</td><td>\e</td><td>␛</td><td>Escape</td></tr> <tr><td>(fs)</td><td>28</td><td>0034</td><td>0x1c</td><td>00011100</td><td>^\</td><td></td><td>␜</td><td>File Separator</td></tr> <tr><td>(gs)</td><td>29</td><td>0035</td><td>0x1d</td><td>00011101</td><td>^]</td><td></td><td>␝</td><td>Group Separator</td></tr> <tr><td>(rs)</td><td>30</td><td>0036</td><td>0x1e</td><td>00011110</td><td>^^</td><td></td><td>␞</td><td>Record Separator</td></tr> <tr><td>(us)</td><td>31</td><td>0037</td><td>0x1f</td><td>00011111</td><td>^\</td><td></td><td>␟</td><td>Unit Separator</td></tr> <tr><td>(sp)</td><td>32</td><td>0040</td><td>0x20</td><td>00100000</td><td></td><td></td><td>␠</td><td>Space</td></tr> <tr><td>!</td><td>33</td><td>0041</td><td>0x21</td><td>00100001</td><td></td><td></td><td></td><td>Exclamation mark</td></tr> <tr><td>"</td><td>34</td><td>0042</td><td>0x22</td><td>00100010</td><td></td><td>\"</td><td></td><td>Quotation mark</td></tr> <tr><td>#</td><td>35</td><td>0043</td><td>0x23</td><td>00100011</td><td></td><td></td><td></td><td>Number sign</td></tr> <tr><td>$</td><td>36</td><td>0044</td><td>0x24</td><td>00100100</td><td></td><td></td><td></td><td>Dollar sign</td></tr> <tr><td>%</td><td>37</td><td>0045</td><td>0x25</td><td>00100101</td><td></td><td></td><td></td><td>Percent sign</td></tr> <tr><td>&amp;</td><td>38</td><td>0046</td><td>0x26</td><td>00100110</td><td></td><td></td><td></td><td>Ampersand</td></tr> <tr><td>'</td><td>39</td><td>0047</td><td>0x27</td><td>00100111</td><td></td><td>\'</td><td></td><td>Apostrophe</td></tr> <tr><td>(</td><td>40</td><td>0050</td><td>0x28</td><td>00101000</td><td></td><td></td><td></td><td>Left parenthesis</td></tr> <tr><td>)</td><td>41</td><td>0051</td><td>0x29</td><td>00101001</td><td></td><td></td><td></td><td>Right parenthesis</td></tr> <tr><td>*</td><td>42</td><td>0052</td><td>0x2a</td><td>00101010</td><td></td><td></td><td></td><td>Asterisk</td></tr> <tr><td>+</td><td>43</td><td>0053</td><td>0x2b</td><td>00101011</td><td></td><td></td><td></td><td>Plus sign</td></tr> <tr><td>,</td><td>44</td><td>0054</td><td>0x2c</td><td>00101100</td><td></td><td></td><td></td><td>Comma</td></tr> <tr><td>-</td><td>45</td><td>0055</td><td>0x2d</td><td>00101101</td><td></td><td></td><td></td><td>Hyphen-minus</td></tr> <tr><td>.</td><td>46</td><td>0056</td><td>0x2e</td><td>00101110</td><td></td><td></td><td></td><td>Full stop</td></tr> <tr><td>/</td><td>47</td><td>0057</td><td>0x2f</td><td>00101111</td><td></td><td></td><td></td><td>Solidus</td></tr> <tr><td>0</td><td>48</td><td>0060</td><td>0x30</td><td>00110000</td><td></td><td></td><td></td><td>Digit zero</td></tr> <tr><td>1</td><td>49</td><td>0061</td><td>0x31</td><td>00110001</td><td></td><td></td><td></td><td>Digit one</td></tr> <tr><td>2</td><td>50</td><td>0062</td><td>0x32</td><td>00110010</td><td></td><td></td><td></td><td>Digit two</td></tr> <tr><td>3</td><td>51</td><td>0063</td><td>0x33</td><td>00110011</td><td></td><td></td><td></td><td>Digit three</td></tr> <tr><td>4</td><td>52</td><td>0064</td><td>0x34</td><td>00110100</td><td></td><td></td><td></td><td>Digit four</td></tr> <tr><td>5</td><td>53</td><td>0065</td><td>0x35</td><td>00110101</td><td></td><td></td><td></td><td>Digit five</td></tr> <tr><td>6</td><td>54</td><td>0066</td><td>0x36</td><td>00110110</td><td></td><td></td><td></td><td>Digit six</td></tr> <tr><td>7</td><td>55</td><td>0067</td><td>0x37</td><td>00110111</td><td></td><td></td><td></td><td>Digit seven</td></tr> <tr><td>8</td><td>56</td><td>0070</td><td>0x38</td><td>00111000</td><td></td><td></td><td></td><td>Digit eight</td></tr> <tr><td>9</td><td>57</td><td>0071</td><td>0x39</td><td>00111001</td><td></td><td></td><td></td><td>Digit nine</td></tr> <tr><td>:</td><td>58</td><td>0072</td><td>0x3a</td><td>00111010</td><td></td><td></td><td></td><td>Colon</td></tr> <tr><td>;</td><td>59</td><td>0073</td><td>0x3b</td><td>00111011</td><td></td><td></td><td></td><td>Semicolon</td></tr> <tr><td>&lt;</td><td>60</td><td>0074</td><td>0x3c</td><td>00111100</td><td></td><td></td><td></td><td>Less-than sign</td></tr> <tr><td>=</td><td>61</td><td>0075</td><td>0x3d</td><td>00111101</td><td></td><td></td><td></td><td>Equals sign</td></tr> <tr><td>&gt;</td><td>62</td><td>0076</td><td>0x3e</td><td>00111110</td><td></td><td></td><td></td><td>Greater-than sign</td></tr> <tr><td>?</td><td>63</td><td>0077</td><td>0x3f</td><td>00111111</td><td></td><td>\?</td><td></td><td>Question mark</td></tr> <tr><td>@</td><td>64</td><td>0100</td><td>0x40</td><td>01000000</td><td></td><td></td><td></td><td>Commercial at</td></tr> <tr><td>A</td><td>65</td><td>0101</td><td>0x41</td><td>01000001</td><td></td><td></td><td></td><td>Latin capital letter A</td></tr> <tr><td>B</td><td>66</td><td>0102</td><td>0x42</td><td>01000010</td><td></td><td></td><td></td><td>Latin capital letter B</td></tr> <tr><td>C</td><td>67</td><td>0103</td><td>0x43</td><td>01000011</td><td></td><td></td><td></td><td>Latin capital letter C</td></tr> <tr><td>D</td><td>68</td><td>0104</td><td>0x44</td><td>01000100</td><td></td><td></td><td></td><td>Latin capital letter D</td></tr> <tr><td>E</td><td>69</td><td>0105</td><td>0x45</td><td>01000101</td><td></td><td></td><td></td><td>Latin capital letter E</td></tr> <tr><td>F</td><td>70</td><td>0106</td><td>0x46</td><td>01000110</td><td></td><td></td><td></td><td>Latin capital letter F</td></tr> <tr><td>G</td><td>71</td><td>0107</td><td>0x47</td><td>01000111</td><td></td><td></td><td></td><td>Latin capital letter G</td></tr> <tr><td>H</td><td>72</td><td>0110</td><td>0x48</td><td>01001000</td><td></td><td></td><td></td><td>Latin capital letter H</td></tr> <tr><td>I</td><td>73</td><td>0111</td><td>0x49</td><td>01001001</td><td></td><td></td><td></td><td>Latin capital letter I</td></tr> <tr><td>J</td><td>74</td><td>0112</td><td>0x4a</td><td>01001010</td><td></td><td></td><td></td><td>Latin capital letter J</td></tr> <tr><td>K</td><td>75</td><td>0113</td><td>0x4b</td><td>01001011</td><td></td><td></td><td></td><td>Latin capital letter K</td></tr> <tr><td>L</td><td>76</td><td>0114</td><td>0x4c</td><td>01001100</td><td></td><td></td><td></td><td>Latin capital letter L</td></tr> <tr><td>M</td><td>77</td><td>0115</td><td>0x4d</td><td>01001101</td><td></td><td></td><td></td><td>Latin capital letter M</td></tr> <tr><td>N</td><td>78</td><td>0116</td><td>0x4e</td><td>01001110</td><td></td><td></td><td></td><td>Latin capital letter N</td></tr> <tr><td>O</td><td>79</td><td>0117</td><td>0x4f</td><td>01001111</td><td></td><td></td><td></td><td>Latin capital letter O</td></tr> <tr><td>P</td><td>80</td><td>0120</td><td>0x50</td><td>01010000</td><td></td><td></td><td></td><td>Latin capital letter P</td></tr> <tr><td>Q</td><td>81</td><td>0121</td><td>0x51</td><td>01010001</td><td></td><td></td><td></td><td>Latin capital letter Q</td></tr> <tr><td>R</td><td>82</td><td>0122</td><td>0x52</td><td>01010010</td><td></td><td></td><td></td><td>Latin capital letter R</td></tr> <tr><td>S</td><td>83</td><td>0123</td><td>0x53</td><td>01010011</td><td></td><td></td><td></td><td>Latin capital letter S</td></tr> <tr><td>T</td><td>84</td><td>0124</td><td>0x54</td><td>01010100</td><td></td><td></td><td></td><td>Latin capital letter T</td></tr> <tr><td>U</td><td>85</td><td>0125</td><td>0x55</td><td>01010101</td><td></td><td></td><td></td><td>Latin capital letter U</td></tr> <tr><td>V</td><td>86</td><td>0126</td><td>0x56</td><td>01010110</td><td></td><td></td><td></td><td>Latin capital letter V</td></tr> <tr><td>W</td><td>87</td><td>0127</td><td>0x57</td><td>01010111</td><td></td><td></td><td></td><td>Latin capital letter W</td></tr> <tr><td>X</td><td>88</td><td>0130</td><td>0x58</td><td>01011000</td><td></td><td></td><td></td><td>Latin capital letter X</td></tr> <tr><td>Y</td><td>89</td><td>0131</td><td>0x59</td><td>01011001</td><td></td><td></td><td></td><td>Latin capital letter Y</td></tr> <tr><td>Z</td><td>90</td><td>0132</td><td>0x5a</td><td>01011010</td><td></td><td></td><td></td><td>Latin capital letter Z</td></tr> <tr><td>[</td><td>91</td><td>0133</td><td>0x5b</td><td>01011011</td><td></td><td></td><td></td><td>Left square bracket</td></tr> <tr><td>\</td><td>92</td><td>0134</td><td>0x5c</td><td>01011100</td><td></td><td>\\</td><td></td><td>Reverse solidus (backslash)</td></tr> <tr><td>]</td><td>93</td><td>0135</td><td>0x5d</td><td>01011101</td><td></td><td></td><td></td><td>Right square bracket</td></tr> <tr><td>^</td><td>94</td><td>0136</td><td>0x5e</td><td>01011110</td><td></td><td></td><td></td><td>Circumflex accent (caret)</td></tr> <tr><td>_</td><td>95</td><td>0137</td><td>0x5f</td><td>01011111</td><td></td><td></td><td></td><td>Low line (underscore)</td></tr> <tr><td>`</td><td>96</td><td>0140</td><td>0x60</td><td>01100000</td><td></td><td></td><td></td><td>Grave accent (backtick)</td></tr> <tr><td>a</td><td>97</td><td>0141</td><td>0x61</td><td>01100001</td><td></td><td></td><td></td><td>Latin small letter a</td></tr> <tr><td>b</td><td>98</td><td>0142</td><td>0x62</td><td>01100010</td><td></td><td></td><td></td><td>Latin small letter b</td></tr> <tr><td>c</td><td>99</td><td>0143</td><td>0x63</td><td>01100011</td><td></td><td></td><td></td><td>Latin small letter c</td></tr> <tr><td>d</td><td>100</td><td>0144</td><td>0x64</td><td>01100100</td><td></td><td></td><td></td><td>Latin small letter d</td></tr> <tr><td>e</td><td>101</td><td>0145</td><td>0x65</td><td>01100101</td><td></td><td></td><td></td><td>Latin small letter e</td></tr> <tr><td>f</td><td>102</td><td>0146</td><td>0x66</td><td>01100110</td><td></td><td></td><td></td><td>Latin small letter f</td></tr> <tr><td>g</td><td>103</td><td>0147</td><td>0x67</td><td>01100111</td><td></td><td></td><td></td><td>Latin small letter g</td></tr> <tr><td>h</td><td>104</td><td>0150</td><td>0x68</td><td>01101000</td><td></td><td></td><td></td><td>Latin small letter h</td></tr> <tr><td>i</td><td>105</td><td>0151</td><td>0x69</td><td>01101001</td><td></td><td></td><td></td><td>Latin small letter i</td></tr> <tr><td>j</td><td>106</td><td>0152</td><td>0x6a</td><td>01101010</td><td></td><td></td><td></td><td>Latin small letter j</td></tr> <tr><td>k</td><td>107</td><td>0153</td><td>0x6b</td><td>01101011</td><td></td><td></td><td></td><td>Latin small letter k</td></tr> <tr><td>l</td><td>108</td><td>0154</td><td>0x6c</td><td>01101100</td><td></td><td></td><td></td><td>Latin small letter l</td></tr> <tr><td>m</td><td>109</td><td>0155</td><td>0x6d</td><td>01101101</td><td></td><td></td><td></td><td>Latin small letter m</td></tr> <tr><td>n</td><td>110</td><td>0156</td><td>0x6e</td><td>01101110</td><td></td><td></td><td></td><td>Latin small letter n</td></tr> <tr><td>o</td><td>111</td><td>0157</td><td>0x6f</td><td>01101111</td><td></td><td></td><td></td><td>Latin small letter o</td></tr> <tr><td>p</td><td>112</td><td>0160</td><td>0x70</td><td>01110000</td><td></td><td></td><td></td><td>Latin small letter p</td></tr> <tr><td>q</td><td>113</td><td>0161</td><td>0x71</td><td>01110001</td><td></td><td></td><td></td><td>Latin small letter q</td></tr> <tr><td>r</td><td>114</td><td>0162</td><td>0x72</td><td>01110010</td><td></td><td></td><td></td><td>Latin small letter r</td></tr> <tr><td>s</td><td>115</td><td>0163</td><td>0x73</td><td>01110011</td><td></td><td></td><td></td><td>Latin small letter s</td></tr> <tr><td>t</td><td>116</td><td>0164</td><td>0x74</td><td>01110100</td><td></td><td></td><td></td><td>Latin small letter t</td></tr> <tr><td>u</td><td>117</td><td>0165</td><td>0x75</td><td>01110101</td><td></td><td></td><td></td><td>Latin small letter u</td></tr> <tr><td>v</td><td>118</td><td>0166</td><td>0x76</td><td>01110110</td><td></td><td></td><td></td><td>Latin small letter v</td></tr> <tr><td>w</td><td>119</td><td>0167</td><td>0x77</td><td>01110111</td><td></td><td></td><td></td><td>Latin small letter w</td></tr> <tr><td>x</td><td>120</td><td>0170</td><td>0x78</td><td>01111000</td><td></td><td></td><td></td><td>Latin small letter x</td></tr> <tr><td>y</td><td>121</td><td>0171</td><td>0x79</td><td>01111001</td><td></td><td></td><td></td><td>Latin small letter y</td></tr> <tr><td>z</td><td>122</td><td>0172</td><td>0x7a</td><td>01111010</td><td></td><td></td><td></td><td>Latin small letter z</td></tr> <tr><td>{</td><td>123</td><td>0173</td><td>0x7b</td><td>01111011</td><td></td><td></td><td></td><td>Left curly bracket</td></tr> <tr><td>|</td><td>124</td><td>0174</td><td>0x7c</td><td>01111100</td><td></td><td></td><td></td><td>Vertical line</td></tr> <tr><td>}</td><td>125</td><td>0175</td><td>0x7d</td><td>01111101</td><td></td><td></td><td></td><td>Right curly bracket</td></tr> <tr><td>~</td><td>126</td><td>0176</td><td>0x7e</td><td>01111110</td><td></td><td></td><td></td><td>Tilde</td></tr> <tr><td>(del)</td><td>127</td><td>0177</td><td>0x7f</td><td>01111111</td><td>^?</td><td></td><td>␡</td><td>Delete</td></tr> </tbody></table> <p><a href="http://www.bahaistudies.net/asma/ascii.pdf">PDF of the 1963 ASCII specification</a></p> <p><a href="https://en.wikipedia.org/wiki/Control_character">Control characters on Wikipedia</a></p> <p><a href="http://www.aivosto.com/vbtips/control-characters.html">Detailed historical information about ASCII and control characters</a></p> i c the light Thu, 25 May 2017 00:00:00 +0000 Rebecca Turner https://becca.ooo/i-c-the-light/ https://becca.ooo/i-c-the-light/ <p>A distance-estimating ray marcher</p> <p>From <a href="http://blog.hvidtfeldts.net/index.php/2011/06/distance-estimated-3d-fractals-part-i/">Syntopia</a>:</p> <blockquote> <p>Classic raytracing shoots one (or more) rays per pixel and calculates where the rays intersect the geometry in the scene. Normally the geometry is described by a set of primitives, like triangles or spheres, and some kind of spatial acceleration structure is used to quickly identify which primitives intersect the rays.</p> <p>Distance Estimation, on the other hand, is a ray marching technique.</p> <p>Instead of calculating the exact intersection between the camera ray and the geometry, you proceed in small steps along the ray and check how close you are to the object you are rendering. When you are closer than a certain threshold, you stop. In order to do this, you must have a function that tells you how close you are to the object: a Distance Estimator. The value of the distance estimator tells you how large a step you are allowed to march along the ray, since you are guaranteed not to hit anything within this radius.</p> </blockquote> <ul> <li><a href="/i-c-the-light.pdf">My paper on the ray marcher I wrote</a></li> <li><a href="https://github.com/9999years/i-c-the-light">github.com/9999years/i-c-the-light</a></li> </ul> <p><img src="/img/i-c-the-light/julia.jpg" alt="A distance-estimated rendering of a pink 3D slice of a 4D quaternion Julia set fractal" /></p>