<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:itunes="http://www.itunes.com/dtds/podcast-1.0.dtd" xmlns:googleplay="http://www.google.com/schemas/play-podcasts/1.0"><channel><title><![CDATA[FREST Substack]]></title><description><![CDATA[The substack for FREST: computing for everyone]]></description><link>https://frest.substack.com</link><image><url>https://substackcdn.com/image/fetch/$s_!MB46!,w_256,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F5fd9d106-47e9-4c50-a15f-707b11ec91ae_550x550.png</url><title>FREST Substack</title><link>https://frest.substack.com</link></image><generator>Substack</generator><lastBuildDate>Sat, 11 Apr 2026 05:11:34 GMT</lastBuildDate><atom:link href="https://frest.substack.com/feed" rel="self" type="application/rss+xml"/><copyright><![CDATA[Guyren Howe]]></copyright><language><![CDATA[en]]></language><webMaster><![CDATA[frest@substack.com]]></webMaster><itunes:owner><itunes:email><![CDATA[frest@substack.com]]></itunes:email><itunes:name><![CDATA[Guyren Howe]]></itunes:name></itunes:owner><itunes:author><![CDATA[Guyren Howe]]></itunes:author><googleplay:owner><![CDATA[frest@substack.com]]></googleplay:owner><googleplay:email><![CDATA[frest@substack.com]]></googleplay:email><googleplay:author><![CDATA[Guyren Howe]]></googleplay:author><itunes:block><![CDATA[Yes]]></itunes:block><item><title><![CDATA[Apps Are Dead; They Just Don’t Know It Yet]]></title><description><![CDATA[(AI Consequences #1)]]></description><link>https://frest.substack.com/p/apps-are-dead-they-just-dont-know</link><guid isPermaLink="false">https://frest.substack.com/p/apps-are-dead-they-just-dont-know</guid><dc:creator><![CDATA[Guyren Howe]]></dc:creator><pubDate>Tue, 10 Mar 2026 11:53:29 GMT</pubDate><enclosure url="https://substackcdn.com/image/fetch/$s_!MB46!,w_256,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F5fd9d106-47e9-4c50-a15f-707b11ec91ae_550x550.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>The App has had its day. The several hundred icons you never use on your computing devices have been sub-optimal relics for decades already, but AI is about to render the whole paradigm entirely unmanageable.</p><p>Apps are containers that trap both data and capabilities. When software was scarce, that made sense. But AI is rapidly making software abundant &#8212; and now the container has become the bottleneck.</p><div class="subscription-widget-wrap-editor" data-attrs="{&quot;url&quot;:&quot;https://frest.substack.com/subscribe?&quot;,&quot;text&quot;:&quot;Subscribe&quot;,&quot;language&quot;:&quot;en&quot;}" data-component-name="SubscribeWidgetToDOM"><div class="subscription-widget show-subscribe"><div class="preamble"><p class="cta-caption">Thanks for reading FREST Substack! Subscribe for free to receive new posts and support my work.</p></div><form class="subscription-widget-subscribe"><input type="email" class="email-input" name="email" placeholder="Type your email&#8230;" tabindex="-1"><input type="submit" class="button primary" value="Subscribe"><div class="fake-input-wrapper"><div class="fake-input"></div><div class="fake-button"></div></div></form></div></div><p>There&#8217;s no sensible way to organise access to apps. I have dozens of apps on my phone that seemed useful when I downloaded them, but now I have no idea what they do and no efficient way of finding out other than to run them. I know I have apps for image processing, but I have no idea which ones those are.</p><p>Hell, Apple&#8217;s &#8220;springboard&#8221; launcher on the iPhone has so little functionality that it is embarrassed by the 1984 original Mac Finder.</p><p>There&#8217;s also no real way to use apps together. That&#8217;s not even really a concept. The clipboard is a 1980s hack, and the share menu is just a slightly quicker form of the same simplistic idea.</p><p>Now enter AI.</p><p>People are already building apps in a couple of hours. I can barely manage the hundred or so apps on my phone today. Apple&#8217;s App Store has <a href="https://www.businessofapps.com/data/app-stores/">about 2.5 million apps</a>.</p><p>When the cost of creating an app approaches zero, the idea that we should organise our digital lives around thousands &#8212; or millions &#8212; of separate apps becomes absurd.</p><h1>The Political Economy of the App</h1><p>The app paradigm didn&#8217;t arise because engineers thought it was elegant. It arose because it matched the economic moats of the software industry. Early on, the moat was distribution &#8212; shipping software was hard, so functionality was packaged as products. Later the moat became reliability and long-term support &#8212; businesses trusted large vendors like IBM, Oracle, and Microsoft to maintain complex systems for years. Then the internet introduced another moat: service infrastructure. Running reliable online systems required enormous operational expertise, so software again consolidated into tightly controlled applications that owned their data, logic, and interface end-to-end.</p><p>But those moats are eroding. Distribution is trivial. Infrastructure is commoditised. And AI is rapidly reducing the cost of producing software itself.</p><p>It turns out the app is not the natural unit of computing.</p><p>It is the natural unit of computing under conditions of software scarcity.</p><h1>Beyond the Walled Garden</h1><p>If the app was the natural structure of computing under software scarcity, what happens when software stops being scarce?</p><p>Once you start wondering what a world without apps might look like, things get interesting. For most of the history of computing, software organised data. But when AI begins dissolving those walls, the natural alternative appears: data organising software.</p><p>Right now I cannot process the data locked inside my apps in any way other than what the owners of those apps permit. We&#8217;re so used to this that it barely even occurs to us that things could be different.</p><p>But imagine something simple.</p><p>Maybe I want to compare how often I&#8217;ve used a particular word across all my writing over time. For a programmer that&#8217;s a small project. For most people it&#8217;s essentially impossible. Hell, not just impossible &#8212; <em>unimaginable</em>. It wouldn&#8217;t even occur to most people to even imagine such a thing.</p><p>Yet computers are pure thought machines. Any computable abstraction we can conceive is available to us. We should conceive more ambitiously.</p><p>Almost everyone in the world now carries an almost-free device capable of presenting real-time simulations of reality that are hard to distinguish from the real thing. These machines support sophisticated human interfaces &#8212; cameras, microphones, styluses, touchscreens, sensors, VR.</p><p>And yet our lives are still organised around windows and clicking icons like it&#8217;s fucking 1984.</p><p>No.</p><p>I refuse to believe that our current computing paradigms are anywhere near the peak of what we might do.</p><p>Try to forget your assumptions about what a computing device looks like and how it behaves. Imagine God&#8217;s iPad. Think about the tasks you actually want to do during your day.</p><p>Flatten the walls between apps. Imagine manipulating any data at any time, through whatever representation or interface makes sense. Imagine a world where Microsoft doesn&#8217;t own your words and your presentations, and you can do far more than what Microsoft has decided you should.</p><p>Set your imagination free.</p><p>You can see it dimly, right?</p><p>Maybe not in perfect detail. But you can certainly see how far it is from what we have now.</p><p>The future of computing will not be apps. It will be fluid computation over shared data.</p><p>Up next: how AI will transform APIs.</p><div class="subscription-widget-wrap-editor" data-attrs="{&quot;url&quot;:&quot;https://frest.substack.com/subscribe?&quot;,&quot;text&quot;:&quot;Subscribe&quot;,&quot;language&quot;:&quot;en&quot;}" data-component-name="SubscribeWidgetToDOM"><div class="subscription-widget show-subscribe"><div class="preamble"><p class="cta-caption">Thanks for reading FREST Substack! Subscribe for free to receive new posts and support my work.</p></div><form class="subscription-widget-subscribe"><input type="email" class="email-input" name="email" placeholder="Type your email&#8230;" tabindex="-1"><input type="submit" class="button primary" value="Subscribe"><div class="fake-input-wrapper"><div class="fake-input"></div><div class="fake-button"></div></div></form></div></div>]]></content:encoded></item><item><title><![CDATA[First-Class Models: The Missing Productivity Revolution]]></title><description><![CDATA[TL;DR: First-class models with branching and merging capabilities represent an almost entirely unused enormous productivity and expressiveness unlock in programming and computer systems.]]></description><link>https://frest.substack.com/p/first-class-models-the-missing-productivity</link><guid isPermaLink="false">https://frest.substack.com/p/first-class-models-the-missing-productivity</guid><dc:creator><![CDATA[Guyren Howe]]></dc:creator><pubDate>Thu, 26 Jun 2025 02:20:00 GMT</pubDate><enclosure url="https://substackcdn.com/image/fetch/$s_!MB46!,w_256,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F5fd9d106-47e9-4c50-a15f-707b11ec91ae_550x550.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>TL;DR: First-class models with branching and merging capabilities represent an almost entirely unused enormous productivity and expressiveness unlock in programming and computer systems.</p><h2>The Current State: Well-Designed Systems, Constrained Users</h2><p>Imagine you&#8217;re building an accounting system from scratch. You&#8217;d design it properly: a normalized database schema, algebraically defined operations for debits and credits, account reconciliation, and comparison functions. You&#8217;d implement data-only, invariant-preserving operations using database functions and procedures, with clean authorization models. The result would be a database perfectly customized for accounting, supporting multiple simultaneous client interfaces in any language.</p><div class="subscription-widget-wrap-editor" data-attrs="{&quot;url&quot;:&quot;https://frest.substack.com/subscribe?&quot;,&quot;text&quot;:&quot;Subscribe&quot;,&quot;language&quot;:&quot;en&quot;}" data-component-name="SubscribeWidgetToDOM"><div class="subscription-widget show-subscribe"><div class="preamble"><p class="cta-caption">Thanks for reading FREST Substack! Subscribe for free to receive new posts and support my work.</p></div><form class="subscription-widget-subscribe"><input type="email" class="email-input" name="email" placeholder="Type your email&#8230;" tabindex="-1"><input type="submit" class="button primary" value="Subscribe"><div class="fake-input-wrapper"><div class="fake-input"></div><div class="fake-button"></div></div></form></div></div><p>This represents the current state of the art&#8212;and it&#8217;s genuinely good architecture.</p><p>But now put yourself in the shoes of the users: knowledge workers, engineers, analysts who depend on this beautifully designed system. Despite having access to well-structured data and operations, they face a fundamental limitation when it comes to the kind of thinking their work actually requires.</p><h2>The Hypothesis Gap</h2><p>Consider an engineer working on a project management system. They&#8217;re constantly asking computational questions that our current systems simply don&#8217;t support:</p><p>- What if I replace this supplier with one offering different prices?</p><p>- If I increase this capacity parameter, how much larger does the engine room need to be? How much more cement do we need?</p><p>- Would using a different manufacturing process have significantly changed our profitability last year?</p><p>Today, when faced with these questions, professionals resort to crude workarounds: rough estimates, Excel spreadsheets copied from the main system, or in rare cases, custom programs written specifically for the analysis. Each approach disconnects them from the authoritative data source and requires manual reconciliation with the primary system.</p><h2>The Solution: First-Class Model Branching</h2><p>What we need is the ability to explicitly manipulate and navigate between different models of our system&#8217;s state. This goes beyond simple version control&#8212;it&#8217;s about making hypothesis exploration a core system capability.</p><h3>Time as the First Dimension</h3><p>Users should be able to view any previous state the system was in, not just as read-only history, but as a living branch they can extend and modify. This isn&#8217;t just data versioning&#8212;when examining past states, users should be able to run the older versions of functions that were available at that time, maintaining complete computational consistency.</p><h3>Hypothesis as the Second Dimension</h3><p>Without changing the authoritative state, users should be able to create simulation branches. Imagine branching your company&#8217;s current accounting state to model next month&#8217;s performance. You manually enter projected sales for California stores, then use regression or AI to estimate likely values for other regions. You can then:</p><p>- Compare this hypothetical future against a baseline projection</p><p>- Diff the states to understand the specific impacts of your California strategy</p><p>- Selectively merge insights back into your planning process</p><p>This is a generalization of what programmers do with Git, but applied to all system state and made accessible to ordinary users.</p><h3>The Navigation Interface</h3><p>Users need intuitive ways to see and navigate branches without getting lost in complexity. While this presents interesting UI challenges (including fraud prevention), the core operations are familiar: branch, compare, merge, and rollback. The key is making these operations meaningful for business content, not just code.</p><h2>The Technical Foundation: Unifying Synchronization</h2><p>The branching and merging users perform manually should be built on the same foundations that handle automatic synchronization in distributed systems. This means generalizing the work done in CRDTs (Conflict-free Replicated Data Types) and other synchronization primitives.</p><p>Currently, we have a fragmented landscape:</p><p>- CRDTs handle concurrent collaborative editing</p><p>- Git provides version control with manual conflict resolution</p><p>- Database replication uses various conflict resolution strategies</p><p>- Most business systems avoid conflicts entirely through restrictive permissions</p><p>By unifying these approaches, we can create composable synchronization primitives where user-driven branching and automatic conflict resolution are different manifestations of the same underlying mathematical operations.</p><p>Some changes (updating contact information) can auto-merge across all branches. Others (modifying core business logic) require explicit human decisions about propagation. The system provides the infrastructure for both, with the choice of automation level being a design decision rather than a fundamental limitation.</p><h2>Beyond Spreadsheet Prison</h2><p>This approach solves what we might call &#8220;spreadsheet escape&#8221;&#8212;the common pattern where users copy data into Excel to explore scenarios, losing connection to the authoritative source and creating reconciliation problems. With first-class modeling, users stay within the system where their explorations maintain semantic relationships and can be selectively integrated back into the main state.</p><p>The engineer wondering about supplier changes doesn&#8217;t need to export data and lose track of dependencies. They branch the current state, modify supplier relationships, and let the system compute cascading effects while maintaining full traceability back to the source.</p><h2>The Productivity Revolution</h2><p>When modeling becomes a first-class system capability rather than an ad-hoc user workaround, we unlock fundamentally new ways of thinking about business problems. Decision-makers can explore scenarios with the same rigor programmers bring to code, but without requiring programming skills.</p><p>The knowledge worker asking &#8220;what if&#8221; questions deserves computational support that matches the sophistication we bring to the underlying data management. First-class models provide that support, turning hypothesis exploration from a painful exception into a natural part of how systems work.</p><p>This isn&#8217;t just better tooling&#8212;it&#8217;s a different relationship between users and their computational environment, one where the system actively supports the kind of exploratory thinking that drives actual business value.</p><h2>This is Frest</h2><p>Attentive readers will realise that this is a description of the UI design principles underlying Frest (the subject of this Blog).</p><div class="subscription-widget-wrap-editor" data-attrs="{&quot;url&quot;:&quot;https://frest.substack.com/subscribe?&quot;,&quot;text&quot;:&quot;Subscribe&quot;,&quot;language&quot;:&quot;en&quot;}" data-component-name="SubscribeWidgetToDOM"><div class="subscription-widget show-subscribe"><div class="preamble"><p class="cta-caption">Thanks for reading FREST Substack! Subscribe for free to receive new posts and support my work.</p></div><form class="subscription-widget-subscribe"><input type="email" class="email-input" name="email" placeholder="Type your email&#8230;" tabindex="-1"><input type="submit" class="button primary" value="Subscribe"><div class="fake-input-wrapper"><div class="fake-input"></div><div class="fake-button"></div></div></form></div></div>]]></content:encoded></item><item><title><![CDATA[Composition to the Rescue]]></title><description><![CDATA[Typical software architecture is driven by absolutely terrible economic incentives to be the brittle mess it is today. There is another way.]]></description><link>https://frest.substack.com/p/composition-to-the-rescue</link><guid isPermaLink="false">https://frest.substack.com/p/composition-to-the-rescue</guid><dc:creator><![CDATA[Guyren Howe]]></dc:creator><pubDate>Thu, 16 Jan 2025 02:36:45 GMT</pubDate><enclosure url="https://substack-post-media.s3.amazonaws.com/public/images/a96b72cc-650d-4189-84e3-0fc0afa9b5f9_1024x1024.heic" length="0" type="image/jpeg"/><content:encoded><![CDATA[<h1><strong>Software's Original Sin</strong></h1><p>Every time I hand-roll another HTML form, I die a little inside.</p><p>Our industry's greatest moral failure isn't security breaches or privacy violations. It's that we've built a priesthood of programmers, hoarding the power of computing behind arcane incantations of JavaScript and Docker containers. We've chosen approaches that suit our highly trained elite rather than ones that democratize computing.</p><p>Where are all the bicycles for the mind?</p><p>We've known how to do better since the beginning. Excel turned millions of accountants into functional programmers. FileMaker let ordinary folks wield the power of relational databases. But instead of building on these successes, we've created systems so complex that even the priests struggle to maintain them.</p><p>Every business application today is essentially the same thing: small islands of simple code floating in a sea of state, connected by brittle ceremonies of data transformation. We pretend we're building grand computational cathedrals, but we're really just state farmers, tending our JSON and praying our API calls don't fail.</p><p>What if it all worked like FileMaker mashed with Excel?</p><p>What if every piece of software could be changed as easily as a spreadsheet? What if you could right-click anything - a chart, a form, a dashboard - see exactly how it was built, and change any part of it? What if "like that, but for my department" was three clicks instead of three weeks of development?</p><h2><strong>The Architecture of Liberation</strong></h2><p>The dirty secret of enterprise software is that most of it is just moving data around and occasionally showing it to humans. We dress it up with sophisticated terms like "microservices" and "event sourcing," but we're basically building ever more complex ways to fail at simple things.</p><p>I think this happens because the boundaries around shared work and ownership within a business just are at the wrong point for software. The vast majority of the software development effort into the world goes into idiosyncratic, short-term, business-driven dreck. Frest defines an architecture style that makes it natural to cooperate on a more optimal scale.</p><p>Our APIs are rigid contracts written in stone. Our security models are Byzantine mazes of permissions. Our user interfaces are change-resistant monuments to someone's best guess from six months ago. And God help you if you want to combine data from two different systems.</p><p>This isn't just inefficient - it's a fundamental betrayal of what computing should be.</p><p>Consider a typical "modern" business dashboard. Behind that seemingly simple chart lies a tangled web of API calls, data transformations, and hard-coded business rules. Want to see the same data slightly differently? That'll be a two-week sprint and three meetings with the architecture team.</p><p>Excel users are laughing at us. They've been doing ad-hoc data manipulation and visualization for decades. FileMaker users can build entire business applications in an afternoon. Meanwhile, we're writing boilerplate code to validate form inputs. Again.</p><h2><strong>The Revolution Will Be Composable</strong></h2><p>The solution isn't better frameworks or more sophisticated development tools. We don't need another way to generate CRUD apps or yet another JavaScript framework.</p><p>What we need is a fundamental rethinking of how software is composed.</p><p>Imagine a world where:</p><p>- Every piece of data has an address, but what you see at that address depends on who you are</p><p>- Everything you see always has a useful, relational, searchable UI</p><p>- Security isn't a bolt-on feature but emerges naturally from how things compose</p><p>- Functions are just relations waiting to be filled in</p><p>- Everything is a query, and every query can be changed</p><p>This isn't science fiction. The building blocks have existed since the 1970s. The relational model. Functional programming. Capability-based security. We just haven't put them together right.</p><p>The missing piece is a composition model that makes all of this natural and inevitable. A way to say "like that, but..." and have the system just understand.</p><h2><strong>The Path Forward</strong></h2><p>We've spent fifty years building software that serves the priesthood. Tools that make sense if you think in abstractions and can hold a mental model of asynchronous state mutations. Tools that actively resist modification by anyone without years of training.</p><p>It's time to burn the priesthood down.</p><p>The next generation of software won't be built on promises of scalability or developer ergonomics. It will be built on a radical premise: that everyone deserves tools they can understand and modify.</p><p>In the next post, I'll show you exactly how this works. How a simple idea about composing models leads to software that's naturally malleable, secure, and distributed. Software that normal humans can actually bend to their will.</p><p>The priests won't like it. But then, they never do.</p><div class="subscription-widget-wrap-editor" data-attrs="{&quot;url&quot;:&quot;https://frest.substack.com/subscribe?&quot;,&quot;text&quot;:&quot;Subscribe&quot;,&quot;language&quot;:&quot;en&quot;}" data-component-name="SubscribeWidgetToDOM"><div class="subscription-widget show-subscribe"><div class="preamble"><p class="cta-caption">Thanks for reading FREST Substack! Subscribe for free to receive new posts and support my work.</p></div><form class="subscription-widget-subscribe"><input type="email" class="email-input" name="email" placeholder="Type your email&#8230;" tabindex="-1"><input type="submit" class="button primary" value="Subscribe"><div class="fake-input-wrapper"><div class="fake-input"></div><div class="fake-button"></div></div></form></div></div>]]></content:encoded></item><item><title><![CDATA[State Farming]]></title><description><![CDATA[Islands in a sea of state.]]></description><link>https://frest.substack.com/p/state-farming</link><guid isPermaLink="false">https://frest.substack.com/p/state-farming</guid><dc:creator><![CDATA[Guyren Howe]]></dc:creator><pubDate>Sun, 14 Jul 2024 00:33:35 GMT</pubDate><enclosure url="https://substack-post-media.s3.amazonaws.com/public/images/8d08e5b8-b0fb-4c9c-9b32-b117ff5104bb_275x183.jpeg" length="0" type="image/jpeg"/><content:encoded><![CDATA[<h1>Coding is State Farming</h1><p>If you just see all the code involved in a typical business app, it is tempting to see it as a single, grand piece of computational architecture.</p><p>While in some sense it <em>must be </em>that, business software in practice is nearly always small pieces of fairly simple code that run asynchronously, serialising and deserialising bits of the state of whatever is being worked on. We write islands of code in a sea of state.</p><div class="subscription-widget-wrap-editor" data-attrs="{&quot;url&quot;:&quot;https://frest.substack.com/subscribe?&quot;,&quot;text&quot;:&quot;Subscribe&quot;,&quot;language&quot;:&quot;en&quot;}" data-component-name="SubscribeWidgetToDOM"><div class="subscription-widget show-subscribe"><div class="preamble"><p class="cta-caption">Thanks for reading FREST Substack! Subscribe for free to receive new posts and support my work.</p></div><form class="subscription-widget-subscribe"><input type="email" class="email-input" name="email" placeholder="Type your email&#8230;" tabindex="-1"><input type="submit" class="button primary" value="Subscribe"><div class="fake-input-wrapper"><div class="fake-input"></div><div class="fake-button"></div></div></form></div></div><p>The instances of running code can be separated by:</p><ul><li><p>time &#8212; perhaps a backup process must be run every night at midnight;</p></li><li><p>space &#8212; computation distributed in physically separate devices must be sent across a network;</p></li><li><p>logic &#8212; Some of the logic of the final computation might be best executed in the database; or</p></li><li><p>resource constraints &#8212; we must do some computation in pieces or in the database because the data involved don&#8217;t fit in memory.</p></li></ul><p>In some cases, a computation might just be suspended in memory and resumed (e.g. when waiting to read a file), but mostly, some values must be serialized in one place and deserialized in another, and code finishes here and new code runs there. Through a file or a database or a network packet, the effect is in many ways the same: computations separated by time and space, connected through the sea of state.</p><p>These computation islands might be <em>very</em> different. You don&#8217;t control the web browser rendering your HTML, for example. Not only are we building asynchronous systems; we are building them out of different code written by different people &#8212; code we may or may not trust.</p><p>The fight to manage the retrieval or generation of the right state so we can carry out the next step of some computation is the source of most of the complexity in business software. An engineer must constantly bear in mind where something is running, with what privileges, able to assume such and so <em>invariants</em> (i.e. things that have been made true by computation before this). And &#8220;state&#8221; can be subtle &#8212; our state isn&#8217;t just the HTTP request we&#8217;re processing, but that at this point in the code, we have authenticated the user, who has these privileges, &#8230;</p><h1>Turing Complete code is trouble</h1><p>Every computation written in a Turing Complete (TC) programming language (as opposed to declarative/logic languages like SQL) which fetches some state in order to do a Very Important Thing is:</p><ul><li><p>a consumer of resources;</p></li><li><p>a source of bugs;</p></li><li><p>a source of latency;</p></li><li><p>a maintenance burden,</p></li><li><p>&#8230;</p></li></ul><p>We can all agree that we should seek to minimise these issues. It&#8217;s why we use high level languages. It&#8217;s why we&#8217;re continually moving toward <em>higher level</em> programming languages (e.g. Rust and its <a href="https://users.rust-lang.org/t/what-are-affine-types-in-rust/23755">affine types</a>). It&#8217;s a big part of why we use frameworks and all the rest.</p><p>But code in TC languages will always be where we get most of our bugs and vulnerabilities. Better languages are great, but <em>better still</em> would be to just have fewer computations that must be written in regular programming languages.</p><h1> Coding as State Farming</h1><p>The database is the most stable part of any business application. Code comes and goes, waxes and wanes, we use different whole applications, but you&#8217;ll still have a good chunk of database state that was put there in the first few days your app ran.</p><p>Maintaining that state is one of the two most important tasks your code does (the other is emitting the right outputs, where that is API calls or user interface generation). We are all in this sense <em>state farmers</em>.</p><p>The tasks we achieve with the code we write can be separated into three kinds:</p><ul><li><p>causing side-effects (e.g. sending an email);</p></li><li><p>creating new state from other state (creating a new appointment based on a HTTP request); and</p></li><li><p>performing complex computations (resizing an image).</p></li></ul><p>The last of these is, kind of, the only one we really <em>must</em> perform in a regular language &#8212;&nbsp; at least, in a way that is vulnerable to all of the issues that code brings.</p><h1>Side Effects through New State</h1><p>Sending emails and other sort of external state change must in the end be performed by regular code. But these effects are in general self-contained and can be represented as a consequence of a clear and simple set of input states.</p><p>This means that we can generally set things up so side effects can be represented <em>as</em> &#8220;creating state from other state&#8221;. So for sending out emails, we can marshall the template and its arguments in the database and then run the small amount of fixed code that causes the side-effect to occur. We can send any sort of email to any sort of address with any sort of content, all without having to change any TC code.</p><h1>Creating State from other State</h1><p>Much of the code we write just rearranges state. A HTTP request comes in from a user, we pull the request apart, and we create state as a result &#8212; a new appointment, say.</p><p>Since we can just rearrange state in a relational query language, we don&#8217;t need to use our TC languages for this.</p><p>However.</p><p><a href="https://www.scattered-thoughts.net/writing/against-sql">SQL is an abomination</a>. It hides the sublime elegance of the relational model inside a language and a set of assumptions that would cause the creators of COBOL to blush.</p><p><a href="https://blogit.michelin.io/an-introduction-to-datalog/">There are alternatives</a>&#8230;</p><h1>Setting SQL aside&#8230;</h1><p>My discussion here leads to two conclustions with regard to SQL:</p><ol><li><p>SQL is awful, but it&#8217;s really well worth using, much more than most folks do, in any event. This also means that you should learn to use your database <em>well</em>, which is more than most folks do either. Postgres in particular has a lot of features that make the awfulness of SQL not quite so bad; and</p></li><li><p>We should be seeking &#8212; <em>demanding!</em> &#8212; alternatives. <a href="https://www.cozodb.org/">CozoDB</a> is the best project I know of right now. Check it out. It&#8217;s really cool.</p></li></ol><h2>Rules</h2><p>SQL is a logic engine, although its design tries hard to hide this. In particular, it&#8217;s a bit odd to use a <em>rule</em> in SQL. A rule expresses how to derive some state from other state. We can say <em>if a customer is on the gold list, they get a 10% discount</em>. Or <em>if an order is created, a shipping task should be created for the warehouse</em>.</p><p>It is useful to think of rules as being either:</p><ul><li><p><em>forward</em>, meaning that whenever a new case of the antecedent of the rule becomes true, we immediately generate and insert its consequent; or</p></li><li><p><em>backward</em>, meaning that when we want to know if the consequent is true, we go looking for matching antecedents</p></li></ul><p>In SQL databases, a forward rule is a <em>trigger</em>, and a backward rule is a <em>view</em> or <em>query</em>.</p><h1>Declarative Triggers</h1><p>In most relational databases, triggers execute code in a regular programming language. In many cases (SQL Server, Oracle), we have a choice between a clunky-but-close-to-the-data language like PL/SQL or TSQL, or Python.</p><p>However, Postgres and SQLite both support triggers in something like pure SQL, giving us <em>declarative rules</em>, letting us have rules while avoiding the issues TC languages bring. Even within the limits of the other databases, we can write state-rearranging triggers in a simple, essentially declarative way, just chaining together SQL queries.</p><h2>Limiting Turing Completeness</h2><p>Much &#8212; most, even &#8212;&nbsp;of what you do in your TC language can be straightforwardly expressed in the data layer, especially once we can represent functions in the data layer.</p><p>A widely used example of a Rails application for tutorials is <a href="https://github.com/JetBrains/sample_rails_app">a blogging app</a>. Here is the code for creating a <em>following</em> relationship from one user to another:</p><pre><code>  # Relationship Controller action
  def create
    @user = User.find(params[:followed_id])
    current_user.follow(@user)
    respond_to do |format|
      format.html { redirect_to @user }
      format.js
    end
  end

  # User Model
  has_many :following, through: :active_relationships,  source: :followed

  def follow(other_user)
    following &lt;&lt; other_user
  end</code></pre><p>Rails is doing a lot of the work here, so this code is simple. Still, it could just be done in SQL, in a lot fewer lines</p><pre><code>INSERT INTO active_relationships(follower_id, followed_id) VALUES(current_user_id, followed_id) RETURNING follower_id, followed_id</code></pre><p>Imagine a very simple framework where the TC language inserts all incoming requests as JSON into a <em>requests </em>table (you would probably be logging the request anyway, so if we stop doing that, we&#8217;re not using significantly different storage). Now,  a trigger on the <em>requests</em> table can extract perform the above insert and then gather values and a mustache template for the resulting web page.</p><p><em>The only reason you&#8217;re balking at this is because SQL is terrible</em>. Modern SQL (particularly SQLite and Postgres) is somewhat less terrible than you imagine if only you&#8217;ll learn to use it &#8212; start with the utterly wonderful <em><a href="https://www.oreilly.com/library/view/the-art-of/0596008945/">The Art of SQL</a></em>. Still, it&#8217;s bad. If we had a modern Datalog with at least the sophistication of SQLite, this would all look almost obvious.</p><h1>Shaping the inputs to code</h1><p>Such TC code as we must write should be written such that the data given to it has <em>exactly</em> the shape needed to carry out the must-be-Turing-Complete part of the computation in the most straightforward way possible. Note that this is just good coding practice anyway.</p><p>The consequence of the above <em>following</em> might be rendering a web page to the user&#8217;s browser. We might have one function that turns values and a template into HTML. Another would take HTML and send it to the user. The data layer can invoke User Defined Functions for the first, and return the second as the result of the action that inserted the original request into the <em>requests </em>table.</p><h1>What does this <em>Mean</em>?</h1><p>I am arguing for writing software differently in two different ways:</p><ol><li><p>We should do more in the query language, limiting the use of TC programming languages to where we must; and</p></li><li><p>We should simplify the TC code we write. Since we have to travel through the sea of state in the database anyway, we should use the power of the query engine to present our TC code with <em>exactly</em> the data in <em>exactly</em> the shape that will make that code easiest to write.</p></li></ol><p>Also, our industry should support development of a better alternative to SQL (i.e. Datalog). The success of <a href="https://surrealdb.com/">SurrealDB</a>, <a href="https://supabase.com/">Supabase</a> and the like demonstrates that developers are open to change in this area. A few million dollars of VC investment could stand up a proper commercial Datalog database which might well take over the world.</p><div class="subscription-widget-wrap-editor" data-attrs="{&quot;url&quot;:&quot;https://frest.substack.com/subscribe?&quot;,&quot;text&quot;:&quot;Subscribe&quot;,&quot;language&quot;:&quot;en&quot;}" data-component-name="SubscribeWidgetToDOM"><div class="subscription-widget show-subscribe"><div class="preamble"><p class="cta-caption">Thanks for reading FREST Substack! Subscribe for free to receive new posts and support my work.</p></div><form class="subscription-widget-subscribe"><input type="email" class="email-input" name="email" placeholder="Type your email&#8230;" tabindex="-1"><input type="submit" class="button primary" value="Subscribe"><div class="fake-input-wrapper"><div class="fake-input"></div><div class="fake-button"></div></div></form></div></div>]]></content:encoded></item><item><title><![CDATA[Why now? So What?]]></title><description><![CDATA[Sequoia founder Don Valentine famously asked founders: &#8220;Why now?&#8221; and &#8220;So what?&#8221;]]></description><link>https://frest.substack.com/p/why-now-so-what</link><guid isPermaLink="false">https://frest.substack.com/p/why-now-so-what</guid><dc:creator><![CDATA[Guyren Howe]]></dc:creator><pubDate>Thu, 17 Aug 2023 02:36:23 GMT</pubDate><enclosure url="https://substackcdn.com/image/fetch/$s_!MB46!,w_256,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F5fd9d106-47e9-4c50-a15f-707b11ec91ae_550x550.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>Regarding FREST:</p><h1><strong>Why Now?</strong></h1><p>Two opportunities:</p><div class="subscription-widget-wrap-editor" data-attrs="{&quot;url&quot;:&quot;https://frest.substack.com/subscribe?&quot;,&quot;text&quot;:&quot;Subscribe&quot;,&quot;language&quot;:&quot;en&quot;}" data-component-name="SubscribeWidgetToDOM"><div class="subscription-widget show-subscribe"><div class="preamble"><p class="cta-caption">Thanks for reading FREST Substack! Subscribe for free to receive new posts and support my work.</p></div><form class="subscription-widget-subscribe"><input type="email" class="email-input" name="email" placeholder="Type your email&#8230;" tabindex="-1"><input type="submit" class="button primary" value="Subscribe"><div class="fake-input-wrapper"><div class="fake-input"></div><div class="fake-button"></div></div></form></div></div><ul><li><p>programming is inefficient and unreliable</p></li><li><p>non-programmers mostly can&#8217;t do their own computing without consulting a programmer</p></li></ul><p>The relational model offers large, relatively easy responses to both opportunities. There are a variety of other opportunities in rethinking how APIs work and how we create user interfaces.</p><h2><strong>Relations are grossly under-used</strong></h2><p>Because SQL is abominable, yet it has been our only way to operate on data relationally, the elegance, security and efficiency of the relational model in representing most of your business logic has been almost entirely ignored.</p><h2><strong>Muggles find relations intuitive</strong></h2><p>The software industry has done an awful job of providing computing abilities to non-programmers. Tools like Visual Basic, while admirable and extremely effective, still require the user to commit to learning the fundamentals of programming, which is non-trivial and challenging or impossible for many.</p><p>The really generally successful end-user computing tools are:</p><p>- Excel and</p><p>- FileMaker/Access/FoxPro/4th Dimension/&#8230;</p><p>Excel is a functional programming tool. The others are relational programming tools. They let the user employ the most useful data types (simple types, relations, lists and functions) through flexible user interface actions.</p><p>Note that functions are just a special kind of relation. Note also that the user is directly manipulating data structures, but expressed in a rich GUI, rather than through code.</p><h2>UIs from APIs</h2><p>It&#8217;s actually pretty <a href="https://frest.substack.com/p/restful-presentation">simple</a> to support a pretty flexible sort of embedding of content from multiple sources. Rather than a rigid, app-like web, we can have a component based UI model.</p><p>Rich content coexists happily within this model: Relationally, a &#8220;document&#8221; can be a single atomic value of a column, although it will often be useful to provide relational views of their contents &#8212; imagine a drawing document accessible as tables of objects, groups, layers, and so on.</p><div><hr></div><h1><strong>So What?</strong></h1><p>Imagine that all of the data in the world was available through a ten times richer version of Access or FileMaker. All of the most useful information in the world is available as tables. Everything useful you can do with SQL, you can do trivially across different data sources and APIs that don&#8217;t need to cooperate (but they can for efficiency).</p><p>Just by intuitive GUI operations, the user can slice and dice the data they are seeing and how it is represented.</p><p>For this new paradigm to be realised, data sources need only <a href="https://frest.substack.com/p/restful-presentation">implement a little more than the typical decent REST API</a> (consisting of type/metadata information and GUI representation).</p><h3>Time Travel? We can do much better than Time Travel</h3><p>There are remarkable new UI possibilities based on the seamless embeddability of FREST components. Since (internal and external) <a href="https://frest.substack.com/p/distributed-relational">API calls are presented as database updates</a>, it is trivial to just keep them. It would then be fairly easy to let the user click on anything they can see and we can show them a tree-style trace of how this thing was built, and we can allow the user to change any of the arguments to change what they&#8217;re seeing or to see a new thing.</p><div><hr></div><h1>Over to you</h1><p>What I need is the time to implement this, while still having a reasonable income. I&#8217;m pretty flexible. A part-time tech or writing gig seems most likely. But there are several possible handsome products and services that could be built on top of this, if you&#8217;re in an investing sort of mood. A tiny investment, even by typical Angel standards, would be enough.</p><p>If you can help change the world by supporting me, please get in touch!</p><div class="subscription-widget-wrap-editor" data-attrs="{&quot;url&quot;:&quot;https://frest.substack.com/subscribe?&quot;,&quot;text&quot;:&quot;Subscribe&quot;,&quot;language&quot;:&quot;en&quot;}" data-component-name="SubscribeWidgetToDOM"><div class="subscription-widget show-subscribe"><div class="preamble"><p class="cta-caption">Thanks for reading FREST Substack! Subscribe for free to receive new posts and support my work.</p></div><form class="subscription-widget-subscribe"><input type="email" class="email-input" name="email" placeholder="Type your email&#8230;" tabindex="-1"><input type="submit" class="button primary" value="Subscribe"><div class="fake-input-wrapper"><div class="fake-input"></div><div class="fake-button"></div></div></form></div></div>]]></content:encoded></item><item><title><![CDATA[RESTful presentation]]></title><description><![CDATA[An elegant tool, for a more civilised age]]></description><link>https://frest.substack.com/p/restful-presentation</link><guid isPermaLink="false">https://frest.substack.com/p/restful-presentation</guid><dc:creator><![CDATA[Guyren Howe]]></dc:creator><pubDate>Wed, 16 Aug 2023 07:31:04 GMT</pubDate><enclosure url="https://substackcdn.com/image/fetch/$s_!MB46!,w_256,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F5fd9d106-47e9-4c50-a15f-707b11ec91ae_550x550.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>One ASPECT of FREST can be nicely separated and used on its own, to great benefit. That is the presentation model.</p><p>FREST is based on truly RESTful values at URLs having representations. It adds some more expectations that produce a very flexible protocol for content embedding, where your mind map can contain a drawing, within which is a spreadsheet, in one cell of which is another (or the same) mind map.</p><div class="subscription-widget-wrap-editor" data-attrs="{&quot;url&quot;:&quot;https://frest.substack.com/subscribe?&quot;,&quot;text&quot;:&quot;Subscribe&quot;,&quot;language&quot;:&quot;en&quot;}" data-component-name="SubscribeWidgetToDOM"><div class="subscription-widget show-subscribe"><div class="preamble"><p class="cta-caption">Thanks for reading FREST Substack! Subscribe for free to receive new posts and support my work.</p></div><form class="subscription-widget-subscribe"><input type="email" class="email-input" name="email" placeholder="Type your email&#8230;" tabindex="-1"><input type="submit" class="button primary" value="Subscribe"><div class="fake-input-wrapper"><div class="fake-input"></div><div class="fake-button"></div></div></form></div></div><p>It&#8217;s really quite simple.</p><p>Every FREST endpoint&#8217;s value can be presented in the following ways:</p><ul><li><p>icon</p></li><li><p>cell</p></li><li><p>row</p></li><li><p>table header</p></li><li><p>full</p></li></ul><p>There will be some other, optional ones.</p><p>Every FREST endpoint supports a request for its metadata (eg using a HTTP OPTIONS request). Among other things, that metadata includes mustache templates for each of the presentations.</p><p>Every FREST endpoint can be retrieved in a JSON representation. That JSON representation can be fed into any of the Mustache templates to produce a representation of the value for each purpose.</p><p>Neither JSON nor well-crafted Mustache templates (meaning, ones from which any sort of &lt;script&gt; tag or property are stripped) can represent anything Turing Complete, nor can they express any kind of interaction with the local system. There is always the option of displaying suspicious content in a frame.</p><p>The metadata also provides details of known major endpoints that take an argument of this type. We can use this to give every UI element a menu/list of functions when it is right-clicked on.</p><p>This is a simple yet profound improvement to an API that supports it &#8212; that API comes with a UI for free, and supports embedding in other content.</p><div class="subscription-widget-wrap-editor" data-attrs="{&quot;url&quot;:&quot;https://frest.substack.com/subscribe?&quot;,&quot;text&quot;:&quot;Subscribe&quot;,&quot;language&quot;:&quot;en&quot;}" data-component-name="SubscribeWidgetToDOM"><div class="subscription-widget show-subscribe"><div class="preamble"><p class="cta-caption">Thanks for reading FREST Substack! Subscribe for free to receive new posts and support my work.</p></div><form class="subscription-widget-subscribe"><input type="email" class="email-input" name="email" placeholder="Type your email&#8230;" tabindex="-1"><input type="submit" class="button primary" value="Subscribe"><div class="fake-input-wrapper"><div class="fake-input"></div><div class="fake-button"></div></div></form></div></div>]]></content:encoded></item><item><title><![CDATA[Join my chat]]></title><description><![CDATA[A private space for us to converse and connect]]></description><link>https://frest.substack.com/p/join-my-chat</link><guid isPermaLink="false">https://frest.substack.com/p/join-my-chat</guid><dc:creator><![CDATA[Guyren Howe]]></dc:creator><pubDate>Wed, 16 Aug 2023 06:44:03 GMT</pubDate><enclosure url="https://substackcdn.com/image/fetch/$s_!2H2-!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fbucketeer-e05bbc84-baa3-437e-9518-adb32be77984.s3.amazonaws.com%2Fpublic%2Fimages%2F9a23d49f-76bd-4f75-baac-0ae5733774bd_1456x743.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>Today I&#8217;m announcing a brand new addition to my Substack publication: the FREST Substack subscriber chat.</p><p>This is a conversation space in the Substack app that I set up exclusively for my subscribers &#8212; kind of like a group chat or live hangout. I&#8217;ll post short prompts, thoughts, and updates that come my way, and you can jump into the discussion. </p><p><strong>To join our chat, you&#8217;ll need to download the <a href="https://substack.com/app/app-store-redirect">Substack app</a>, now available for both iOS and Android.</strong> Chats are sent via the app, not email, so turn on push notifications so you don&#8217;t miss conversation as it happens.</p><div><hr></div><h2>How to get started</h2><ol><li><p><strong>Download the app by clicking <a href="https://substack.com/app/app-store-redirect">this link</a> or the button below.</strong> Substack Chat is now available on both iOS and Android.</p></li></ol><p class="button-wrapper" data-attrs="{&quot;url&quot;:&quot;https://substack.com/app/app-store-redirect&quot;,&quot;text&quot;:&quot;Get app&quot;,&quot;action&quot;:null,&quot;class&quot;:&quot;button-wrapper&quot;}" data-component-name="ButtonCreateButton"><a class="button primary button-wrapper" href="https://substack.com/app/app-store-redirect"><span>Get app</span></a></p><ol start="2"><li><p><strong>Open the app and tap the Chat icon.</strong> It looks like two bubbles in the bottom bar, and you&#8217;ll see a row for my chat inside.</p></li></ol><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!2H2-!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fbucketeer-e05bbc84-baa3-437e-9518-adb32be77984.s3.amazonaws.com%2Fpublic%2Fimages%2F9a23d49f-76bd-4f75-baac-0ae5733774bd_1456x743.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!2H2-!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fbucketeer-e05bbc84-baa3-437e-9518-adb32be77984.s3.amazonaws.com%2Fpublic%2Fimages%2F9a23d49f-76bd-4f75-baac-0ae5733774bd_1456x743.png 424w, https://substackcdn.com/image/fetch/$s_!2H2-!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fbucketeer-e05bbc84-baa3-437e-9518-adb32be77984.s3.amazonaws.com%2Fpublic%2Fimages%2F9a23d49f-76bd-4f75-baac-0ae5733774bd_1456x743.png 848w, https://substackcdn.com/image/fetch/$s_!2H2-!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fbucketeer-e05bbc84-baa3-437e-9518-adb32be77984.s3.amazonaws.com%2Fpublic%2Fimages%2F9a23d49f-76bd-4f75-baac-0ae5733774bd_1456x743.png 1272w, https://substackcdn.com/image/fetch/$s_!2H2-!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fbucketeer-e05bbc84-baa3-437e-9518-adb32be77984.s3.amazonaws.com%2Fpublic%2Fimages%2F9a23d49f-76bd-4f75-baac-0ae5733774bd_1456x743.png 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!2H2-!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fbucketeer-e05bbc84-baa3-437e-9518-adb32be77984.s3.amazonaws.com%2Fpublic%2Fimages%2F9a23d49f-76bd-4f75-baac-0ae5733774bd_1456x743.png" width="542" height="276.5837912087912" data-attrs="{&quot;src&quot;:&quot;https://bucketeer-e05bbc84-baa3-437e-9518-adb32be77984.s3.amazonaws.com/public/images/9a23d49f-76bd-4f75-baac-0ae5733774bd_1456x743.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:743,&quot;width&quot;:1456,&quot;resizeWidth&quot;:542,&quot;bytes&quot;:501468,&quot;alt&quot;:&quot;&quot;,&quot;title&quot;:null,&quot;type&quot;:&quot;image/png&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:true,&quot;internalRedirect&quot;:null,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" title="" srcset="https://substackcdn.com/image/fetch/$s_!2H2-!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fbucketeer-e05bbc84-baa3-437e-9518-adb32be77984.s3.amazonaws.com%2Fpublic%2Fimages%2F9a23d49f-76bd-4f75-baac-0ae5733774bd_1456x743.png 424w, https://substackcdn.com/image/fetch/$s_!2H2-!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fbucketeer-e05bbc84-baa3-437e-9518-adb32be77984.s3.amazonaws.com%2Fpublic%2Fimages%2F9a23d49f-76bd-4f75-baac-0ae5733774bd_1456x743.png 848w, https://substackcdn.com/image/fetch/$s_!2H2-!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fbucketeer-e05bbc84-baa3-437e-9518-adb32be77984.s3.amazonaws.com%2Fpublic%2Fimages%2F9a23d49f-76bd-4f75-baac-0ae5733774bd_1456x743.png 1272w, https://substackcdn.com/image/fetch/$s_!2H2-!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fbucketeer-e05bbc84-baa3-437e-9518-adb32be77984.s3.amazonaws.com%2Fpublic%2Fimages%2F9a23d49f-76bd-4f75-baac-0ae5733774bd_1456x743.png 1456w" sizes="100vw" loading="lazy" fetchpriority="high"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container restack-image"><svg role="img" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg"><g><title></title><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container view-image"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><ol start="3"><li><p><strong>That&#8217;s it!</strong> Jump into my thread to say hi, and if you have any issues, check out <a href="https://support.substack.com/hc/en-us/sections/360007461791-Frequently-Asked-Questions">Substack&#8217;s FAQ</a>.</p></li></ol><p class="button-wrapper" data-attrs="{&quot;url&quot;:&quot;https://open.substack.com/pub/frest/chat&quot;,&quot;text&quot;:&quot;Join chat&quot;,&quot;action&quot;:null,&quot;class&quot;:null}" data-component-name="ButtonCreateButton"><a class="button primary" href="https://open.substack.com/pub/frest/chat"><span>Join chat</span></a></p>]]></content:encoded></item><item><title><![CDATA[How the CIA Ruined Programming]]></title><description><![CDATA[We&#8217;re only just now starting to recover. And we have a long way to go.]]></description><link>https://frest.substack.com/p/how-the-cia-ruined-programming</link><guid isPermaLink="false">https://frest.substack.com/p/how-the-cia-ruined-programming</guid><dc:creator><![CDATA[Guyren Howe]]></dc:creator><pubDate>Wed, 05 Apr 2023 04:24:59 GMT</pubDate><enclosure url="https://substack-post-media.s3.amazonaws.com/public/images/165d6df7-d378-4fcb-bcd9-6a7027d5f39f_2048x2048.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>My entire adult life, since I first learned to program, I felt like Neo in the Matrix:</p><blockquote><p><strong>&#8230;there's something wrong with the world.</strong> <strong>You don't know what it is, but it's there.</strong> <strong>Like a splinter in your mind - driving you mad.</strong></p></blockquote><p>I felt that this couldn&#8217;t be right, that programming surely doesn&#8217;t have to be so hard, so tedious, so error prone. <em>Particularly</em> business software. Every time I had to hand-roll another HTML form, I died a little inside. Shouldn&#8217;t the computer be doing more of this <em>for</em> me?</p><div class="subscription-widget-wrap-editor" data-attrs="{&quot;url&quot;:&quot;https://frest.substack.com/subscribe?&quot;,&quot;text&quot;:&quot;Subscribe&quot;,&quot;language&quot;:&quot;en&quot;}" data-component-name="SubscribeWidgetToDOM"><div class="subscription-widget show-subscribe"><div class="preamble"><p class="cta-caption">Thanks for reading FREST Substack! Subscribe for free to receive new posts and support my work.</p></div><form class="subscription-widget-subscribe"><input type="email" class="email-input" name="email" placeholder="Type your email&#8230;" tabindex="-1"><input type="submit" class="button primary" value="Subscribe"><div class="fake-input-wrapper"><div class="fake-input"></div><div class="fake-button"></div></div></form></div></div><h1>The answer was there all the time.</h1><p>I&#8217;ve investigated every alternative programming approach I could find. All of the approaches that showed promise were &#8220;declarative&#8221; &#8212; they let you solve problems without having to write Turing complete code. GUI design using a GUI editor like Visual Basic. Direct data manipulation using Access or FileMaker. HTML is declarative, as is CSS.</p><p>But they&#8217;re all piecemeal and narrow efforts. What I realise now that I always wanted was a more general approach to making more of programming declarative.</p><p>The tragedy is that this approach exists. It is well-known, well-studied, simple, effective, and <em>old</em>, insofar as such things are in our industry. This approach is the <a href="https://www.seas.upenn.edu/~zives/03f/cis550/codd.pdf">relational model</a>, first described by E. F. Codd in 1969.</p><p>The theory underlying the relational model starts with how a small collection of simple and well-known data manipulation algorithms can be assembled into a declarative First Order Logic<a class="footnote-anchor" data-component-name="FootnoteAnchorToDOM" id="footnote-anchor-1" href="#footnote-1" target="_self">1</a> engine. This is remarkable. The theory leads not only to a simple way to implement a logic engine, but also to how to optimise its performance.</p><p>We <em>should</em> be using this universal approach to declarative programming <em>everywhere</em>. Just about every place in your code that&#8217;s a case statement or a lookup in a Map should be a much richer relational thing. But it&#8217;s not, and it&#8217;s because of the CIA.</p><h1>Because SQL. Because the CIA.</h1><p>I don&#8217;t actually blame anyone at IBM or Oracle or the CIA (the principle players in this story) for <a href="https://gizmodo.com/larry-ellisons-oracle-started-as-a-cia-project-1636592238">what happened</a>. At the time, the decisions everyone made were reasonable.</p><p>The timeline goes something like this:</p><ul><li><p>1969: Codd publishes <em>A Relational Model&#8230;</em></p></li><li><p>1974: Chamberlain and Boyce, working at IBM, <a href="https://web.archive.org/web/20070926212100/http://www.almaden.ibm.com/cs/people/chamberlin/sequel-1974.pdf">first describe SEQUEL</a>, which we now know as SQL</p></li><li><p>1979 IBM starts producing products implementing SEQUEL.</p></li><li><p>1979 Relational Software (now ORACLE) produces Oracle, aiming to sell it to the CIA, the Navy, and other US government agencies</p></li></ul><p>The name <em>Oracle</em> <a href="https://in.rediff.com/money/2006/aug/11oracle.htm">came from a </a><em><a href="https://in.rediff.com/money/2006/aug/11oracle.htm">CIA code name</a></em>. And targetting US government agencies appears to have been based at least in part on <a href="http://www.nytimes.com/2002/01/31/opinion/31iht-edlarry_ed3_.html">political considerations</a> of Larry Ellison, the founder of Oracle.</p><p>Again, at the time it was developed, SEQUEL was a reasonable, even somewhat great idea. Programming languages are still developing rapidly today. In the 70s, it was popular &#8212; and not clearly wrong &#8212; to develop programming languages that resemble English (hello, COBOL!). Bear in mind that the cutting edge in user interfaces at the time was the CRT terminal, which <a href="https://en.wikipedia.org/wiki/IBM_2260">first appeared in 1964</a> and which was only becoming ubiquitous around the time Codd was publishing about relations.</p><p>If you wanted non-programmers to access data, you wanted a simple language that could be used from a terminal. This was the original intent of SQL. For that purpose, at that time, it was pretty good!</p><p>At the time SEQUEL was being described, there were other, much better database languages being developed for programmers. Datalog, the best such language started to come together at a <a href="https://archive.org/details/logicdatabases0000symp">conference in 1977</a>.</p><p>Datalog and SQL bear almost no resemblance. It&#8217;s hard to imagine they&#8217;re based on the same underlying theory. Datalog is terse, elegant and flexible, none of which can be said of SQL.</p><p>What happened is that the founder of Oracle, at the time when database standards and technologies were a movable subject, used guaranteed and generous support from the CIA and other agencies to achieve dominance with his SQL-based product.</p><h1>What might have been&#8230;</h1><p>If your only exposure to the relational model is SQL, go read about <a href="https://archive.org/details/logicdatabases0000symp">Datalog</a>. Comparing it to SQL is the best way to understand the ghetto that SQL has driven the computer industry into.</p><p>Where we are today is that the only significant way we employ the relational model is through SQL. Along with the terrible, no good, very bad language itself comes a bunch of other fairly arbitrary assumptions arising from the original use of such tools in banks and the like. So an SQL database is a heavyweight, local file-based, bureaucratic thing designed primarily around concerns of data integrity in the face of multiple users (transactions, MVCC and the like).</p><p>These are fine features for a lot of use cases, but are unnecessary bloat for a lot of others. No wonder the NoSQL movement has developed other products around other storage models, such as not needing a predefined schema, or distributed and eventually consistent stores. Such a pity, then, that their rejection of the arbitrary rigidities of SQL led them to abandon the relational model at the same time. Such wasted effort!</p><p>Imagine instead that we had standardised on Datalog. Your favourite programming language lets you manipulate declarative logic as easily as you do arrays and strings.</p><p>Relational models are naturally distributed:</p><div class="embedded-post-wrap" data-attrs="{&quot;id&quot;:103651486,&quot;url&quot;:&quot;https://frest.substack.com/p/distributed-relational&quot;,&quot;publication_id&quot;:1405522,&quot;publication_name&quot;:&quot;FREST Substack&quot;,&quot;publication_logo_url&quot;:&quot;https://substackcdn.com/image/fetch/f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F5fd9d106-47e9-4c50-a15f-707b11ec91ae_550x550.png&quot;,&quot;title&quot;:&quot;Distributed == Relational&quot;,&quot;truncated_body_text&quot;:&quot;The style of distributed architecture described here is at the heart of the design of FREST. The basic story Assume we have four systems. System A wants to invoke a function on system D, but the function in D needs arguments from systems B and C. The typical pragmatic solution is for A to&quot;,&quot;date&quot;:&quot;2023-02-21T06:52:34.001Z&quot;,&quot;like_count&quot;:2,&quot;comment_count&quot;:0,&quot;bylines&quot;:[{&quot;id&quot;:45307374,&quot;name&quot;:&quot;Guyren Howe&quot;,&quot;previous_name&quot;:null,&quot;photo_url&quot;:&quot;https://substackcdn.com/image/fetch/f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Ff1af23bc-ef15-4253-873b-ce076cbaadbb_144x144.png&quot;,&quot;bio&quot;:&quot;Software engineer/philosopher&quot;,&quot;profile_set_up_at&quot;:&quot;2023-02-10T06:03:28.537Z&quot;,&quot;publicationUsers&quot;:[{&quot;id&quot;:1367741,&quot;user_id&quot;:45307374,&quot;publication_id&quot;:1405522,&quot;role&quot;:&quot;admin&quot;,&quot;public&quot;:true,&quot;is_primary&quot;:false,&quot;publication&quot;:{&quot;id&quot;:1405522,&quot;name&quot;:&quot;FREST Substack&quot;,&quot;subdomain&quot;:&quot;frest&quot;,&quot;custom_domain&quot;:null,&quot;custom_domain_optional&quot;:false,&quot;hero_text&quot;:&quot;The substack for FREST: computing for everyone&quot;,&quot;logo_url&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/5fd9d106-47e9-4c50-a15f-707b11ec91ae_550x550.png&quot;,&quot;author_id&quot;:45307374,&quot;theme_var_background_pop&quot;:&quot;#6B26FF&quot;,&quot;created_at&quot;:&quot;2023-02-10T06:04:01.594Z&quot;,&quot;rss_website_url&quot;:null,&quot;email_from_name&quot;:null,&quot;copyright&quot;:&quot;Guyren Howe&quot;,&quot;founding_plan_name&quot;:null,&quot;community_enabled&quot;:true,&quot;invite_only&quot;:false,&quot;payments_state&quot;:&quot;disabled&quot;}}],&quot;twitter_screen_name&quot;:&quot;unclouded&quot;,&quot;is_guest&quot;:false,&quot;bestseller_tier&quot;:null}],&quot;utm_campaign&quot;:null,&quot;belowTheFold&quot;:true,&quot;type&quot;:&quot;newsletter&quot;,&quot;language&quot;:&quot;en&quot;}" data-component-name="EmbeddedPostToDOM"><a class="embedded-post" native="true" href="https://frest.substack.com/p/distributed-relational?utm_source=substack&amp;utm_campaign=post_embed&amp;utm_medium=web"><div class="embedded-post-header"><img class="embedded-post-publication-logo" src="https://substackcdn.com/image/fetch/$s_!MB46!,w_56,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F5fd9d106-47e9-4c50-a15f-707b11ec91ae_550x550.png" loading="lazy"><span class="embedded-post-publication-name">FREST Substack</span></div><div class="embedded-post-title-wrapper"><div class="embedded-post-title">Distributed == Relational</div></div><div class="embedded-post-body">The style of distributed architecture described here is at the heart of the design of FREST. The basic story Assume we have four systems. System A wants to invoke a function on system D, but the function in D needs arguments from systems B and C. The typical pragmatic solution is for A to&#8230;</div><div class="embedded-post-cta-wrapper"><span class="embedded-post-cta">Read more</span></div><div class="embedded-post-meta">3 years ago &#183; 2 likes &#183; Guyren Howe</div></a></div><p>You would still have a lot of the benefits that SQL databases have: logic represented relationally can be trivially employed from all the programming languages, for example.</p><p>Wherever in your business application you are enforcing rules, you can do so more naturally and entirely declaratively, making those rules more sharable and malleable.</p><p>Most of a modern business application can naturally be represented in First Order Logic. Such representations are deliberately not Turing complete &#8212; meaning you can&#8217;t have off-by-one errors or null dereference errors or any of the problems that arise when feeble human minds try to craft the behaviour of a Turing complete system.</p><p>Security concerns are naturally expressed and enforced declaratively (even SQL can do this much better than your ad hoc code can).</p><p>In short, pervasive and facile deployment of the relational model in software engineering would be a very different software engineering &#8212; simpler, easier, more reliable, more easily distributed, and more secure.</p><p>And we would be in that world today had Larry Ellison not done a deal with the CIA in the 70s.</p><div class="footnote" data-component-name="FootnoteToDOM"><a id="footnote-1" href="#footnote-anchor-1" class="footnote-number" contenteditable="false" target="_self">1</a><div class="footnote-content"><p>Waving my hands only slightly, First Order Logic is the richest logic possible such that it is guaranteed that a computer can work out every possible consequences of the facts and rules in the system. Anything richer, and you run into the Halting Problem/G&#246;del&#8217;s Theorem issues.</p></div></div>]]></content:encoded></item><item><title><![CDATA[What is FREST?]]></title><description><![CDATA[FREST is simpler computing for ordinary folks and programmers alike.]]></description><link>https://frest.substack.com/p/what-is-frest</link><guid isPermaLink="false">https://frest.substack.com/p/what-is-frest</guid><dc:creator><![CDATA[Guyren Howe]]></dc:creator><pubDate>Wed, 08 Mar 2023 04:26:41 GMT</pubDate><enclosure url="https://substackcdn.com/image/fetch/f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F5fd9d106-47e9-4c50-a15f-707b11ec91ae_550x550.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>Frest is currently a software design philosophy, or you might say a &#8220;pattern&#8221;. Code will be coming in due time.</p><p>Computing today is composed of finely interwoven layers of <a href="https://en.wikipedia.org/wiki/Transmission_Control_Protocol?wprov=sfti1">great</a> <a href="https://en.wikipedia.org/wiki/Relational_model">abstractions</a> and <a href="https://en.wikipedia.org/wiki/File_system?wprov=sfti1">horrible</a> <a href="https://en.wikipedia.org/wiki/SQL?wprov=sfti1">cruft</a>. The relational model and SQL. URLs and the filesystem. Direct manipulation of abstractions with a nice GUI, and rolling yet another HTML form by hand.</p><p>FREST isn&#8217;t radical. Or rather, being radical isn&#8217;t really radical. The history of the computing industry is one of radical experimentation leading to radical change, in a kind of punctuated continuum. The radical idea becomes mainstream after the crazy pioneers have proven it.</p><p>Today&#8217;s most popular programming languages put ideas that were fringe and theoretical just a few years ago right in your face. Advanced type systems. The Actor model of parallelism. Synchronisation of distributed data. AI.</p><p>Before Typescript, advanced type systems were largely the stuff of academic papers. A couple of years after Typescript&#8217;s release, advanced type systems are suddenly everywhere.</p><p>The computing industry is in a constant struggle between worse-is-better and, well, better-is-better. There are both good and not-so-good economic, cultural and engineering reasons for our industry&#8217;s conservatism.</p><p>This project seeks to go where the puck is going <em>much faster</em>. There are a small number of mature ideas we should adopt in one go. In particular:</p><h3>1. The network is the basis</h3><p>Everything important is addressable and serialisable. Every field in every table in your database. Every high-level function in your business model. Every conceptually distinct element in your UI.</p><p>In FREST, the filesystem isn&#8217;t the starting point. The filesystem isn&#8217;t even necessary<a class="footnote-anchor" data-component-name="FootnoteAnchorToDOM" id="footnote-anchor-1" href="#footnote-1" target="_self">1</a>. Addressable content is the better way. Even <em>within single applications</em>, major articulation points should be connected by URLs<a class="footnote-anchor" data-component-name="FootnoteAnchorToDOM" id="footnote-anchor-2" href="#footnote-2" target="_self">2</a>.</p><p>Even code isn&#8217;t loaded from a file; it&#8217;s <em><a href="https://deno.land/manual@v1.30.3/basics/modules#remote-import">at an address</a></em>. <em>Everything is at an address</em>.</p><h3>2. Direct manipulation of abstractions</h3><p>The most successful products for non-programmers to do computing are those which allow folks to directly manipulate highly expressive abstractions.</p><p>FileMaker and Access (and these days, say, Notion) are end user relational data bases with some functional features. It turns out that when given a nice GUI for manipulating such things, non-programmers can employ relational database concepts with considerable facility.</p><p>Then, consider spreadsheets. Spreadsheets let non-programmers do functional programming. Spreadsheet jockies don&#8217;t use variables; they use <em>values at addresses</em>. Few have heralded the amazing news that Microsoft recently <a href="https://www.ablebits.com/office-addins-blog/write-recursive-lambda-function-excel/">added an easy to use Y combinator to Excel</a>. Excel was already the best product Microsoft ever made<a class="footnote-anchor" data-component-name="FootnoteAnchorToDOM" id="footnote-anchor-3" href="#footnote-3" target="_self">3</a>.</p><p>Non-programmers, it turns out, can do effective functional and relational programming with minimal training. Why did our industry keep trying to push &#8220;simple&#8221; imperative programming paradigms as computing for non-programmers? Visual Basic is fine, but where&#8217;s Visual Scheme?</p><p>One of the main questions FREST seeks to ask and answer is: What other highly expressive abstractions can we present to non-programmers in an intuitive way?</p><p>The FREST project&#8217;s bet is: quite a few: <em>actors</em>, <em>encryption</em>, <em>append-only data stores</em>, <em>logic</em>,&nbsp;&#8230;</p><p>Note that the features of a typical business application that are amenable to this approach are almost precisely those that compose the grinding tedium of everyday corporate programming. So this is <em>very much</em> about making lives easier for developers as well.</p><h3>3. There&#8217;s always a GUI. Always.</h3><p>Excel and FileMaker let users directly manipulate abstractions, always with an appropriate GUI presentation (even if it&#8217;s just a textual function editor).</p><p>Everything addressable in a FREST system is able to present itself in any of a small number of composable ways, such as:</p><ul><li><p>icon</p></li><li><p>descriptive link</p></li><li><p>table cell</p></li><li><p>table row</p></li><li><p>table or form header</p></li><li><p>&#8220;full page&#8221;</p></li></ul><p>FREST is agnostic about its presentation layer, just requiring that there should be one. Most obviously in the first case, FREST would probably present HTML versions of these things, perhaps as Web Components.</p><p>This sort of collection of presentations is sufficient for robust and subtle embedding. For example, a table can be embedded as an icon or descriptive link in some richer context &#8212; an outliner, or a web page element, or an object in a drawing program. Within those UI paradigms, the content can be expanded and manipulated.</p><p>All that a programmer has to do if they introduce a new type is to provide those presentations of it. Usually, this can be done through existing presentations. A programmer who writes a function need only describe its input and output types (and any security constraints) and it will be available to use anywhere applicable, through the GUI or the API.</p><p>The industry has tried several times (eg Apple&#8217;s OpenDoc and Microsoft&#8217;s COM Objects) to enable embeddable live content. FREST bets that modern web and other technologies provide an opportunity to do it right this time.</p><p>Consider introducing a really novel content and presentation type &#8212; an outliner, say. The author of the outliner can write the outline presentation and manipulation logic and storage, and then trivially support <em>any other content at all</em> within the outline. An outline node could be a table that can be collapsed to an icon and expanded at will.</p><h3>4. The UI is always navigable</h3><p>FREST content might present all sorts of interesting buttons, menus and other chrome for interacting with content. But at the very least, every FREST value responds to a right click or long press or the like, presenting a <em>meta</em> view of the thing the gesture is addressed to.</p><p>There will in time likely be all sorts of semantic, pragmatic and other content on the meta UI. But at the very least initially, the meta view will allow the user to discover and invoke all the things that can be done with the value. It lets the user switch between the standard presentation and others (a table of data might be viewable as a chart, for example). It lets the user choose between functions that take the value as an argument. This might create a new single value, or a new column in a table.</p><h3>5. Functional Programming</h3><p>If ordinary folk can do functional programming, and professional programming languages are all turning into LISP/Scheme/ML, then FREST will too. The addressable functions of which a FREST system is composed include functional programming constructs that allow composition of existing things into new things.</p><p>I&#8217;ll discuss this in more detail later, but briefly:</p><ul><li><p><a href="https://en.wikipedia.org/wiki/Partial_application">partial application</a> is central;</p></li><li><p>functions present themselves as forms;</p></li><li><p>user interface elaborations are preformed through functional operations, for example, starting from a table of addresses, a user can add a column of maps of those addresses, which is to say the user applies a (functional) <em>map</em> operation to the data underlying the table;</p></li><li><p>API operations are functional programming operations. API clients can create new values, including functions</p></li><li><p>API operations are asynchronous. If a client of an API wants a value, it passes an address for the return value to be sent to. FREST is a continuation-passing architecture. This means, for example, that any function call can be a remote call back.</p></li></ul><h3>6. Relations</h3><p>Functions are user-friendly, but relations are much more so!</p><p>Try to forget SQL. I&#8217;ll have a whole &#8217;nother rant about that later. Just think in terms of simple relational operations such as you might use through the user interface in Access or FileMaker. Joining relations, querying them, obtaining new values through functions of old ones.</p><p>SQL has poisoned our entire industry. Relations are elegant and beautiful and it is possible and fruitful to represent much more of our software through relations. Any aspect of software represented relationally is simpler, easier to get correct, and more fungible than it is through a traditional programming language.</p><p>FREST says APIs should be relational. GraphQL is our industry already heading in this direction. The result is APIs of much greater subtlety, flexibility and efficiency. More on that soon.</p><h3>7. Composition and Security</h3><p>Security and composition are two sides of the same idea in FREST. This model will be described in a separate piece soon.</p><p>Briefly, a client of a FREST system is faced with values (which might be functions, rows, whole relations, addresses, anything at all) that are missing, that are provided as a default, or that are fixed. The client can provide or override any value that isn&#8217;t fixed.</p><p>This is why even the internal relationships within a FREST system are accessed through addresses: so they can be externally overridden.</p><p>Many things will have a default value that can be changed. Most of the logic and layouts of UI generation will be a default. Many behaviours will be defaults. The user will be able to right-click something in their UI, navigate all the defaults that produced that thing, and change any of them.</p><p>This is where the relations comes in. To take just one example, the sequence of operations that occur when you add an item to your shopping basket should be described in a relation, not buried in code.</p><p>Imagine if you could customise your Amazon user interface by right clicking and removing things, surfacing things that are hidden, changing the actions when an item is added to a shopping basket, and so on. FREST promises to make software like that easier to write than the current rigid Amazon experience was to write in the first place.</p><h1>In Summary</h1><p>FREST is a set of protocols that define network-first, relation-first, GUI-first data manipulation and programming environments.</p><p>Work is underway on example implementations in more than one language.</p><p>FREST is about to change everything. Stay tuned.</p><div><hr></div><div class="subscription-widget-wrap-editor" data-attrs="{&quot;url&quot;:&quot;https://frest.substack.com/subscribe?&quot;,&quot;text&quot;:&quot;Subscribe&quot;,&quot;language&quot;:&quot;en&quot;}" data-component-name="SubscribeWidgetToDOM"><div class="subscription-widget show-subscribe"><div class="preamble"><p class="cta-caption">Thanks for reading the FREST Substack! Subscribe for free to receive new posts and support my work.</p></div><form class="subscription-widget-subscribe"><input type="email" class="email-input" name="email" placeholder="Type your email&#8230;" tabindex="-1"><input type="submit" class="button primary" value="Subscribe"><div class="fake-input-wrapper"><div class="fake-input"></div><div class="fake-button"></div></div></form></div></div><p></p><div class="footnote" data-component-name="FootnoteToDOM"><a id="footnote-1" href="#footnote-anchor-1" class="footnote-number" contenteditable="false" target="_self">1</a><div class="footnote-content"><p>In all of this, we&#8217;ll build on what we have, at least initially, so early FREST systems will exploit filesystems and SQL databases and the lot. But systems have been built without a traditional filesystem. A great example: BeOS&#8217;s filesystem was also a relational database, and was much the better for it.</p></div></div><div class="footnote" data-component-name="FootnoteToDOM"><a id="footnote-2" href="#footnote-anchor-2" class="footnote-number" contenteditable="false" target="_self">2</a><div class="footnote-content"><p>We&#8217;ll discuss why in due course.</p></div></div><div class="footnote" data-component-name="FootnoteToDOM"><a id="footnote-3" href="#footnote-anchor-3" class="footnote-number" contenteditable="false" target="_self">3</a><div class="footnote-content"><p>Excel is quite a bit of engineering, but why can&#8217;t anyone make a better word processor than Word? I just need a nice UI with tables and basic layout and stuff, and strong style sheets. Word is just okay on these criteria, but everything else I&#8217;ve ever used isn&#8217;t better.</p></div></div>]]></content:encoded></item><item><title><![CDATA[Distributed == Relational]]></title><description><![CDATA[Perhaps surprisingly, distributed systems are naturally relational]]></description><link>https://frest.substack.com/p/distributed-relational</link><guid isPermaLink="false">https://frest.substack.com/p/distributed-relational</guid><dc:creator><![CDATA[Guyren Howe]]></dc:creator><pubDate>Tue, 21 Feb 2023 06:52:34 GMT</pubDate><enclosure url="https://substackcdn.com/image/fetch/$s_!4US3!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd71285ca-168f-4ec1-8180-cadb2c762e0f_698x484.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>The style of distributed architecture described here is at the heart of the design of FREST.</p><h1>The basic story</h1><p>Assume we have four systems. System A wants to invoke a function on system D, but the function in D needs arguments from systems B and C.</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!4US3!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd71285ca-168f-4ec1-8180-cadb2c762e0f_698x484.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!4US3!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd71285ca-168f-4ec1-8180-cadb2c762e0f_698x484.png 424w, https://substackcdn.com/image/fetch/$s_!4US3!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd71285ca-168f-4ec1-8180-cadb2c762e0f_698x484.png 848w, https://substackcdn.com/image/fetch/$s_!4US3!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd71285ca-168f-4ec1-8180-cadb2c762e0f_698x484.png 1272w, https://substackcdn.com/image/fetch/$s_!4US3!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd71285ca-168f-4ec1-8180-cadb2c762e0f_698x484.png 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!4US3!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd71285ca-168f-4ec1-8180-cadb2c762e0f_698x484.png" width="698" height="484" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/d71285ca-168f-4ec1-8180-cadb2c762e0f_698x484.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:484,&quot;width&quot;:698,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:37833,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/png&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:false,&quot;topImage&quot;:true,&quot;internalRedirect&quot;:null,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!4US3!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd71285ca-168f-4ec1-8180-cadb2c762e0f_698x484.png 424w, https://substackcdn.com/image/fetch/$s_!4US3!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd71285ca-168f-4ec1-8180-cadb2c762e0f_698x484.png 848w, https://substackcdn.com/image/fetch/$s_!4US3!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd71285ca-168f-4ec1-8180-cadb2c762e0f_698x484.png 1272w, https://substackcdn.com/image/fetch/$s_!4US3!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd71285ca-168f-4ec1-8180-cadb2c762e0f_698x484.png 1456w" sizes="100vw" fetchpriority="high"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container restack-image"><svg role="img" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg"><g><title></title><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container view-image"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><p>The typical pragmatic solution is for A to <em>sequentially</em> request the information from B and C, and then send all the arguments to D.</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!JP77!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fc3205d9e-d05f-4bb4-9fab-1b63eb51572c_698x508.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!JP77!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fc3205d9e-d05f-4bb4-9fab-1b63eb51572c_698x508.png 424w, https://substackcdn.com/image/fetch/$s_!JP77!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fc3205d9e-d05f-4bb4-9fab-1b63eb51572c_698x508.png 848w, https://substackcdn.com/image/fetch/$s_!JP77!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fc3205d9e-d05f-4bb4-9fab-1b63eb51572c_698x508.png 1272w, https://substackcdn.com/image/fetch/$s_!JP77!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fc3205d9e-d05f-4bb4-9fab-1b63eb51572c_698x508.png 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!JP77!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fc3205d9e-d05f-4bb4-9fab-1b63eb51572c_698x508.png" width="698" height="508" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/c3205d9e-d05f-4bb4-9fab-1b63eb51572c_698x508.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:508,&quot;width&quot;:698,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:50115,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/png&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:false,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:null,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!JP77!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fc3205d9e-d05f-4bb4-9fab-1b63eb51572c_698x508.png 424w, https://substackcdn.com/image/fetch/$s_!JP77!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fc3205d9e-d05f-4bb4-9fab-1b63eb51572c_698x508.png 848w, https://substackcdn.com/image/fetch/$s_!JP77!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fc3205d9e-d05f-4bb4-9fab-1b63eb51572c_698x508.png 1272w, https://substackcdn.com/image/fetch/$s_!JP77!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fc3205d9e-d05f-4bb4-9fab-1b63eb51572c_698x508.png 1456w" sizes="100vw"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container restack-image"><svg role="img" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg"><g><title></title><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container view-image"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><p>This is not taking maximum advantage of parallelism, but is simple to write and good enough for most purposes. Notice that invoking <em>h</em> takes 5 sequential operations.</p><p>A maximally efficient solution would have <em>A</em> send messages to <em>B</em> and <em>C</em>, which would in turn send their results to <em>D</em>.</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!uQyj!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb8506060-9450-4691-9b04-0f6e9c6c8f8d_696x508.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!uQyj!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb8506060-9450-4691-9b04-0f6e9c6c8f8d_696x508.png 424w, https://substackcdn.com/image/fetch/$s_!uQyj!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb8506060-9450-4691-9b04-0f6e9c6c8f8d_696x508.png 848w, https://substackcdn.com/image/fetch/$s_!uQyj!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb8506060-9450-4691-9b04-0f6e9c6c8f8d_696x508.png 1272w, https://substackcdn.com/image/fetch/$s_!uQyj!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb8506060-9450-4691-9b04-0f6e9c6c8f8d_696x508.png 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!uQyj!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb8506060-9450-4691-9b04-0f6e9c6c8f8d_696x508.png" width="696" height="508" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/b8506060-9450-4691-9b04-0f6e9c6c8f8d_696x508.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:508,&quot;width&quot;:696,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:48107,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/png&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:null,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!uQyj!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb8506060-9450-4691-9b04-0f6e9c6c8f8d_696x508.png 424w, https://substackcdn.com/image/fetch/$s_!uQyj!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb8506060-9450-4691-9b04-0f6e9c6c8f8d_696x508.png 848w, https://substackcdn.com/image/fetch/$s_!uQyj!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb8506060-9450-4691-9b04-0f6e9c6c8f8d_696x508.png 1272w, https://substackcdn.com/image/fetch/$s_!uQyj!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb8506060-9450-4691-9b04-0f6e9c6c8f8d_696x508.png 1456w" sizes="100vw" loading="lazy"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container restack-image"><svg role="img" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg"><g><title></title><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container view-image"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><p>Now we have two parallel sequential operations. <em>A</em> isn&#8217;t waiting around with some suspended process taking up resources; it is done. The effort required from <em>B</em> and <em>C</em> is the same as in the sequential solution, but they can execute simultaneously.</p><p>Using current paradigms, manually writing the logic for every endpoint to participate in every such exchange is prohibitively complex. <em>B</em> and <em>C</em> might be written by different teams!</p><p>What if I tell you there is a simple existing technology that makes the maximally distributed solution natural to write?</p><p>The message <em>A</em> &#8594; <em>B</em> has to be able to specify where <em>B</em> should send its result. The same with <em>A</em> &#8594; <em>C</em>. It&#8217;s easy enough for <em>A</em> to include a destination URL in its messages for this purpose.</p><p>It&#8217;s inside <em>D</em> where things get interesting. Its function <em>h</em> is being invoked not in one go, but sort of <em>gradually</em>. It will receive messages from <em>B</em> and <em>C</em>, in any order, and when both messages have been received, then <em>h</em> can execute.</p><p>So <em>D</em> has to be able to cache the arguments to function calls. This means that <em>B</em> and <em>C</em> need some way to indicate that the message they&#8217;re sending are for the same function invocation. So there will have to be an identifier for the function invocation.</p><p>As an example, we might submit the arguments to a function as query parameters in a URL. Perhaps <em>/h/&lt;id&gt;?a=&lt;arg1&gt;</em>. A will have generated a UUID that it sent to both <em>B</em> and <em>C</em> in the destination argument for their results.</p><p>Assume for simplicity that functions take a fixed number of required arguments.</p><p>Notice that invoking a function now involves storing any of a fixed set of key-value pairs to a location specified by a (function) name and id. The location specified by the id should be created if it doesn&#8217;t exist; otherwise its values are updated.</p><p>But this is logically identical to performing an <em>upsert</em> operation on a table!</p><p>Seen this way, the function invocation is naturally expressed as a trigger on a table that first checks following an update whether the row has no null columns, and if so, executes the function, passing the row values as arguments.</p><h1>Relations + Functions + Continuations</h1><p>A distributed computing system, then, is naturally expressed as a set of relational stores with triggers.</p><p>Note that the receiving function for the results might be another trigger table, but can be any sort of HTTP endpoint<a class="footnote-anchor" data-component-name="FootnoteAnchorToDOM" id="footnote-anchor-1" href="#footnote-1" target="_self">1</a>, even on an entirely different service.</p><p>This design is what Functional Programming has long called <em>continuation-passing style</em>. Many languages have recently adopted the <em>await</em> keyword as a way of synchronising asynchronous systems. What <em>await</em> is silently doing in such languages is turning the rest of the function after the <em>await</em> into a continuation function. <a href="https://en.wikipedia.org/wiki/Continuation-passing_style">Continuation Passing Style</a> is entirely mainstream at this point.</p><h1>Don&#8217;t worry about SQL</h1><p>It is worthwhile to address in passing a concern that many folks will have at this point: SQL is awful, and writing triggers is a pain in the ass. This is absolutely true but not terribly important.</p><p>First: any SQL platform worth using (this means, basically, not MySQL<a class="footnote-anchor" data-component-name="FootnoteAnchorToDOM" id="footnote-anchor-2" href="#footnote-2" target="_self">2</a>) supports writing triggers in Python, which no-one considers a terrible language.</p><p>Second: PostgreSQL is the best database for programming by such a wide margin, it&#8217;s astonishing. PostgreSQL supports triggers in any of <a href="https://wiki.postgresql.org/wiki/PL_Matrix">16 programming languages</a>.</p><p>Third, PostgresSQL, SQL Server and SQLite (using host language custom functions) can notify external/host programs of data changes (using <a href="https://www.postgresql.org/docs/15/sql-listen.html">LISTEN/NOTIFY</a> in Postgres, <a href="https://learn.microsoft.com/en-us/dotnet/framework/data/adonet/sql/query-notifications-in-sql-server">Query Notifications</a> in SQL Server, <a href="http://docs.oracle.com/database/121/JJDBC/dbchgnf.htm#JJDBC28815">Change Notifications</a> in Oracle and triggers with custom functions in SQLite).</p><p>It is thus possible with a lightweight framework to effectively support trigger functions written in a straightforward way in a nice programming language.</p><p>Also, the world might (<em><strong>finally!</strong></em>), just a little, be waking up to just how awful SQL is, and we see projects such as the wonderful <a href="http://surrealdb.com">SurrealDB</a> to provide relational databases not based on SQL.</p><h1>Querying</h1><p>Since we have to have a relational store, it will be natural to support relational querying. SQL is abominable, so we&#8217;ll have something better, probably <a href="https://en.wikipedia.org/wiki/Datalog?wprov=sfti1">Datalog</a> or some moral equivalent.</p><h1>Functions, you say?</h1><p>I&#8217;ll elaborate more on this in later posts, but to provide a taste of what&#8217;s to come, I&#8217;ll briefly discuss some of the interesting consequences of this model of function invocation.</p><p>First, note that we&#8217;re allowing API clients to create endpoints for their own purposes. There is nothing, for example, to prevent a single function invocation being triggered by data coming from multiple different organisations (modulo a good security story, which we have but this will have to be discussed later).</p><p>Since we&#8217;re doing function calling, a natural thing would be for our servers to support functional programming operations, such as <em>map</em> and <em>fold</em>. Allowing API clients to do filtering and such things server side will let them make custom API endpoints that reduce load for both them and the service provider. No need to have clumsy endpoints that provide everything and the kitchen sink that the API clients might not need.</p><p>We will allow a function invocation to be partially completed and then <em>locked</em>, meaning it <em>can&#8217;t</em> be fully completed. Rather, a message can be sent to the address of the locked row, but must include a different id as an argument. The values from the locked row can then be copied to the new id, along with any other arguments supplied. In this way, we can allow API clients to employ partial functions <em>they can define</em>, and combined with the likes of <em>map</em> and <em>fold</em>, customise our service in a way that is both simple and flexible.</p><p>This is programming, but without using a programming language. We could implement a LISP or something to make this easier, but we&#8217;re not letting API clients just run their own code on our servers<a class="footnote-anchor" data-component-name="FootnoteAnchorToDOM" id="footnote-anchor-3" href="#footnote-3" target="_self">3</a>.</p><p>Because SQL has so poisoned the well, relations have been grossly underutilised in programming. The model for programming we&#8217;re developing here will inevitably lead to expressing <em>much</em> more of business logic in a relational manner, producing software that is simpler, more correct, and more malleable than software that expresses business logic in a regular programming language. More on this soon.</p><p>Finally, some <em>very</em> interesting possibilities arise out of supporting a reactive UI on top of such a system. Since we&#8217;re using a database, we can automatically generate a user interface much as Access or FileMaker do. Programmers will be freed from such quotidian tedium as hand-rolling <em>yet another</em> HTML form. More on this, soon, also.</p><h1>Conclusion</h1><p>Truly efficient distributed systems are most naturally expressed through functions as triggers invoked from upsert operations on addressable relations.</p><p>Having adopted this paradigm, interesting possibilities arise out of adding functional programming operations and reactive user interfaces.</p><p>These considerations form the heart of the design of FREST.</p><div class="footnote" data-component-name="FootnoteToDOM"><a id="footnote-1" href="#footnote-anchor-1" class="footnote-number" contenteditable="false" target="_self">1</a><div class="footnote-content"><p>We shall not require HTTP; it will eventually be possible to run FREST over SOAP or gRPC or whatever; HTTP is suitable for exposition, however.</p></div></div><div class="footnote" data-component-name="FootnoteToDOM"><a id="footnote-2" href="#footnote-anchor-2" class="footnote-number" contenteditable="false" target="_self">2</a><div class="footnote-content"><p>MySQL is actually strictly less capable than SQL Server, PostgreSQL and Oracle, and given its intended uses, even SQLite. It is beyond me why anyone sane would choose to use it.</p></div></div><div class="footnote" data-component-name="FootnoteToDOM"><a id="footnote-3" href="#footnote-anchor-3" class="footnote-number" contenteditable="false" target="_self">3</a><div class="footnote-content"><p>We should, though! Platforms such as <em><a href="http://deno.land">Deno</a></em> are arising that support robust sandboxing, and this is a wonderful development.</p><p></p></div></div>]]></content:encoded></item><item><title><![CDATA[Welcome to FREST!]]></title><description><![CDATA[Computing for everyone]]></description><link>https://frest.substack.com/p/welcome-to-frest</link><guid isPermaLink="false">https://frest.substack.com/p/welcome-to-frest</guid><dc:creator><![CDATA[Guyren Howe]]></dc:creator><pubDate>Sat, 18 Feb 2023 00:14:21 GMT</pubDate><enclosure url="https://substackcdn.com/image/fetch/$s_!MB46!,w_256,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F5fd9d106-47e9-4c50-a15f-707b11ec91ae_550x550.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>Software has been in crisis since its beginning. Software is unreliable, slow and clunky, both for users and programmers, and has been since its beginning.</p><p>Our biggest moral failing as an industry is that we have favoured software engineering approaches that suit our highly trained priesthood, rather than ones that democratise computing. Where are all the bicycles for the mind?</p><div class="subscription-widget-wrap-editor" data-attrs="{&quot;url&quot;:&quot;https://frest.substack.com/subscribe?&quot;,&quot;text&quot;:&quot;Subscribe&quot;,&quot;language&quot;:&quot;en&quot;}" data-component-name="SubscribeWidgetToDOM"><div class="subscription-widget show-subscribe"><div class="preamble"><p class="cta-caption">Thanks for reading FREST Substack! Subscribe for free to receive new posts and support my work.</p></div><form class="subscription-widget-subscribe"><input type="email" class="email-input" name="email" placeholder="Type your email&#8230;" tabindex="-1"><input type="submit" class="button primary" value="Subscribe"><div class="fake-input-wrapper"><div class="fake-input"></div><div class="fake-button"></div></div></form></div></div><p>The reasons for all this are cultural and economic, not technological. Folks have been <a href="https://en.wikipedia.org/wiki/Lisp_(programming_language)">developing</a> <a href="https://en.wikipedia.org/wiki/Smalltalk">better</a> <a href="https://en.wikipedia.org/wiki/HyperCard">ways</a> of writing software since the beginning, but as an industry, we have adopted them at a glacial pace.</p><p>FREST is a <em>punk</em> architecture. It rejects the culture and economics that produced the crisis, in favour of a radically simpler way.</p><p>For the user, it means access to computing without having to go wallet in hand humbly to the engineering priesthood. Regular folk will be able to tie together the working abstractions programmers employ ever day, simply and directly. Much as the spreadsheet democratised functional programming, and Access and FileMaker freed ordinary folks to employ relations, FREST is a general architecture for presenting abstractions in a GUI.</p><p>For the programmer, the tedium that is the day to day of programming work will fade away. Most of that tedium is replaced by the direct manipulation of abstractions. No more hand-rolling HTML. Much less writing code to adapt this library to that library.</p><p>FREST selects the most productive programming abstractions, and supports manipulating them at a very high level, both through code and through a GUI. Manipulating data and its user interface also <em>en passant</em> constructs an API. In FREST, the API and the user interface are two sides of the same thing.</p><div><hr></div><p>On this substack, I will write short pieces that explain the various parts of this vision:</p><ul><li><p>analysis of the cultural and economic issues that have led us astray;</p></li><li><p>identification of the dead ends and failures that software perpetuates that should be discarded;</p></li><li><p>identification of the technologies that get it right, that form the foundation of FREST; and</p></li><li><p>the design of FREST, from its user interface to the engineering approaches that make it possible.</p></li></ul><p></p><div class="subscription-widget-wrap-editor" data-attrs="{&quot;url&quot;:&quot;https://frest.substack.com/subscribe?&quot;,&quot;text&quot;:&quot;Subscribe&quot;,&quot;language&quot;:&quot;en&quot;}" data-component-name="SubscribeWidgetToDOM"><div class="subscription-widget show-subscribe"><div class="preamble"><p class="cta-caption">Thanks for reading FREST Substack! Subscribe for free to receive new posts and support my work.</p></div><form class="subscription-widget-subscribe"><input type="email" class="email-input" name="email" placeholder="Type your email&#8230;" tabindex="-1"><input type="submit" class="button primary" value="Subscribe"><div class="fake-input-wrapper"><div class="fake-input"></div><div class="fake-button"></div></div></form></div></div>]]></content:encoded></item></channel></rss>