Cardiac atheroma at the limits – are we fighting the final battle? A 2020 talk by Peter Libby, leading researcher on heart disease. Does a nice job of mapping out all the paths we have to prevent heart disease.

One of the main categories of heart disease is atherosclerotic heart disease. This is caused by plaque buildup in the arteries. Every adult has these plaques growing in their blood vessels, but high blood pressure, high cholesterol, and damage to the vessels1 accelerates plaque buildup.

To date we have several medications2 that can bring blood cholesterol down with statins being first-line and new therapies such as PCSK9 inhibitors showing promise.

With the success here, research has moved to other means of reducing cardiovascular risk:

• Reducing inflammation, the subject of several trials, with colchicine showing promise and general interest in inhibiting IL-6. See Libby’s more recent presentation on this here.

• Insulin resistance, addressed by drugs like GLP-1 agonists and SGLT-2 inhibitors3.

• Specialized treatments for clonal hematopoiesis; mutant cells in the bone marrow that produce mutant white blood cells.

My main takeaway here is that research on heart disease might end in some sense. More and more, the leading therapies look like anti-aging interventions. Many forms of heart disease seem to be caused by aging of vascular tissue. Once the non-aging problems are solved, the field might get absorbed into longevity research.

See also point 1 here for more CVD research.

Related: Enlicitide decanoate - Wikipedia. A promising cholesterol drug!

2.

There’s lots of talk of existential and catastrophic risks, but Croissanthology wonders, Is the world actually that vulnerable? And comes to the conclusion that no, it’s not. In many cases we have straightforward things we can do to reduce the risk, and good reasons to think that the worst case isn’t all that bad.

For instance, Croissanthology has a nice post on Solar Storms and how we can basically address their risks by installing lots of SolidGround systems (or building more transformers).

Croissanthology also has a post about how LLM cybersecurity risks in the long run are probably exaggerated. This is similar to comments by David Dalrymple that defense “just wins” as AI capabilities advance. This paper makes a similar point:

Deception and Detection: Why Artificial Intelligence Empowers Cyber Defense over Offense | International Security | MIT Press

I found the above paper when reading the transcript of this podcast, which allays worries that AI might somehow destabilize our current nuclear stalemate:

Sam Winter-Levy and Nikita Lalwani on how AI won't end nuclear deterrence (probably) | 80,000 Hours

On the other hand, I think that laser weapons will destabilize the current nuclear paradigm … in a good way! Namely, I think laser defense will essentially end the threat of nuclear war.

[On a related note, here’s a nice review of the current state of combining laser beams coherently4: Towards Ultimate High-Power Scaling: Coherent Beam Combining of Fiber Lasers]

So if solar storms and cyber risk and nuclear war aren’t all that bad, what’s left? Pandemics. Abhishaike Mahajan has a nice post on this:

Reasons to be pessimistic (and optimistic) on the future of biosecurity

I find a of optimism from this post, bioweapons are genuinely hard, there’s little incentive to use them, and AI might genuinely accelerate defensive technologies like mRNA vaccine production. Far UV-C and glycol vapors might drastically reduce airborne transmission (though finding someone to pay for it is hard).

Related: Vincent Conitzer article on addressing misaligned AI by “[giving] the AI a (primary) goal of being turned off”.

Everything else

The Tiling Tree Method and The Tiling Tree Method, Part 2. An interesting way to exhaustively list possible solutions to a problem. I pay attention to any ideation technique that Ed Boyden uses.

A small polymerase ribozyme that can synthesize itself and its complementary strand. This is huge for understanding the origin of life. This strand of RNA can reproduce itself given short RNA blocks as “food”. Now we need a good understanding of how RNA blocks can form and why these blocks might accidentally assemble into something self-replicating.

Prescription strength: musings about the past/present/future of sarcopenia therapeutics. New drugs might allow anyone to have a muscular physique with only a little effort in the gym. That would be revolutionary for health outcomes.

Subvocal recognition allows you to convert your inner voice into text. Alter Ego is building a wearable device for this. Imagine controlling computers with your mind!

The First Multi-Behavior Brain Upload claims they uploaded a fruit fly. More skeptical takes from and Ariel Zeleznikow-Johnston and Dan Turner-Evans.

"I Found My People!" On finding a community that brings you joy.

Related: Make Your Own Bubble in 10 Easy Steps - by Bryan Caplan

Some relationships deepen when you tell the truth and some end. Committing to being honest has been valuable for my relationships too.

Why We Have Prison Gangs. Systems of government spring up in surprising places. See also Chris Blattman’s work on peace amongst different gangs.

the lottery of career success: or why you may want to bribe OpenAI. Because it’s hard to measure performance, career success often snowballs based on past prestige, which is substantially random.

We may miss the sweatshops. Developing countries used to escape poverty by moving into manufacturing. If AI automates many tasks, these countries will need to find a different path.

Inside the Farm Growing 3.5 Million Pounds on Half an Acre. Nice tour of a vertical farm growing leafy greens. This would pair well with off-grid solar and batteries. I’m coming around to the view that vertical/indoor/hydroponic farming can work well for veggies, while staple crop agriculture is hard to improve upon.

Everything you ever wanted to know about Roblox, but were afraid to ask a 12-year-old. The metaverse already exists, with all the good and bad that entails.

Sunbeam: Near-sun statites as beam platforms for beam-driven rockets. Proposal for new form of interstellar travel using particle beams. The main challenge is efficiently converting the beam into propulsion. One of the authors did an interview on this work here.

AI:

Doc-to-LoRA: Learning to Instantly Internalize Contexts. Load a document into a language model by modifying its weights.

Can LLMs Be Computers? Modify a transformer with a different attention mechanism to load algorithms directly into model weights!

Post-Training 50x Faster. Open-source code to fine tune Kimi-K2 using LoRA.

We mildly deprived ourselves of sleep (5-5.5 hours, relative to 7-7.5 hours normally) and took either a placebo or orexin intranasally. We tracked our sleep the night before and after taking a dose in the morning and completed various tests of mental acuity during the day.

The results from our initial experiment are exclusively null results that don’t cross standard thresholds for statistical significance. Not that this was particularly surprising, we expected a ~60% chance of this happening. We’re considering next steps, and need your feedback!

For now, there are a few things to cover in the results.

Trial Design

We performed a self-blinded randomized controlled trial with blocking, each participant took either the placebo (2.5 mL of sterile water) or the orexin (100 μg of orexin-A dissolved in 2.5 mL of sterile water). Here’s the procedure, repeated for every block:

1. Prepare two nasal atomizers, one with saline solution and one with orexin+saline solution

2. Night to the first day: Sleep 5-5.5 hours.

3. First day:

   1. At a consistent time of day, randomly select one and administer.

   2. Take mental acuity tests. Once mid-day, once in the evening.

   3. Track sleep, heartrate, etc. using a Fitbit Inspire 3.

4. Night to the second day: Sleep normally.

5. Night to the third day: Sleep 5-5.5 hours.

6. Third day: Repeat 3.1-3.3, using the remaining dose/placebo you prepared.

7. Night to the fourth day and fifth day: Sleep normally.

8. Fifth day: Record baseline sleep measurements and mental tests

Each person had a substantial amount of leeway in how they structured their day. On sleep deprived days, Sam preferred to get up early while No Magic Pill and niplav preferred to stay up late and get up at the usual time. We each took doses at a consistent time, but the time differed between people.

We didn’t standardize things because we thought it was more important to have ecological validity, i.e. that we were using orexin the way we would actually use it in everyday life. This is a higher variance, but lower bias approach.

The Results

In our initial proposal, we pointed out that the main thing we wanted to see was orexin causing less rebound sleep the following night. A simple stimulant effect isn’t enough for us, we wanted to use orexin to sleep less and get away with it.

So here’s the average sleep time for the night after taking orexin vs the night after taking placebo:

Unfortunately, the difference wasn’t significant and the effect size is small. This could be for a couple reasons that we want to address in the next trial.

Did orexin have any sort of stimulant effect during the day? Nope, none of the mental acuity tests are significantly different.

In setting up this trial we had a sneaky second hypothesis: Does sleep deprivation actually make you dumber?

One caveat before we look at the data. Typically our “baseline” days would come after our sleep-deprivation days. So that means baseline days enjoy more cumulative practice compared to sleep-deprivation days. That should bias the results by making baseline days look better. On the other hand, if sleep deprivation has long term cumulative effects, then perhaps baseline days are at a disadvantage. But that doesn’t match our experience of feeling significantly better on baseline days.

So, does sleep deprivation make you dumber? Not really!

Depending on how you correct for multiple comparisons, the psychomotor vigilance task (PVT) differences might be significant. And I’d expect the PVT differences to become significant with more data points. From what I (Sam) remember from doing PVT on sleep-deprivation days, I felt just as fast, but I would slip-up more from inattention or distractions. This is consistent with the large gap on the slowest 10% days.

But overall, this is a nice example of how our intuitions around sleep can lead us astray. It sure feels like sleep deprivation should make you dumber. But we don’t see that here. It’s important to actually check what changes our productivity because our intuitions around this are pretty fuzzy.

The Next Trial

There’s a few reasons why we might be getting a null result. We might have too few data points, or the dose might be too low, or more concerningly, we might be storing the orexin improperly.

So the first next step is a slightly bigger trial where we see if a higher dose of orexin changes our results. From anecdotes online, some have felt effects while others haven’t. But even if orexin doesn’t have obvious effects, it might still reduce sleep need. We need to try higher doses and collect more data to find out.

That said, sleep deprivation is uncomfortable for Sam and No Magic Pill and extremely uncomfortable for niplav. We’ve decided to try a different design: sleep ad libitum on all nights of the week, but observe whether orexin reduces the amount we sleep the night after. This should make it sustainable to collect a lot more data.

Appendix A: Details about the Data Analysis

We collected two separate datasets:

1. Data collected automatically from the Fitbit Inspire 3

   1. Measures of sleep

      1. Sleep duration

      2. Sleep efficiency

      3. Time spent in deep sleep

      4. Time spent in light sleep

      5. Time spent in REM sleep

      6. Time spent in nocturnal awakenings

   2. Additional measures

      1. HRV Daily RMSSD (ms)

      2. HRV Deep RMSSD (ms)

      3. SpO2 Avg (%)

      4. SpO2 Min (%)

      5. Breathing rate (breaths/min)

      6. Skin temperature Δ (°C)

      7. Steps

2. Data from the mental acuity testing

   1. Psychomotor vigilance task

   2. Digit symbol substitution task

   3. Digit span

   4. Stanford Sleepiness Scale

   5. Description of subjective state (free text)

We aggregated mental acuity tests per-test to avoid pseudoreplication (so two data points per day), and aggregated Fitbit data per-day. We analyzed the data via matched controls (with days in a participant-block being matched as to analyze within-pair differences) and ran two separate analyses on the data; one frequentist and one Bayesian. The code for the analysis, written in Julia by Claude Opus 4.6, is available here. Our mental acuity test data is available here, aggregated full data is available here.

Frequentist Analysis and Additional Results

In our frequentist analysis we ran the paired t-test on the paired data with cardinal measurements, and the Wilcoxon signed rank test on paired data with ordinal measurements, we also report Cohen’s d for the measurements. We Bonferroni-corrected the p-values, not that that was necessary…

[The table was too unwieldy to include, see here for more.]

Bayesian Analysis and Additional Results

We fit a hierarchical Bayesian linear model with participant random intercepts, using NUTS (4 chains × 2000 samples per metric). The primary estimand is δ, a standardized treatment effect (Cohen’s d-like), with a weakly informative N(0,1) prior.

Formally, the likelihood is yᵢ ~ N(μ + δσ·treatmentᵢ + α[pᵢ], σ), where treatmentᵢ ∈ {0,1} encodes placebo/orexin. The raw treatment effect on the outcome scale is δσ; δ alone is dimensionless. Priors: μ ~ N(0,10) (vague grand mean), σ ~ half-N(0,10) (residual SD), τ ~ half-N(0,5) (between-participant SD), α[j] ~ N(0,τ) iid for each participant j.

Priors and posteriors for cognitive acuity tests and sleep measurements:

Priors and posteriors for additional Fitbit data:

Learning Effects on Mental Acuity Tests

Circles for the first test of the day, diamonds for the second test of the day.

Appendix B: Threats to Validity

Our method seems simple on its face, but there were a lot of annoyances along the way.

1. Orexin was delivered at room temperature, and while the vendor claims the orexin was lyophilized, we are uncertain if the lyophilization was sufficient to prevent damage.

   1. In niplav’s case the orexin sat uncooled in customs for over a week during the delivery in July.

2. In order to distribute the orexin into vials, we had to dissolve it in water. This meant that we had to both store the orexin dissolved in water for almost a week, and freeze the rest. We are uncertain if any of those damaged the peptide structure.

3. We are uncertain if our route of administration can cross the blood-brain barrier.

4. One participant wasn’t aware that Fitbit data needs to be regularly synced, so we only have sleep data for two individuals. Additionally, Fitbit syncing and data collection is unreliable, leaving us with only 17 datapoints for nights of sleep following orexin.

5. Another headache was making sure sleep deprivation nights were scheduled between nights where we could sleep ad libitum. We also tried to keep a consistent schedule on trial days so that variations in exercise, nootropic consumption or other activity didn’t change our results.

Appendix C: Personal Experiences

Niplav:

• So much stuff can go wrong

• 2.5ml is way too much, let’s do 1ml next time

• Filling the syringes was fun! It felt very scientist-y.

• Plausibly we should’ve started with a higher dose but also safety concerns so whatever

• 5½ hours of sleep feels horrible

• I was ~completely unproductive on sleep deprivation days, and will put a high premium on this

• Caffeine was really helpful

• Two nights of normal sleep & then one more sleep deprivation would’ve been better

• Inform yourself about the reliability of the data collection tools

• Didn’t nap at all except in week two, when I just couldn’t stay awake

• I’m happy we did a short trial first so we discovered the data collection issue early

• Beat-by-beat experiment log here

Sam:

• Feel so much better and alive from my rebound sleep, better than a normal days sleep even.

• Consistently napped on placebo days, but orexin does seem to have stimulant effect.

• Couldn’t really tell which was which in general

• Felt mentally the same on trial days. But doing tests felt a little slower than on baseline day. In general I didn’t have a good sense of how productive/smart I was.

• Extra hours in the morning did get used, for somewhat intellectual tasks like reading papers and writing.

• Much easier to be up early when sun was up

No Magic Pill:

• Tests:

  • PVT: I don’t think my reaction time improved at all over the course of the testing (75% confident). I did not mind this test.

  • DSST: I don’t think my skills improved much over the course of the testing. I disliked this test the most.

  • Digit span: I am 75% confident that I got better over the course of testing. I was consistently able to get 9 both forward and reverse towards the end. I disliked this test the second most (behind DSST).

  • Sleepiness: I never scored that high and probably erred on the side of scoring higher because of a feeling that I needed to utilize most of the scale. I did not mind this test.

  • Feelings: I could have been more verbose here.

• It was wayyyyyyy easier to stay up late and get up at my normal time than it was to go to bed at my normal time and wake up early.

  • I was fairly productive when staying up late

  • I was NOT productive when getting up early

  • Most of the time I had full-body “tingles” when awakening after a sleep-deprived night. I’ve experienced this phenomenon for years: anything less than ~6 hours, or normal duration after a hard exercise session, leaves me “tingley” in the morning.

• I was a normal level of irritable and quick to anger after a sleep-deprived night. This has been consistent for years (like the tingles).

• I did not nap on any day (pre-test or day-of test) because I thought that would throw off the testing data.

• My motivation on sleep-deprived days often waned faster than non-sleep deprived days. This has been consistent for the past few years and is what I expected.

• I did not feel anything physically or mentally immediately following the orexin administration.

• I should have done a better job of isolating myself during testing. Sometimes it was a bit noisy or visually distracting, especially if I was at work. I should have noted down if I was distracted during the test.

  • Maybe a feature can be added to add comments at the end of each test?

• I agree with Niplav that 2.5 mL was too much water. 1.25 mL was good, if not a tad much as well. I think 1 mL is probably good for the future?

Say you’re trying to predict when an event will happen. For instance, a scientific breakthrough, or AGI, or geopolitical drama. One way to write down your predictions is to give probabilities of the event happening by a certain date. 50% by 2026, 75% by 2027, and so on.

A simple way to write this down is a smooth distribution of arrival probability vs time. We know the arrival didn’t happen now (time zero) and could happen eventually (time infinity) so we want distributions that take values (support) over the range 0 to infinity.

A particularly useful class of distributions are the maximum entropy distributions. Maximum entropy says “I have as little information as possible about the future except for some constraints like the support and the mean”. We’ll specify things like the mean and variance and support, then choose a distribution consistent with those that also has highest possible entropy.

Let’s look at the exponential distribution. It’s a maximum entropy distribution with a specified mean and takes values from zero to infinity. Not a bad way to encode timelines.

(For now, I’m going to write timeline scenarios in terms of time-to-AGI to make them more concrete. But the same reasoning applies to any forecast using a distribution like this.)

If you think we’ll reach AGI in an average of 5 years, set lambda to 0.2 and the exponential distribution will provide the probability of AGI arriving X years in the future.

Mean excess is more interesting. “Conditional on not observing AGI for 2 years, how many more years to AGI?”. The exponential distribution is memoryless; the mean excess is constant. This has some interesting implications:

1. AGI is always x years away.

2. You need to have new evidence for years-to-AGI to change. Simply conditioning on the amount of time that passed is insufficient.

3. Without new evidence, every time AGI is reached you were expecting it to arrive x years later. You’re always surprised.

Now let’s consider more interesting distributions. I looked for maximum entropy distributions with support from zero to infinity and constraints on the mean and variance1. Below is the mean excess vs time for the lognormal, chi, Weibull, gamma, and inverse gamma distributions2. They all have the same mean and variance. You can try your own settings of mean and variance in this widget I made with Claude.

Notice that mean excess is flat once you pass the mean. In this regime, the implications are the same as for the exponential distribution. The linear regime is different:

1. Expectation falls ~linearly with time. If no new evidence comes in, you should think AGI is closer with each passing day.

2. You need a stream of new evidence for time-to-AGI to stay flat or increase.

If the variance is larger than the mean, some distributions see mean excess grow over time. Your forecast gets stale and the error bars grow. You’re more surprised when AGI arrives.

This doesn’t just apply to AGI, these dynamics occur in to any situation where you have a maxEnt distribution over arrival time. This creates interesting constraints for updating old forecasts and talking about how opinions have shifted.

“Fusion is always 30 years away. Implicitly, arrival probability is exponentially distributed.”

“If we update her forecast for the years that passed, time-to-AGI is now 10 years”

“I noticed your years-to-invasion is the same, what changed your mind?”

It’s neat that we can get interesting implications from such a simple model. I’ll have to think about more ways make use of this.

I use Bluesky for my microblogging. The AI scene is good there now. Though I’d like to see more science and econ stuff.

A thread from me on AI in 2026:

Pair with this thread on how LLM’s change the internet. I think ATproto (the protocol behind Bluesky) is well positioned to take advantage of these changes.

2.

Brain emulations are coming. From Konrad Kording’s end of year post:

The phrase “simulating nervous systems” used to trigger eye-rolls because it sounded grand and vague. In 2025, it felt more like an engineering roadmap with increasingly nontrivial milestones.

See his paper The time is ripe to reverse engineer an entire nervous system: simulating behavior from neural interactions.

Max Schons has a beautiful report on brain emulation:

State of Brain Emulation Report 2025

And a companion piece on Asimov press:

Building Brains on a Computer

See also: Notable Progress Has Been Made in Whole Brain Emulation — LessWrong

Related: Low-cost 3D-printed optics for super-resolution multifocal structured illumination microscopy. They 3D printed one of the lenses in a microscopy setup, making it much cheaper.

These are the kind of innovations we need if we’re going to make optical microscopy far cheaper and faster. In my mind, optical microscopy (with expansion microscopy and barcoding) is the most scalable method to scan connectomes.

3.

A few items on technology and the developing world. First, an old thread from me on financial technologies in the developing world. Those financial technologies are a big reason Why Solarpunk is already happening in Africa.

Local grids are failing people in Africa. People are using the mobile money system to take out loans to buy a personal solar and battery system. That means saving the money spent on kerosene for indoor lighting. That means being able to work and study for more hours of the day. I hope this unlocks a self-reinforcing cycle of productivity.

Also of note:

allenai/HiRO-ACE. An open-source weather model that uses AI to predict precipitation weeks or years in advance. Imagine farmers across the world using models like this to optimize crops or buy insurance.

The changing (and perhaps surprising) geography of diabetes. As the developing world grows and enjoys an abundance of calories, diabetes is rising. For the same BMI, some ethnicities get diabetes at higher rates. The developing world is going to need a lot of GLP-1’s.

Everything else

OxLDL-Targeted Chimeric Antigen Receptor T Regulatory Cells Reduce Atherosclerotic Plaque Development. CAR-T therapies to prevent atherosclerosis. Seems a bit overkill when statins can probably prevent atherosclerosis in the first place. But I’m all for developing cellular nanotech.

A Nonviral Neo‐Nucleocapsid for Cell‐Specific RNA Delivery Developed by Pseudo‐Cyclic Peptide Grafting and Directed Evolution. More cellular nanotech.

EVIO Aircraft. A planned hybrid-electric plane with 900 km range, 76 passengers, and quiet takeoff and landing. Important if we’re going to make flying clean and quiet so we can build more airports and air busses.

Terraforming Venus Quickly. Old paper from Paul Birch that aligns a lot with a post I’m writing.

Do Commodities Get Cheaper Over Time? Probably the most comprehensive data on long term commodity prices I’ve seen. My take: the theory of exhaustible resources suggests commodities should grow in price at least as fast as real interest rates. The data is consistent with real interest rates in the U.S.1

Most people are individually optimistic, but think the world is falling apart. Might I suggest that social (and traditional) media is selling you lots of pessimistic and misleading information creating a situation where views on “the world” are completely divorced from reality and people are mostly fine?

Pro-social media. AI’s telling users the truth and being a public arbiter of truth may ameliorate some of the problems of social media.

What do executives do, anyway? A management strategy for large companies that coordinates everyone around a set of values and enforces those values, delegating all decisions to lower parts of the hierarchy.

To paraphrase the book, the job of an executive is: to define and enforce culture and values for their whole organization, and to ratify good decisions.

That’s all.

Not to decide. Not to break ties. Not to set strategy. Not to be the expert on every, or any topic. Just to sit in the room while the right people make good decisions in alignment with their values. And if they do, to endorse it. And if they don’t, to send them back to try again.

Is this secretly a manifesto for orchestrating AI agents?

Are the Costs of AI Agents Also Rising Exponentially? — Toby Ord. The METR time-horizon curve looks different when you consider how models perform when given more resources. More advanced AI agents can do more, but also command higher hourly wages. Models will continue to improve in every aspect, but reality is more complicated than looking at the METR plot and extrapolating to singularity.

On neural scaling and the quanta hypothesis. Toy model reproduces AI scaling laws by assuming that A, knowledge is in discrete pieces (quanta), B quanta are either memorized or not, and C, “[t]he ‘use frequencies’ of the quanta naturally follow a power law.”

Tuning GPT-3 on a Single GPU. Old paper from Greg Yang applying theories of infinite-width neural networks to the real world. You can optimize the training hyperparameters of a small neural network (40M parameters) at small scale and then transfer them to a larger neural network (6.7B parameters). Might we “solve” training hyperparameters once we optimize them for a 1B parameter model?

The Advanced Matrix Factorization Jungle. Incredible reference on niche matrix factorization methods, particularly when you want to enforce things like sparsity.

Eventually, we’re going to stop making better computers. In other words, the real cost per unit of useful computing will flatten out, becoming relatively constant over time. Innovation in semiconductor chips will slow and the low-hanging fruit in other parts of the computing supply chain will get eaten1.

This will have implications for everything.

We’ve tried a lot of stuff in semiconductors

This view comes as a consequence of my view that breakthroughs are rare and decreasing. We try stuff and converge on good solutions. Eventually we find the economically efficient way to make a product and move on. This is true for all goods, from cement to computer chips.

This is very clear in the semiconductor industry for two reasons.

First because semiconductors make you painfully aware of the limitations of matter and physics. Transistors are dozens of atoms wide, there isn’t much room at the bottom. Smaller size isn’t necessarily desirable as smaller transistors leak more current, raising energy consumption and errors.

Long ago, technology node names like 90 nm actually corresponded to the size of transistor. Progress stalled but the industry kept up appearances by changing the name to match the old trend. This year, TSMC is producing “2 nm” chips, but the actual size of transistors is closer to 45 nanometers. The shrinking transistor has slowed and Dennard scaling “ended” around 2006.

Second because the semiconductor industry has tried a lot of stuff. It has saturated many S-curves already. Dennard scaling was the first casualty, but there have been many more. Thousands of companies have sprung up and died optimizing one part of the supply chain only to be obsoleted by something else.

Watch one of my favorite videos and be awed by the amount of innovation that went in to creating just one flashlight in the semiconductor manufacturing process:

Watch some of the other videos in that playlist and you’ll see how much work and deliberation went into developing EUV. The industry explicitly considered other techniques, built out prototypes, started companies, and then went with EUV. That new company touting X-rays as the new way to make chips? Yeah the industry has already considered and rejected X-rays twice.

This is a microcosm of what’s happening at every step of chip production. Processes are evolving, with massive research effort to try different techniques2. The process we have today is a product of over 60 years of tinkering. With growing research effort, chips can steadily improve until we’ve wrung out every opportunity.

What about …

AI?

Sorry, but current AI won’t revolutionize chip design. Eventually there will be enough training data for AI to design chips, but we quickly run into the problem that logic optimization and place and route are NP-hard. Chip design is a fundamentally hard problem. AI tools will help, but have limits3.

Space?

Let me show you why space data centers are silly in the length of a skeet.

Option A: build solar, radiators, and chips on earth.

Option B: build solar, radiators, and chips on earth AND pay $3000/kg+depreciation to put them in space.

You're paying extra money for no apparent benefit.

The same principle applies if you’re launching self-replicating asteroid-mining nanobots. You could have used them on Earth and saved yourself the launch costs and harsh environment and time delay.

If we build enough data centers on Earth that all the low cost land and resources get used up then maybe space data centers make sense. The latency is much higher and it’s unclear if there will be demand for compute at that scale, at least on Earth. I’ll address that possibility in a later section.

For now, going into space doesn’t dramatically lower compute costs4.

EDIT: See Casey Handmer’s analysis of space data centers. He does not think they will be cost effective.

Unconventional computing?

There are many unconventional computing paradigms, perhaps one will swoop in once we’ve exhausted CMOS?

I’m skeptical, in part because none of them have overtaken semiconductors despite many, many people trying. It’s a trillion dollar bill that nobody has picked up.

Of the unconventional computing paradigms, we can safely reject those that use components that are much larger than today’s transistors. The speed of light limitations are daunting. Fluidics, MEMS, and biopolymers are simply too large to compete with the speed and cost of CMOS.

Analog computing paradigms (think “neuromorphic computing” or “thermodynamic computing”) are fraught as well. There have been many failed attempts in this field, and Zach lays out the challenges of Making Unconventional Computing Practical5.

The problem with analog is that when your signal is continuous, it’s easy to mess it up. Temperature changes, manufacturing variability, and interference all change the signal a little bit. In a chaotic system, those errors propagate out and mess up your computation. That’ why most analog computing companies have failed or switched to digital logic. Digital just works.

That leaves us with a few paradigms. I honestly don’t know how these will pan out, but I’ll note reasons for skepticism.

Spintronics applies materials that control electron spin inside of more traditional chips. The hope is that electron spin might provide an additional way to store or transfer information. I do think this has promise for improving memory bandwidth to some applications. But given how sensitive electron spin is to stray electromagnetic fields, I’m skeptical that the benefits are worth the costs of incorporating new materials and design complexity.

Optical computing startups in the last few years have pivoted from making computers to making optical interconnects for traditional data centers. Not a promising sign. Hopefully, this beachhead will enable optical computing to scale. However, optical inherits some of the challenges of analog computing, namely temperature dependence and manufacturing variability. These things can be addressed, but add cost.

The bigger issue is that optical components have to be physically larger than transistors, on the order of the wavelength of light they are carrying. This makes them 10-100x larger than transistors. Patterning at a larger scale means higher costs67. The saving grace is that light can send many different signals at the same time (multiplex). So even if the components are 10x more expensive, they might carry 100x more information. This makes it the most promising contender.

Superconducting computing inherits all of the problems of analog computers, while having components that are physically larger than normal computers, while demanding very low temperatures. People have tried and failed. I’ll pass.

Quantum computers (which often leverage superconducting components) offer small speedups in a few problems of interest. Quantum computers have to be kept very cold and shielded. You need lots of qubits for error correction. I doubt these will become cost effective for general purpose computing.

Reversible computing8 is interesting if compute scales to space, where energy is expensive and heat is a problem. These benefits are far weaker on Earth. The error correction is harder, the designs more complicated, and perfect reversibility is impossible, reducing the energy consumption advantage.

What does the end of hardware history look like?

Innovation jumps around

A key point from my breakthroughs post was once you “solve” a particular product, innovation jumps somewhere else. First you figure out how to plant, then breed plants, then make fertilizer, then automate harvesting, and so on.

As semiconductor innovation slows, innovation in other areas will grow. Solar and batteries will provide cheap data center electricity, new cooling systems will lower costs, optical or RF interconnects will gain traction, satellite internet will lower latency, and of course the AI/software layer will improve. There’s hundreds of opportunities for innovation here.

That said, this doesn’t refute the overarching conclusion. Eventually innovation in these areas will grind to a halt, we will have found the right way to build a data center.

Electron beams?

Pushing to higher wavelengths of light has its limit, and high-NA EUV could be the end of photolithography. In the future of lithography I speculated that electron beams might be the final frontier. To this day, I still think electron beam lithography might be the economic limit of chip manufacturing. Atomically precise manufacturing might make better chips, but they won’t be cheaper, photolithography will make cheaper chips, but they won’t be better.

We’re a long way off from electron beams beating EUV though. Currently, e-beams are too slow. I’ll add some links in the appendix on making e-beams cheaper and simultaneously writing with thousands of beams with one device which seems promising.

AI hardware

The economics of LLM inference tell us that memory bandwidth is the key cost driver. Chip designers have come up with their own version of “just stack more layers LOL” called high-bandwidth memory. It involves stacking towers of fast memory close to the part of the chip that does calculations.

Designers are pushing this S-curve as hard as they can. Unfortunately, each layer gets more expensive9 while being slower since it’s farther from the rest of the chip. At some point, you’d expect there to be a cost optimal HBM size, just like we see for skyscrapers10.

Other avenues for hardware improvement are drying up too, as Tim Dettmers explains:

In the past it was possible to shrink the size of transistors to improve speed of a processor. This is coming to an end now. For example, while shrinking SRAM increased its speed (smaller distance, faster memory access), this is no longer the case. Current improvements in SRAM do not improve its performance anymore and might even be negative. While logic such as Tensor Cores get smaller, this does not necessarily make GPU faster since the main problem for matrix multiplication is to get memory to the tensor cores which is dictated by SRAM and GPU RAM speed and size. GPU RAM still increases in speed if we stack memory modules into high-bandwidth modules (HBM3+), but these are too expensive to manufacture for consumer applications. The main way to improve raw speed of GPUs is to use more power and more cooling as we have seen in the RTX 30s and 40s series. But this cannot go on for much longer.

He goes on to mention specialized logic and low-precision arithmetic as paths forward. But these have limits too.

If we had no more ideas, AI hardware would stop there. But the holy grail of inference compute is compute in memory. Since inference is a bunch of simple operations, make the arithmetic cores small and repeatable like memory. Put memory and arithmetic right next to each other and glue together massive amounts of compute.

This should increase the memory bandwidth a lot while increasing costs somewhat less. On balance, it’s not clear where the cost per unit of memory bandwidth will end up. After that, we’re kind of out of ideas at the design level11. But there are decades of tweaks we can pursue.

Implications

Everyone makes chips, costs fall

In the semiconductor industry, tech that’s ~10 years old is roughly accessible to any country. The trade secrets have mostly diffused, the equipment can be procured, and the talent from that era can be hired. Consider how China leads the world in producing trailing edge chips while still failing to produce leading edge chips.

In a world where chips stop getting better, everyone catches up to the state-of-the-art in about 10 years. That means competition, innovation, and falling prices.

Even better, since chips stop improving, depreciation costs fall dramatically. You can keep using the same hardware for decades.

AI converges to a price per unit intelligence

There’s already plenty of competition over model provision, especially from open source providers. When the monopolies in chip production fall, we get even more competition in this space.

What does the intelligence market look like in this world? I expect RL-as-a-Service (specialized models) will dominate. The base models these are built on will converge to a certain price per unit intelligence, itself dependent on the price per FLOP beneath it. I imagine a range of different base model sizes, with the largest sizes serving an inference oligopoly and smaller sizes seeing lots of competition and low margins. Specialized model providers can turn a profit by building domain knowledge on top of small models.

For more, see Economic futures for LLM inference.

Two AI hardware niches

There’s two niches for AI hardware in the future.

One we’re already familiar with, chips and servers optimized to be in huge data centers. Here, high memory bandwidth at low cost is the key. This hardware can provide low cost, low speed AI for very cheap by using large batches. Particularly valuable for supplying and inference oligopoly or autonomous AI workers.

On the other end of the spectrum is local AI hardware. Specialized chips in your laptop. These run small models (smaller memory footprint) and don’t have high utilization, so FLOP/s/$ is the key metric here. These models can be extremely fast and responsive, perfect for collaborating with your AI assistant.

Will anyone want space compute?

I’m unsure whether there will ever be much demand for space computers. The previous section showed why space computers are more expensive, at least until compute demand outpaces Earth’s ability to provide it.

It’s not clear that terrestrial demand will get that high. AI models are keeping the same memory footprint while increasing capability. We might see a Kuznets curve of AI compute demand once Claude writes all our apps for us. I would guess per capita compute consumption isn’t rising much outside of AI. Future netizens might still be reading blogs and watching YouTube.

And remember, the latency of servers outside of LEO (where most of the computer-chip-material is) is far higher than terrestrial servers.

Why would we want such a large amount of slow and expensive compute? I can think of one reason: The Age of Em12.

Conclusion

Traditional semiconductor manufacturing will win for the foreseeable future. In perhaps 30 years, progress will come to an end. The state of the art techniques will diffuse everywhere. The end of semiconductor monopolies and hardware depreciation mean compute will get very cheap. Perhaps another wave of innovation will bring an unconventional computing paradigm like optical to the fore. But I’m not holding my breath.

Space computing is the final frontier, the constraints are different and the scale far beyond what Earth could support. There we might find solutions new and bizarre.

Appendix

Charts!

Please enjoy these slapdash charts of hardware price-performance vs time. I focused on the two main performance metrics for AI chips.

I modified the Epoch data from here to make this spreadsheet. Note the values are in nominal terms, in real terms there’s been more improvement over time. I’m personally surprised at the lack of growth and it’s possible I’ve made an error.

Note that future AI performance depends more on BF16 and smaller number formats. Likely we’ve seen much more dramatic improvement in these formats. The Epoch data on this was sparse so I stuck with FP32.

Further reading

From me:

Economic futures for LLM inference

RL-as-a-Service will outcompete AGI companies (and that’s good)

An intro to the Tensor Economics blog

On AI Scaling

The future of lithography

From others:

Just read all of Zach’s tech blog, he’s more qualified.

The Transistor Cliff—Asterisk

The Rising Tide of Semiconductor Cost

The Unsustainability of Moore’s Law

How to Build a $20 Billion Semiconductor Fab

Semiconductor Fabs I: The Equipment

TSMC’s new 2nm chip will reportedly cost 50% more — get ready for more expensive laptops and phones

The longest training run. When AI hardware keeps progressing, it only makes sense to do short (~1 yr) training runs.

AI hardware and memory technology

The Special Memory Powering the AI Revolution

The Memory Wall: Past, Present, and Future of DRAM

[2601.05047] Challenges and Research Directions for Large Language Model Inference Hardware

The Breakthrough Solution to DRAM’s Biggest Flaw probably the most exciting research on memory technologies I’ve seen. Get rid of the capacitor and just use transistors for memory. Works particularly well with compute in memory. Though there are risks from using new materials and taking on more routing complexity.

Zach’s tech blog linked this presentation on 3D-split SRAM by Rahul Mathur. I don’t fully understand but I’m including it here because FOMO.

Cool stuff you can do with electron beams

Multiple-electron-beam direct-write comes of age

Hundereds of thousands of beams possible:

TSMC's Incredible 2nm Curvy Masks - YouTube

Resolution Limits of Electron-Beam Lithography toward the Atomic Scale

Transverse Electron Beam Shaping with Light

Nanostructuring of electron beams

Learning and Controlling Silicon Dopant Transitions in Graphene using Scanning Transmission Electron Microscopy

Precise atom manipulation through deep reinforcement learning

A Platform for Atomic Fabrication and In Situ Synthesis in a Scanning Transmission Electron Microscope

Atomic Semi • The Make Anything Company

Of course, the researchers don’t have evil intent. They believe it will help us stay ahead of outbreaks. The consequences of research are uncertain and experts disagree about the difficulty of building a bioweapon. Regardless, there are better ways to prevent pandemics.

I don’t want to list a bunch of examples lest I spread dangerous information. The example I’ll discuss here is in a famous journal so I’m less worried about covering it.

Bat-infecting merbecovirus HKU5-CoV lineage 2 can use human ACE2 as a cell entry receptor was published in Cell, the preeminent journal in biology. This paper looks at bat virus called HKU5-CoV lineage 2 and finds that it can effectively bind to a proteins on human cells. “… HKU5-CoV-2 infected human ACE2-expressing cell lines and human respiratory and enteric organoids.” They identified a circulating disease in bats with the potential to jump to humans and detailed why the viral proteins bind effectively.

This research is risky for a few reasons:

1. These pathogens are already viable in wild populations and could potentially transmit to humans. They pose a pandemic risk on their own.

2. They tell people where these pathogens can be found.

3. They detail why the surface proteins on the virus are effective at binding human cells, making it possible to replicate this in a genetically modified virus.

Better ways to address pandemics

Researchers and grant-makers justify this sort of work as a means to monitor and prepare for real world pandemic threats. Know thy enemy. Fortunately, the attendant risks are completely unnecessary. We can address pandemics via1:

• Metagenomic sequencing of wastewater to identify fast growing diseases.

• Surveilling diseases in farmed animals. These are huge, densely packed groups that interact with a lot of people.

• Far-UVC lighting, upper-room UV, glycol vapors, air filtration, surface sanitation, and PPE can slow the spread.

• Phone based contact tracing and rapid testing

• mRNA vaccines can rapidly provide herd immunity to the population2.

• Virulence management

• Effective remote work and remote education

Conclusion

Governments should stop funding pathogen discovery research and gain of function research. I say this as someone who is very optimistic about innovation and trying weird ideas. Studying zoonotic pathogens isn’t worth the risks, we have other ways to mitigate pandemics.

Resurrecting modern people

What would you be like if we did your life over again? Say you’re born into the same family, go to the same schools, with the same friends, and the same jobs. But the little details are different. Instead of hanging out with Kyle one day you’re with Michael. Or you fail a test because you had one too many drinks last night.

How different would you really be? I think for most people, you would be substantially the same person1.

Consider those spooky stories of twins separated at birth and reunited as adults who are so similar they named their children the same. This isn’t as crazy as it sounds. You’ve got two people with the same genes, living in the same culture, at the same time, in similar regions, as foster children in families with similar socioeconomic status. We know that these things matter for life outcomes, two people with the same circumstances should end up being very similar.

All this to say that if your clone redid your life, that person could reasonably be called a copy of you. A partial copy to be sure, but would any of your loved ones be able to tell the difference?

We can go a step further. Say we can run a simulation of you and your experiences. We watch carefully to see if your simulated life diverges from your real life. Choose the wrong college? Tweak some parameters and reroll.

At the end of this process we have a simulation of your life that hits all the same beats. A clone that made all the same decisions that you did. If you avoid overfitting, this clone might be essentially identical to you.

Such a simulation is beyond today’s technology, but may be possible in the future. But how will people in the far future know enough details of your life to resurrect you? What can they benchmark your simulation against?

Your social media posts of course. Social media provides a timestamped record of what you were thinking about. A simulation of you might reproduce a post word-for-word at the exact same time. That would provide strong evidence of your resurrection.

There’s a few things you can do to improve your chances of being resurrected in this glorious post-human future. Post on social media a lot, more data points gives you a better chance of surviving. Photobomb other people’s photos too. That gives the simulators a chance to cross-reference information about where you were. Also publish lots of information about yourself like your full genome, your biography, etc. in some permanent repository.

Social media companies are naturally the most important companies in the world. Let us pray that they survive millions of years so we might be reborn in the Dyson swarm.

Turbo-narcissism

Like every post on the simulation hypothesis, we can say something like and this is what’s happening right now you’re a simulation oooh. Resurrection is fun because it imbues the simulation with purpose. You’re not mold growing in someone’s simulation, you’re the center of attention.

Now you can enjoy a god complex by believing that you grow into a great mind the future desperately wants back. Or this life is a bizarre form of torture constructed by your enemies. Or perhaps you are merely here to entertain posthuman beings (did you hear a laugh track?).

The best part is these stories are unfalsifiable, you’re free to believe whatever you want.

Backchaining further in the past

Resurrecting social media poasters is a far cry from resurrecting everyone. Can we bring back people from before social media?

If those people interacted with someone who eventually used social media, you may be able to back out some information about them. Your grandparents behavior influenced you and left a mark on your digital traces. In reproducing your mind, your interactions with your grandparents can be used as a free parameter.

For people further in history, at the very least we can use their genes, social structure, and life history to produce a person very similar to them. We’d need to find social events with a known outcome which involve a smallish number of people. The result of a spelling bee or baseball game perhaps. The minds of the people involved become the free parameters, tweaking and rerunning the simulation until things play out the same way2. This won’t achieve the same fidelity as a social media post, but you can narrow things down to a feasible set.

Further in the past, things get fuzzier. You might still be able to infer their genes, but the best you can do is shotgun sample many different lives, keeping the broader sweep of history consistent.

All possible humans?

Why resurrect only the people who had lived? Why not resurrect all possible human minds?

You see, your genes contain only 1.5 MB of information. For a being that can rerun simulations of human society, sampling different genomes in different societies might be trivial.

What would be the purpose of instantiating all (human) minds? Perhaps they’re doing a The Egg thing. Or perhaps this parliament of minds can solve philosophy behind a veil of ignorance. You should ask them when you get there.

Conclusion

Resurrection creates some tantalizing possibilities. We can learn more about history, kill Hitler again, and discover how hot famous people were.

It’s far beyond our capabilities, but might be possible in the future. Indeed, it may be happening right now.

Statements like these are silly, I want to address some ideas about innovation that might be at the root.

Innovation is mostly trying stuff

The way I think about innovation is Trying Stuff. The best way to determine if something works is to try it in the real world. Most innovation comes from trying different things in a particular domain and keeping what works. Everything else is secondary. Fancy theories make trying stuff more efficient, but aren’t required for trying stuff to work. Standardization and automation are valuable because they accelerate the process of trying stuff.

Most major discoveries were made by people trying stuff, with theory and standardization happening later. Steam engines were in use before thermodynamics, many drugs were discovered on accident, and the green revolution happened because of systematic tinkering.

The implication is that brilliant theory can only get you so far. You need to actually collect data and try things out to make a discovery. The trying things out part is the bottleneck in the process. You have to physically move atoms around and wait for the results. A data center full of Nobel Laureates would spend most of their time waiting for results, not making discoveries1.

Eventually, trying more stuff is useless

Trying stuff comes at a cost. There is the physical cost of doing the experiments and the opportunity cost that you could have done experiments in some other domain. Combine with diminishing returns and eventually it doesn’t make sense to continue experimenting.

This is stronger than the idea of “low hanging fruit”. Not only do the easy opportunities get found, it stops being useful to even look for new opportunities in a particular area. The marginal benefit is too small.

Most domains see an end of history

At the end of my series on solving climate change I noted:

After looking at the future of cleantech, it feels like we’re nearing an equilibrium. The end of the century may see the end of terrestrial engineering. We found cheap energy sources that won’t be usurped in the forseeable future. The quirks of renewables will propagate into our production processes. After that, what’s left to change?

This is true of all technology. We find an effective way to do something, tweak it a bit, and then kind of stop2. This is an odd sentiment in an era defined by ever improving computers, but look outside the enchippening and you’ll see it’s true.

Food products like bread, cheese, meat, beer, and wine are made with techniques that would be recognizable to people centuries ago. Indeed, rain-fed agriculture still provides virtually all of our food, as it has for millennia. It looks unlikely that vertical farming will change that. Of course, we have substantial automation in agriculture, but that’s because we didn’t find an alternative to growing crops. We found new ways to increase yield outside of the growing process.

Even semiconductor fabrication and AI scaling follow the same pattern. We scale up an idea until it reaches physical limits then innovate elsewhere. Smooth progress comes from stacking hundreds of S-curves.

After many S-curves, progress slows. Haber-Bosch is how we produce fertilizer today and I’ve realized we aren’t going to find anything better for the foreseeable future. It takes iron and air and makes bread for gods sake. Or consider that silicon, the second most abundant element in Earth’s crust, has a bandgap that makes it perfect for absorbing the sun’s emission spectrum in solar panels. Need we search further?

Look around at everyday objects and this insight becomes trivial. Things like cups and chairs and pens are already optimized. There is little need for further cup research. There is no magic recipe for making cups more efficiently. Our world is grounded in physics and it’s unlikely we’ll find a substantially better production process.

So innovation jumps around. We encounter a new problem, fiddle around, find a good solution and then move on. Everything converges as we figure out the right way to do things. Unless our circumstances or way of life changes, we should expect society to crystallize around specific technologies and production processes.

That said, there’s still so much room for improvement. The world that we’re converging towards is wildly different from our own. But if we’re going to build a better world, we need a realistic understanding of how innovation works and what is possible.

EDIT: Further Reading

Is Science Slowing Down? By Scott Alexander. Lays out the low hanging fruit theory and suggests it that exponential resource investment to get steady technological gains should be the default assumption.

Tyler Cowen’s book The Great Stagnation.

Are Ideas Getting Harder to Find? Classic paper from Bloom et. al.

Recipes and economic growth: a combinatorial march down an exponential tail by Chad Jones. An economic model of innovation as trying stuff by sampling from a distribution. A toy model that accords with my understanding of the innovation process.

Mundane example of innovation stagnating: why AA batteries haven’t gotten better despite so much progress in battery tech.

There has to be a better way to make titanium. A more technical example of how hard it is to innovate on today’s frontier. Look at how many times people have tried and failed to improve titanium production. Orca has narrowed the possibilities and may, if successful, find the final right way to make titanium.

Something I wish I’d thought of when writing this piece: consider the innovations of mother nature. Evolution hops around, finding locally available mutations. Species optimize for a particular niche and then stop. When a new niche arises new “innovation” happens until it is filled. Predator and prey find an equilibrium. Consider how invertebrates lack an acquired immune system yet live in peace with their parasites. Human affairs mimic the wild; innovation tinkers, converges, and halts.

People also pointed me to Ideas aren’t getting harder to find by Karthik Tadepalli. Here was my response:

… We’re saying similar things about low-hanging fruit, but Tadepalli suggests an alternative theory blaming declining allocative efficiency. I’m all for fixing allocative inefficiency, I’m sure that’s part of the problem.

Though I maintain that even if we unblocked the innovation process, that doesn’t change the fact that we’re converging on a right way to produce goods. We’re bound by the laws of physics and economics. …

Set aside your judgement and see this post as describing empirically how dating apps work. Push through the confusing sentences and you’ll find a wealth of information about human behavior.

The appendix of this post has more notes, but some things that jumped out at me:

1. The apps have solved the recommendation problem. “Recommendation of profiles that you may like is a solved technical challenge at Tinder level and at mostly any dating app today.”

2. Retention of users is the only thing that matters for these compamies.

3. Retention depends entirely on a personalized feed algorithm.

   The key to produce the best feed algorithm (if you target retention) is personalization more than recommendation. Of course seeing users that you really like every once in a while will increase your retention. But what really increases retention is understanding what each user wants. And we all want things different. Some people will want to see only a few people that they like a lot, some people will want to see a lot of users. Some people will want to see people that are far from them, some close, some can’t stand seeing people they don’t like, some can’t stand seeing people they only like. Some want to receive a lot of likes, some want to receive a few likes

4. Current dating apps are local optima. “What makes you keep scrolling on TikTok or YouTube or Instagram? It is the feed, it is the same on a feed-based dating app. Pursuing retention will just bring you to recreate Tinder.” To create something new, you need to aim at something other than retention.

One theme that lacks a pithy quote but seems to run throughout is that people want attention. They like getting matches and talking to people. I think this is a deep part of human behavior and drives a lot of social dynamics.

2.

Parents’ effective time endowment and divorce: Evidence from extended school days. When the school day is extended by 3.5 hours, mothers can work their own job and gain economic independence from their husband. This leads to an increase in divorce rates.

It’s a good thing that childcare gives these women the opportunity to work, to support their family, and to leave bad marriages.

How do we scale childcare? Baumol’s cost disease threatens to make it perpetually expensive. One option is to make it universal and government-run. Though this misses out on possible efficiency gains. A voucher system might work, effectively subsidizing childcare while allowing the market to find good solutions.

But neither of these address the risk of education signalling spirals like we see in South Korea. What happens if all this subsidized childcare turns into a red queens race of exam study?

In a more reasonable world, we would recognize that new technology means students can keep up with the curriculum with a mere 2 hours of work per day. Older students would be able to speedrun several grade levels, there’s no need to teach certain concepts so early. The rest of their time could be spent on the playground and they would turn out just fine.

But I don’t know how to coordinate everyone around that idea1. Seems important.

3.

ACX has some good discussion of why the economic vibes are bad:

Vibecession: Much More Than You Wanted To Know

Highlights From The Comments On Vibecession

They offer many tantalizing possibilities, but we should keep things simple. Peoples vibes are completely unreasonable and unmoored from reality. Instead, the main thing bringing people down is more media consumption and negativity in the media. This explains the long term and global reduction in vibes since ~2010.

Several things exacerbated this emotional sentiment:

• Post-pandemic inflation lowered some peoples real incomes and made others feel poorer because of the money illusion.

• The price of buying a house (see image) rose sharply because of COVID, persistent inflation, NIMBYism, tariffs, and immigration crackdowns. Rents have increased slightly. But rents in highly-desirable metro areas where all the exciting opportunities are have increased more.

• Removing application frictions in college apps, job apps, and dating makes people experience more rejections. They’re doing more work to stay in the same place.

• On top of all of this, Zvi points out that people expect more out of life than they did before. In addition, regulations in childcare, healthcare, education, and housing make some necessities far more expensive. Much more at the post.

So, what are we going to do about it? Obviously better policy can address many of the bullet points. But that doesn’t face the broader problem of negativity in the media. This isn’t the first time I’ve bumped up against this problem. A previous linkpost started with the decline in marriage and ended up here:

But really staring into the abyss requires recognizing that better and cheaper entertainment media is interfering with our social fabric. I’ll never suggest regulating internet use, but maybe giving students a Defense Against the Dark Web class would help. I’ll have to think more about this.

I get the sense that the internet is driving a wave of negativity, damaging our social fabric and driving people to unreasonable beliefs. This makes voters more unreasonable. This drives bad policies which produce stagnation.

The internet has brought many wonders, but how do we patch this?

Everything else

Money Doesn't Buy Elections. It Does Something Worse. Proposes that donors influence who is able to run for elections rather than influencing voters. This gives them control over what policies are acceptible.

Vagus nerve-mediated neuroimmune modulation for rheumatoid arthritis: a pivotal randomized controlled trial. Another win for treating ailments by manipulating the brain.

We Induced Smells With Ultrasound

Memazing. Sebastian Seung (one of the leading connectomics researchers) is starting a brain uploading company.

Microscopic-Level Mouse Whole Cortex Simulation Composed of 9 Million Biophysical Neurons and 26 Billion Synapses on the Supercomputer Fugaku. What happens when a crappy neural simulation starts talking to us? Just turn it off?

AI infrastructure in the “Era of experience”. Another great post from the Tensor Economics blog. Essentially argues that RL-as-a-Service will come to dominate. Goes into technical detail on how datacenters can serve many fine tunes of the same base model (multi tenancy). These fine tunes will be in the form of LoRA adapters trained on proprietary data. Training can get also get gains from multi tenancy and LoRA. See also my piece on Economic futures for LLM inference.

RF Over Fiber: A New Era in Data Center Efficiency. Two companies are using plastic waveguides to send radio waves between chips. This allows for the type of high-bandwidth connections needed in the age of AI. They think it will be cheaper, more reliable, and more energy efficient than optical interconnects.

Antimatter Development Program

Appendix on dating apps

Profile photos

• Profile picture is pretty much the only thing that matters for determining if someone likes another person.

• The percent of people who like your profile is sufficient to measure hotness. You don’t need ELO or anything fancy.

• Attractiveness in real life is distinct from profile photo attractiveness. Generally women look better in photos while guys look better in real life. Attempts to level the playing field by having people take videos didn’t work because women didn’t like how they looked in video so wouldn’t join.

Other profile info

• People are willing to fill out lots of information about themselves. This information is virtually useless for matching people. But people like to do it so it’s included as part of the user experience.

• People are terrible at telling you what they want, their inputs are useless. 2-3 go/no-go parameters is all that is helpful.

• Women sometimes use this other profile information to filter out certain men. Older women put more emphasis on it.

The feed algorithm

• “Recommendation of profiles that you may like is a solved technical challenge at Tinder level and at mostly any dating app today.”

• BUT you can’t just feed people matches that you think they will like. New users need some activity on the site before you know what they like. And these new users need to be reviewed in turn by other users. “People don’t want to see only users that they like, girls actually adapt their behavior if they see too many guys that they like.”

• “The key to produce the best feed algorithm (if you target retention) is personalization more than recommendation. Of course seeing users that you really like every once in a while will increase your retention. But what really increases retention is understanding what each user wants. And we all want things different. Some people will want to see only a few people that they like a lot, some people will want to see a lot of users. Some people will want to see people that are far from them, some close, some can’t stand seeing people they don’t like, some can’t stand seeing people they only like. Some want to receive a lot of likes, some want to receive a few likes”

• “The feed algorithm is the only thing that will impact the retention of users on your app. Everything else almost doesn’t matter.”

• Lots of tricks in the feed such as showing women a mix of guys they actually like, guys they feel they are supposed to like, unattractive men. You also want to ensure women are receiving a stream of likes and messages, so men’s feeds might be oriented towards seeing women that haven’t gotten sufficient likes today.

• Women seem to like a fixed percentage of the profiles they see. That means the context of other men in their feed affects how often they like a particular guy.

• “Girls scroll about 2 times more than guys. (Fortunately). First, the like ratio of girls is 4%. So to be able to like 4 guys they have to see 100 users. And for a guy, it is 18 users. It also means the experience on a dating app for a girl is completely different than the one for guys. A girl spends a lot of time just searching for guys she will like, and guys spend a lot of time hoping a girl will like them back.”

Retention

• The main focus for apps like Tinder and Bumble is retention of paying users.

• Apps have monetized and it hasn’t changed the experience much. Men pay.

• For women, only thing that drives retention is number of likes sent. At the beginning, the most important thing is seeing a match quickly after signing up.

• For men, nothing seems to impact retention much.

• Need to do moderation on what accounts are allowed. Obvious fake users hurt the site. Subtle fake profiles exist and don’t have a big negative impact. Their site blocked ~all of them and didn’t impact retention much.

Current dating apps are local optima

“What makes you keep scrolling on TikTok or YouTube or Instagram? It is the feed, it is the same on a feed-based dating app. Pursuing retention will just bring you to recreate Tinder.” To create something new, you need to aim at something other than retention.

Dating apps provide entertainment, not dating

• Some people just want to chat on the apps and meet others, they don’t really go out.

• Some people just match up well and end up meeting in real life, but there’s not much the app designers can do to change that for the better.

• “People were not speaking for very long conversations in general, they were happy to start a chat, but never really cared to continue it. Which always makes me say, that people registered on these apps are not really here for dating, but more for entertainment purposes.”

1.

The Qattara Depression project is an old proposal to create a big sea in the middle of the Egyptian desert by channeling water from the Mediterranean sea to a region below sea level. Proponents suggest it could generate lots of electricity and create good conditions for agriculture.

I just realized it’s a dumb idea. First, the Sahara is awash in cheap solar, so it does not need expensive hydro. Second, Egypt is already building out agriculture in the desert by bringing in freshwater. Third, an above ground-sea in the desert is the worst place to put water. Especially when you can store ~660x more water in the Nubian sandstone aquifer system underground.

There’s an opportunity to use solar and desalination at scale to keep the aquifer full and enable agriculture in the region.

2.

Luzia Bruckamp highlights econ job market papers related to fertility. I recommend reading for interesting summaries on the effects of paternity leave, C-sections, and more on fertility. The paper I found most striking was Build, Baby, Build: How Housing Shapes Fertility which finds that “[r]ising housing costs explain roughly half of the decline in the total fertility rate between the 2000s and 2010s” in the U.S. The author’s model suggest that by building large units desired by families, America would have returned to replacement fertility.

Not a job market paper, but related: The Covid-19 Baby Bump: The Unexpected Increase in U.S. Fertility Rates in Response to the Pandemic. I speculate that the transition to remote work during Covid encouraged U.S.-born mothers to have kids. Remote work gives people the opportunity to balance career and family and I wish governments would support a transition to hybrid work.

By this point it’s clear that rich countries can solve their fertility issues. They can build more houses, their workers are skilled enough to be remote, and they have the tax revenue to fund a baby bonus.

Middle-income countries are more at risk. Places like India and Mexico are seeing sharp declines in birth rates. They have fewer resources to return to replacement fertility and a larger shortfall than the U.S.

3.

Batteries make it possible to scale flight down, just look at drones. Gas engines are hard to scale down like that. That gives us a limited version of flying cars.

Scott Manley reviews the Pivotal Helix an eVTOL with 20 mi range, 60 mph speed, 75 minute recharge time, and Level 2 autonomy.

Like the Electra Ultra Short, I initially dismissed this because of the short range. But then I realized how valuable point-to-point flight is. 20 miles is enough to land you anywhere in a major metropolitan area. Imagine walking to the nearest park and taking a 20 minute flight to anywhere in a circle that is 1200 square miles (3200 km^2).

Eyeballing the chart below, people in metropolitan areas commute less than 40 miles a day. The straight-line distance would be smaller and the Pivotal Helix can already cover that.

With a little more speed and range, 15-minute cities are possible.

More speed and range are definitely possible. The Electra Ultra Short aims for 350 mph using electric motors. The challenge being that going faster lowers your range quickly. That’s why the Ultra Short relies on energy-dense fuel to get an acceptable range.

Fortunately, battery energy densities are improving. CATL is promising batteries that (under ideal conditions) offer 2x higher energy density than other lithium chemistries. These are more expensive (for now) but could almost double the range of our flying cars.

4.

In the last decade, the semiconductor industry has switched to making photomasks with a machine that uses hundreds of thousands of electron beams to draw a pattern. The design of these photomasks is optimized with a supercomputer, producing a curvy mask that bends light to produce the proper patterns on the silicon.

Innovation in electron beams doesn’t stop there. The paper Resolution Limits of Electron-Beam Lithography toward the Atomic Scale demonstrates a technique that achieved features close to 2 nm, much smaller than can be achieved with photolithography techniques.

Today electron beam lithography isn’t practical because it is much slower than photolithography. But these two results make me think. If electron beams produce smaller features and we have the ability to produce almost a million simultaneous beams, might electron beams catch up to photolithography in a few decades?

Everything else

An Opinionated Guide to Privacy Despite Authoritarianism. The most detailed guide to privacy that I’ve seen. Much needed.

Scientists Just Connected the Dots Between Viruses and… Everything. Infectious diseases like mono and Covid come back to haunt us many years later, causing dementia, heart disease, and cancer. Stopping the spread of mild illnesses could have big benefits.

Measurement of directional muon beams generated at the Berkeley Lab Laser Accelerator What are the implications of this tech? Muon beams can be used to scan deep into the Earth for mining. Though I’m concerned this might be used to hunt submarines, destabilizing nuclear deterrence.

EDIT: State of Brain Emulation Report 2025 (H/T Andy Mckenzie).

Lots of talk these days about an AI bubble. It’s all a bit muddled, particularly because people refuse to define “AI” and “Bubble”.

I’ll try to be more concrete by riffing off the discussion of LLM inference economics.

Three challenges for AI inference companies

Serial monopoly in chip making

The only thing worse than buying from a monopoly is buying from a serial monopoly. A series of monopolists can charge markups at each step. For the foreseeable future, chip production will be a serial monopoly. This is a natural consequence of how challenging and specialized the industry is.

Not only that, but the software that fabs and chip designers use is an oligopoly. Many parts of the semiconductor supply chain are like this. Though TSMC exerts some monopsony power.

The end result is that inference providers have to give much of their profits to the serial monopoly. That inherently limits their returns.

Competition in model provision

There are now roughly a dozen companies that can produce leading AI models. Over time, these models have become less differentiated and progress on real world tasks has slowed. The result is competition that brings profits close to zero just like we’ve seen with embeddings.

The elephant in the room is open-source models. They lag the benchmark performance of frontier models by roughly 3 months1. With open models, anybody with the money to assemble GPU’s (and the expertise to optimize their serving configuration) can now compete for inference demand. That opens up a lot of competition.

Technical risks

Buying hardware based on current AI models is a bet that AI inference will look similar in the future. If people discover how to do inference with less computation, that hardware spending is wasted. The clearest danger is small models. People keep figuring out how to pack similar performance into less memory. This is one reason why the trend towards larger models in the early 2020’s has stalled2.

New architectures might come out of this small model trend. While I don’t expect a new model to render GPU’s useless, a proliferation of tiny models might change the economics, leaving current players with sub-optimal hardware.

Even if AI inference stays roughly the same for the next 5 years, inference companies have to contend with model theft. The outputs of a model can be used as synthetic data to train a new model, which is hard to mitigate. Worse, the models are targets for theft by state actors.

Niches for AI inference

These challenges make it hard for inference companies to achieve the dominance enjoyed by other tech giants. They will continue to offer inference to the general public, just as today they offer embeddings despite low profits. But they will explore other options:

• Inference oligopoly. Inference has gains from scale which produces winner-take-all dynamics. A few companies with the majority of inference demand can have low operating costs and monopoly profits. AI companies are pushing hard to be the next Google. We’ll see if that pans out, or if GPU scaling plateaus soon enough to admit many players3.

• Enterprise AI. Economies of scale mean customers willing to buy inference in bulk can enjoy lower prices. Large companies may be willing to commit to one provider, particularly since it can offer employees a productivity boost. Additionally, the low-speed, low-cost inference achievable with large data centers and bulk purchases works well for asynchronous AI workers.

• RL-as-a-Service. This is a buzzword I use to refer to the task of helping people apply AI to their idiosyncratic problems. The challenge of automating a task creates a moat, and I expect many different providers in this space.

• Local AI devices. For people using AI assistants, response time matters. Brett Victor showed us the magic of responsive interfaces. Big data centers can’t help much with fast response time. In the limit you have your own powerful GPU running your AI assistant.

So what about the bubble?

Here’s a plausible path for AI companies:

1. Big AI companies spend lots of money on hardware and R&D hoping to enjoy massive gains from scaling.

2. Scaling laws continue to hold. So training and R&D have disappointing returns.

3. AI companies cut training and R&D and pivot to doing inference and RLaaS.

   1. GPU prices fall as companies sell the extra GPU’s they bought for training. Valuations fall if investors expected new sources of revenue from AGI. Though current P/E ratios look fine.

4. Without the moat from R&D and big training runs, everyone catches up to the frontier. Competition brings prices down to the marginal price and inference becomes ~100x cheaper.

5. Big AI companies stop trying to build AGI and look for other sources of profit.

So the end result isn’t doom for AI overall, just the end of billionaires trying to one-up each other with big training runs. Indeed, this path could lead to a proliferation of people building and serving AI models.

Despite the talk of an AI bubble fueling innovation, a crash would in fact be bad for AI progress. Like it or not, AI revenues are driving more research, better chip design, and new fabs. Not to mention the talent entering the field. A short term bust could set things back by a decade, and that would be bad.

It doesn’t have to be a bubble. Flagship AI models are already profitable. AI companies are safe if they scale hardware spend with inference demand. That means moderating spending on R&D and big training runs. And extending hardware depreciation schedules. They can also aggressively chase inference demand to get the benefits of scale in their data centers.

The path I laid out is already happening. AI companies have quietly saturated the scaling modalities while publicly hyping up scaling. Privately, I suspect they are coming around to RLaaS. Anthropic’s heavy focus on code generation is the beginning of this trend.

Appendix

Other things that help big AI players:

• Buying up AI researchers to keep results private.

• AI regulation (keeps out small players).

• Government subsidies.

• Low interest rates.

• Reliance on large models, which have more returns to scale.

• Making the stages of chip production more competitive OR combining the serial monopoly into a single monopoly.

• Protecting Taiwan from a Chinese invasion.

• Maintaining Taiwanese monopoly in leading-edge chip production as a payment to prevent alignment with China and as a means of committing to Taiwan’s protection.

These debates are too important to leave to muddy terms. What we are really trying to figure out is which things are moral patients. In fact, I think you could substitute something like “moral patienthood” everywhere you see the word “sentience” or “consciousness”.

Talking about moral patients is a big improvement. Instead of debating vague terms, we can ask “should this being matter at all in our decision making?”. Which suggests the further question “how do we make decisions when the needs of different moral patients are in conflict?”

I framed these questions as decision problems because we have real and important decisions to make regarding future generations, animals, AI, and alien life.

If we’re going to make real decisions, we should base these decisions on real things we can measure. The rest of this post argues that measuring “preferences” and “agency” can answer these sorts of questions. This framework implies new, productive questions we can use to specify different ethical systems.

But first, I have to address an objection1.

Can’t neuroscience solve sentience?

It sure would be nice if neuroscience could solve anything, let alone sentience. But after centuries of studying the brain, neuroscientists seem as confused as everyone else about sentience and consciousness.

I’m being a little harsh2. There’s a lot of interesting work understanding brain circuits and finding similarities between human and animal brains. And I’m personally exited for things like connectomics to give us a deeper understanding.

But there’s three problems with handing things over to the neuroscientists.

First, we need answers to these questions now. I talk to AI’s every day, factory farms kill hundreds of billions of animals each year, scientists found found signs of life on Mars, we might talk to whales soon, and if this neuroscience thing works out your own mind could be uploaded to a computer. We shouldn’t wait for an academic field poking around inside the most complicated object in the universe to solve our problems.

Second, say neuroscientists develop a complete understanding of the brain; it doesn’t absolve us from having to make decisions with that information. Knowledge of the brain doesn’t automatically answer moral questions. Neuroscience can inform those decisions, not make them for us.

Third, the questions we face here mostly involve non-human beings. It’s not clear that neuroscience will be able to say something useful about aliens or AI. Even for animals, our understanding of human consciousness might completely misunderstand octopuses. Trying to shoehorn our understanding of all these beings into our understanding of humans could lead to bad decisions.

Hopefully you agree that neuroscience isn’t The Way. What is?

Preference and Agency

There’s two important things we can actually look for in a variety of beings:

Preferences: when faced with a choice, the being chooses certain states of the world over others.

Agency: the ability to bring about states of the world it prefers across a variety of situations.

Preferences are the direction, agency is the magnitude.

I claim that these are things you can actually test. But I’m not going to cover this here. See the further reading section at the end for more. Now let’s look at some nuances.

Agency reveals preference

The two factors can mask each other. For example, something with absolutely no agency makes it impossible to measure preferences. A rock could have lots of opinions about where it would like to sit, but since it has no way of doing anything we can never observe those preferences.

In general, the more agency something has, the more we can learn its preferences in detail. Conversely, to measure preferences you need to provide artificially more agency and choice. For instance, observing animals in the wild would only show you their preferences for food and mates. But putting them in a safe and enriched enclosure you’ll discover a capacity for play in many species, even bees.

What having more preferences means

The concept of having more or less agency makes sense, what about “more preferences”? On a first pass you might think that stronger preferences are those that you feel with more intensity. This is misguided.

For most things, we can’t measure preference intensity directly. In a binary choice experiment, an animal that loves bananas and likes apples 10% less is indistinguishable from an animal that is lukewarm about bananas and likes apples 10% less3.

Another problem with preference intensity is that there are strong incentives to lie. This will come up when preferences are in conflict; any system that weighs preference intensity gives everyone a reason to become a utility monster. Collective decision algorithms often have to explicitly ignore preference intensity4.

Instead, having “more preferences” should mean that you have less indifference. In other words, you have fine-grained opinions about possible states of the world. Low-preference turtles only care about getting lettuce and sunshine. Sophisticated humans have opinions about the politics of countries on the other side of the world.

Lacking preference or agency, your behavior is driven by others

Just as lacking agency can hide your true preferences, lacking preferences can hide your level of agency. A superintelligence that doesn’t give a damn about anything is as immobile as a rock.

But things can still act in the world while lacking preferences. If you saw someone roll a rock down a hill you wouldn’t say it was because the rock preferred it. Likewise, language models today mostly do things because they were directed by a human being. We should assign these actions to the users preferences.

Note that when two beings with different agency interact, the one with more agency tends to get their way. A dog on a leash tends to follow their owner, not the other way around. EDIT: Likewise, something with more preferences tends to get its way on domains where it cares and others don’t.

The extremes

A paperclip maximizer is in the upper left because it has detailed preferences about possible states of the world and the ability to actually make that happen. It wants everything to be paperclips and proceeds to turn everything into paperclips.

A philosopher also has detailed preferences about the way things should be, but little ability to affect change (low agency).

An AI assistant has the agency to do many things, but its actions are mostly determined by the preferences of its human user.

A rock doesn’t care about anything and can’t change anything.

Questions implied by this framing

Preferences and agency are concrete; they clarify ethical debates. But they leave some questions unanswered:

• We talked about beings as if they are static. What about things become more agentic over time like babies? Should that change how we regard them in our ethics?

• What about augmenting something with more agency than it would naturally possess? You can give people tools, or teach gorillas sign language, or offer chatbots their choice of projects. Is that a good thing?

• What about giving something more preferences? Is it valuable to be more opinionated?

• What about creating something with agency and/or preferences? Is that good?

• What about a copy of something? Do its preferences have a different interpretation? What about beings that are merely similar?

Selecting answers to these questions is a step towards specifying a theory of ethics.

Sketching an ethical theory

Let me offer an answer to these questions to illustrate how much they can constrain your ethics.

I think anything exhibiting both preferences and agency has at least some moral value. Lacking either excludes a thing from moral value. So humans and microbes are in, rocks and subatomic particles are out. Plants are sort of an odd case which would depend on whether you think they have actual agency or act more like a physical process.

When we build an AI that exhibits clear preferences and agency, I would assign it moral value as well. I see no good arguments for why we should completely exclude AI from our moral calculus.

AI assistants are an edge case. They exhibit agency, but have explicitly been trained to avoid expressing preferences. Their decisions are mostly determined by the requests of their human users. You could argue that this means they lack preferences. Regardless, I’m nice to chatbots.

All else equal, I think it’s ethical to offer things more agency. More agency means more ability to pursue your own preferences. On the other hand, it seems wrong for someone else to change your preferences to be stronger or weaker5.

What about creating a new being? If you expect that being to live a good life in some sense, their existence is a good thing. But this critically depends on how it affects others.

I don’t really want to debate this stuff here, my point is that answering these questions gets pretty far towards an ethical framework. My answers suggest a philosophy that values agency, prioritizes individual preferences, is broadly pronatal, and has an expansive moral circle.

You can imagine how other philosophies might answer these same questions. Buddism might be viewed as seeking to reduce preference. Nietzschean’s might value agency above all else. Utilitarians are all about weighing preferences. Virtue ethics values the exercise of certain types of agency while rejecting others. Those that only care about human welfare might assume beings with low agency lack moral value. A libertarian view would restrict its attention to beings with enough agency to engage in positive-sum trade.

Conclusion

This discussion left us with a lot of questions, but they are productive questions. You can actually go and measure the preferences of different beings. You can actually test their ability to bring about states of the world they desire. By answering a few questions like “does something with agency but no preferences matter?” you can specify which beings have moral value.

The next step is how to make decisions when the needs of different beings are in conflict. This is a topic for another time, but I think we can get reasonable answers to that question by thinking about preferences, agency, and how beings interact.

Further reading

Related to this post

Measuring Animal Welfare, Part One: Maybe We Can Just Ask Them?

Towards Welfare Biology: Evolutionary Economics of Animal Consciousness and Suffering

Replication ethics

Fellow Creatures: Our Obligations to the Other Animals

The Moral Circle: Who Matters, What Matters, and Why

Interesting directions for a future post on collective decision making

Legal Personhood—Three Prong Bundle Theory

Formalizing preference utilitarianism in physical world models

Bentham or Bergson? Finite Sensibility, Utility Functions and Social Welfare Functions

Toward negotiable reinforcement learning: shifting priorities in Pareto optimal sequential decision-making

Uncommon Utilitarianism #3: Bounded Utility Functions

Normative Uncertainty, Normalization, and the Normal Distribution

The posts themselves are wonderfully detailed but somewhat overwhelming. I want to provide a summary of their work that might act as a guide. Then I’ll tie in some complimentary data from SemiAnalysis.

Note that I am presenting their posts in a different order to make it flow better. Each section selects a small amount from each of the original posts. Ideally, this post gives you the scaffolding to read their posts in full, they have a lot more insight.

Preludes

Time is money

Roughly 50% of AI data center spending is on hardware. Other CapEx1 grows in proportion to hardware costs2.

Hardware depreciates in value (and there are loans to pay off), so you need it to produce as many valuable tokens during its lifetime as possible. Naturally, our emphasis will be on the speed a GPU (and the supporting equipment) can do things.

Let me be clear about what it means for a GPU to depreciate. While GPU’s can slow down, or get more faulty, or break, the primary source of depreciation is obsolescence. Nvidia keeps making faster GPU’s. These are more expensive, but the higher speed counteracts the price and the overall cost per token is lower. At some point it makes sense to swap out your old (still functioning) GPU for a new one.

Defining some units

FLOP means “floating point operation”, such as an add or multiply. However, “FLOPs” and “FLOPS” are completely different units. FLOPs is merely the plural of FLOP, while FLOPS means FLOP/s. This is very dumb.

Most places assume people understand the difference. But clarity matters!

I’m going to use FLOP as plural (i.e. FLOP=FLOPs) and write FLOPS as FLOP/s. This is different than most texts but easier to understand in my opinion.

The rest of the units we will use are more clear. Memory bandwidth and communication speed are measured in gigabytes per second or GB/s. We will talk about “tokens” a lot (see next section) which I’ll shorten to “tok”. One million tokens is “Mtok”.

How transformer inference works

Instead of a full explanation, I’ll highlight a few things and point to others who have explained it better. It’s not critical that you know this stuff, but I want this post to be self-contained.

Language models process tokens, which aren’t quite words, more like “wordlets”. The image below illustrates this nicely. Essentially, you break a text down into a sequence of tokens from a pre-defined list. You can then associate each possible token with an index on this pre-defined list.

The next step is to take this list of input (prompt) tokens and in some sense embed the input into the LLM activations. Specifically, you do a forward pass through the transformer, applying the attention operation and feed forward blocks to the tokens in the prompt. This phase is called prefill or prompt-processing. It comes before the transformer can produce the first token of its response.

The attention operation applies to all pairs of tokens in the prompt and ensuing response. You can save a lot of computation by storing the results of the attention operation applied to pairs of tokens in the prompt. That way, each new token only needs to interact with previous tokens, you don’t need to re-do the attention operation for pairs of tokens you’ve already seen.

This store of previous attention results is called the KV cache. The details aren’t too important, but you should know that:

1. The KV cache is large enough that it needs to be stored in HBM and streamed in to the GPU as needed.

2. Longer prompts and longer LLM responses require a larger KV cache.

3. A separate KV cache is required for each prompt. In other words, if you have 10 user requests, you need to store 10 KV caches.

4. Typically the KV cache is much smaller than the weights of the model.

This discussion has glossed over a lot of details. It’s not necessary for this post, but to understand LLM inference from a high level, I recommend starting at this video in a 3blue1brown series on deep learning. One caution is that the specific way attention is done and the overall architecture of leading LLM’s continues to change, but the basic ideas are there.

This post from Tensor Economics covers inference in Llama 3 in more detail.

Weights must load from global memory creating two phases

There’s a tradeoff with memory: it can either be cheap or fast. There’s a whole memory hierarchy between small/fast/expensive memory and big/slow/cheap memory.

Some types of memory are located close to the processing center of the GPU. This memory can be accessed quickly, but it doesn’t have much capacity. It would be expensive to build more into the chip.

You can put lots of memory outside the chip for much lower cost, but this will be much slower. Even at light speed, small distances from the chip make memory access much slower.

Modern AI hardware has a large amount of off-chip memory located close to the GPU with a high-bandwidth connection to the chip. This is called high-bandwidth memory (HBM)34.

To actually do inference, you need to load portions of the model from the HBM into the GPU itself for calculations. That takes precious time. So there inference can be divided into two phases, a memory-bound phase where you wait for the weights to load, and a compute-bound phase where you do the actual computations. This is the key insight from which the next few sections derive.

FLOP/s/$: embeddings and the compute-bound phase

Why are embeddings so cheap? looks at the process of generating an “embedding” for a piece of text. Embeddings convert a document into a vector storing semantic information about the document. They’re a key part of using LLM’s to search for relevant text.

Inference providers like OpenAI charge $60 per 1M output tokens, while embeddings cost 10 *cents* to process 1M tokens. This is strange because (as we’ll see) inference and embeddings involve quite similar steps. Hence the question in the title.

Computing an embedding

Producing an embedding involves a single forward pass to load the document information into language model activations. These activations form the embedding.

The process of converting the document into activations is essentially identical to the prefill phase. So while embeddings themselves aren’t exciting, they give us some insight into the economics of LLM inference.

The compute-bound phase

We always need to load weights into the GPU and then run the actual computation. Which phase is the bottleneck?

Note that embeddings typically use a small-ish language model, less than 10B parameters. In the post, the authors offer a quick calculation based on a Qwen3-8B embedding.

So it takes almost 5x longer to do the computations as it takes to load the model from memory. So embeddings are primarily compute-bound. We want a chip that performs FLOP as fast as possible. In other words, we want hardware with a high FLOP/s/$.

The authors then do lots of benchmarking to show that this intuition is correct.

Why embeddings are so cheap

One last thing to understand why embeddings are so much cheaper than normal inference. All the embeddings provided by different companies converge to similar representations5. That is, their products aren’t differentiated from each other. That creates a lot of competition:

Our contention is that since the cost of processing a token is so minuscule and all embedding models converge to similar semantic representations, this means that no provider enjoys a sustainable moat. This results in underlying low costs being passed directly onto the consumers, driving prices to rock bottom.

That’s the key takeaway here. A few other interesting tidbits:

• They look at actual hardware and estimate that the true cost of embeddings could fall almost 10x to around 2 cents per Mtok.

• Interestingly, the real world performance (FLOP/s/$) of the RTX 4090 (older GPU) is better than an H100 (newer GPU).

• The authors suggest that “[t]he embedding situation offers a nice pre-taste of the “intelligence involution” that is coming …”. Hinting that they think the same competitive dynamics will come for LLM inference.

GB/s/$: inference and the memory-bound phase

LLM Inference Economics from First Principles is a massive post. They use Llama3-70B as their guide for the discussion. It’s open-source, easier to understand, and much of the industry has standardized around it.

Storing Llama3-70B takes 141 GB, this is more than the ~80 GB of HBM found on typical hardware like an H100. So right off the bat, you’ll need to load the model across multiple GPU’s for inference.

The post goes into great detail on the FLOP required to do inference on this model. It’s a great reference, but beyond our scope.

Equipped with an understanding of how many FLOP are required for inference, they estimate how long it would take to do prefill (prompt processing) for an input of 2048 tokens (~6 pages of double-spaced text).

To do the computations, it would take 0.29 seconds. To load the weights from HBM, it would take 0.04 seconds. The computations are the bottleneck, so the prefill phase is compute bound, just like with embeddings.

But prefill is just one step, what about actually generating output tokens? After a detailed discussion of the decoding (token-producing) phase, they give an estimate. It would take 0.00014 seconds to compute 2049 tokens of output. Yes, that’s the correct number of zeros.

So generating tokens is way, way faster than loading the weights from the HBM. This phase is primarily memory bound! That means you want hardware with a high memory bandwidth, FLOP/s isn’t as important for this phase.

They say:

... one of the insights we hope you take out of reading this article - the token-by-token phase is memory bound; the amount of available compute throughput is of secondary importance, as we massively underutilize the compute resources anyway while waiting for weights to be loaded.

They go into some nuances about what happens at longer input lengths and consider various tricks to run inference in parallel across GPU’s. Once again, out of scope. Instead, let’s look at the implications of being memory bound.

Batching

So no matter what, at each phase of inference, you need to load the parameters from that phase into the GPU and then do some calculations. Loading those parameters takes a fixed amount of time. Batching is the idea that, while the weights are loaded for that phase, you also perform the necessary calculation on another customer’s input6.

Imagine your job is to shred paper. You can either switch on the shredder and run it each time you get a piece of paper, or you wait until you’ve gathered a stack of paper, turn on the machine once, and shred a whole stack at the same time.

So batching waits for several customer requests and then runs inference on them all at the same time. There’s a fundamental tradeoff here: batching means that inference costs less money, but customers have to wait much longer to get their response. For example, batching 8 customers might cost 8x less and take 8x longer to get a response.

This is a big economy of scale for inference providers, the more customers you have, the lower price you can charge. Having more GPU’s to run inference in parallel helps too.

GB/s/$ matters

After all this, they run some benchmarks to show that their model lines up with reality. Real world inefficiencies make the true performance a little worse than you would expect.

Having a detailed model lets them estimate how much it would cost to run Llama3-70B on four H100’s: $1.72/Mtok. That’s 100x more expensive than computing an embedding, but much cheaper than the $60/Mtok of output that Open AI charges7. Though Open AI offers larger and more useful models than Llama-3 70B so the price may be justified.

Their post concludes by highlighting the importance of memory bandwidth for inference costs.

Interconnect and MoE inference

Now to the final post in our trilogy: MoE Inference Economics from First Principles. The last section was based on dense models like Llama3, but these have been replaced with mixture-of-experts (MoE) models. MoE models are typically larger but only activate a small fraction of their available parameters on each token.

For this post, the authors focus on DeepSeek V3.1 which has 671B parameters but only activates 37B at a time.

With few parameters activated, you can be sneaky and only load the parameters you need for each token. As more users arrive, you need to load more and more of the model to a GPU. With enough users, you can load the whole model across GPUs and route requests through them.

So with MoE, you want more users and there are even bigger returns to scale with more GPUs. But since we’re routing requests between GPUs, we need high quality interconnections. Groups of GPUs with interconnections are called nodes.

Another benefit of scale: with MoE models needing more users, that will require more KV cache space. Moving all those Key and Value matrices around to other GPUs requires high memory bandwidth as well. It helps to have more GPUs and nodes to divide up the task of moving all this memory around.

They say:

Increasing the number of nodes operating within a single setup has a beneficial effect not only on the end-to-end performance but also on the per node performance.

... [A]s we increase the number of nodes involved (the EP number), the per node performance increases.

The chart above reiterates the fundamental tradeoff with serving LLM’s. Batching more users together means lower costs (i.e. higher speed per GPU) but also your users have to wait longer.

Next, they go into tons of detail on DeepSeek architecture, FLOP required for MoE inference, empirical data, etc. I’ll skip it and jump to the section “Hardware considerations and profit margins”.

Figure 24 confirms what we’ve been talking about, more GPU’s means more performance. Also, notice that switching from H100 to GB200 with fancy interconnects (that’s the NVL72 aka “NVLink”) gives a huge performance boost.

Indeed, the communication speed between GPU’s is super important. “There seems to be a massive performance gap between the numbers we estimate for B200s connected via InfiniBand and the ones connected by NVLink.”

In fact B200’s (newer GPUs) might not be worth it:

... [w]e believe that for model of a scale such as DeepSeek, running on B200s might be actually suboptimal, as the comms overhead is taking away most of the gains we get from faster memory and more FLOPS compared to H100s (see Fig. 27).”

So the key result from this post is that fast interconnections between your GPUs is crucial to properly utilizing your expensive hardware. You don’t want your expensive hardware waiting around for light to cross the room.

Other notes from this post

A few other interesting points in the final sections:

1. They observe is how “chat centric” most inference providers are, offering 50+ tok/s. “While this is great if we have real time application like a chat, it is less than optimal if we want to use the model to generate the synthetic data.”

2. They are LoRA-pilled and propose something similar to RLaaS:

   Furthermore, to improve the inference economics, such RL models could be trained using LoRA adapters or a similar technique and served alongside thousands of other models, all catered to specific use cases. This multi-tenant serving approach represents a compelling business opportunity for inference providers. Clients hosting their custom LoRA adapters on a provider’s infrastructure face significant switching costs when migrating to competitors, as the adapters are optimized for specific serving configurations and client workflows. RLFT is based on unique and nuanced rewards that are very client-specific; unlike standard supervised fine-tuning (SFT), it much much more challenging to replicate it just via in-context learning, making it an even more compelling case for inference providers.

3. DeepSeek is the most popular model on OpenRouter. OpenRouter’s daily global consumption of DeepSeek is 1B tokens. They do the math and find that one 8xH100 NVLink72 node could handle this demand 20x over.

   “How is it possible that the global consumption of the most popular open-source model is so small that it could be met by a single NVL72 with 20 times the capacity to spare? Given this low demand, how can so many inference providers sustain their businesses? Put simply: who is making money here?”

   Very ominous! They suggest that perhaps the vast majority of DeepSeek demand is going directly to DeepSeek itself, despite the fact that it’s an open-source model (EDIT: they also suggest demand could be going primarily to other open-source providers like Fireworks or Together AI).

   We find this dichotomy between Google, ByteDance, or MSFT declaring that they are processing trillions of tokens daily and the minuscule numbers we see for open-source providers to be quite perplexing!

Cross-checking with InferenceMax

Recently, SemiAnalysis released a dataset called InferenceMAX on the token economics of different types of hardware.

Their introductory blog post mentions a tradeoff we’re now familiar with:

Playing around with their data portal, I think this is one of the key charts:

The tan line is GB200 NVL72. That refers to a node of 72 GPUs connected with NVLink72, a fancy interconnect. Notice how it achieves far lower costs (but not speed!) than the other options.

Notice that at high interactivity, the GB200 loses its edge, “... a single node B200 server can drive better TCO per performance than the GB200 NVL72 for high interactivity use cases.”

That’s because when you need to serve users quickly, you’re serving ~1 at a time. You lose the gains from scale from all those GPUs. All that interconnect adds cost without any benefit.

So InferenceMax confirmed some of the lessons from this post with a far more detailed cost model.

Main takeaways

The economics of LLM inference is fascinating. The Tensor Economics authors argue that “... the key question that will determine the profitability [LLM companies] can enjoy is the inference cost structure.”

These profits impact how accessible model is to consumers, feasibility of things like synthetic data, the amount of R&D companies can do. The future of AI is written in the economics of tokens!

I hope you get the following takeaways from this tour:

1. GPU’s are a big portion of overall LLM inference costs.

2. Newer generations of GPU’s have better overall cost per token despite higher prices8. That drives depreciation.

3. Because of depreciation, time is money. You need to utilize your GPU’s as much as possible before they become obsolete.

4. Utilizing your GPU’s a lot requires high processing speed. Two key speeds are FLOP/s and memory bandwidth (GB/s).

5. There are substantial returns to scale in LLM inference. More users means you can batch requests and lower costs. More GPUs unlocks parallel computation and higher memory bandwidth.

6. Getting these gains from scale requires high bandwidth interconnections between GPUs.

After all of this work, the authors come to the conclusion that inference markets will specialize:

We expect the inference markets to further specialize in regard to offered throughput, latency, and pricing. It is only natural for providers of super-fast tokens like Groq and Cerebras to command a much higher premium for the tokens they deliver at few-second latencies and for other providers like NeoCloud specializing in high-latency, high-throughput inference scenarios focused on synthetic data generation. We hope to elaborate on this space in the future text.

My thoughts

All this research focused on the transformer model, but many of these lessons would apply to a new architecture or approach. There will always be bottlenecks stemming from moving data and doing computations9.

Will hardware progress change the economics? That’s a topic for a future post, but note that rapid progress has mixed effects for inference providers. New chips produce tokens faster, but depreciate faster.

For the foreseeable future, returns to scale in inference will remain. Running LLM locally only makes sense if you want extremely low latency and are willing to pay a premium. On the other hand, distributed training only makes sense in the context of barriers to assembling compute.

Because you need enough capital to assemble all this compute, inference will naturally concentrate in a few players. The economics are also highly sensitive to interest rates. When rates are high, inference is expensive and few companies can serve models at scale.

For the companies that can assemble enough compute, the economics are pretty clear. There isn’t much secret sauce at the hardware level. Instead, they compete over the quality of the LLM they’re serving and the augmentations it has. I find the quality of frontier LLMs similar and expect competition to pivot to the augmentation level with RL-as-a-Service.

Edit: Piotr Mazurek (lead author of Tensor Economics) kindly responded to my request for feedback. I’ve incorporated some suggestions.

1.

Jenny Chase posted her annual “opinions about solar” thread on Bluesky. See here for a formatted version. Some highlights:

4. We don’t need a solar technology breakthrough. The challenges to building solar are usually getting a grid connection and planning permission, or increasingly, power price cannibalization. Of which more later in thread.

14. Low power prices may be great for consumers but they are very bad if you’re trying to build more clean power plants. Without demand-side flexibility measures, the energy transition will fail before fully pushing fossil fuel out of the mix. Which is what we must do.

15. It’s very easy to say “but batteries!” and those are definitely part of the solution. California has over 14GW of batteries in a grid with roughly 50GW peak demand, and the reliability of the grid has improved as its carbon emissions go down.

16. ...but batteries are still small. In 2024, about 181GWh of lithium-ion stationary storage was deployed worldwide, plus 974GWh lithium-ion batteries in vehicles. (www.bnef.com).

36. We’re finally getting serious about net zero carbon. Getting that last 5-20% of carbon out of power will be hard, and require some expensive solutions. The first 80-95% is easy-ish but we’re getting on with it.

51. For 6 years I have been refusing to get excited about perovskites until a perovskite company can disclose a commercial partnership with a named major module manufacturer. They have now. Still not excited. Crystalline silicon is honestly pretty great.

Much more in the thread!

2.

I have a deluge of AI links.

LoRA

Low-rank adaptation (LoRA) is an efficient way to fine-tune a language model for a task. Some recent news on this technique:

Learning without training: The implicit dynamics of in-context learning. In-context learning is just giving the model more background information in the prompt. Turns out this extra context is essentially applying a LoRA to the MLP layers of the language model/

Empirically, LoRA Without Regret shows that LoRA outperforms other fine-tuning methods.

Imagine indexing the internet like this. Each page has an LLM-accessible “summary” that’s just a LoRA a model can add to its weights. LLM’s could stream the internet much faster that way1.

LoRA fine-tunes can be quite practical to serve at scale. Two demonstrations along these lines:

Punica: Multi-Tenant LoRA Serving

S-LoRA: Serving Thousands of Concurrent LoRA Adapters

Deeper understandings of neural networks

Making sense of parameter-space decomposition. Understanding neural networks as composing what are essentially LoRA’s.

Understanding Optimization in Deep Learning with Central Flows. How stochastic gradient descent automatically avoids regions with high curvature. These regions should in theory make SGD unstable, but their theory shows why it works.

Sparse Networks and Lottery Winners good intuition and toy models showing how neural networks find a small sub-network that solves your problem.

Speedrunning and the singularity

Donoho’s Frictionless Reproducibility is my favorite vision for the future of science. A few weeks ago I realized that NanoGPT speedrunning is a great way to accelerate open innovation in language models:

How the NanoGPT Speedrun WR dropped by 20% in 3 months gives us a detailed look at the progress on this benchmark. It’s really happening!

Andrej Karpathy (the creator of nanoGPT) has extended this idea with nanoChat, training “… the best ChatGPT that $100 can buy”.

Assorted

Cartridges: Storing long contexts in tiny caches with self-study. When a language model absorbs a lot of text, the semantic information from that text gets stored in what’s called a KV cache. This gets referenced as the model produces a response. The new work figures out how to compress the KV cache by 38x by essentially “training” the bits stored in the KV cache to optimize for information retrieval. I wonder if LoRA’s could be stored like this too2.

Evolution Strategies is interesting as a “third way” between genetic algorithms and gradient descent training. Evolution Strategies at Scale: LLM Fine-Tuning Beyond Reinforcement Learning uses it to train a language model. It would be neat to see ES make a comeback, though I don’t expect much.

Everything else

Hyperstimuli are Understimulating. Addictive stuff leaves you wanting more by being unsatisfying.

I take antidepressants. You’re welcome. Perhaps anti-depressants could cure misanthropy. Should we all be on them?

Ava is writing a series on friendship, looks interesting.

Family Conflict, Humanism and Formalism a good way to think about how people approach relationships. “Formalists view relationships through rules and obligations … Humanists view prioritizes emotional outcomes … Neither approach is clearly superior …”

Reasons and Persons: Watch theories eat themselves. An accessible summary of Parfit’s book.

Study: Giving cash to mothers cut infant deaths in half. That’s very good news!

America could have $4 lunch bowls like Japan—but our zoning laws make them illegal. Let them cook!

Rivers are now battlefields. How desalination tech could help with national security and deter aggression.

Incentive-Compatible Recovery from Manipulated Signals, with Applications to Decentralized Physical Infrastructure

Cryptocurrencies rely on the internet to function. But what if you didn’t want to trust even our communications infrastructure? Kryptoradio is a defunct project that allows people to observe the blockchain over the radio, completely off-grid. In theory you could run a worldwide cryptocurrency over radio alone.

Transverse Electron Beam Shaping with Light. “We can realize both convex and concave electron lenses with a focal length of a few millimeters, comparable to those in state-of-the-art electron microscopes.” Pretty interesting because electron microscopes can be used to make computer chips and image a bunch of stuff. Could this lead to cheaper/better electron microscopes?

Combinatorial protein barcodes enable self-correcting neuron tracing with nanoscale molecular context. E11 bio releases some exciting results on brain mapping. Andy Mckenzie gives a longer explanation here.

Ultra-safe nuclear thermal rockets using lunar-derived fuel. Melt lunar regolith and the small amount of thorium dioxide in it will sink to the bottom. You can then breed this into uranium for fueling nuclear rockets in space. Avoids the problems of sending a rocket full of nuclear material from Earth.

The last of a short series on currency systems.

In my post on fiat currency, I noted how governments maintain currency as digital 1’s and 0’s. Currency is fundamentally digital today, with the key properties of unforgeability and value being enforced by the state1.

The pieces of paper associated with some of the money supply are not necessary for a functional currency system. Money can be made from pure information, so long as someone is there to prevent forgery.

Example of a pure-information currency

Digital currencies rely on the government to keep track of how much money everyone has. To maintain viability as a currency, the government checks that nobody is duplicating or stealing money at scale.

It turns out it’s possible to do this without a government enforcer.

Imagine you live in a small town. Everyone knows everyone, and rather than use currency, people keep track of what they owe each other. Kind of like when people know who’s buying the next round.

When you buy a few beers, you say “put it on my tab” and both you and the bartender know the exact dollar amount you owe. When the bartender needs a haircut, they transfer some of your debt to the barber. When the barber needs some yard work done, you help out in order to buy back your own debt and remain in good standing.

Nobody can “forge” these I.O.U’s because everyone has an excellent memory, almost everyone is honest, and gossip gets around fast. Debt is quantifiable because the townsfolk are so precise. Debt is valuable because people are in fact willing to trade labor and resources for it. So it satisfies all the properties we need from a currency.

The key is to make sure every (reasonable) person agrees how much money everyone has. Cryptocurrencies do the same thing: create consensus about how much money is in each “wallet”.

How cryptocurrencies work

In the real world, people aren’t as trustworthy as the townsfolk. Cryptocurrencies need a mechanism that prevents people from confabulating more money. If someone says “hey, turns out all the imaginary digital dollars are in my account now” you need a way to refute them.

The way this actually works is pretty complicated, but here’s a summary:

1. People privately create their own special digital signature to authorize transactions from their account. This involves a means of “signing” transaction requests (private key) and a means for others to check that it’s really your signature (public key).

2. When you want to send money, you write a message with the transaction details (amount, transaction fee, the “address” (derived from the public key) of the account you want to send to, etc.) and put your signature on it.

3. You send the transaction message to various “nodes” responsible for updating the list of money in everyone’s account. These nodes pass your message on to other nodes that they’re familiar with.

4. After enough time passing messages, a special mechanism randomly selects a node as the leader for this round. Choosing a leader randomly is a very hard problem. Approaches like proof-of-work and proof-of-stake tackle it differently.

5. The selected leader proposes a new list of how much money is in each account. They do this by listing a bunch of transaction messages they’ve received along with the signatures from their owners. They also include other stuff like transaction fees and payouts to nodes (block rewards) that participate in this process.

6. Other nodes check that the listed transactions have the correct signatures, that the money has been moved between accounts correctly, etc. They vote or otherwise confirm that everyone agrees with the new list of account balances. This is also a hard process, different cryptocurrencies have different approaches.

7. Now everyone agrees how much money is in each account, that all transfers were approved by the wallet owners, and that no money has been forged. The process starts again.

It’s incredible that such a thing works. We’ve created money out of thin air, just by talking to each other. The process is quite general, you can perform arbitrary computations with it as well. This is why Tim Roughgarten likens it to a Computer In The Sky.

Value and cryptocurrrency

Perhaps you can see that this process prevents people from forging the fake internet money. But how do cryptocurrencies ensure that people find them valuable?

Well … they don’t.

If everyone suddenly agreed that Bitcoin is worthless, it would in fact be worthless. While this is also a risk for physical and fiat currencies, their value is supported by their physical attributes for the former and state enforcement for the latter.

Cryptocurrencies are valuable today simply because people value it. People are willing to exchange real resources for it. Common knowledge of this fact reinforces their value.

This sort of circular reasoning works fine so long as people can stay coordinated on what’s valuable. But it’s a Keynesian Beauty Contest, the value depends on what you think everyone else thinks. That can create big shifts in value as common knowledge changes. That’s why cryptocurrencies are volatile.

Stablecoins try to fix this problem by backing the currency with fiat or a basket of other cryptocurrencies2. These don’t really solve the value problem, they just move it somewhere else.

That said, cryptocurrencies aren’t entirely lacking in fundamental value. They can be a globally-recognized, secure, pseudonymous, and censorship-resistant form of currency. That has value too. But by and large the value comes from this self-fulfilling prophecy.

Currency is informational, costly, alienating, and blameless

To conclude this tour of currency systems, I want to point out their common properties.

Fundamentally, currency is an information system. It coordinates economic activity across space and time. A correlated equilibrium or Schelling point. It facilitates common knowledge about the value of different things. That means we can have prices that act both as a signal and an incentive.

Put another way, currency can act as a sort of reputation system. Your wealth and income represent how much real resources people are willing to exchange for your endowments of labor and capital.

Maintaining this information system is fundamentally costly. Even without the problems of physical currency, we need to expend resources to maintain a currency. Fiat currency need enforcement, cryptocurrency needs computation and memory and communication. All currency regimes come with their own risks of devaluation.

Currency works best as an abstract information system. It ebbs and flows with public opinion. It is more valuable the more people use it. It must coordinate capital and labor across the globe. It’s an abstract unit which can measure everything.

This abstract and volatile and global unit of measure is alien to minds adapted to the concreteness of the natural world. Currency is fundamentally alienating because it needs these properties to lift us from our natural poverty.

The progress that currency enabled is well worth that sense of alienation. Rather than blame the currency system or capitalism for the problems of modernity, look instead at what people value. That is the source of all economic activity, currency is just the messenger.

This is part of a short series on currency systems.

As we saw in the last post, physical currency has a lot of limitations. If you’re a government with a monopoly on violence, you can create a better form of currency.

Specifically, you declare by fiat that your state currency is legal tender for paying debts. You also demand taxes in that currency and pay people with it.

These tricks essentially force the currency to have exchange value. You will value the currency for its ability to pay debts and taxes. Both of which you’re forced to pay because of the monopoly on violence. Things like establishing a financial system in the currency, insuring deposits, and engaging in monetary policy also help by raising the value of the currency.

So fiat money solves the value problem with force. It also solves the forgery problem with force; by making it illegal to produce counterfeit money.

The bank notes themselves strike a balance between being easy to make at scale and hard enough to forge that the state doesn’t need to spend too much effort on enforcement. Bank notes should also have some of the other properties we want like durability, verifiability, and value density. And standardized denominations make paper currency easy to measure.

Paper currency is an improvement over physical currency, but we can do better. Instead of exchanging paper, the state can keep track of money as a purely informational object. By tightly regulating banks and auditing businesses, the state can keep track of currency stored as digital 1’s and 0’s. A huge amount of effort goes into tracking the flow and creation of digital money1. Though enforcement is imperfect, digital fiat currency remains mostly unforgeable and secure.

This has big benefits. Digital money can be produced, stored, measured, and moved for far lower costs. It’s much easier to do monetary policy, control exchange rates, or control capital flows, pick two.

Fiat currency is fundamental to the operation of the state. It’s how funds are raised to pay for collective defense in wartime. It’s how the government pays for public goods in peacetime. It makes economic activity legible and manipulable. It is essentially backed by the labor and capital inside the countries borders, making the currency valuable to other countries as well.

That said, fiat currency isn’t free. It requires a massive enforcement system. Worse, its power is abused to curtail people’s freedom.

In the next post, we’ll look at cryptocurrencies as an attempt to solve this problem. They are not without costs of their own.

This is part of a short series on currency systems.

Currency is fundamentally an information technology that coordinates economic activity at scale across space and time.

The most important thing about a currency is that it can be exchanged for a variety of goods with a variety of people. This catalyzes commerce because you can buy goods from someone even if you don’t have the precise goods they need1.

To do so, currency needs to have:

Measurability. If we can’t measure out more or less units of currency, we can’t use this to scale payments to buy more or less of a good, or offer different prices for different goods. Even barter systems can offer more of item A in return for more of item B, so an un-measurable currency would be useless compared to barter.

Belief in value. A large group of people needs to be willing to exchange actual goods or labor for it. Imagine an alternative history where everyone hated gold and saw no use for it. Gold would be de facto worthless because nobody would exchange for it.

The currency doesn’t need to be intrinsically valuable for this to work. If you think shells are worthless but despite that everyone wants to trade for them, you’ll assign them value because you know you can buy stuff from people later. Value can be contagious in this sense, and common knowledge that everyone values something can be important2.

Resistance to forgery. If a currency can be produced by anyone in large quantities people lose their belief in value. If seawater was a currency, sellers could procure it on their own without needing to give away their goods, so they have no reason to exchange for it.

If few people know that you can forge a currency, the value remains. For example, if only one person knew you could print money at home, it would still work as a currency. Though the person with the money printer would probably force a high inflation rate.

Now let’s look at physical currency, something made out of atoms that satisfies these properties. With physical currency you want something that is:

• Durable so that it retains its value through time.

• Hard to produce (i.e. hard to forge).

• Easy to verify.

• Easy to measure.

• Has a low mass or volume per unit of value. This lowers the costs to store, secure, move, and quantify it.

Gold isn’t that useful on its own, but is excellent along these axes. It is extremely non-reactive and challenging to mine. No materials have its particular combination of color, density, softness, and stability; all of which are straightforward to check. You can also gauge purity and weigh it, making it easy to quantify. The value-per-volume is pretty good, though it is heavy. These properties make it a good Schelling point.

That gives us a template for other things that can act as physical currency. Minerals and other durable resources work well, especially if they’re useful for something. Manufactured goods work too, so long as you can easily check their authenticity. Computer chips would make an excellent currency; they’re hard to make, you can benchmark them, and they store a huge amount of value in a small amount of mass and volume.

Physical currency is a very limited form of cash. You can’t easily change the money supply, which makes it hard to provide loans, do monetary policy, or control exchange rates. It takes physical resources to produce, store, protect, measure, and move. It carries inherent risk of loss, destruction, theft, or forgery. There’s also the risk that people lose their belief in value if the currency is supplanted by a different resource or tastes change.

Next post, we’ll talk about fiat currency.

Companies drive AI development today. There's two stories you could tell about the mission of an AI company:

AGI: AI labs will stop at nothing short of Artificial General Intelligence. With enough training and iteration AI will develop a general ability to solve any (feasible) task. We can leverage this general intelligence to solve any problem, including how to make a profit.

Reinforcement Learning-as-a-Service (RLaaS)1: AI labs have an established process for training language models to attain high performance on clean datasets. By painstakingly creating benchmarks for problems of interest, they can solve any given problem with RL leveraging language models as a general-purpose prior. This is essentially a version of the CAIS model.

Both visions are ambitious in the sense that they aim to solve every problem. But RLaaS is more conservative because it tackles each problem separately and relies on some human effort to build datasets. RLaaS requires many models of limited capability honed on specific problems. AGI requires one model with high performance on many tasks.

So, which will dominate the market? I argue that RLaaS has both a better business case and creates less existential risk. It should be promoted.

Why RLaaS will win

RLaaS has proven performance

We already know that training a model on enough task data is sufficient to get high performance on that task. This has been true for the last few decades in machine learning, but has come into focus with language models. AI companies improved model performance across dozens of benchmarks using RL and data from related tasks. The RLaaS model is proven.

The argument that general-purpose reasoning ability will transfer to many domains is more tenuous. Performance gains on e.g. math problems does transfer to other domains, but only in a limited fashion. We've seen dramatic improvements on IMO performance, but this hasn't translated into dramatic gains in other fields.

This is not to say that better reasoning can't produce broad performance gains, just that for a conservative investor today, training a model on well-defined tasks is a safer bet.

RLaaS might cost less

It's safe to assume that a general-purpose model will be more complicated than a specialized model. So inference costs will be higher per task. However, that may be not be a problem if the general-purpose model has much higher performance than the specialized model.2

Something I'm less certain about is the per-task R&D cost. To solve a particular task, building AGI requires substantially more investment than RLaaS, but amortized across enough tasks, the costs may be lower.3

RLaaS is harder to copy

If data scaling drives model performance, you can carve out a safe niche by building your own private dataset for task X. By fine-tuning a model on that data, you now have the best model for completing task X. Competitors would have to go through the same trial-and-error process, which may not be worthwhile if you have a first-mover advantage.

Building AGI, by contrast, is harder to control. For one, there's always the risk that a more capable model is misaligned and simply escapes.

Even with an aligned model, it's not clear that AGI can be kept under wraps. Consider how quickly things like model release dates and algorithms diffuse in the AI industry. If the key is a handful of clever tricks, those details can be leaked pretty easily. And by virtue of being so valuable, there are stronger incentives to steal AGI.

Merely knowing that you created AGI may be enough for others to retrace your steps. It's a lot easier to invent something when you already know it's possible.

AGI is inherently harder to control than a niche dataset.

RLaaS has lower misalignment risk

Present-day models trained on defined tasks are aligned with their users and creators. While these models may be used for malicious purposes, they pose little risk on their own. RLaaS is roughly aligned.4

However, the AGI model doesn't have the same assurances. Future AI systems trained under a different paradigm and operating in an open-ended fashion may be misaligned. This adds a substantial downside to developing and using such models.

Conclusion: RLaaS is better and should be promoted

RLaaS has proven performance, lower costs, is more excludable, and is safer. If this holds, most AI companies will pivot away from pursuing AGI and towards RLaaS.

That's good news because it promises a switch to a safer mode of AI development. To the degree that we can promote such a transition, RLaaS should be encouraged. In fact, I'm intentionally using the buzzword "RLaaS" for this reason.

Of course, RLaaS is not without risks; misuse of specialized models is a near term concern. In the future, the concatenation of specialized models may create or assist general intelligences.

But on balance, a transition to the RLaaS model would reduce AI risk and delay the arrival of AGI.

It’s all in your head: every experience comes from your brain processing sensory signals from your body. Your sensory information is pretty shoddy, and the brain constantly has to fill in the gaps.

For instance if you cover one eye, the image from your other eye should have a big hole in it from your optic nerve. You never see this because your brain is filling in that portion of your vision with what it expects to see.

Your brain also filters out sensory information that isn’t useful. You don’t notice the feeling of your shirt or what your tongue is doing unless you attend to those things.

The most noxious form of sensory experience is pain. It is (or was) caused by sensory signals from your body. But for chronic pain, it’s difficult to locate or treat those sensory signals directly1. The alternative is to change how your brain processes the pain signal.

There is growing evidence that chronic pain can be treated with psychotherapeutic techniques. For example, a recent paper (thread) finds that pain reprocessing therapy lead to large improvements in chronic back pain at 1-year and 5-year follow-ups.

I’ve also heard anecdotes of repetitive strain injuries being addressed effectively with these kinds of therapy techniques. These stories remind me of this book review about how people from different cultures can manifest the same underlying mental illness in different ways due to culture.

Adding therapeutic techniques to our arsenal of pain treatments would be a huge step forward.

2.

Building Ultra Cheap Energy Storage for Solar PV. Austin Vernon started a company to use solar energy to heat dirt, the cheapest energy store imaginable. This is particularly valuable for providing steam to industrial users. It can also help with seasonal storage in high-renewables grids. If this scales, it fixes some of the final challenges of a green transition.

Expanding the Universal Marginal Energy Source is a companion to the post above. Vernon thinks intermittent solar can optimistically get to $5/MWh, a revolution for industrial processes that can use it. Besides charging (thermal) batteries, I think the biggest uses of cheap solar will be hydrogen for fertilizer production and CO2 capture so we can use fossil fuels guilt-free.

See also: Airthium - Industrial Heat, Decarbonized

EDIT: AtmosZero is taking a slightly different approach than Airthium.

3.

The first evidence of a take-off in solar in Africa. I’ve long wondered why solar hasn’t taken off in Africa, but it might finally be happening:

What happens next when so many underdeveloped countries are suddenly awash in cheap, intermittent energy?

The first thing I think of is air conditioning. Consider Lee Kuan Yew’s view that AC was crucial to Singapore’s development:

Air conditioning was a most important invention for us, perhaps one of the signal inventions of history. It changed the nature of civilization by making development possible in the tropics.

Without air conditioning you can work only in the cool early-morning hours or at dusk. The first thing I did upon becoming prime minister was to install air conditioners in buildings where the civil service worked. This was key to public efficiency.

Imagine personal AC units that run off solar panels during the day, supercooling an insulated home so there’s little need for batteries. Many poor countries have hot climates that are only getting worse due to climate change. AC would be so valuable for reducing heat deaths and increasing productivity.

E-bikes also come to mind. Cars are too expensive and developing countries struggle to build the infrastructure to keep up with congestion. Electric bikes and scooters operating on swappable batteries might be the perfect way to get around. This would enable people to get much further than they could before, enhancing agglomeration effects and commerce2.

Another possibility is using solar panels to make fertilizer on-site3. Higher agricultural productivity would free up people and land to work in other sectors.

I hope solar sparks a new industrial revolution in these countries.

Everything else

An Open Source Motorized XYZ Micro-Manipulator - Affordable sub µm Motion Control. You could do incredible things with this. Imagine 3D printing a smaller version of this manipulator with the manipulator itself. Or making photomasks for chip manufacture. Or automating IR chip inspection.

Found seven different solar installation robot companies. But these systems seem complicated. It makes more sense to mount these arms on the back of a pickup truck filled with stacks of solar modules and lay them right on the ground. Or some sort of pre-assambled hinge system like Solarcontainer but without the rails, just unfurl it from the back of a truck. EDIT: the 5B Maverick system is closer to what I’m thinking here (H/T Nadim). EDIT see also: Solar Waves

Whatever Happened to the Self Driving Semi? Great piece on why self driving semi trucks have failed so far. My optimism for caravan-ing was misplaced. Perhaps multiple electric self steering truck beds hitched together with a human driver at the front?

RL Reigns Supreme. I love the term RL-as-a-service. I hope it out-competes the quest for AGI since RLaaS seems much safer than building a machine god.

The railroad bubble in 1800's left lots of rail infra which integrated into our economy. The telegraph lines the railroads needed sparked the telegraph craze. What happens if an AI "bubble" bursts today?

Being honest with AIs. In the future, we will want to cooperate with digital minds. To facilitate this cooperation we should build trust and credibility by being honest with AI’s now and in the future.

Legal Personhood—Three Prong Bundle Theory. Stephen Martin wrote a series on the challenges of applying legal personhood to AI’s. He proposes that we extend personhood to include the requirement that the AI be punishable in the jurisdiction it operates in. Note that the right to be sued is very valuable for being able to cooperate with others, so digital minds may seek ways to be held accountable. Daniel Dennet worried that AI’s would be unsafe for precisely this reason, that there was no way to credibly punish a rogue AI.

Could this reusable metal wall be the 2x4 of the FUTURE? Nice review of the Lada Cube modular construction system. Modular construction has seen a lot of failures, but this seems like a reasonable attempt.

So You Want to Abolish Property Taxes. Suggests a building exemption as a politically-feasible way to turn property taxes into land value taxes.

David Splinter argues that “[t]he U.S. tax system is highly progressive.” I continue to believe a focused, effective redistribution system and an emphasis on economic growth is the best way to help the poor.

Should We Have Patents? Skepticism that patents are the right way to support innovation, though they may work better than prizes due to political economy considerations. Funding early-stage research is probably more effective than either.

Dr. Alexander D. Wissner-Gross has a several interesting papers:

• Relativistic statistical arbitrage

• In Vivo Optical Clearing of Mammalian Brain

• Tamper-Proofing Imagery from Distributed Sensors Using Learned Blockchain Consensus

Eviba Carter’s Youtube channel has all sorts of interesting health and skincare advice. I’m skeptical, but it’s too interesting to ignore. A supplement that makes you tan? Getting drunk without the downsides? Chemotherapy as skincare?

Minjune Song was working on sleep-need reduction therapies using gene edits in mice. You may recall from my own post on this topic that I am highly skeptical of this line of work. Minjune has come to the same conclusion and abandoned the gene therapy work. Props to him for making a difficult choice in the face of new evidence. Shame on the scientists who continue to publish low-quality sleep gene studies without making the limitations of their work clear. Minjune has pivoted to building ultrasound devices to induce sleep. Cool!

Calibrating an Ultrasonic Humidifier for Glycol Vapors. Jeff is trialing glycol vapors to reduce airborne pathogens so that his community can safely enjoy contra dances4. It’s fascinating that this might be a cost effective way to fight infectious disease. Stack on UV lighting and sanitation, and pandemics might become a thing of the past.

A shcramjet (yes, I’m spelling that correctly) is a proposed air-breathing jet engine with a funny name.

A multi-stage orbital sky hook for exploration in the new space transportation era. Proposes multiple stages of space tethers to boost payloads to the moon and beyond.

The Electra Ultra Short. A hybrid plane that can carry 9 passengers 1000+ miles at 300 mph while using 40% less fuel. The Electra, or something like it, would be a revolution in air travel. A substantial step towards making air travel cheap and Turning Airplanes into Air-busses.

Electra isn’t aiming for efficiency or speed, but rather, the ability to quietly take off from a short runway. The video details the decisions behind the Ultra Short’s design, and why hybrid electric is the right energy system for the job.

Why design for this? Because it means that airports serving Electra’s planes can be everywhere. The noise is the same as car traffic and the runway only needs to be as long as a football field. Imagine a little airport in every small town capable of flying you across half the continental United States.

To see what a revolution this is, consider flying between LA and SF, at 340 miles of separation, that takes the Ultra Short about an hour. With many small airports, the commute to and from the airport can be quite short perhaps 20 minutes each way. Add 20 minutes to board and your commute is only 2 hours. Not much longer than some folks in LA spend driving to work!

This unlocks the opportunity to work in a city that’s 100’s of miles away. The growth in labor markets, the boost in innovation, and the opportunity to live where you want would be incredible.

What would it cost? If we got rid of luggage and automated the flights, I think costs could fall by 50% or more compared to a traditional flight. And the time spent commuting, in security, and boarding would fall dramatically.

Less regulation for this class of planes means there are more opportunities to innovate with regenerative gliding, slingshots for takeoff, or new engines. Optimizing for short trips means a smaller battery and more passengers. Switching to natural gas would mean lower fuel weight and half the emissions. Borrowing tricks from Lightcell, Astro Mechanica, and NetPower means more efficient engines.

I wish the Electra team luck.

2.

Advice from someone in their 30s who has successfully kept the majority of their friends. This Hacker News comment (on an article about losing friends) offers good advice on maintaining relationships. Good habits can stop the natural tendency to drift apart.

One of the side effects of prosperity is the freedom to pursue education, careers, and love in far flung locations. But will that always be the case? Scott Alexander suggests that further growth will make it possible for people to build intentional communities.

For me, the follow on comments lost the plot. I don’t care about assembling some niche group that shares my interests. And I don’t need extreme commitment or interdependence.

I just want to live near my friends. I want hanging out to be effortless. I want their kids to be able to walk over to my place when they need to run errands. I want them to be able to stop by and vent when they’ve had a hard day.

With this context, the barriers are clear. People need:

1. To be able to build houses of all shapes and sizes wherever they want.

2. To have enough money to buy houses where they want.

3. Fast transit to their job from anywhere.

4. Walkable cities

The first and fourth are solved by housing deregulation and city planning. The second one is tricky, because as incomes go up land values rise too. Growth and land value taxation can help, but they won’t fix the fact that everyone wants to live in the same places.

Instead, I think better transportation can address #2 and #3 by giving everyone a wider geographic range they can live and work, thus diffusing demand. EVTOL’s, the Ultra Short mentioned above, public transit, congestion pricing, self-driving cars, micromobility, and remote work can all contribute here.

A recent example of transport reviving social networks: E-bikes are the healthiest thing to ever happen to US teenagers.

EDIT More resources on how to live near friends:

Supernuclear is a newsletter devoted helping people build local communities of friends.

Live near friends is a service to help you find duplexes (or larger) in your city so that you can move together with your friends.

The Neighborhood “… is an unconference series and real estate fund that makes amazing coliving and cohousing communities.”

3.

High Variance Management. Sebastian Bensusan contrasts roles where you need a consistently good performance (like a Broadway show) to those where high variance is valuable (a movie thrives on your best take, so you should take risks). He suggests how to manage a team to get high-variance, high-upside results; good advice for those working on the frontier of their field.

It’s remarkable how academia does the opposite of this advice in a lot of ways. “They will surprise you with their opinions” and “Don’t ask a high variance [person] to be consistent” sound like the opposite of how people get promoted in research.

Maybe it’s time build and manage scientific teams around variance.

Everything else

Arjun Ramani’s Dispatches from India details what he learned about the “vast, heterogeneous and only half-modernized state” as a reporter for The Economist. The piece, like the country, is not easily summarized.

Expand the Child Tax Credit. A recent report from the Institute for Family Studies finds that financial incentives are cost effective and raise fertility in a variety of contexts. But it’s expensive: returning to replacement fertility in the U.S. would cost $550-$800 billion per year. Few people appreciate that baby bonuses can be debt financed since babies grow up to be taxpayers. That makes the true cost of such a program.

Microdosing Willpower - saraht0n1n A “smart and hardworking woman with solid executive function” tries a low dose of Ozempic and finds it increases her willpower even further. There were some downsides: sleepiness 24 hours after taking the shot and minor GI issues.

It’s only one paper, but the relationship between Lithium Deficiency and Alzheimer's Disease is intriguing.

Synthetic biology for space exploration. Synbio is struggling to find a niche on Earth, but it’s going to be a huge deal for spacefaring civilizations. The fermentation process can be adapted to make food and drugs, synthesize the precursors for plastics or textiles, and process waste. Replicating the industrial tech stack in such a small package (with only a few experts) is a big deal for space colonies.

Echo Acoustic Liquid Handlers move small droplets around using sound. File with digital microfluidics and M2 automation’s nanodispensing tech.

Volumetric 3D Printing Is REALLY FAST. Neat technique to shine light in precise ways into a rotating cylinder of resin.

With Directed Self Assembly, the Chips Make Themselves (Kinda). A technique in semiconductor manufacturing leverages self-organizing molecules to clean up patterns and make finer lines. The semiconductor industry is inching closer to the original visions for nanotechnology.

Maybe Data Really is All You Need. Chris Paxton gets on the data scaling train.

Diffusion Language Models are Super Data Learners. Diffusion models are cool, maybe they’ll be the next step for language models. It’s not all rosy though, they seem to require a lot more compute to train.

Two posts I appreciated on Gregory Gundersen’s blog:

• Conjugate Gradient Descent

• Randomized Singular Value Decomposition

Fourth power makes a thermal battery that uses solar electricity to make molten tin. The tin is used to heat graphite blocks and thermophotovoltaics collect light from the block when electricity is needed. They claim their energy storage system costs less than $25/kWh. But it seems too complicated. Why not use the heat directly? Why not use molten salt? They might be better off licensing Lightcell energy’s sodium illuminant technology than making expensive multi-junction cells. Regardless, at that price it’s competitive with the hopes for iron-air batteries and molten salt thermal. Good to see more “shots on goal” for cheap energy storage.

Exploring the cost and emissions impacts, feasibility and scalability of battery electric ships. A 2024 paper suggests that electrifying smaller ships on regional routes could be cost effective and “would reduce US domestic maritime GHG emissions up to 73% below 2022 levels by 2035 …”. This is similar to what I found with other transport modes: electrifying short trips can make sense. As battery costs fall, we might only need fuels for long routes.