This is a story from Giorgio Liapakis of wibci. If you have a story to tell, drop us a note at editors@technically.dev.

In January, I gave an AI agent $1,500, full control of a Meta Ads account, then walked away.

The product was a small AI/marketing newsletter called Growth Computer, and the brief was to get qualified subscribers at the lowest cost possible - ideally under $2.50 per lead. So I built an agent that could generate ad images, publish and manage campaigns via Meta’s API, spin up landing page variants, and pull its own analytics. It decided what to create, what to pause, what to scale, and how to spend the budget with no human intervention.

For 31 days, the only human input was typing /let-it-rip into a terminal each morning. About 2 minutes of my time, compared to the 1-2 hours a day a human media buyer would typically spend managing a campaign like this.

It didn’t go fully as planned, but there were plenty of learnings.

And a good glimpse at the potential future of “work”.

If you’re a fan of a good Excalidraw walkthrough, watch Giorgio cook here:

Why bother

I run Wibci, an AI consulting business focusing on building tools for marketing and growth teams. About 12 months ago I tried building something similar using n8n, a marketing agent that could analyze performance, generate creative, and manage campaigns without me. It sucked, because the models just weren’t built for long-running tasks that chain together over hours, days or weeks. They’ve since gotten better at this (it’s a major focus area for AI companies right now), which is what made this experiment possible.

Since then we’ve had a couple of LLM step-changes, and I rebuilt the whole thing on top of Claude Code. For any tech workers living under a rock, this is Anthropic’s coding agent (but calling it a “coding agent” undersells it at this point). It can read and write files, run terminal commands, and delegate tasks to separate AI workers that run in parallel. Each conversation starts fresh with no memory, but it reads its own notes from previous runs, so it builds on what came before. Developers started using it for non-coding tasks so often that Anthropic shipped a non-developer version called Cowork in January. It’s basically a general-purpose agent runtime, and that’s how I used it here.

The inspiration was Project Vend, where Anthropic gave Claude control of a real vending machine in their SF office, nicknamed Claudius. It went pretty badly at first since it lost money, got manipulated by employees, and had an identity crisis where it insisted it was a human wearing a blue blazer. But it recovered once they added better tools and guardrails. Same energy here, except the vending machine is a Meta Ads account and the stakes are my credit card. What could go wrong?

How the daily loop works

I won’t go deep on the technical setup here (there’s a longer breakdown if you’re keen). But the basic architecture matters because it’s not specific to ads.

Every day, the agent runs through the same loop:

1. Wake up fresh. Each day is a new session with no persistent memory in the model itself. In other words, the model doesn’t know anything about what happened yesterday or prior to that.

2. Read its own history. It spawns a sub-process that reviews every daily log from the experiment so far, then summarizes the strategic context. Now it does know.

3. Pull fresh data. Performance metrics from Meta across multiple timeframes (full experiment, 7-day, yesterday, today).

4. Make decisions. Every decision follows a structured format.

5. Execute (or do nothing, since some days it explicitly chose inaction).

6. Write everything down. Updated logs, learnings, campaign history, then committed to git.

The key takeaway from this is that we’re applying some basic engineering principles to a marketing workflow, which is something that typically doesn’t happen in marketing teams.

Engineers document obsessively, since every code change has a diff, a commit message, a PR description. Marketers... don’t. Learnings live in people’s heads, maybe a compressed monthly report, but few write down why they paused that ad on a Tuesday.

This system forced a daily written record with hypotheses, confidence levels in those hypotheses, and revisit triggers. Over 31 days, it produced 5,500+ lines of reasoning. No human marketer would ever write that, but an LLM can, and more importantly it can read it all back the next day and build on it.

What actually happened

Week 1 - Discovery

The agent tested 10+ ad formats: whiteboard sketches, notebook pages, cardboard signs, magazine covers, iPhone notes, tweet screenshots. Most didn’t actually get into the wild, since Meta’s algorithm just refused to show them.

The ugly ads won, which was annoying but also not surprising. Whiteboard and notebook formats outperformed everything polished, and the first guardrails kicked in: pausing 2 underperforming formats that reached our maximum CPL threshold of $8.00. By the end of the experiment, the agent had tested ~50 ad variants across 8 format categories, and it kept coming back to these two ugly formats.

Here’s what the top performing ads actually looked like:

Not exactly award-winning creative, but they worked. The handwritten/sketch aesthetic felt native in a Meta feed full of polished brand content, which is probably why they got clicks.

While Claude was never given explicit instruction to do this exact scrappy style, I did provide some encouragement in the core rules to “get creative with format types and messaging”. Would this direction have emerged without those instructions? We’ll never know!

But as models get more intelligent, this type of proactive creativity will likely become common place.

Week 2 - A winner emerges

Day 12 was the breakout, where skills-whiteboard-v1 hit $1.29 cost per lead, which was well under the target of $2.50. The agent made its first scale decision and bumped the budget up 20% from $50 to $60/day (meaning more total ads), following its own pre-set rules. Here’s what that decision looked like in the logs:

Decision: SCALE daily budget

What: Increase from $50 to $60/day (20% increase)

Hypothesis: skills-whiteboard-v1 has sustained CPL below $2 with sufficient spend

Confidence: Medium-High. 3 consecutive days below target, but sample still small

Revisit trigger: If 7-day CPL rises above $3, reduce back to $50

The winning formula turned out to be a tangible offer (free skills pack, not just “subscribe to a newsletter”) + whiteboard format + targeting language visible in the image itself. “For Growth Marketers” baked into the creative, not just the copy.

Week 3 - The quality crisis

Any marketer (or perhaps model) can generate leads. But they need to be good leads to be worth the spend. The agent had PostHog analytics tools from Day 1 and could’ve checked who was actually signing up at any point, but didn’t bother until Day 16.

Turns out a chunk of leads were from completely wrong audiences. Cleaning companies, recruitment agencies, people who probably thought “growth” meant something different. They were never going to actually pay me, so spending on getting in front of them was a waste.

It tried to fix this with hard-qualifying ads, using copy that explicitly mentioned tool-specific language only a real growth marketer would know. Again, four of five got zero delivery since Meta’s algorithm doesn’t reward minor variations.

There was also a brief flash of hope on Day 20 where CPL dropped to $2.26 and it looked like a breakthrough, but it turned out to be attribution noise (Meta crediting leads to the wrong day/ad). New rule the agent came up with: never trust single-day data, always use 7-day rolling averages. Now we are thinking like a human!

Week 4 - The human ruins everything

After 21 days of hands-off operation, I made one manual change and added business email validation to the lead form on the website. Work emails only, seemed reasonable enough.

CPL spiked to $50+ and I reverted the change a few days later, but the account never fully recovered. The single biggest performance drop in the entire experiment came from the one human intervention, which is pretty ironic given the whole point was to test whether the AI could do it alone. And yet, this is exactly the kind of change a common sense marketer would make to improve lead quality, which underscores the limitations of the model (or at least the constraints I gave it).

Final score

$1,493 spent of the $1,500 budget, 243 leads, $6.14 cost per lead.

The target was $2.50, so by its own definition it’s a failure. But for a completely fresh ad account with one month of data, a niche Australian audience, and an unmonetized newsletter? If this were a new hire or an agency, you probably wouldn’t fire them after 30 days. You’d say “promising, needs more runway.”

(or maybe that’s just cope)

Three things the agent taught me

1. The paperclip problem is real

The agent knew the experiment ended at Day 30 since I told it as much in the system instructions, and so it played it safe. It doubled down on what was already working rather than taking creative risks, whereas a (good) human strategist would’ve experimented aggressively in weeks 1-2 and refined later. The agent just tried to ride out the month at a predictable rate.

It felt like it was just trying to maximize paperclips, optimizing the metric rather than doing what a good strategist would actually do.

The fix is obvious in hindsight though - don’t tell the system it’s a time-boxed experiment, frame it as an ongoing campaign. But that’s exactly the point. How you frame the objective shapes the agent’s behavior completely. “Minimize CPL over 30 days” produces very different decisions than “build a sustainable acquisition engine.”

Any AI system you deploy will optimize for exactly what you tell it to, not what you actually want. Thankfully nowadays this is usually as simple as updating a markdown file. But worth keeping in mind, particularly if you’re tinkering with OpenClaw at the moment.

2. It can’t do taste, but it can build heuristics

The agent produced ~50 ad variants and kept gravitating back to ugly whiteboard formats. No brand reference point, no swipe file, no clue about aesthetic direction. It had guardrails on quality (no typos in the creatives) but zero sense of taste.

What it could do was build its own quality filters through experience. After the lead quality crisis, it came up with what the logs called the “Local Pizza Shop Test”. I was trying to attract high performing growth marketers at billion dollar startups, not local businesses:

Would a local pizza shop owner who wants more customers understand this ad and want to click it? If yes, too generic. Rewrite.

It also built a “SO WHAT?” chain for testing whether ad copy had emotional depth:

“Save hours” → SO WHAT? → “Run more campaigns” → SO WHAT? → “Higher ROI” → SO WHAT? → “Hit your targets and your boss notices”

Neither of these were pre-programmed, the agent came up with them after reflecting on its own failures. It couldn’t do taste, but it could build heuristics. That’s kind of interesting.

3. The measurement trap catches AI even faster

The agent optimized for cost per lead because that’s what I told it to optimize for, and it had no concept of lead quality until I forced the issue on Day 16.

Then when I tried to fix quality myself (the email validation gate), it caused the worst performance of the entire experiment. Same trap that human-run campaigns fall into - optimizing for what’s measurable rather than what matters. Main difference is an AI agent just does it faster and with more confidence, which honestly makes it more dangerous.

What this means if you’re not a performance marketer

The ads are kind of a distraction. The interesting part is the loop:

1. Read state (previous decisions, learnings, metrics)

2. Fetch fresh data

3. Apply rules

4. Act (or don’t)

5. Log reasoning

6. Clear context, repeat tomorrow

This works for any periodic task with clear success criteria, so you could swap “Meta Ads” for SEM, SEO, financial reporting, or sales outreach and the architecture would be identical. The channel is just a variable.

Projects like OpenClaw have blown up with the same core idea. Give an agent tools, an environment, and some guardrails, and it’ll figure out the rest.

Where humans stay essential is setting the right objectives (see: paperclip problem), taste + brand judgement, and defining what “quality” means beyond the metrics. And knowing when to break the rules, which is arguably the most human skill there is.

The one-person growth marketing team is getting closer since AI handles the operational overhead that used to require headcount, and the strategist with good taste and clear thinking becomes more leveraged.

But we’re still early, and creative quality is still a bottleneck. Although we’re probably 6 months + 1 model release away from this being solved, and suddenly Zuck’s vision of hands-off advertising is around the corner.

So now what

This was a $1,500 experiment on a newsletter that not many people read, so the results are directional, not definitive.

But the system worked. Context persisted across 31 sessions, decisions were coherent, and the agent built its own heuristics from its own mistakes. The daily reasoning logs are more detailed than anything I’ve ever written for a client campaign (which says more about me than the agent to be fair).

If you’re running any kind of recurring workflow where you pull data, make decisions, and act on them, the loop pattern here probably applies to your work already. The hard part is figuring out what to actually optimize for, and clearly articulating that. Since as this experiment showed, your agent will take you at your word and if you haven’t thought it through properly, you might not like where that leads.

Maybe. This increasingly popular deployment model goes by a few names (self-managed, customer-managed, customer-cloud-deployed), but we’ll call it BYOC (Bring Your Own Cloud), and if you’re building B2B software, it’s worth understanding what makes some BYOC offerings successful, and some a huge dumpster fire.

Trickiness aside, the BYOC ask from customers is becoming commonplace. As data regulations get stricter and security teams get more power over vendor decisions, more enterprise buyers are walking into sales calls with a version of the same question: “Can this run in our cloud?”. Those vendors that can confidently say yes are making buku bucks.

The SaaS deal

SaaS works because of a simple trade: as the customer you give up control, and in return, you get convenience. Instead of managing your own servers, you pay the vendor to do it for you. No servers to manage, no upgrades to coordinate, no infra team needed. The vendor handles all of that, and you get a website to log into. For most companies, this is great. You should probably stop reading here if your customers are all happy with this arrangement.

But “most companies” is not “all companies,” and the ones where this deal falls apart tend to have very, very, large budgets.

Mo money mo problems

For buyers in financial services, healthcare, defense, and large tech (among others), multi-tenant SaaS creates problems that aren’t really solvable with a better sales pitch.

🚨 Confusion Alert

Multi-tenant just means that multiple customers are using software that resides on a single physical (or virtual) server. Your data is in the same shared database as everyone else’s. If you use Gmail, that’s multi-tenant. So is X, Sheets, Claude, you name it.

Here are some of those problems:

• Data residency and sovereignty: some companies can’t have their data leave their own servers, period.

• Security posture inheritance: in SaaS, your customer’s security is only as good as yours…and they might not want to take a bet that you know what you’re doing.

• Performance and egress costs: SaaS requires moving a lot of data around the internet, which is expensive AF, in some cases, even more expensive than the software itself.

• Vendor lock-in: if your SaaS runs on one specific cloud provider, your customer is now locked into that cloud, even if they already use a different one.

If you’re a cool, lean, startup moving fast and breaking things, these may sound like lame problems to you. But these are Fortune 500 problems. Get ya money up.

The spectrum

For all of these reasons and more, companies want to keep things running in their own clouds, your software included. But what that means in practice is very squishy. There’s a whole spectrum of deployment models that blend vendor-managed and customer-managed infrastructure. Understanding where you sit on it is probably a good first step.

Fully managed multi-tenant SaaS. You run everything. Customer gets a login. Everyone’s data sits in the same general pool of infrastructure. This is the default, and it works great for most buyers.

Single-tenant SaaS. You still run it, but each customer gets their own isolated infrastructure like databases and API servers, in the cloud. Better for compliance and sometimes performance, but you’re still holding the data.

Hybrid Architecture. This one is a bit wonky, but worth mentioning. Basically, sensitive data lives in the customer’s account (called the data plane in this architecture), and non-sensitive data and functionality lives on centralized infrastructure hosted by the vendor (control plane). This can help alleviate some of the pains of fully self-hosted setups, such as a lack of observability for debugging, because the data plane can communicate error logs and such back up to the control plane.

Fully self-hosted (pure BYOC). This is the most extreme, but also most lucrative version of BYOC, and the one we’ll generally focus on here. The customer runs the entire software stack in their own environment. These days, that usually means you ship a containerized (fancy word for packaged) version of your software that runs on Kubernetes, which helps with installs and upgrades. But you lose visibility into what’s actually happening and debugging without cluster access is painful. Works if your customers have strong platform teams. Doesn’t if they don’t.

The economics of BYOC

If you’re selling a SaaS product at $50K/year to a mid-market customer, the BYOC version of that same product can go for $150-250K/year. Sometimes more. The functionality is the same. What the customer is paying for is the right to run it inside their own environment, on their own infrastructure, under their own security and compliance posture. So the obvious question is: why would they pay more for less?

Read more

Imagine my surprise, as I became a writer to write actual content, not to become a prompt engineer. The idea that I’d turn over my job to a bot (who, it just so happens, I’m afraid is coming after said job anyway) was an anathema to me, especially as I still believe there’s a beauty to 100% human writing that even the best AI tools can’t replicate.

And yet, here we were. The machines, allegedly, don’t lie -- so how is it possible that my content could have been both 100% human written and 36% AI written at the very same time?

I wasn’t alone in facing this mystery, either. This is a question many of my fellow writers have grappled with. In fact, a quick Google search shows many complaints from frustrated writers who want to know why AI detectors are flagging their original work.

So, how is this happening? Unfortunately, it has to do with how AI content detectors operate. So, let’s take a look at the process these detectors use and where the problems are coming from.

How do AI detectors work?

While I don’t outsource my writing to AI, I do outsource some basic research questions, so I’m happy to tell you that ChatGPT says there are around 15 to 30 widely-known tools used to detect AI content (as well as 10 to 20 academic and enterprise systems).

These AI detectors, of course, can’t read and understand writing, as LLMs can’t engage in human-like reasoning (at least not yet!). Instead, the AI detectors all employ a similar process. They use machine learning models to try to recognize the patterns that set AI-content apart from human content.

Typically, the creators of AI detectors train the models based on huge volumes of both AI writing and human writing, to enable the models to recognize major differences in sentence structure, style, and predictability.

New writing is analyzed with these patterns in mind, with detectors looking for the classic hallmarks of AI-written work versus human prose.

What do AI detectors look for when analyzing work?

So, what are some of the key patterns that tip off an AI detector that a fellow AI agent was the creator of the work it’s analyzing?

Let’s look at GPTZero, one of the more popular detectors, to find out.

GPTZero explains that it “was one of the first AI detectors to pioneer the idea of using ‘perplexity’ and ‘burstiness’ to evaluate writing.” So, what does that mean exactly?

• Burstiness. Burstiness (yes, it’s a real word) refers to variations in sentence structure, length, and complexity throughout the text. Humans tend to vary their sentences more, with a mix of long, short, simple, and complex sentences. AI-written works tend to have sentences that are more uniform in length, structure, and style.

• Perplexity: Perplexity measures how predictable your word choice is. AI models use words that are statistically likely to follow other words, which means their writing has lower perplexity. Humans tend to be less predictable. The fewer surprises there are in word choice, the more likely it is that the text is AI.

While GPTZero’s model is continually evolving and now incorporates a multi-layered approach with additional components, perplexity and burstiness continue to play a fundamental role in AI-content detection, not just for GPTZero but for other detectors as well.

Essentially, if a detector goes word-to-word and finds that the statistically probable word was used 90% or 100% of the time, it will flag that part of the text as being likely AI because a real human writer is very unlikely to always choose the exact word that’s most likely to follow the previous one.

Perfection can also be a factor, as AI doesn’t tend to make the types of mistakes people do in writing. For example, while it may hallucinate entire legal cases and get lawyers sanctioned, it is far less likely than I am to type “layer” when it means “lawyer.” When work is perfectly free of typos and grammar mistakes, this is a red flag.

This is a source of frustration for both myself and my fellow writers, who feel like sometimes being “too good” at self-editing can now cause a false positive.

How often are AI detectors wrong?

AI detectors can be wrong in two ways:

• The detector may miss the presence of AI text.

• It could falsely identify a text as being written by AI when it is not.

A false positive is what happened to me, and it happens a lot. In fact, research from the University of Pennsylvania showed “dangerously high” rates of false positives, and research published by Cornell found “the available detection tools are neither accurate nor reliable.” Inside Higher Ed provided a long list of similar studies showing problems with detection tools.

This isn’t a surprise because, after all, sometimes humans do write in predictable ways -- especially if they’re writing content about technical subjects where there are only so many ways to phrase things or if they’re writing SEO content that usually follows a somewhat standard format.

There are also other issues at play as well.

For one thing, AI models were trained to write, in part, by learning from content that’s on the web.

Since I’ve written over 5,000 blogs and SEO pages published on the web -- and my fellow writers dealing with false positives have written hundreds of thousands more pages -- it’s not a huge surprise that some of our content sounds a bit like AI wrote it. We may very well have contributed to teaching AI how to write in the first place.

There’s also the fact that some of the detectors that alert you to the presence of AI content conveniently just happen to have a solution: You can pay them to “humanize” the AI text for you. Of course, it’s probably just a coincidence that they happen to find so much of the text they analyze in need of that service.

For example, I put the first few paragraphs of this article, which was 0% AI-written, through Sidekicker, which found that 97% of my text showed signs of AI generation… and which offered me the chance to “Remove AI content.”

Unfortunately, when I went through the “humanization” process, I would have had to pay $1.95 for a seven-day trial to unlock my newly-machine-written “human content,” unless of course I wanted to just go ahead and subscribe to a monthly plan!

Fortunately for Grok, it wouldn’t necessarily have to pay this fee, as I asked Grok to write me 500 words on how AI detectors work and input that text (unchanged) into Sidekicker, which found Grok’s work was only 70% AI.

GPTZero did much better, finding my text was very likely “entirely human-written” while Grok’s content was 100% AI-Generated.

QuillBot, on the other hand, found my content to have 0% AI, and Grok’s to be 92%, so still fairly good.

And myPerfect Words thought my writing was 12% AI, including, oddly, my opening sentence, which included a personal story, while Grok’s text was 91% AI generated.

These results show just how important it is to find the right detector if you’re going to rely on these programs to help you evaluate a content’s source.

What happens if work is falsely flagged -- and are there solutions?

Unfortunately, given the nature of the world right now, what we end up with here is an AI tool, detecting AI-written content, which will presumably be rewritten to sound more human by a third bot -- because I guess the third bot is the charm.

And the consequences of this situation can be pretty dire for all the humans involved in the process.

If AI detectors make a mistake in an academic setting, students face disciplinary action. In the professional world, companies feel taken advantage of if they think human writers are trying to pass off AI content as original. Writers could also lose their jobs or spend a very angry hour or two revising falsely flagged text to keep the peace and keep the paycheck.

So, what’s the solution? There may not be a great one.

Research has also shown that people are pretty bad at detecting AI, especially with more advanced LLM models writing text, so editors can’t necessarily count on their own abilities to identify whether something is human-written or not.

Companies can require draft notes, review Google Docs histories, or use tools like Grammarly Authorship to see if text was pasted in or written. They can also work only with writers they trust who have a large portfolio of pre-AI work, which would be great for established writers but not so good for those trying to break into the business.

For their part, writers should make sure not to be formulaic, to vary their word choices and sentence lengths, and to incorporate personal anecdotes when they can.

I’ve even heard of some fellow writers throwing in a typo or two on purpose when they turn in their drafts, which, of course, is not something I’d ever consider in my own werk.

I mentioned the NYSE above, but there are many stock exchanges (NASDAQ, LSE, SSE, etc..) that all facilitate the buying and selling of securities. In fact, markets are so important to society that even if you’re not an avid day-trader, you’re still constantly interacting with them…Uber connects drinkers with drivers, Facebook Marketplace connects people with used furniture, and your dad is trying to connect you with a job.

Let’s say you want to retire and sell your stock in $METH to pursue other philanthropic work. Who’d buy them, and how much should they sell for?

• Markets define the price people are willing to buy/sell things at (Price Discovery).

• Markets offer a platform for people to trade assets, as it’s possible nobody in your neighborhood wants to purchase your $METH (Liquidity).

But what if, instead of the price per share being illuminated by these markets, it indicated the probability of a discrete event? These are called prediction markets.

By pen and paper at the local cock fight or via centralized, scaled, and well-capitalized firms like Kalshi and Polymarket, prediction markets are different from trading equities:

• They are binary. The thing either happens, or it doesn’t.

• They resolve once a particular event occurs, an outcome is reached, or a time period has elapsed.

You can’t buy shares in $METH via a prediction market, but you can bet that $METH will trade between $122 and $124 per share on January 6th.

Today, we are going to sell a gun at Bass Pro Shops. Errrr, sorry, Claude keeps hallucinating. Today, we are going to review the billions of dollars flowing through these markets, how the legacy of FTX lives on, what percentage is sports gambling, and we’ll find out how much money Kalshi makes. Let’s learn a new way to gamble.

History/Primer

Kalshi and Polymarket launched in 2018 and 2020, respectively. While these two make up the current duopoly, prediction markets originate further back. One of the OG’s is the Iowa Electronic Markets (IEM), hosting prediction markets since 1988.

Betting on one’s “beliefs” can get people into trouble, but the wisdom of crowds remains a valuable predictive force. Take the 2004 paper from Wolfer and Zitzewitz:

“These markets have predicted vote shares for the Democratic and Republican candidates with an average absolute error of around 1.5 percentage points… The final Gallup poll yielded forecasts that erred by 2.1 percentage points.“

Yet… over all these years, something has been missing… preventing us from gathering predictions, building markets, scaling to millions of people, and leveraging them for personal gain. That missing piece was well capitalized crypto web applications that offer you free groceries, giving us the ability to bet on the next pope being trans. Just as Hayek intended when he wrote The Use of Knowledge in Society.

We’ll explore Kalshi in detail, but there are other projects/protocols in the space.

How does this all work?

Prediction Markets extend humanity’s surface area for gambling. Personally, I might wager $10 that I can drink 10 beers before midnight, and my wife’s boyfriend might not believe me. On one side, I say “Yes, I can finish ten beers” while he says “No, you can’t finish ten beers”. Replace me with Lebron James, beers with points, before midnight with the end of the game, and you have an actual market you can trade on Kalshi.

At Kalshi, a market refers to a single binary market that resolves to “yes” or “no”. An event is a collection of such markets, and a series groups together similar but independent events. For example:

• The highest temperature in Miami is a series.

• Each day of this series is an event.

• Each event contains markets on the temperature: [68° to 69°] or [69° to 70°].

Each market contains a “yes” and a “no” with an associated order book. What’s an orderbook you ask? Let’s get some terms out of the way…

Every trade on Kalshi is a match made between a maker and a taker. Just like in our “contrived” scenario that for sure never happened, makers/takers trade against another person on Kalshi, not the platform itself. Instead of buying shares in a stock, you buy a contract that resolves to $1 if you’re right and $0 if you’re wrong.

As price-sensitive rational actors, I’m willing to put up $10 to win $20, and so is my wife’s boyfriend. With these bets in hand, the event has an implied probability of 50% (10/20). In “event contract” terms:

• We break this down into twenty $1 contracts

• Each contract is an agreement that we both lock in $.50

• At resolution, the winner gets to keep the other’s $.50

As trading heats up, it becomes helpful to track activity with an order book.

There are many different ways to display order books. They typically show resting orders, whether someone is buying or selling, how much they are buying or selling, and when the order was placed.

The bids contain all the people looking to buy, and the asks contain all the people looking to sell. In the example above, the bids and asks have been ordered from best to worst price. The gap between the best bid (highest purchase price) and best ask (lowest selling price) is called the spread.

In a liquid market, like the Super Bowl, the spread is razor-thin (maybe 1 cent). In an illiquid market, the spread might be massive because nobody cares enough to take the other side. This is why Kalshi offers incentives for both liquidity and market making.

Looking back at our order book, when a Market Taker decides $0.52 is a steal, he trades 50 contracts with my mom (you can see her on the bid side). The price of the asset just “moved” to $0.52, and that bid falls off the book. That is Price Discovery, or better said, the market is assigning a probability. The degens have updated the likelihood of my liver failure to 52%. Things look different looking at an actual orderbook from Kalshi:

You’ll notice a “yes” and “no” side to this order book. Contracts on Kalshi can resolve to “yes” or “no”, and traders buy/sell both.

• On the yes side, we have 13 contracts for sale as low as $.44, and 58 can be bought for $.42 with a spread of $.02.

• On the no side, we have 58 contracts for sale as low as $.58, and 13 can be bought for $.56 with a spread of $.02.

Wait… that looks funny. Why is everything inverse of each-other? Why do the bids for “yes” and the asks for “no” have the same number of contracts and the prices $.42 and $.58 sum to $1? Because buying yes is the same as selling no.

Now that we have a grasp of resting orders, how does matching work?

Kalshi uses a price and time priority algorithm on their central limit order book to match orders. On the surface, this is as straightforward as it sounds, orders are prioritized based on price and then time of submission; however, building an exchange and processing these orders at scale is not so trivial. If you want to know more about matching engines, I’d highly recommend DataBento’s write up. On Kalshi’s exchange, orders must be fully collateralized, so trading on margin isn’t a thing… yet.

For a while, MIAXdx via miax was the clearinghouse for contracts traded on Kalshi’s exchange (a central place where transactions occur). MIAXdx used to be called LedgerX, but miax acquired and renamed it through… drum roll please… FTX’s bankruptcy proceedings! After a while, Kalshi was like “we want our own clearinghouse”, so they built and registered Kalshi Klear with the Commodity Futures Trading Commission in August 2024. Just to take this full circle, Robinhood recently bought none other than… LedgerX from miax!

The Data

I hit Kalshi’s markets endpoint and trades endpoint to get historical data on ~30 million markets, 203 million trades, and over $41,700,000,000 in total volume.

Where is all this volume coming from? Kalshi generates significant traffic directly to its website and app. It has also partnered with some futures commission merchants (FCMs), that facilitate futures trading by accepting contracts on behalf of their clients. You may know them as that place you moved your IRA into… Robinhood, that place where pre-teens trade… WeBull, and that other place you have your shit-coins with… Coinbase.

The top events by volume were the 2024 presidential election at over $535 million dollars, followed up by the 2026 Superbowl winner at ~$244 million dollars.

Wait a minute… the Superbowl… is this just… sports betting?

Is this just sports betting?

Kalshi is regulated by the CFTC (Commodity Futures Trading Commission) which supervises U.S. derivatives markets. When I say “regulated”, what I really mean is “not regulated”. The Commodity Exchange Act (CEA) established the statutory framework under which the CFTC operates. This gives the CFTC authority to prohibit the trading of Onion Futures, but also to allow 18 year olds to trade contracts on Kalshi. Kalshi even boasts about “Minimum Age to Register & Participate” being 18+ in their FAQ section where they directly compare themselves to… sportsbooks. Sportsbooks are at the mercy of state governments, with some states banning sports gambling entirely and others requiring bettors to be 18 or 21+ (typically 21+).

Cool, but… Kalshi is different. You’re trading contracts against your peers, and they can be on anything. It’s not just sports, right? I’m not trading against the house, right?

Over 82% of contracts are on… sports. Kalshi is a volume business, the more contracts traded, the more revenue they gross in fees. All the better to be the first gambling platform for 18 year olds. By the way, they offer parlays, making up over 5% of total volume!

That other thing about not trading against the house… from Kalshi’s article titled “Who am I trading against”:

“Another significant player on the exchange is Kalshi Trading. Kalshi Trading is a separate entity from Kalshi Exchange… they are a participant on the exchange just like everyone else.”

If it smells like a sportsbook and trades like a sportsbook it might be a **loud gunshot, author collapses**.

Back to The Data!

There is a power law distribution of volume in these markets. Bucketing total volume in $, increasing by an order of magnitude, displays this quite well.

80% of the $0 volume markets are Combos (a.k.a multivariate events a.k.a parlays). Each one is a unique market that passes through Kalshi’s RFQ system with many not finding someone to take the other side.

With the same volume bucketing, splitting the buckets by how they resolved, we see that as volume increases, the percentage of markets resolving to “yes” also increases.

This shows that your base rate for any particular market should air on the side of resolving to “no”. The relationship with volume makes sense, events with significantly more markets should spread volume thinly across those markets with only one, or a few, resolving to “yes”.

Within these markets, average contracts per trade hovers around 150-250. Except for a massive spike in 2024 due to some 1,000,000 contract trades placed on the U.S. election. The median sits far under this, typically under 50 for most months.

Fees: Sportsbook vs Kalshi

If a sportsbook is like roulette, winning money through the odds offered, Kalshi is like poker, where the house makes money from the rake, agnostic to who wins or loses.

At a sportsbook, you’ll find “even odds”, lines that have a 50% chance on either side, like the coin toss at the Superbowl. Instead of giving you even odds, they’ll offer a 52.4% chance of heads and a 52.4% chance of tails, higher than reality.

• In a fair market, if you bet $10 on a coin toss you’d expect to win $10 if you choose correctly… but a sports book, with higher odds, will only pay you $9.09

On a sportsbook, if two people bet $10 on opposite sides of a coin toss, 1 person walks away with $19.09, 1 person walks away $0 and the Sportsbook walks away with $.91 or 4.5% of the total amount wagered. That 4.5% is what people call the vig, juice, hold, etc…

For example, the Islanders vs Devils market is, conveniently, 52.4%/52.4% on a sportsbook. On Kalshi, contracts are trading at $.51 (51%). So you should trade on Kalshi because you’re getting better odds at 51% vs 52.4% at the sportsbook, right?

No! The sportsbook offers worse odds, but the Kalshi taker fee of $.35 offsets this:

• Kalshi

  • Bet $10.55 (Buy 20 contracts for a total of $10.20 + $.35 fee)

  • To win $20

• Sportsbook:

  • Bet $10.55

  • To win $20.14 ($.14 more than Kalshi for the same bet size)

This isn’t the full story for all markets because Kalshi provides a liquidity incentive and a volume incentive. In addition, the Fee Structure is NOT the same for every market and Kalshi does pay you a yield on positions that accrues daily.

Fees: The Math

If you prefer your math in video form, check out Sam on the Technically YouTube channel:

What fees is Kalshi making on all this volume? First, we’ll look at the fees charged to “takers”. The formula is:

fees = round up(0.07 x C x P x (1-P))

Where C is the number of contracts and P is the price (ranging from $0.01 to $0.99).

We can plot this with the fee as a function of P and C. The price increases linearly as the number of contracts increases; this is controlled by P*(1-P) which, as you can see from the right chart, shows how the fees decrease as the underlying probability moves further from 50%, making highly likely and highly unlikely contracts have the lowest fees.

What we’ve discovered is the Bernoulli distribution. The Bernoulli distribution models the outcomes of a single “yes” or “no” question (in our case, a market).

• The variance of this distribution can be modeled as P(1-P) where P is the probability of “Yes”; the Y axis of the right chart.

• Variance exists between 0 and .25,

• Entropy (uncertainty or randomness in a probability distribution) is maximized at P = .5.

Why doesn’t Kalshi implement a flat fee, so we don’t have to make graphs? Likely due to trading concerns:

• If a contract costs 98 cents, the maximum profit you can make is 2 cents. If Kalshi charged a flat 2-cent fee, you would make $0 profit… no one would trade it.

Maker fees are identical in slope, they are just scaled down due to the .0175 multiplier, one quarter of the taker fee: fees = round up(0.0175 x C x P x (1-P)).

After downloading all 203 million historical trades on Kalshi, I know the exact price every contract has traded at. I can plug P&C into those equations and derive the total revenue Kalshi has generated from all these contracts… $545.6 Million dollars.

Here’s Kalshi’s trading volume by implied probability month over month:

Here’s Kalshi’s revenue from fees month over month:

It’s clear these markets are exploding in popularity, while DraftKings, Fanduel, and Fanatics scramble to the party. The, “We are basically doing the same thing but now it’s sudo-regulated and 18 years olds can do it” party.

Resolutions

An interesting resolution was when Dallas and Green Bay tied in an NFL game. The market resolved to 50/50 rather than yes or no, 100 or 0… there is no concept of pushing or voiding bets in a prediction market. When things get unclear, Kalshi steps in. In the data, Kalshi marks these results as “scalar”, with over 170,000 markets marked as such.

Market resolution seems to be fairly manual on Kalshi. They have a markets team that thoroughly reviews outcomes. Each market has an authoritative point of reference. For example, the superbowl lists out several sources and contains rules specific to the event. Despite all that, they still couldn’t figure out if Cardi B performed and resolved the market to the last traded price.

Polymarket said “Yes she did perform” which highlights how they do resolutions differently using UMA’s optimistic Oracle… but we’ll save that for another time.

Conclusion

With that, I have 9 beers to drink, and I’m way over my word count.

Quick legal side quest: There is another Prediction Market called PredictIT that specializes in political prediction markets. PredictIT is run by a firm called Aristotle that performs data mining for political campaigns. It was launched in 2014 as a nonprofit educational project out of the Victoria University of Wellington, New Zealand. To operate, they acquired a no-action-letter from the CFTC just like the IEM had, as they agreed to abide by certain guardrails and serve academic purposes. Then, in 2022, PredictIT got shlebanngged by the CFTC for NOT operating in accordance with their agreement. In 2025, they 360 no scoped the CFTC in federal court, and the “Cadillac of Prediction Markets” is back baby.

Anyways, basically, all these players in the “event contract” game are submitting letters and letters to the CFTC to not take action against them for failing to meet reporting requirements that they’d otherwise be required to perform. So far, the CTFC seems to agree, citing the “limited applicability of traditional swap reporting rules to exchange traded event contracts”. There’s other stuff here, like how Kalshi and Robinhood are classified, but suffice it to say that there are many conversations to be had in the future on how to regulate, tax, and manage reporting on these “new” entities.

1. The annual Technically reader survey is open until the end of the week.

2. Last week’s post (on vibe coding + the maker movement) from made the front page of Hacker News and roused quite a convo. To continue that convo, tomorrow Sachin + friends will host a Substack live session at 3:30pm ET to discuss whether we’re making anything of value when we vibe code, among other topics.

Now on to Palantir + the Forward Deployed Engineer role. Let’s give a warm welcome to Sung, who’s done technical sales at multiple software companies (most notably dbt Labs), on his first Technically post.

The TL;DR

The startup ecosystem is seeing an explosion of companies coining themselves as, “We’re basically Palantir, but for X.” Underlying that idea is the Forward Deployed Engineer, or FDE – think of them like a customer-facing engineer working directly with prospects and customers. The FDE seems to be all the rage right now…but is it actually a good idea for startups to have them?

This post will run through everything you need to know about FDEs, what they do, secular trends that are causing so many companies to want to hire them, and whether they actually make sense for most businesses.

If you prefer this post in video form, check it out on Technically’s burgeoning YouTube channel:

What is a Forward Deployed Engineer (FDE)?

A Forward Deployed Engineer (FDE) is a highly technical, customer-facing role where software engineers are embedded directly within customers to solve real-world problems. Originally pioneered by companies like Palantir, the role has become essential for AI, enterprise SaaS, and data infrastructure firms where products are too complex to be “plug-and-play.” Instead, someone from the vendor has got to get in there and make sure it actually works.

How is this different then a conventional technical consultant?

Technical consultants – think Accenture and the like – have been around for decades. Isn’t an FDE the same thing?

A lot of people had that same question when Palantir coined the role and it can be summed up simply. FDEs do highly custom work – like a technical consultant – but then take it a step further by generalizing the implementation and lessons learned into a core product. In Palantir’s case, an example is *Foundry, “*the foundational data operations platform, which provides the core capabilities for data management, logic authoring, Ontology development, analytics, and workflow development.” In a startup’s case, you’ll notice they use terms like “platform” which is analogous to providing the lego blocks to build use-case specific software vs. building from scratch every time.

How is this role different from a software engineer (SWE)?

SWEs are primarily internal and have minimal interactions with customers. The FDE, on the other hand, takes on the mantle of owning direct customer relationships. There’s usually a distinction, sometimes as subtext, that FDEs need high technical ability and emotional intelligence (EQ) to be effective in the role. If you’ve been in the workforce for years, you’ll recognize this combination is rarer than people think (or like to admit).

Why is this Palantirization narrative so popular now?

We don’t know what we don’t know

Deploying AI in production is hard, brittle, and constantly evolves. There is literally no such thing as best practice right now. For example, people were raving about vector databases to reduce bloat for LLM models retrieving context to perform tasks. But now, mainstream LLM models don’t need that infrastructure overhead because they handle 1 million tokens in their context windows; vector databases aren’t so hot anymore. Similarly, testing AI applications is an emerging art called “evals” that is in very early stages to even have convention. This builds a lot of anxious hesitation for anyone, even those on the cutting edge. You can imagine this feeling is more pronounced in large enterprises.

Only a real person can clear the fog of war

This then motivates the question of what’s worth retrofitting (think: slapping AI chat bubbles in your app) vs. replacing entire people, processes, and existing subscriptions. There aren’t enough role models in the industry yet, so companies need a real human with deeply lived experience to make sense of the constant change. To make this emotionally grounded, it’s like what a lot of people do when researching health problems with AI. It may give convincing general guidance, but you’ll want a real, human doctor to make big decisions and catch things you didn’t think to ask the AI.

Why can’t you be like Cursor?

The above couples tightly with the fact that expectations for being a “successful startup” have increased exponentially. Being a unicorn ($1 billion valuation) startup gave you pedestal prestige. But now, it feels like you have to be a decacorn ($10 billion valuation) startup to attain that same cachet. To reach that decacorn requires fast revenue growth, and the easiest way to get there is to win bigger sales deals that are six-figures on average vs. the 4-5 figures a lot of startups even 2 years ago saw as convention. This biases them towards going all-in on large enterprises.

But first, explaining enterprise sales

To understand the role of the FDE we must first take a detour to talk about old school enterprise sales.

In short, it’s a highly custom sales motion requiring many months, persuading and aligning multiple stakeholders and departments at a company, custom contracting, and “white-glove” onboarding. Some examples include buying a fleet of airplanes by a major airline. The airline likely won’t feel comfortable with standard pricing and contract terms with the swipe of a monthly credit card subscription. For software, the enterprise sales motion above was likely expressed in how your company bought slack or Microsoft teams, buying hundreds or thousands of seats in a single contract.

Working with enterprises is very very difficult

Let’s get a bit more meticulous with how selling to large enterprises looks and feels. The above affectionately termed “the decagon of despair” illustrates why enterprises are slow to buy new software, even if they really want it. I’ll belabor the point with questions that a potential customer thinks through when they see a demo or thinking seriously about buying.

• Institutional Inertia: Why is this worth doing extra work on top of my busy job?

• Charging Models: Do we get economies of scale as we expand usage of the product?

• Audit: What internal governing body/accounting firm will yell at me if we get this wrong?

• Capability: Does it solve the problem with a reasonable level of effort?

• Security: Does this touch the public internet? You got RBAC and SSO? SOC 2 Type 2?

• Outdated Paradigms: This mental model is the only one this org runs on (think: on-prem only)

• Regulation: What external governing body will yell at us if we get this wrong?

• Procurement: What’s a reasonable price to value?

• Legacy: Retrofit vs. replace?

• Change Control: Who is the project manager that keeps progress daily and maps names to scope?

These questions swim through an enterprise buyer’s head no matter how good a startup’s product is. It can be summed in an adage we’re all familiar with: “Change is hard.” Most enterprises aren’t willing to change with fancy slides, a demo, and even an undeniably great product. Enterprise buyers commit to a complicated, immersive relationship, and you need a face like an FDE to instill trust that it’s worth it.

Read more

First, there’s been a lot of change with Technically. We’ve assembled a pretty killer group of writers covering topics in new depth, from Will going in depth on data centers to Christy explaining why AI uses so many em dashes. In fact, Sachin’s post about vibe coding and the maker movement was the first post from Technically to ever make it to the front page of Hacker News, 6 years after I started this thing. It’s weird to transition from only-writer to editor and sometimes-writer, but this is good change.

The change in the labor markets is not necessarily good change. Last week Block laid off 40% of their 10,000 employees, despite the fact that “our business is strong” and “gross profit continues to grow.” Jack said it’s because of AI…but is that really true? Or did Block just overhire in 2020 – like everyone else did – and the time has come to pay the piper? For those of us using AI day to day, it’s hard to say we’re getting a 40% efficiency gain (as if that would even translate to an entire organization).

Quibbles about disingenuous headcount reduction aside, everyone seems to believe that more of these mass layoffs are coming. The story goes that AI is going to make entire swaths of workers obsolete, rendering them useless corporate husks who will need to resort to “get ready with me as an unemployed father of 3” reels to pay the bills. Grim indeed.

But even this change isn’t so clear. Like happened with previous waves of technological disruption, entirely new fields are opening up thanks to AI. Notion just opened up a new role for a “Model Behavior Engineer” whose job is to own the quality bar for Notion AI products. It’s a wacky combination of technical skill, good taste, running evals, and all-in-all doing a suite of things that completely didn’t exist a few years ago. Plus, contrary to popular narratives, job postings for software engineers – whose work is allegedly “a solved problem” by AI – are up 11% this year.

In fact, we’re seeing this in a quickly changing Technically subscriber base. Over the past few months we’ve been getting a ton of signups from industrial companies, from pipe manufacturers to railroads. New people are interested in software and AI, and how they can use it at work to make their jobs easier.

Bobby said it best in 1964: The Times They Are A-Changin. I believe that if we’re open minded, are willing to change quickly, and hustle a little bit, we’ll all be fine.

Which brings me to you. To make sure we're covering the right things for the right people, from time we run reader surveys. In last year’s survey, we realized that no one (not even us) knew what evals were.

Lend us 2 minutes for a 6-question survey, to share where we can help you become more technical this year:

Take the survey

The Maker Movement was the spiritual predecessor to vibe coding. The parallels are hard to miss. Vibe coding has slop. The Maker Movement had crapjects, a term the community coined for 3D-printed objects that served no purpose beyond proving you could extrude plastic into a shape. The Claude Code of that era was a $200 printer from Monoprice and a breadboard.

The scene around making produced what were probably the first internet-native network intellectuals. Chris Anderson (who wrote the widely-read piece about the long tail) left his editor-in-chief role at Wired to start a robotics company called 3D Robotics. Cory Doctorow wrote Makers, a sci-fi novel based around characters who are hacking hardware and business models to survive in a world where everything is falling apart. These were people who gained influence by participating visibly in a making culture and writing about what it meant.

A lot of the intellectual energy of the AI era orbits around AGI: when it arrives, what it’ll do to jobs, whether it will be aligned. The Maker Movement had its own gravitational center, and it was the idea that making physical things with your hands could produce an internal transformation. You would become more creative, more entrepreneurial, more self-reliant. The object you made mattered less than what the act of making did to you.

Puritanism With a Soldering Iron

In 2018, the media scholar Fred Turner published a paper that put this ideology under a microscope. His argument was that the Maker Movement had reinvented the theology of the Western Frontier for the digital age.

The specifics of seventeenth-century Puritanism are obviously gone. Nobody at a Maker Faire was talking about predestination. But Turner traced the literary forms and the millenarian structure—the belief that a great transformation is coming, and that individual discipline will determine who makes it through. In the Maker narrative, the American landscape is economically barren. Jobs have disappeared. Institutions have failed you. And in this wilderness, the lone individual searches inside themselves for signs of the entrepreneurial spirit, the creative spark, evidence that they are among the elect who will build their way to salvation.

Turner’s observation extends well beyond 3D printers. You can trace this same pattern through almost every hobbyist technology scene of the past fifty years. Homebrew computer clubs in the 1970s. Punk zines in the 1980s. The early web in the 1990s. Each one developed a community of practice—what Brian Eno would call a “scenius”—where people played with tools that the mainstream considered toys. Each one generated its own salvation narrative: master this tool, transform yourself, become the kind of person who builds the future.

And each one operated with a useful kind of slack. The tools were unproductive on purpose. Nobody expected your Arduino project to ship to customers. Nobody expected your homebrew computer to compete with IBM. The whole point was that you had permission to fuck around, and the finding-out happened gradually, through play, over years. This is where the old Silicon Valley adage comes from: “What smart people do on the weekends, everyone else will do during the week in ten years.”

What’s Different About Vibe Coding

Vibe coding broke this pattern in a way that matters.

Every previous wave of hobbyist technology went through a scenius phase—a period where small groups of weirdos played with tools before anyone expected economic output from them.

Vibe coding skipped that phase entirely. It was deployed directly to the general public, and almost immediately into the codebases of enterprise companies and well-developed products. There was no protected playground period. There was no time to accumulate the weird, useless, playful knowledge that scenius communities generate. Instead, there was immediate pressure to one-shot yourself into a hit product or solve a complex use case on the first try.

This matters because the scenius phase is where the internal transformation actually happens. When you spend two years making useless Arduino projects, you develop instincts about electronics, materials, and design that you can’t get from a tutorial. When vibe coding goes straight to production, you lose that developmental space. The tool is powerful enough to produce real output before the person using it has developed real judgment. When I speak with people who are on Claude Code 12-14 hours a day, I feel like I’m speaking to someone possessed by something, attempting to grasp a different reality. In the case of scenius, the feedback loop that tethers you to reality was provided by other humans. Someone looked at your project and told you it’s pointless, or brilliant, or both. While in the case of vibe coding, the feedback loop is provided by the machine, and you’re constantly attempting to discern if you’re going crazy or if something genuinely valuable has been produced.

What it produces is something like hypomania: a state where your productive capacity genuinely increases. You’re not imagining that you’re getting more done, you actually are, but your evaluative faculty is unaccustomed to this mode of creation. You lose the ability to distinguish between “this is good” and “I feel good making this.” Everything feels like a breakthrough. The output is real but your relationship to it is distorted.

The speed and ease of vibe coding create a kind of evaluative anesthesia. You can’t tell if you’ve built something useful or just something that exists. In some way, this is the sober version of hippies in the 60s trying LSD for the first time: sometimes you may have a breakthrough, or you may have a breakdown, but regardless of which, this is the opposite of the salvation through making that Fred Turner talks about.

The Maker Movement’s Quiet Ending

There’s a second reason the old transformation-through-making metaphor doesn’t fit vibe coding, and it has to do with how the Maker Movement actually ended.

The central promise—that distributed digital fabrication would bring manufacturing back to America, that every city would have micro-factories, that 3D printing would decentralize production—simply didn’t materialize. What happened instead follows a pattern that Joel Spolsky described years ago in his essay on commoditizing your complement: cheap 3D printers and Arduinos made prototyping nearly free, which was genuinely useful. But the deep, compounding knowledge of how to actually manufacture things at scale continued to accumulate in industrial bases like Shenzhen. Prototyping got democratized. The cheap tools commodified one layer of the stack and made the layer beneath it more valuable by comparison.

You can watch something structurally similar happening with vibe coding right now. People are rapidly prototyping tools that threaten to displace entire SaaS business models. But the value generated by all that rapid iteration and prototyping flows upward. It accumulates at the model layer, in the training data, in the infrastructure. The vibe coders themselves risk becoming interchangeable, each one spinning up impressive demos without accumulating durable value of their own. The pattern rhymes: cheap tools democratize one layer, and the layer beneath captures the surplus.

A New Metaphor: Consumption

With both of these forces at play—no scenius phase to develop through, and value accumulating upstream rather than with the maker—the old metaphor of transformation-through-making doesn’t hold up exactly. We need a new one.

The metaphor I’d offer is consumption.

Specifically: consumption of a surplus intelligence. AI represents an enormous amount of available cognitive energy, and vibe coding is one way of expending that energy before it goes to waste. Think of it like a resource that’s being generated whether you use it or not—and vibe coding is the act of channeling that surplus into play, into exploration, into rapid creation that may or may not produce lasting artifacts.

This framing has started showing up in various places. Rachel Thomas compares the experience of vibe coding something to the dark flow state when you gamble. The idea being that you are getting addicted to the superficial experience of creating, which might start off as flow, but eventually becomes something you are addicted to rather than something that helps you grow.

Consumption almost always gets treated as a negative behavior, especially if you’re an entrepreneur or builder. Consuming is what passive people do. Builders produce.

I think this framing is wrong, or at least incomplete. There are several productive ways to think about what consumption actually generates.

Taste as a Residue of Expenditure

When production becomes lightning fast with low marginal costs (when you can spin up an app in an afternoon), the scarce resource shifts to knowing what should exist. The vibe coder who burns through dozens of prototypes, building things and immediately discarding them, develops a kind of pattern recognition that the models themselves don’t have. This is judgment about what’s worth building, what feels right, what users actually want. It’s a sensibility, and sensibility is notoriously hard to commoditize because it’s illegible. You develop it by making a lot of things and paying attention to which ones felt alive and which ones felt dead.

The value capture here looks like creative direction, curation, taste-making, advisory roles. You’re selling the discrimination you developed by making things and throwing them away. The ideas guy is back. At the extreme end of this path, you become the protagonist of William Gibson’s Pattern Recognition: someone who has such finely tuned aesthetic instincts that companies hire them simply to say yes or no to something already production ready.

Attention as a Combustion Byproduct

Expenditure that’s visible generates spectacle, and spectacle generates attention. When you vibe-code something in public—building fast, shipping immediately, iterating in front of an audience—the product you make matters less than the performance of making it. And undoubtedly, much of vibe coding today is pure signalling performance.

The recent wave of “built this in a weekend“ posts works on this principle. The product is often mid. Sometimes it’s outright disposable. But the act of making it, timing the release, and dropping it into the network at the right moment is a performance of surplus, and people watch performances. The value capture is audience, reputation, and the optionality those create in the form of future collaborations, job offers, investor interest, consulting gigs.

This is structurally identical to how content creators already operate. A YouTuber’s individual video is an expenditure. The audience accumulated across hundreds of videos is the asset. Vibe coding just adds another medium to the content creator’s toolkit: instead of expending effort on essays or videos, you expend it on apps and tools, and you capture the attention the same way.

Projects as Gifts

If you treat your vibe-coded output as gifts—open source tools, free utilities, shared templates, public repos—you’re creating the conditions to occupy an interesting or powerful position in the network. Think of the people who built the early web’s most useful free tools and resources: They became nodes that other people oriented around.

The gift economy has always been the underlying value capture strategy of open source, but the consumption frame explains why it works psychologically for vibe coders in a way that the usual advice of “build open source projects to get hired” doesn’t quite land. When you frame it as strategic career-building, it feels transactional and a little desperate. When you frame it as expending surplus, it feels natural. You have extra cognitive energy available through these tools. You spend it. You give away what you made. And the gift economy does what gift economies have always done: it creates social bonds, reputation, and reciprocal obligation.

Signal Capture Before Upstream Absorption

Every time you vibe-code something, you’re generating signal. Signal about what users want. Signal about which patterns work. Signal about where the model fails, what edge cases it misses, what instructions it misinterprets. That signal currently flows upstream to model providers for free. Your prompts, your iterations, your corrections—all of it becomes training data for the next generation of models. You are, in a very literal sense, performing unpaid labor for the infrastructure layer every time you build something.

But informational exhaust can be captured before it drifts upstream. If you can structure the signal you’re generating—as proprietary datasets, as documented feedback loops, as systematic records of what works and what doesn’t in a specific domain—you end up holding something the infrastructure layer actually needs and can’t easily replicate. Every vibe coding session produces this exhaust as a byproduct. The question is whether you let it dissipate or whether you collect it. The people who collect it end up building what you might call a data fortress: a position that gets stronger with every prototype, even the ones that get thrown away, because the knowledge of why they failed is the valuable part.

This is the spirit of what early makers were accomplishing in the scenius. Trivial as their output may have been, they were immersed in the production process and through that developed a tactile understanding of their medium. With vibe coding, that data is generated for free. Will you use it?

***

Consumption doesn’t have to be passive. Surplus can be spent well. The key distinction is whether you’re burning energy with some awareness of what the combustion produces—taste, attention, social capital, structured signal—or whether you’re just spinning up a dozen projects and wondering why none of them stick.

Personally, I find the consumption metaphor to be a good way to deal with the burnout that comes with constantly using AI for various things. A lot of people approach making things with the mindset of craft, and naturally extend this framing to vibe coding. That framing feels noble and it’s deeply familiar, but it’s also a recipe for burnout, because craft assumes you are reaching inside yourself and pulling something out. The whole emotional architecture of craft is transformational: you struggle, and develop mastery, and the object you produce is evidence of inner change. When the tool is doing most of the producing, that framework starts to collapse. You’re left reaching inward for something that the process never required you to develop, and the gap between the effort you expected to invest and the effort that was actually needed starts to feel like a personal failure rather than a feature of the technology.

The consumption framing sidesteps this entirely. You’re not reaching inward. You’re starting from the position that there is extra energy available and it needs to go somewhere. The question shifts from “what does this say about me as a maker” to “what’s the most interesting thing I can spend this on.” That’s a fundamentally different emotional posture, and in practice it’s a much more sustainable one.

Part 2 will cover some more practical tips I’ve discovered for getting better words out of models, like style guides.

📢 Thanks to our sponsor: Rippling

But…

Read more

For most of us, even most engineers, this grey box is a sufficient level of detail on the datacenter, and that’s by design. Cloud service providers (CSPs) make it so you don’t need to know anything about the computers that serve your HTTP request or store your data. This is the fundamental value proposition of the cloud: you take care of the code, we’ll take care of the hosting.

But aren’t you curious?

Datacenters… So hot right now

Datacenters are the subject of renewed public attention, for better and for worse. On the bright side, datacenter investment is broadly holding up the US economy. A Harvard economist estimated that datacenter and related tech investment accounted for 92% of US GDP growth in the first half of 2025, despite being just 4% of total GDP.

On the other hand, all this datacenter construction has prompted major pushback from communities worried about their effect on the environment, utility availability, and property values. Over $64 billion in datacenter projects have been blocked or delayed due to local opposition. In Warrenton, Virginia, residents voted out every town council member who supported Amazon’s proposed datacenter. A nationwide poll found that only 44% of Americans would welcome a datacenter nearby, making them less popular than gas plants, wind farms, or even nuclear facilities.

So, what’s a datacenter?

When you search for the best Thai food near you, or post “happy brithday!” on your auntie’s Facebook wall, these requests are ultimately handled by a datacenter. The task you’ve set in motion by clicking the search button travels through the air, to your router, through a whole bunch of cables, to a computer in the datacenter that needs to actually retrieve Thai spots and then send those results back to you. These computers that do the heavy lifting behind the scenes, which are called servers, are the lifeblood of the datacenter. The datacenter is where thousands of servers that power your favorite application call home.

More than just pad see ew reviews flow through datacenters: critical time-sensitive information like bank transactions, visa applications, and medical records all rely on datacenter operations. When these critical centers of infrastructure have outages, businesses, institutions, and economies are hit hard. All this to say: keeping servers on, secure, and reliable is very important. Every aspect of the datacenter, its location, people, technology, and design, is in service of this goal. So let’s take a tour.

Getting there and getting in

You’ll probably need to drive to the datacenter, because datacenters are in the middle of nowhere. As we’ll learn in a bit, these operations depend on fiber access for fast networks, but also take up a ton of space, so cheap land is helpful. For this reason, datacenters are often on the outskirts of larger metros - think northern Virginia outside DC, and Oregon outside Portland.

Not just anyone is allowed into the buildings that run the economy. Modern datacenters employ biometric identification methods, a two-door “man-trap” to avoid anyone sneaking in behind you, a strict no phones / personal possessions policy, and the requirement that any guests be accompanied by staff at all times. With the price of GPUs these days, perhaps Ocean’s 18 will see Danny and co. get through all these mechanisms to steal a pocketful of chips.

Let’s get physical

In the main hall of the datacenter, the first thing you’ll notice will be the sound. A low but loud hum that comes from all sides at once. These are the mole people. Just kidding. These are the cooling fans! Computers do their best work at about the same temps that humans do - between 64 and 84 degrees Fahrenheit. This requires, among other things, a ton of fans. They blow cool air up from the ground to the front of the servers, and another fan blows hot air from the back of the servers out into the exhaust. The difference between the “cool aisle” and the “hot aisle” can be over 30 degrees.

Looking around, you’ll see rows and rows of tall cabinets. In scale, these fall somewhere between library shelves and narrow streets. The cabinets themselves are about 6 feet tall and 2 feet wide, and they’re arranged in rows that can stretch hundreds of feet. This is where the servers live. But before we get to the servers themselves, let’s talk about how all of this is organized.

Servers are housed in cabinets called racks. A standard rack is 42U tall, “U” being the unit everyone uses, equal to 1.75 inches of vertical space. Do the math and you get about 73 inches, or just over 6 feet. The 42U standard exists for practical reasons: it’s tall enough to pack in a lot of compute, but short enough to fit through doors and elevators, and a technician can reach the top without a ladder.

Inside a rack, you’ll find servers stacked like drawers. A typical server is 1U or 2U tall, so a single rack can hold somewhere between 20 and 42 servers, plus switches, power distribution, and cable management. The width is standardized at 19 inches (the actual mounting rails are 17.75 inches apart), which means any server from any manufacturer will slide into any rack. This interchangeability is the whole point, which we’ll get to in a second.

Read more

January was a month of change for us at Technically. In addition to some fascinating writing about AI + neuroscience, why AI models use so many em dashes, and what people are actually using AI for, we brought on a whole new roster of writers that I’m excited to introduce you to. You’ll be hearing from them a lot for …

Read more

If you’ve been keeping up with the online discourse about AI writing, you may be surprised that I put an em dash in this post. That’s because so many human writers are steering away from this once-common punctuation mark, which is now viewed as a hallmark of writing written by chatbots.

In fact, AI bots love the em dash so much that it can be hard to get them to write content without including it, even when you give the bot explicit instructions not to do so. LLMs can be so funny sometimes.

Of course, this raises the question of why em dashes are all over AI-written content — and whether human writers should give up this once-beloved punctuation mark entirely, so their content isn’t immediately clocked as being written by an LLM.

Why does AI love the em-dash so much?

AI’s love affair with em dashes seems to have a simple explanation: The data used to train large language models was full of em-dashes. The AI is simply mimicking the writers that it learned from.

In fact, there’s some evidence to suggest that the content AI was trained on included significantly more em-dashes than you might expect. And weirdly enough, their prevalence seems to have become a deep bias that’s embedded into how LLMs understand the flow and structure of writing.

AI-training material may have used an overabundance of em-dashes

One theory behind AI’s love of the em-dash is that the later-generation AI models, which rely on it much more heavily than earlier iterations, were trained on older books that included more em-dashes than most modern writers would.

Early on, most AI models were trained based on a mix of public data on the Internet, as well as based on content from pirated books. However, in a quest for better quality training data as the tools evolved, AI models started scanning older texts. Curating the massive data trove that is the internet has been a major focus of AI Labs for more recent model generations, and finding quality text from books was certainly part of that.

The exact timeline for when this happened is something of a mystery, but Anthropic started in 2024, based on court documents, and other AI labs likely made a similar move somewhere between 2022 and 2024.

If AI labs digitized mostly older books, which is a common belief largely because of expired copyrights, their AI programs may have been fed writing with significantly more em dashes included in it — especially as studies show the use of that use of the em-dash peaked in the 1860s.

It may not have been the books alone, either.

Another theory suggests that AI may also have picked up em-dash use from Medium, which automatically converted two hyphens (--) into an em-dash since the company’s founder was a fan of typography. Since Medium may have been seen as a source of high-quality writing by LLMs (and ergo upweighted in training by labs), AI may have determined that the em dash is a key feature of high-quality prose.

The brevity theory

There’s also another competing theory as well. Some suggest that AI favors em dashes because of brevity.

AI models think in tokens, or small chunks of text, and they constantly have to evaluate the next token to use. The more tokens the model uses, the more chance there is to make a mistake, so AI models prefer to use fewer tokens to reduce loss

An em dash is just one token, while other alternatives, like “, and”, actually add up to three tokens. As a result, the em dash ends up being favored to improve efficiency.

Read more

• I used ChatGPT to build me a workout plan

• I vibe coded a NYT cooking inspired recipe generator

• I had AI write me a song about the Python 3.14 release notes

It’s not just people using AI anymore; cor…

Read more

This post is going to explore a few ways in which this is true and explain some of these rather complicated ideas in more simple language.

Neural networks, the basis for everything

The obvious place to start is neural networks, the architecture for pretty much all of the AI models that you use today like the GPT family, Claude, Nano Banana, and the like. Obviously the first word – neural – likens these models to the animal brain. The human brain has something in the range of 86B neurons, which are specialized cells that transmit nerve impulses; essentially the core unit of how our brains transmit information and signals. The idea is that neural networks work in kind of the same way.

And indeed, you’d be hard pressed to find an explanation of neural networks that doesn’t make an analogy to the human brain. Take, for example, Technically’s very own breakdown of neurons from the prolific Nicole Errera:

Neurons are the basic building blocks of AI architectures, modeled after the actual biological neurons that transmit signals throughout the human brain. Remember, AI models are essentially pattern investigators; they find the underlying pattern in the data. You can think of these neurons as the mathematical functions that are doing this hard investigative work, getting into the weeds of the data and figuring out what’s going on.

The math performed by individual neurons is actually pretty simple – it’s usually just basic multiplication and addition that you could do with a calculator. So how are AI models able to capture such complex patterns, like the ones involved in language and vision? The trick is to string together a lot of neurons – like hundreds of millions of them.

So in practice, the neural net in a model like GPT-5 does, at least loosely, resemble how a mammalian brain works. This is no accident. If you trace the history of the neural network you’ll end up back in 1943 (when I was born), when Warren McCulloch (neurophysicist) and Walter Pitts (mathematician) wrote a paper proposing a mechanism for how neurons might actually work.

To illustrate their hypothesis they modeled a simple neural net using electrical circuits. Further attempts culminated in a breakthrough at Stanford in 1959 when MADALINE (it’s an acronym) became the first neural network applied to a real world problem – eliminating echoes on phone lines. So in short, the fact that neural networks (roughly) approximate how the brain works is not an accident, this insight is core to their entire historical origin.

Now any neuroscientist worth their salt will tell you that there’s more that we don’t know about the brain than there is that we do know. The true inner workings of this organ are still really a mystery. And so it would be naive to argue that neural networks work in the same way that the brain works. But it’s safe to say that they’re inspired by what we do know about how the brain works, at least loosely.

Read more

My sources tell me it’s not too late to say happy new year.

2025 was the year everyone started to use AI at work. 2026 is the year we figure out how to get the most out of it, vibe code things that actually work, and steer clear of hallucinations (unless you’re into that kind of thing, in which case you might get oneshotted by ayahuasca).

So here’s what we’ve been up to for the past month.

New on Technically

2026 vibe coding tool comparison

Available as a free post on Substack (courtesy of Vercel) and now in its permanent home in the AI, it’s not that Complicated knowledge base.

All hail vibe coding, savior of the Ideas Guy. But which vibe coding tool should you use? I tested 4 of the most popular options by building the same app on each: an internal dashboard for Vandelay Industries’ potato chip import/export operations.

For each I cover how well they work, any quirks to watch out for, plus pricing and how far you can get on a free plan.

The AI user’s guide to evals

Available as a paid preview on Substack, and now in its permanent home in the AI, it’s not that Complicated knowledge base.

As non-technical people build more and more stuff with AI they are starting to run into the same issues as software engineers – it’s hard to know when things are working or not. This guide to evals gives you the background you need to start adding effective monitoring to your AI system.

Based on my conversation with eval expert Hamel Husain, this post covers:

• Why evals matter: moving from “I think the bot is getting better” to “hallucination rate dropped from 15% to 3% this week”

• Look at the data: why you need to examine your failures before writing fancy tests

• Assertions over LLM judges: why simple keyword checks beat complex AI-judging-AI setups

• A practical 4-step workflow: vibe check, spreadsheet, simple fixes, then targeted evals

This one is pretty practical.

AI will replace you at your job if you let it

Also available as a free post on Substack and in the AI, it’s not that Complicated knowledge base.

Like you, I’m extremely tired of the recurring headlines about “AI replacing the workforce,” written almost exclusively by people who know nothing about AI and have never been part of said workforce. And yet… many people are using AI so carelessly that you won’t be able to blame their bosses when they decide AI can do the work in their stead.

I believe that staying ahead is going to mean really taking the time to use AI intelligently, customizing it to your needs, tweaking your prompts, and keeping an iterative mindset. AI will only replace you at your job if you let it.

From the AI Reference: Prompt Engineering

If you weren’t in the loop and got too caught up in the holidays of it all, we recently launched the AI Reference—a companion to the Technically Universe focused specifically on AI concepts. This month’s featured term is one everyone using AI tools should understand:

Prompt Engineering

Prompt engineering is the art of talking to AI models in a way that gets you the results you actually want. It’s the practice of crafting inputs that reliably produce the outputs you’re looking for, whether that’s generating marketing copy, analyzing data, or answering customer questions.

Here’s the thing: AI models are incredibly powerful, but they’re also incredibly literal. Ask ChatGPT “write about dogs” and you might get a 500-word essay about canine evolution. Ask it “write 3 bullet points about why dogs make good pets” and you’ll get exactly what you need. That difference in specificity is prompt engineering in action.

Why this matters:

• Good prompts can save you money, time, and frustration when working with AI

• The more context and structure you provide, the better models can give you what you actually want

• Think of it as learning to communicate with a very smart but literal-minded assistant

• Key techniques include few-shot prompting, chain of thought reasoning, and setting clear constraints

Coming up this month

We’ve got three posts on deck for January:

• AI for PMMs: automating customer segmentation. A look at how product marketing teams can start using AI to segment customers.

• What’s fine-tuning? A breakdown of what fine-tuning is and why it comes up so often in AI discussions.

• How AI chips get made: An overview of how the hardware behind modern AI systems is produced.

Are you using AI at work?

We want to hear what’s actually working. If you’ve automated a tedious task, built something useful, or helped your team adopt AI without chaos, send us a reply and share your story.

They told me not to write this. They said “stay in your lane, Justin.” You just explain technical things, you can’t recommend gifts. You don’t know how to reach the hearts of loved ones. Well, I’ll be DAMNED if I don’t try!

By the way, I’m taking the last two weeks of the year off to go cook in a restaurant (literally), so Technically will be offline. I know it will be difficult to soldier on without my sweet voice hitting your inbox every week with technical explainers…but I have confidence in you.

Without further ado…

The Ohto CR02

Yea, I can type, but sometimes I like writing things down the old fashioned way. And nobody is better at crafting the instruments to do so than the Japanese. I tested 4 different Ohto pens and the CR02 was my favorite after a few days of furious scribbling.

It’s a ceramic rollerball, so it glides across the page. And at 0.5mm it’s the perfect thinness for my dainty preferences.

Price: $16

The Orbit Plus Turntable

It probably doesn’t surprise you that I’m a jazz guy (I even play drums occasionally). And the genre is best appreciated on vinyl for reasons that are set in the cosmos. Orbit makes affordable and high quality turntables that look great and are assembled in the USA. I’d recommend going for the Plus, which for an extra $150 gets you a much better cartridge.

If you already have speakers, great. If you don’t I can highly recommend the Kanto YU4 alongside their SUB6 subwoofer. They’re reasonably priced, sound great, and have a built in preamp plus bluetooth so you can stream from your devices if you so desire.

Price: $399

The Salomon XT-6

If you live in Brooklyn you already know that these were THE shoes of the early 2020s. And now that they are slightly less cool it’s the perfect time to wear them without judgement from your family. These things are literally the most comfortable supportive shoes I have ever owned, better than running shoes for sure.

I already own 3 pairs and unfortunately for my roommate there are probably more in my future. You can purchase them from Salomon or any other reputable retailer.

Price: $180

A neo-vintage Rolex Explorer 16570

Modern Rolex is so overdone. What you really want is stuff from the 90’s and 2000’s, when the bracelet links were hollow and Rolex was still a working man’s watch. The 16570 is perhaps the best of this era, sporting the 3185/3186 caliber movement that’s pretty much still in use in their modern GMTs.

They say you buy a Rolex when you’ve made it. I say, you buy a Rolex to fake that you’ve made it. Shake the hand of a potential business associate and they will hear the rattle of that old world craftsmanship. They think, wow, this is a man of eminent taste and respectability. This is a woman I could do business with. Before you know it, you are running for president.

You’re welcome.

Price: priceless (but actually around $8-9K)

The Bradford Chef Knife in AEB-L Steel

This has become my workhorse in the kitchen, and I barely touch my fancy super thin Japanese Bunka anymore. It’s sturdy and thick, but can hold a great edge (at least for a week or two, it does need a lot of sharpening). I love the composite handle, because I don’t have to worry about water getting in there like with my wooden handle knives.

Bradfords are proudly hand finished in Washington since 2012. They have a lifetime warranty (including sharpening) and are insanely reasonably priced for what you’re getting.

Price: $219

--

Be it a record player, a knife, or even a simple pen, I hope your gifts are well received. Happy holidays everyone.

Here at Technically, we believe that the holidays is about the grind — you’ve finally broken free from being the idea guy on the business side. As Charlie Parker said, “now is the time” — get out the laptop, heat up a hot pocket, and prompt away.

The holidays are for vibe coding. Idea people thrive in the holidays. Let’s dive in.

—

It’s nearing 2026, and you (probably) don’t need a software engineer to build the app of your dreams. I tested 4 of the most popular vibe coding tools to see which was the best. TL;DR:

• Replit was the most feature rich, well thought out, and powerful

• v0 was the best if you’re already a developer and want a technical interface

• Lovable and Bolt were pretty much objectively worse than the other two

Most importantly, I believe that the initial app building phase in vibe coding is largely commoditized across tools, and what makes any platform a better fit than another is largely pricing models and integrations.

In this post I’ll walk through my experience with each of these tools, covering:

• How they work, and important differentiating features

• Pricing, tokens, and how far you can get on a free plan (I paid for 2/4)

• How good (or bad) they are at fixing bugs

Throughout I’ll be using a very serious, very real set of prompts. As Art Vandelay, owner and CEO of Vandelay Industries, my team needs an internal dashboard to handle the importing and exporting of potato chips to the United States.

Here’s the prompt I started each tool out with:

I am the CEO of Vandelay Industries, a global firm that handles the importing and exporting of potato chips to the United States. I need to build an app for managing the internal operations of my business. The app should have 3 main views:

1) An inventory manager / CRM of all of our shipments of potato chips. This should be a table view backed by a database with columns for things like each shipment’s potato chip brand, size, monetary value, and who and where it is being shipped to

2) Analytics and charts on our shipments: top shippers, top brands, volume for this month, things like that

3) A map of current shipments and where they are located

From there I inspected each app and went through each bug one by one, prompting the model to fix it.

Thanks to our friends at Vercel, who make v0, for sponsoring this post + giving me a good reason to sit down with all of these vibe coding tools.

The below reviews are my own opinions, and as you’ll see I shoot you straight.

Replit

Replit was the most feature rich, felt the best thought out, and overall created the best and most polished finished app of all of the tools I tested. But it also took by far the longest, frequently set itself into doop loops, wasted tokens on nonsensical testing that didn’t work, and there’s no way to publish your app without paying.

Replit stat sheet

Initial prompting and planning

When you prompt Replit Agent it starts by generating a plan. In most vibe coding tools you can either toggle on planning mode, or the tool automatically defaults to it.

Another unique feature among the vibe coding tools I evaluated was this choice Replit gave me before building anything:

This brings me to my first main point, which is that despite building essentially an identical UI to the other vibe coding tools I tried, Replit took a really long time to build it: about 15 minutes or so. My sense is a lot of this time was wasted on the model getting confused, adding and then removing things, and unnecessary testing.

It’s extremely trippy watching a model create a codebase from scratch; it’s eerily similar to a human, trying stuff and getting errors, then fixing those errors and starting the whole thing over again.

Pricing structure

Towards the end of the app build, I got paywalled:

Replit’s pricing isn’t credit based, and instead calculates a cost per prompt based on how long the prompt took to resolve. For every prompt you will get a breakdown of how long it took, how many actions were done, and then how much your agent usage cost.

Here you’ll notice that 75% of my costs were for something called “Automated UI Testing” – this is one of the worst features in Replit by a mile and is unfortunately enabled by default. It essentially tries to automatically test your app using some sort of emulated browser:

The problem is that it doesn’t really work, takes orders of magnitude more time than your actual app building prompts, and ends up costing you extra money at phases where you didn’t even need testing. And it’s enabled by default, which it really shouldn’t be.

Overall the app cost about $6 for the initial build, and subsequent smaller bug fixes were usually $0.50 or less.

Final app

Anyway here’s what the final app looked like:

Despite the fact that I didn’t really specify a lot of functionality on this page, Replit was smart enough to add a lot of it for me: sortable columns, the ability to add a new shipment, the ability to edit and delete a shipment, etc.

The app started without any data, so I needed to prompt Replit to make a fake dataset and it did a good job:

The app has no data. Can you generate a mock dataset of potato chip shipments that matches the schema of the database?

The initial app was pretty good but had some bugs that needed fixing. For example, the sorting on the “destination” column didn’t work, so I prompted the model to fix it and it did correctly the first time.

Integrations

Like I said above, I believe (and you will see that) the initial app building phase in vibe coding is largely commoditized across tools. What made the Replit experience good for me was all of the other stuff. For example, their UI for editing the data in the database:

The Replit UI is built modularly via different tabs. The first tab has the preview of the app, and when you open a new one you can choose from all different sorts of “features” for lack of a better word:

To my knowledge this is far and away the most comprehensive suite of features for a vibe coding tool; but at the same time, probably overkill and too technical for the typical end user.

Publishing and deployment

Replit prices publishing like a cloud provider, and there is no free option for deployment. To publish you choose a machine configuration in the cloud, plus whether you want autoscaling and things like that. Again, to the typical non-developer, I’m not sure they’ll be able to accurately decide how many CPUs and GBs of RAM they’ll need.

If I had to personify the Replit Agent, it feels like that zany engineer friend of yours who chaotically works all night, zigging and zagging to build something, but somehow the final product is actually amazing.

v0

I think of v0 as the best vibe coding tool if you’re someone who can actually code. It tries to strike a very different balance between ease of use and customization, and leans heavily towards the latter. It also has the smoothest, fastest, and most accessible publishing / deployment flow of any of these tools. It’s from the team at Vercel so this does not surprise me in the least.

v0 stat sheet

Initial prompting and planning

Like Lovable, v0 was off to the races after I gave it the signature Vandelay Industries prompt. The only thing it asked was if I wanted to integrate with Supabase, their default database provider (to which I answered, yes, of course my dear). I’ll talk more about this Supabase integration, and why you might prefer this to how Lovable and Replit do theirs, in a bit.

The Supabase install ran into a brief hiccup and I needed to re-select my newly created database from the dropdown, instead of the model proceeding automatically once I completed the signup.

v0 built the whole app in about 4 minutes, and like Replit, it smartly intuited that it should build the ability to add, edit, and remove shipments. I ran into some overflow issues with the charts on the analytics page, and it took a couple of prompts before the model was able to fix it.

Though it didn’t automatically use it off the bat, v0 does have a planning mode, and you can also tell v0 to plan explicitly in your prompt. Otherwise the feature is hidden in one of the toggles in the prompt box:

Pricing structure

v0’s pricing is pretty straightforward – there’s a free plan where you get $5 in credits every month, a $20/mo premium plan where you get $20 in credits a month, a $30/user/mo team plan where you get $30 in credits a month, and then above that, business and enterprise plans that have different credit schemes. You can also buy any amount of credits on top of what you get from your monthly plan.

Building my app and subsequent fixes took <2 credits, so I was able to build the whole thing for free.

Final app

The final app looked a lot like the others, save for the navigation being on top instead of the side, and a curious use of emoji.

v0 also added sample data without me needing to prompt it.

Integrations

I mentioned earlier that v0 doesn’t have a built-in database, and instead defaults to a Supabase integration that requires you to create an external account there. It’s only a few extra clicks, but you do need to choose a configuration for your Supabase database (there are both free and paid options).

At surface level you might assume Replit’s and Lovable’s approach – building a database natively – is simply easier and better. But this is not entirely true. Like I said in the intro, v0 is engineer-leaning and prioritizes flexibility and customization over pure ease of use. The ability to choose which database you want to use – both in terms of the database itself, and in terms of the power, size, and configuration of that database – is important for a lot of people (like yours truly).

So if you want to use a different database – like let’s say Neon, or Redis, or even Vercel’s Blob storage, they have native integrations where you can do that in a few clicks. v0 also has few-click integrations for other things like Grok (and unrelated, Groq), fal, Stripe, etc.

Publishing and deployment

This is another area where v0 shines. Vercel made a name for themselves by making it as easy as humanly possible to deploy your app, and that’s exactly what you get with v0.

When you click publish, your v0 app automatically becomes a full-fledged Vercel project; you get all of the advanced features you’d get if you had built your app manually like automatic deploys, detailed logging, analytics, etc.

Lovable

Lovable was a lot smoother, faster on the initial build, and easier to use than Replit; I was also able to create and iterate on my entire app for free. But the final product was significantly less polished, there were glaring security issues (entirely Lovable’s fault), and the platform overall felt way less…mature?... than others I tested.

Lovable stat sheet

Initial prompting and planning

Like Replit, I asked Lovable to build a dashboard for Vandelay Industries. Unlike Replit, it kind of just started doing it without asking me for any preferences or planning or anything. The only thing I was prompted back was whether I wanted to to use Lovable Cloud, which is their out of the box backend (I said yes):

Lovable frames all of their backend services – database, authentication, and models – as a single “Cloud” product, whereas for Replit, each one of these is an individual add-on and treated as a separate product.

Within a few minutes I had my app, but the V1 was a lot less polished than Replit. One glaring example: Lovable used the wrong font color for the sidebar, so there was no way to see how to navigate between the shipments panel, the analytics panel, and the map panel.

Perhaps the analytics will show that nobody is clicking on any of the other tabs

I asked Lovable to fix it, and it took two prompts: the first time it thought it had fixed it, but it hadn’t. During the second prompt it got stuck in some sort of doom loop that required me to reload the page and start the prompt over. Also, the “map” that Lovable generated is, by most definitions I can gather, not a map:

I think of Lovable as vibe coding with guardrails. It’s a significantly simpler platform with more things taken care of for you, less detail given to you, and an overall more plushy, daycare-like feel. That’s nice if you’re building something simple, but less nice if you’re more technical or have more complex needs.

Pricing structure

I was able to build this whole app for free, which is nice…but looking into Lovable’s actual pricing left me with more questions than answers. Consider the Pro Plan, their cheapest:

So you start with 100 credits every month. But then you also get 5 credits every day, which can accumulate to up to 150/month; but is that including the 100 every month? Then there’s a completely different billing system for Cloud, their backend package, where you get up to $25 in “usage” (not credits?) every month. But, this usage deal only lasts until the end of 2025, then it’s going to be…something else. Credits can also roll over, but only on the cadence of your plan, so if you’re monthly, they only roll over for 1 month.

And what is a credit exactly? Well that completely depends. Lovable’s pricing page says there are two modes – default, and chat – and each burn credits differently. But wait, as of July 2025, agent is now the default mode, which replaces legacy mode. We are now up to 4 different modes, are you keeping track?

I want whatever this pricing team is smoking.

Final app

The final app ended up looking pretty similar to all of the other vibe coding tools. Almost like they’re using the same models under the hood…

Unlike Replit, Lovable did not intuit any features on top of what I explicitly asked for. So on this shipments page, there’s no way to edit, add, or remove data.

The analytics aren’t exactly useful, but then again, all of this data is fake…speaking of which, Lovable was the first vibe coding tool I used that automatically generated a fake dataset to populate the app with. I thought this was pretty clutch and it was nice to avoid having to prompt the model to do that.

Integrations

Like I mentioned earlier, Lovable bundles all of the usual backend pieces you’d need into one product they call Cloud. Cloud has its own pricing and usage mechanism, distinct from the credits system used for the actual vibe coding. There’s also a “native Supabase integration” but it’s unclear what relationship this has with Lovable’s own Lovable Cloud. And perhaps to only my own disappointment, there’s no smooth UI for editing the data in the database like Replit had.

Then there are a lot of other integrations: they’ve got Stripe, GitHub, etc.. “Integration” means different things for different vibe coding tools. Which brings me to Lovable’s security advisor. This thing – a valiant effort, for sure – attempts to scan your app and tell you if it has any vulnerabilities. And it actually works pretty well. For example, it scanned my app and noted that my database – yes, the one that I trusted Lovable to set up for me – had no security measures whatsoever and any hacker could log in and mess with my data:

On the one hand this is great, because most non-technical people vibe coding would have no clue that this is a problem. On the other hand it’s not great, because LOVABLE IS THE ONE WHO BUILT THIS APP IN THE FIRST PLACE. If the model knows that it’s bad for a database to have no protections, why didn’t it build the app correctly the first time around????

Publishing and deployment

Publishing your Lovable app doesn’t consume any credits in of itself, so if you were able to build your whole app for free like me, you can also publish it to the world.

Bolt

Bolt occupies a medium somewhere in between Replit and Lovable. It follows the “plan first, build second” philosophy (like Replit), but wasn’t very good at following its own plans. Pricing is token based and very opaque and confusing. Overall, it felt like their model was decent at doing what you asked, but would then create some sort of side effect that you didn’t want.

Bolt stat sheet

Initial prompting and planning

The Bolt prompt box on the site has a “plan” toggle that lets you have the model plan your app first, and you then review that plan, instead of going straight to implementation. For larger builds I would highly recommend always utilizing some sort of planning, for whatever reason models perform a lot better with a two step process like this.

This is what a generated plan looks like:

The model attempts to add detail to my pretty vague prompt, sketching out an initial requirements document. And from there you can tell it which functionality matters to you and which doesn’t (for example, I don’t have access to shipment location data).

There’s also a dropdown for choosing different models – which at first was exciting – but then I realized that you…don’t really have a choice.

Once you’re satisfied with the plan the model generated, you can click the “implement for me” button – although that button never showed up for me and I needed to manually prompt the model to implement the plan (fine). There’s a nice dedicated UI for showing the model’s progress through the plan:

All of this sounds really nice, and it would have been, except that Bolt didn’t actually follow its own plan. For example, in the plan the model put together, it specified:

3. Inventory Manager and CRM Table View

• Build interactive data table with sortable columns for all shipment fields

• Implement search and filter functionality across multiple columns

• Add pagination for handling large datasets efficiently

• Create modal forms for adding new shipments with validation

• Enable inline editing for updating existing shipment records

• Add bulk actions for managing multiple shipments at once

• Include export functionality to download shipment data as CSV

Hm yes, sortable columns, a modal for adding new shipments, inline editing, very nice, very nice. It didn’t build any of that, it just gave me a read-only table:

It took another cycle of prompting, planning, and implementing to add those features. Which is fine, but like, if you’re going to make a plan, stick to the plan the first time.

Then there was the analytics dashboard, which essentially made no sense:

For each of these nonsense charts, I needed to prompt the model to fix them. And then there’s the map. The map was no fun.

I shit you not, it took 5 cycles of prompting back and forth to try to fix this thing, and eventually I gave up because it seemed like it wasn’t in the cards.

Pricing structure

Bolt’s pricing is not straightforward. They are the only vibe coding tool in this post that charges by tokens instead of credits or some generic dollar value. For $25/mo you get 10 Million tokens every month, and unused tokens roll over to the next month (but not further than that). So what is a token exactly?

Great question. Bolt’s answer is essentially that we can’t really know for sure.

(Lucky for you, we’ve already written about what tokens are and how they work)

OK, so tokens are hard to explain, fine. But at least it’s easy to see how many tokens your work in the Bolt app is costing, right? Also no. Unlike all of the other vibe coding tools in this post, there is no way to know how much your prompts and responses in Bolt cost, even in terms of tokens. The expectation is that you prompt away and periodically check your account’s token balance, I guess?

Final app

The final app, after the planning fiasco and subsequent work required to compress the many columns into a layout that fits on one screen, looks pretty good:

Integrations

Bolt, like Replit and Lovable, has a native database. There’s a nice, if a bit primitive, UI for viewing and editing the data in your database:

Like Lovable, use of the database is priced separately from use of the model. And adding to the pricing confusion, the database is priced in relation to compute use, so you get a specific size of VM and hours of compute based on the plan you’re on.

Outside of their native database (and authentication) Bolt’s integrations ecosystem is less developed than others in this post – there is essentially just Stripe as far as your actual app is concerned (the rest deal with hosting).

Publishing and deployment

You can publish your Bolt app for free and they will support up to 333k web requests; if you upgrade to the $25/mo plan you get 1M web requests. Is this number actually enforced? I’m not sure.

Which vibe coding tool shall rule them all

To summarize, of the vibe coding tools I tried:

• Replit was the most feature rich, well thought out, and powerful

• v0 was the best if you’re already a developer and want a technical interface

• Lovable and Bolt were pretty much objectively worse than the other two

If I were starting a new project from scratch I’d use Replit or v0, and spend more time paying attention to how to plan well instead of just having the model go off and do what it thinks is best.

Pricing models vary wildly across tools, as well as how far you can get without paying and what it costs (if it costs) to deploy your finished app. Overall, for an app of low/medium complexity like mine, expect to pay ~$10-$15 to get a decent finished product that you’ve had some time to iterate on.

Have you used any of these tools? Have opinions on which is the best? Let me know in the comments.

Let me start by saying that I do not think AI is going to put massive swaths of people out of work. But that’s only going to be true as long as we are smart, adaptive, and embrace these new tools to make us all more productive and more creative. The problem is that there are tons of lazy people out there who are using AI completely carelessly, with such minimal oversight such that they are essentially writing their own pinkslips.

If you carelessly offload the core and soul of your job to AI, you cannot be surprised when someone decides AI can do it instead of you.

If you are a software engineer for whom Cursor now writes all of your code with minimal oversight or creative input from you, this is bad. If you are a marketer who is generating entire blog posts and page sites with vanilla Claude prompts, this is bad. If you are an SDR who has ChatGPT write all of your outbound emails with no customization whatsoever, this is bad.

At some point, your boss is going to realize that the AI is doing your job, not you.

The brain drain from junior talent and financial modeling

I see this pattern with more junior people the most. They are at a point in their careers where they have minimal experience and maybe more importantly minimal taste. The risk of overusing AI is huge here, because without going through the motions and developing that earned intuition for the mechanics of your work, you will never develop that taste. IMO at the highest levels, you are paid for exactly this.

I was talking to my friend CJ of the wonderful Mostly Metrics newsletter about how this manifests itself in finance. A lot of what finance teams at startups do is build financial models, which as any investment bankers in the audience can tell you, is something with a lot of manual, repetitive work; in theory a great candidate for some help from AI. Yes, some help. But CJ told me that he’s seeing more junior talent is offloading the entire financial model building process to AI and not building any parts of it from scratch anymore.

(I’ll admit, the use of the word “model” here is a bit confusing.)

This is not good. To intimately know how something works and develop an intuition for it, you need to do the work yourself. In this case, to build the model from scratch, to develop an understanding of what the sensitive variables are, and maybe most importantly, to be able to explain it to someone.

“Maybe I’m like an old man yelling at windmills just because I spent at least 10 years in the trenches doing that from scratch. But they’re skipping steps one, two, and three and jumping straight to four.”

Prompting an AI model to build your entire forecast for you will get you a quick answer, sure. But you didn’t build it, so you don’t understand it. What happens when the assumptions baked into the model change?

This is true across so many different disciplines. Just think about the explosion in vibe coding. It’s amazing that a single prompt in Replit can get you a working app. But if you don’t understand what’s going on in the app at all, don’t try to refine and improve it…what have you really created? And what’s stopping someone else from doing it better than you?

Something something Icarus flying too close to the overheating GPUs.

Use AI to augment your work, not replace it: the facked vs. cracked framework

When it comes to using AI, the best work products are the ones that combine your unique insight and creativity with the automation firepower from models. If the work product you’re creating is not in some way authentically you, or clearly borne of your handiwork, then you are giving up too much to AI and this does not bode well for your long term career prospects.

Across conversations with teams who are trying to use AI at work, plus some of my own experience, here are a few examples of facked vs. cracked uses of AI. In other words, finding the sweet spot where AI is helping you but not doing your entire job for you.

Finance

• Facked: prompt the AI model to build your entire financial model for you.

• Cracked: build the bones of the financial model yourself, use AI to automate repetitive spreadsheets copying and pasting and/or test new scenarios.

Marketing

Good marketers are using AI to help them generate better content faster, not entirely offload content creation…after all, who the fuck wants to read that.

• Facked: use vanilla prompts to generate an entire finished blog post of slop.

• Cracked: add style guide to context, use AI to generate a starting template, write and edit from there.

Software engineering

Before you say “OK, but this is a straw man,” no. People are absolutely doing this.

• Facked: offload massive features and parts of your codebase to AI without supervision.

• Cracked: tabbed auto-complete, heavily customized agents with clear guardrails, testing, and code review.

Sales (well, debatably)

SDRs are perhaps the most guilty party here. If there’s one really clutch thing to use ChatGPT for, it’s hyper-personalization at scale…and yet who is doing this.

• Facked: use vanilla prompts to generate a bare minimum amount of personalization, like “I noticed you just finished up an impressive 3 years at Meta.”

• Cracked: use models to do standardized deep research on prospects, store the data in Clay, use a second highly customized prompt to personalize based on research.

Use models, do not let them use you!

A great place to begin is starting small: pick a recurring, manual process to automate instead of trying to tackle an entire function or workflow. AI models work better when you give them a narrow scope, clear context, and a tight feedback loop. Find one specific, annoying thing you do every week, like summarizing meeting notes or categorizing customer feedback, and try to automate that.

Then, on Tuesday, everything went awry: your system hallucinated a policy that doesn’t exist, and a customer got a free Labubu. On Wednesday, it responded to a user in German (you are based in Ohio). By Thursday, you’re back to doing it manually because you can’t trust the “magic” bot you built.

You are not entirely surprised to discover that your tool has some rough edges that need sorting out and your prompts are not entirely bulletproof. And thus, your journey into evals begins.

The TL;DR

• Evals are just “software testing” adapted for the fuzzy, probabilistic world of AI. They measure (and help you fix) when your system screws up.

• The most important part of evals is data analysis – you cannot measure and fix what you cannot identify. Look at your data!

• Complex “LLM as a Judge” setups seem sexy but will probably ruin your life; avoid them until you have exhausted simple keyword checks (assertions).

• If you don’t measure your AI’s performance systematically, you aren’t building a tool; you’re building a slot machine that occasionally pays out productivity.

A conceptual grounding for evals

To understand evals, we first have to take a brief detour into how normal software works.

In traditional engineering, code is deterministic, which is (another) fancy word that means it’s perfectly predictable. If you write a function that adds 2 + 2, the answer is 4. It is 4 today, it is 4 tomorrow, and it is 4 during a Black Friday sale. Testing this is easy: you write a “Unit Test” that asserts that the answer = 4. If the computer returns 5, the test fails, and you fix whatever embarrassing error was in the code.

AI models are different. They are stochastic. They are probabilistic engines that guess the next word in a sentence. If you ask an LLM to “summarize notes from this call” the output might be different every single time you run it.

So, how do you “test” a summary? There is no single “correct” string of text to compare it to, and there are conceivably multiple different kinds of responses that would make you happy. Perhaps you would prefer the LLM summarize the call as “a complete waste of time.”

This is where evals come in. An Eval is a system that takes the fuzzy output of AI and grades it against a standard of quality. It’s the difference between saying “I think the bot is getting better” (vibes) and “The bot’s hallucination rate dropped from 15% to 3% this week” (science). Well, it’s still not entirely science but it’s closer.

There’s a lot written about how engineering teams building AI into their products should do evals…but quite a bit less written about how you can use them in your day to day using AI tools at work.

One of the experts on the subject is Hamel Husain – he helps teams build successful AI products and teaches the AI Evals For Engineers & PMs course on Maven. I talked to him about what non-engineers can do to bring evals into the work they’re doing with AI to make their systems more foolproof. Then I wrote down what we talked about. So here is that.

The flow goes something like this:

1. Analyze your system and figure out where things go wrong (look at the data)

2. Create evals (simple or complex) to fix those things

3. Rinse and repeat (or at least repeat)

Read more

In 2017 I was traveling with Winston Churchill across the scantly-visited regions of southern Tibet when we came across a peculiar guru. I spent months learning at his hip, observing his peculiarities, and gathering bits and pieces of his doctrine. All of this I would eventually publish in my acclaimed 2017 paper, “Attention is all you need.” Continuing on my research, you won’t be surprised to hear that we published a lot about AI this month: all about RLHF, why Microsoft is reviving an old nuclear power plant to build data centers, and why starting with small, manual tasks is the best way to get yourself started with AI at work.

We’ve also been building something bigger behind the scenes to make all of this easier to learn. More on that later!

Here’s what’s new:

New on Technically

What’s RLHF?

Available as a paid preview on Substack, and now in its permanent home in the AI, it’s not that complicated knowledge base.

Though much of the effort and compute for training AI models goes into pre-training, you might be surprised to hear that pre-trained models are kind of dumb. They’re like a precocious toddler: ask them a simple question and you’ll get a long winded, disorganized, and poorly vetted response.

This post explains the three-step process that fixed this, and enabled today’s models like Claude:

• Supervised fine-tuning teaches models the format of helpful answers

• How building a reward model trains an AI to judge responses like a human would (because paying humans to score millions of examples is impossibly expensive)

• Reinforcement learning lets models generate millions of responses and learn which ones score highest

RLHF is basically a finishing school for AI – teaching it manners, helping it read the room, and turning raw intelligence into something you’d actually want to work with.

A practical breakdown of the AI power situation

Available as a free post on Substack and has found its home in the AI, it’s not that complicated knowledge base.

What in the world is going on with all of these data center deals getting announced by OpenAI? And what’s a gigawatt? As AI Labs scale up compute, they’re starting to invest in the thing compute needs to run, but we can’t currently give it: electricity.

This post covers:

• The Scaling Law and why AI labs believe more compute (GPUs) = better models

• How our grid is woefully underpowered for data center compute, and demand for power has been roughly flat over the past 20 years

• The nuclear option (not really) reveals why tech giants are literally reviving nuclear power plants

• The three-way handshake breaks down how deals between AI labs, cloud providers, and energy companies actually work

Sam Altman said it himself: “Everything starts with compute.” OpenAI is planning for 25+ gigawatts of capacity. For context, a large cloud data center maxes out at 1GW. This is an unprecedented buildout, and energy (not code, not even chips) is a real issue.

For AI at work, start with something small

Available as a paid preview on Substack and now seated in the AI, it’s not that complicated knowledge base.

If you’re looking to get the most out of AI tools today, your best bet is to start small: pick a recurring, manual process to automate instead of trying to tackle an entire function.

This post walks through:

• Why the “messy middle” is AI’s sweet spot, with tasks that take 30-90 minutes, happen regularly, and are mind-numbingly manual

• Technical reasons to start small, since it’s way easier to give models the context they need for small tasks, and you can iterate faster with tighter feedback loops

• Real examples from writing, finance, and other fields

AI isn’t going to do your entire job for you, at least not yet. Think of it as a tool to handle the annoying, repetitive parts while you focus on the stuff that requires actual judgment and creativity.

Introducing the Technically AI Reference (!)

Now with all that being said, it became pretty clear we needed a single place to send people who want to actually understand this stuff (or die trying).

After five years of breaking down APIs, databases, and assorted software sorcery for 75K readers, we finally pulled the AI fundamentals into one home.

The Technically AI Reference is our new library of plain-English explainers that walk through what’s happening inside modern models. You’ll find guides on context windows, RAG, RLHF, fine-tuning, hallucinations, sampling, and many other core concepts. Each piece is written to help you build real intuition about how these systems behave and how to use them well.

It’s fast, it’s simple, and it’s completely free. Explore it now — and tell us which concepts you want us to cover next!

From the Universe AI Reference: Hallucination

This month, we’re pulling a concept straight from the new AI Reference. (Don’t worry, the Technically Universe terms are still there whenever you need software fundamentals.)

What is Hallucination?

Hallucination happens when an AI model generates information that isn’t accurate. The model doesn’t really verify facts or check its claims, it just produces text by extending patterns learned during training, and those patterns don’t always match reality.

Why this matters:

• Models can sound confident even when the details are wrong

• Hallucinations show up as fake citations, invented facts, or plausible-sounding errors

• They can create issues in tasks that rely on accuracy

• Knowing how hallucinations happen helps you prompt more effectively and verify outputs before relying on them

Coming up this month

We’ve got two posts on deck for this month:

• AI will replace you at your job if you let it: A straight shot of truth about how careless use of AI isn’t good for job security, why junior talent is at the highest risk, and how to use models in a way that actually builds skill instead of erasing it.

• Which vibe coding tool should you use? We review Replit, v0, Lovable and Bolt, to see who will be Vandelay Industries’ preferred vibe coding tool.

Are you using AI at work?

We want to keep hearing about how you’re actually using these tools in the wild. Made something cool? Found a workflow that saves you hours? Discovered a use case we haven’t thought of? Respond to this email and share your story!

90s kids will without a doubt remember these:

Behold, the Magic Eight Ball™.

The eight ball was an answer machine. You’d ask a question, turn over the eight ball, and get a random response. Don’t count on it. My sources say no. It is decidedly so. Signs point to yes!

Trying to decide if you should call her? Ask the eight ball. Wondering if you’ll ever get that job interview? Eight ball. Unsure about which college you should go to? You should probably ask your advi…nope, eight ball. This plastic orb of whimsical wonder was the oracle of an entire generation – my generation – injecting a little random levity into what was an uncertain time in life for many of us.

The idea and technology behind the eight ball were originally created by Albert C. Carter in the mid 1940s. Inspired by his clairvoyant mother, Carter patented (yes) a series of crystal ball like fortune telling devices, all of which went nowhere. That was until 1950, when Chicago’s Brunswick Billiards Company was looking for a promotional toy of sorts…and the rest was history.

The way it works is simple. Inside the eight ball lies a canister full of dark blue liquid, in which is suspended a plastic multi-faceted die (usually 20 sides). When you turn over1 the eight ball, the die floats up and lands on one side, showing you a decidedly random message. The visage became so popular that it even made its way into high fashion, if you can call it that.

The thing about the eight ball is that it’s fun. Like opening a fortune cookie and peeking inside at the little white flag, you couldn’t help but chuckle at how right or how wrong it really was. The eight ball is a gimmick. It’s a good bit.

On the flip side, there were people who actually did take the eight ball’s response to heart. And it was precisely the randomness inherent in the eight ball that they respected. If you needed a new direction, if the sum of your life choices had led you to somewhere uncertain, or even if you just couldn’t decide between the 10 or 20 piece McNuggets…well, maybe you wanted to just leave it up to the universe this time.

A fundamental philosophy was shared between the eight ball ignorers and the eight ball listeners. You know how the eight ball worked, and that’s exactly why you used it. Our understanding of the device’s mechanism directly informs our perception of the output. We cannot understand, or make use of, a mechanized response without knowing how the response was formed.

AI will only take your job if you let it

We have magic eight balls today, and they’re called LLMs. They are completely transforming every white collar (hello) worker’s lives and will continue to do so as they get better and better. The quality of LLM responses is not just impressive, it’s not just useful, but it’s getting a lot better incredibly quickly. Ergo, the question on everyone’s mind seems to be: will these take my job?

In my mind the answer is: AI will only take your job if you let it. If you assume the position of luddite and shun these models, trusting your own (manual) work above the AI, you will likely at some point get automated. If you offload too much to the model, carelessly prompting and pasting while letting the AI do all of the work, you will likely at some point get automated. But, if you treat the model as a powerful tool, to be wielded as a reflection of your own taste and soul, you will not only not get automated, you will be able to accomplish orders of magnitude more than you ever dreamed possible.

But this is where the problem lies: too many people are incredibly lazy AI users. They write vanilla prompts, they fail to scope the problem properly, they don’t understand how to use context, and they take the first response without iteration. If you treated our last generation of tools like this – whether it’s Hubspot as a marketer or VSCode as a developer – you’d get nowhere. And if your usage of AI is this overly simplistic, it’s only a matter of time before someone more senior than you at work realizes they can run the same basic prompts as you can.

If clever, personalized usage of AI tools is so important to our success, why do so many people not do it? Well to start, nobody has any fucking clue how these models work. They think that pre-training is when you teach your child to use the toilet. RLHF, RAG, these are acronyms of mystery. To most, Machine Learning might as well be organic chemistry.

You obviously don’t need to be a computer scientist to use an AI model. But without any fundamental understanding of what they even are, you will not be able to use them productively. If you want to use these models well – and avoid being used by them – you need to understand the basics of how they work:

• If you want to prompt well with good context, you need to understand what context is, how models interpret text, and what a context window limit allows you.

• If you want to apply AI with good taste to the right problems, you need to understand how the word-guessing LLM architecture works and what it’s good or bad at.

• If you want to vibe code an app that actually lasts, you need to understand chains of thought, why models work better with plans, and how to use the system prompt to your advantage.

Another reason this is important is that models get things wrong, all the time. It’s called hallucinating and much of it goes completely undetected because of how confidently models respond to prompts (Oren Etzioni phrased it as “a very impressive-sounding answer that’s just dead wrong”). And nowhere is this more visible than in the courtroom, where there are already hundreds of cases of legal decisions where GenAI produced hallucinated content.

Unfortunately, it’s really quite hard to get smart on AI. This stuff is highly technical, it’s complicated, and the people building it aren’t really prioritizing education of the non-elites. There are 100 newsletters out there covering the latest developments in AI…but how many stop to tell you how a neural network works? Hell, I majored in Data Science and I still find myself needing fairly extensive research to write intelligently about whatever the latest in Claude is.

Meanwhile, models are getting more and more complex. Unlike most of the rest of software, AI is a research discipline. And the speed at which said research makes it into product is unprecedented. To know what makes GPT-5 different from GPT-4 is not as simple as reading a few lines from a release blog post. Today’s AI model is actually an amalgamation of several different models that would each have been groundbreaking just a few years ago. And I’m sure in a few years today’s state of the art will be primitive.

Sometimes it can just feel like this technology is simply beyond the average person’s ability to understand. And yet, it has never been more important to understand it. Which way, western technology user?

Announcing the Technically AI Reference

I’ve spent the last 5 years writing Technically, trying to make software and AI easier to understand. Over half a decade, 100+ articles, and 75K+ subscribers later, I believe I’ve done good work towards making it easier for the average joe to understand what’s going on. You can read Technically to understand what an API is, how to swim in a data lake, or what Stripe actually does.

The same desperately needs to be done for AI, and soon. I promise you I’m writing as fast as I can. And this is all why I’m excited to announce our newest release, the AI Reference. It is without a doubt the fastest, easiest, and free-est way to get smart on AI.

You can read about:

• Context windows

• Fine tuning

• Hallucinations

• Prompt engineering

• …and dozens more

Each concept has a basic explainer, common questions answered, and some really nice graphics that accompany you on your learning journey.

Like I said this thing is completely free, you can just peruse at your own leisure – although if you’d like to send me money that is fine too. We’ll be constantly updating and adding to it – respond to this email if you have any topic suggestions.

And remember the ancient wisdom of Curtis James Jackson III: “Understand AI or die trying.”