You need to remove those files from git's cache.

You can remove specific files or folders:

git rm --cached <file>
git rm -r --cached <folder>

Or just clear the whole cache:

git rm -r --cached .

Then commit your changes so git knows which files it should track going forward:

git add .
git commit -m "fix: stop tracking ignored files"

Related

• Clear .gitignore cache • Ainsof So'o
• Ignore files that have already been committed to a Git repository • Stack Overflow
• How do I make Git forget about a file that was tracked, but is now in .gitignore? • Stack Overflow

Parents and children

To achieve that nesting, my solution was to manually give each note that represented a subtopic (or subsubsubtopic) a parent value (via markdown frontmatter) and then move any child notes (i.e. notes with a parent value) into their parent's children array:

/**
 * Given an array of collection items, returns the array with child items nested under their parents.
 */
function nestChildren(collection: Note[]): Note[] {
  // Step 1: Create a mapping of item IDs to item data
  const slugToNodeMap = collection.reduce(
    (nodesBySlug, item): Record<string, Note> => {
      // Add an empty children array to the item's existing data
      nodesBySlug[item.id.toLowerCase()] = { ...item, data: { ...item.data, children: [] } }
      return nodesBySlug
    },
    {} as Record<string, Note>,
  )

  // Step 2: Build the nested item tree
  const tree = collection.reduce((roots, item): Note[] => {
    // Find the node matching the current collection item
    const node = slugToNodeMap[item.id.toLowerCase()]

    if (item.data.parent) {
      // If the note has a parent...
      const parentNode = slugToNodeMap[item.data.parent.toLowerCase()]

      if (parentNode) {
        // ...add the item's data to the parent's children array
        parentNode.data.children.push(node)
      } else {
        console.error(`Parent slug "${item.data.parent}" not found (this should never happen).`)
      }
    } else {
      // If the item has no parent, treat it as a new root-level note
      roots.push(node)
    }

    // Return the updated tree and keep iterating
    return roots
  }, [] as Note[])

  // Return the final, nested tree
  return tree
}

/**
 * Returns all notes with child notes nested under their parents.
 */
export const getNestedNotes = async (): Promise<Note[]> =>
  nestChildren(await getCollection('writing', note => isNote(note)))

Incremental left padding

My solution for visually indicating how those notes related to each other was to multiply each note's left-padding by 1.4x its descendent level:

<ul class="list-notes">
  {
    notes.map(note => {
      const getLinkText = (item: Note): string => item.data.title || item.id

      const getChildren = (item: Note, level: number) =>
        item.data.children && (
          <ul>
            {item.data.children
              .sort((a: Note, b: Note) => a.data.title.localeCompare(b.data.title))
              .map((child: Note) => (
                <li
                  class="before:content-['└'] before:pe-[0.3rem] mt-[0.1rem]"
                  style={`padding-left: ${level === 0 ? 0 : 1.4}rem`}
                >
                  <a href={`/${child.slug}/`} class="link">
                    {getLinkText(child)}
                  </a>

                  {/* Recursively render grandchildren, etc */}
                  {getChildren(child, level + 1)}
                </li>
              ))}
          </ul>
        )

      return (
        <li class="mb-2 break-inside-avoid-column">
          <a href={`/${note.slug}/`} class="link">
            {getLinkText(note)}
          </a>

          {getChildren(note, 0)}
        </li>
      )
    })
  }
</ul>

Maintaining note hierarchies is a chore

That was a fun programming puzzle to solve, but unfortunately it made my note-taking more difficult by forcing me to decide where each new thought belonged in the hierarchy before I could jot it down.

To remove that friction, I've adopted to a flat (and eventually searchable and filterable) approach to note-taking instead. But I wanted to share this former nested-topics solution in case anyone else finds it useful.

<p class="!my-0">Really, really no vertical margins</p>

tmux -V

Not tmux -v.

Not tmux --version.

Not tmux version.

Why you gotta be so special, tmux?

SELECT DISTINCT column
FROM table_name

I found that helpful today when I wanted to understand the significance of a value I was looking at by comparing it to the alternatives.

Happily, it turns out creating your own pipe mechanism is a one-liner:

from functools import reduce

result = reduce(lambda acc, f: f(acc), (fn1, fn2, fn3), value)

Which you can reuse by wrapping it in a function:

from functools import reduce
from typing import Callable, TypeVar

_A = TypeVar("A")
_B = TypeVar("B")

def pipe(value: _A, *functions: Callable[[_A], _A]) -> _A:
    """Pass a value through a series of functions that expect one argument of the same type."""
    return reduce(lambda acc, f: f(acc), functions, value)

And calling it with any number of functions:

assert pipe("1", int, float, str) == "1.0"
# => i.e. str(float(int('1')))
# => i.e. int("1") -> float(1) -> str(1.0) -> "1.0"

So you can stop thinking up names for throwaway variables like these:

def str_to_float_str(value: string):
    as_integer = int(value)
    as_float = float(as_integer)
    as_string = str(as_float)
    return as_string

assert str_to_float_str("1") == "1.0"

Credit to Statically-Typed Functional Lists in Python with Mypy by James Earl Douglas and the returns.pipeline.flow source code for the inspiration.

Update: Nov 29, 2024

While the pipe function above with the Callable[[A], A] type hint works fine if every function in the pipeline outputs the same type (A), the example I showed above doesn't actually work out very well! Mypy notices that some of the functions (int and float) output a different type than we started with (str), so we aren't actually passing A all the way through.

After trying a number of workarounds (and getting some good advice on Reddit), I learned that you can either tell Mypy what's going on by painstakingly articulating every possible overload:

_A = TypeVar("A")
_B = TypeVar("B")
_C = TypeVar("C")
_D = TypeVar("D")
_E = TypeVar("E")

@overload
def pipe(value: _A) -> _A: ...

@overload
def pipe(value: _A, f1: Callable[[_A], _B]) -> _B: ...

@overload
def pipe(
    value: _A,
    f1: Callable[[_A], _B],
    f2: Callable[[_B], _C]
) -> _C: ...

@overload
def pipe(
    value: _A,
    f1: Callable[[_A], _B],
    f2: Callable[[_B], _C],
    f3: Callable[[_C], _D]
) -> _D: ...

@overload
def pipe(
    value: _A,
    f1: Callable[[_A], _B],
    f2: Callable[[_B], _C],
    f3: Callable[[_C], _D],
    f4: Callable[[_D], _E],
) -> _E: ...


def pipe(value: Any, *functions: Callable[[Any], Any]) -> Any:
    return reduce(lambda acc, f: f(acc), functions, value)

Or, you can just use expression, which already does this for you.

I'm going to do the latter, but this was a fun exercise in the meantime. 😎

In brief, here are the steps:

1. Create a Google Form
2. Link it to a Google Sheet
3. Copy the pre-filled link to your form and edit it to end with &submit=Submit and say formResponse instead of viewform
4. Use a Text shortcut action to insert your data into that link
5. Use a Get contents of URL action to send your data to your form results sheet

A response link for a form with one question will look something like this:

https://docs.google.com/forms/d/e/<form-id>/formResponse?usp=pp_url&entry.<question-id>=<answer>&submit=Submit

Here's what my shortcut to save the current URL looks like:

dependencies = [
	"google-cloud-bigquery[bqstorage, pandas]",
]

2. Create a BigQuery client

from google.cloud import bigquery

bq_client = bigquery.Client(project="project")

3. Query your data

query = """
SELECT column_one, column_two
FROM bq_table
WHERE column_three = true
"""

df = bq_client.query(query).to_dataframe()

I looked it up on my usual domain registrar (Hover): $20.99

I looked it up on CloudFlare: $12.99

Sold.

(The free security features don't hurt either.)

Then, I discovered those are the dependencies of the packages I installed. And the dependencies of those dependencies. And so on.

Seems kinda obvious now.

# Which packages and casks are installed?
brew list

# Why is <package-name> installed? Which packages are using it?
brew uses <package-name> --installed

# Which dependencies came with <package-name>?
brew deps <package-name> --tree

function act {
  # do stuff
}

Instead of like this:

act() {
  # do stuff
}

It turns out function parentheses are optional in bash and zsh. Though, using parentheses is more POSIX compliant if that's relevant for your use case.

In fact, all of these variations are equivalent:

function act () {
  command
}

function act() {
  command
}

function act {
  command
}

act () {
  command
}

act() {
  command
}

act () command

act() command

#!/usr/bin/env bash

echo "♥" $(pmset -g batt | grep -Eo '[0-9]+%')

2. Make the file executable

chmod +x battery.sh

3. Call it from tmux.conf

set -g status-right "#($HOME/.config/tmux/battery.sh)"

Until you check, the most accurate type to assign that data is one that means "I don't actually know".

(This post is a response to questions I received on Reddit suggesting we always know a given value's data type. For any data coming from outside your program, I think that assumption is risky!)

Making Assumptions

Let's say you're taking input from a user and expecting it to be a string. A naive approach would be to assume the data will always be what you expect:

const getUserInput = (): string => {/*...*/}

const unsafe = () => {
  const data = getUserInput()
  data.toUpperCase()
}

No warnings, no problem, right? Not exactly... There are no warnings because we told the type checker the user input is always a string. That's why it's happy to let us call string methods on data.

But what if data is sometimes undefined (or anything other than string)? In that case, this code will experience an uncaught TypeError at runtime saying, Cannot read properties of undefined (reading 'toUpperCase'), which may leave your program in a broken state.

This is where "unknown" can help — and unfortunately where a lot of people reach for an "any" type. Be careful! "any" is the opposite of "unknown" and effectively disables type checking by telling the type checker all assumptions about a value are safe. You probably don't want that.

Unknown to the Rescue

Some languages explicitly include a type called "unknown" (e.g. TypeScript has one), while others have a type you can treat similarly (e.g. Python's object type effectively means "unknown").

Whatever your language gives you, the general approach is the same:

1. Assign the "unknown" type to the unverified data
2. Explicitly validate the data's relevant characteristics before you use them
3. Be happy when your tooling warns you about unsafe assumptions you're making

No Assumptions

Let's ask the type checker to help us be more careful:

// 🤞 Should be a string, but who knows...
const getUserInput = (): unknown => {/*...*/}

const unsafe = () => {
  const data = getUserInput()
  data.toUpperCase()
  // 🚨 'data' is of type 'unknown'
}

Perfect! We want those type warnings. We're calling a string method (toUpperCase) on a value we haven't confirmed is a string. That's risky.

To resolve the warning, we need to validate the assumption we're making about data's type:

const getUserInput = (): unknown => {/*...*/}

const safe = () => {
  const data = getUserInput()

  if (typeof data === 'string') {
    data.toUpperCase() // confirmed safe
  } else {
    // handle invalid input
  }
}

With each assumption you validate, the type checker "widens" its understanding of your data from its narrow starting point (unknown) to a type with more characteristics (e.g. string).

And now you can be sure what type of data you have.

The object type

The object type in Python "is a base for all classes" containing the "methods that are common to all instances of Python classes." And since in Python "types" and "classes" are effectively the same thing, object is both Python's base class and base type.

While Python doesn't explicitly advertise object as the language's "use when you're not sure" type, it works well for that use case because it only includes the features all types share.¹ Beyond that, it makes no assumptions about what type features a value might have.

In practice, using the object type tells Mypy to warn you if you start making unvalidated assumptions about what you can safely do with that value:

def unsafe(value: object) -> None:
    value.get("some_key")
    # 🚨 "object" has no attribute "get"

To pass the type-checker, you need to confirm value has a get method before you use it:

def safe(value: object) -> None:
    if not isinstance(value, dict):
        return None

    value.get("some_key")
    # other logic...

You won't need object for data you create yourself, but it can make validating external data much safer and easier.

You can do that by defining a generic type variable:

from typing import TypeVar

unknown = TypeVar("unknown")

This is useful anywhere you want Mypy to remind you to make no assumptions about a value's type:

def unsafe(value: unknown) -> None:
    value.get("some_key")
    # 🚨 Mypy: "unknown" has no attribute "get"

def safe(value: unknown) -> None:
    if not isinstance(value, dict):
        return None

    value.get("some_key")

Using a Generic to mimic unknown is a safer option than Any, which effectively disables type-checking by assuming anything you try to do with a value will work.

Update

Thanks to this helpful discussion of this post, I've learned that Python's built-in object type already works as an "unknown" type, so there's no need to resort to hacking generics to replicate one.

I'll leave this post up to avoid a broken link, but let's all agree to use "object" instead!

You merge your PR only to discover you accidentally missed a bug and now prod is broken. Gah! Is there a quick way to undo that merge?

Yes! What you need to do is “revert” your PR. Here’s how to do that using the GitHub UI.

Reverting a PR

1. In the browser, go to your merged PR
2. Click the “Revert” button near the bottom of the page to create a new revert- branch that reverses your changes
3. When prompted, open a PR for that revert- branch
4. Merge the PR

That’s it. 😀 Crisis averted. Your main branch is back to how it was before your feature.

Restoring your original feature branch

If you want to debug and re-open an improved version of your original PR, here’s what you need to do:

1. Click the “Revert” button at the bottom of the reversion PR you just merged to create a new revert-revert- branch that includes the same changes as your original feature branch (i.e. revert the reversion)
2. Pull that revert-revert- branch locally (don't open a PR yet) and make any changes you like to this new copy of your feature branch (not your old one!)
3. When ready, push your changes and open a new PR for the revert-revert- branch with the repaired version of your feature
4. Merge when ready

Reverting a commit

So far, these steps have assumed the change that broke prod came from a PR. But what if you committed directly to main?

That's a simpler fix. You'll just need to revert that bad commit (no PR required):

git checkout main
git log # find the hash of your bad commit ("q" to exit)
git revert HASH
git log # admire your new "Revert X" commit (optional)
git push

Or using Lazygit:

1. Go to the Branches panel and check out main
2. Go to the Commits panel and highlight the bad commit
3. Press t to revert your commit (you'll see a new commit appear reversing your bad one)
4. Press P to push the new commit to your remote branch

Try, try again

These steps have helped me out of multiple “oh 💩” moments at work. I hope they help you too!

Reverting your reversion may seem a bit strange the first time you do it, but hopefully it will make sense as you think about it. 🙂

Related

• Reverting a pull request • GitHub Docs
• How to PR and merge again after reverting PR using Github Revert Button • StackOverflow
• Undo a Git Commit • Justin Joyce

Learning how to configure Neovim can be overwhelming. One way to get started is to install a few pre-built configurations (like LazyVim, NvChad, AstroNvim, LunarVim or the official Neovim Kickstart) and see what you like.

Most installation instructions will tell you to replace everything in your ~/.config/nvim directory with the new configuration. But once you do, you lose the ability to launch Neovim with your previous config.

With that approach, you can only have one Neovim config installed at a time.

But what if you want to compare two configs? Or maintain different configs for different purposes (e.g. one for work and one for personal projects)?

Install each config in its own directory

To be able to use more than one config, you'll need to make a couple changes to your setup:

1. Instead of installing a new configuration in ~/.config/nvim, install it in a custom ~/.config subdirectory
2. Each time you open Neovim, specify which config you want by setting the NVIM_APPNAME environment variable in your launch command

For example, assuming you've installed LazyVim in ~/.config/nvim-lazyvim, you'd launch it with this command:

$ NVIM_APPNAME=nvim-lazyvim nvim

Neovim uses NVIM_APPNAME to determine which config directory to load. If you don't include it (or set it to an invalid value), Neovim will use the default config in ~/.config/nvim.

Switching configs using alias, select or fzf

Lets assume your ~/.config directory includes these subdirectories:

~/.config
├── nvim-astrovim
│   └── init.lua
├── nvim-kickstart
│   ├── init.lua
│   └── lua
│       ├── custom
│       └── kickstart
├── nvim-lazyvim
│   ├── init.lua
│   └── lua
│       ├── config
│       └── plugins
├── nvim-lunarvim
│   └── config.lua
└── nvim-nvchad
│   ├── init.lua
│   └── lua
│       ├── core
│       ├── custom
│       └── plugins
└── nvim

To quickly open Neovim using each config, you could create an alias for each launch command:

alias v='nvim' # default Neovim config
alias vz='NVIM_APPNAME=nvim-lazyvim nvim' # LazyVim
alias vc='NVIM_APPNAME=nvim-nvchad nvim' # NvChad
alias vk='NVIM_APPNAME=nvim-kickstart nvim' # Kickstart
alias va='NVIM_APPNAME=nvim-astrovim nvim' # AstroVim
alias vl='NVIM_APPNAME=nvim-lunarvim nvim' # LunarVim

Or use select to list your configs so you can choose one:

vv() {
  select config in lazyvim kickstart nvchad astrovim lunarvim
  do NVIM_APPNAME=nvim-$config nvim $@; break; done
}

Which would produce a menu like this:

Or you could get fancy and use a fuzzy finder like fzf to do the same thing:

vv() {
  # Assumes all configs exist in directories named ~/.config/nvim-*
  local config=$(fd --max-depth 1 --glob 'nvim-*' ~/.config | fzf --prompt="Neovim Configs > " --height=~50% --layout=reverse --border --exit-0)

  # If I exit fzf without selecting a config, don't open Neovim
  [[ -z $config ]] && echo "No config selected" && return

  # Open Neovim with the selected config
  NVIM_APPNAME=$(basename $config) nvim $@
}

Here's what that fzf menu would look like:

Conclusion

Configuring Neovim can be daunting, but being able to install a few configs and compare them is a great way to gather inspiration for your own custom config.

Good luck!

Related

• NVIM_APPNAME • Neovim docs
• Neovim Config Switcher • Elijah Manor
• Neovim Switcher Gist • Elijah Manor
• Lazyman: Neovim Configuration Manager • Ronald Record
• You Should Use a Neovim Distro If You Are New • ThePrimeagen

If you spend time wondering what to write in the blank “Description” field every time you open a pull request, adding a PR template to your repo will speed you up.

By pre-filling that “Description” field with a few quick prompts to answer, you’ll always know what to write and your reviewers will always get the info they need.

How to add a PR template

1. Create a folder called .github at the root of your repo
2. Create a file called .github/pull_request_template.md
3. Use GitHub’s Markdown syntax to add any prompts you think will be useful to that file

That’s it! Each time you open a pull request, the description section will now be pre-filled with your template.

What to include in your template?

Feel free to add any prompts you like! I prefer to keep it simple:

## ✅ What

<!-- A brief description of the changes in this PR. -->

## 🤔 Why

<!-- A brief description of why we want these changes. -->

## 👩‍🔬 How to validate

<!-- Step-by-step instructions for how reviewers can verify these changes work as expected. -->

## 🔖 Related

- [Jira task](url)
- [Slack thread](url)

(Those comments are only visible while editing.)

Don’t overdo it

Whatever you do, don’t make your PR template too long. If you notice your team keeps skipping certain prompts, don’t hesitate to remove them. Only include prompts that actually save everyone time.

If your reviewers always have to ask which ticket each PR relates to, adding a ticket link prompt to the template is probably a good idea. But adding 50 well-intentioned checkboxes will only teach your team to ignore the template entirely.

So, keep it short. 😊

Related

• Creating a pull request template for your repository • GitHub Docs
• Create a GitHub PR Template • egghead.io
• GitHub Markdown Cheatsheet • Adam Pritchard
• Writing on GitHub • Github Docs

Sometimes you want to upload a video, but you can’t because its file size is too large. When that happens, you can use ffmpeg to shrink the video size with one command.

Here’s how:

1. Install Homebrew

/bin/bash -c "$(curl -fsSL <https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh>)"

2. Install ffmpeg

brew install ffmpeg

(Or download FFmpeg for your OS.)

3. Run ffmpeg on your file

# General pattern
ffmpeg -i path/to/original/file path/to/new/file

# Example
ffmpeg -i "Desktop/Screen Recording 2022-11-08 at 4.30.40 PM.mov" Desktop/code-walkthrough.mp4

That’s it — I was able to shrink a 10-minute macOS screencast from 1.4 GB to 148 MB just by running that command and waiting 6 minutes.

Where to go from here

This approach works best when your final video quality doesn’t need to be very high.

If you’re creating high-resolution content, you’ll want to dive deeper into ffmpeg’s configuration options or use a tool like Handbrake to help you select the quality you need.

Related

• How to Reduce Video Size With FFmpeg — John Otieno

CSS filters are great

CSS filters are super cool. They can take something that looks like this…

…and make it look like this:

Love it. Effects like these are the fun part of writing CSS.

And blur() is just one of many fun filter options. The rest are listed here.

CSS filters become especially useful when you want to animate items on and off the page, or in and out of the foreground. For example, you may want to gradually blur items as they animate out. Or gradually blur the page background when a user opens a dialog or modal.

Safari woes

Unfortunately, once you start animating the CSS filter property, you’re going to notice your animation is awfully choppy in Safari.

I ran into this issue at work while animating CSS filters. And I recently heard Scott Tolinski lament on Twitter and the Syntax podcast that he was excited about using CSS filters but was forced to abandon them because of this Safari rendering issue.

But there’s a fix! That’s why I’m writing this post, if only to help Scotty. 😎

The fix

To fix the issue in Safari, take this…

.blurry {
  filter: blur(4px);
  transition: filter 0.3s ease-in-out;
}

…and change it to this:

.blurry {
  filter: blur(4px);
  transition: filter 0.3s ease-in-out;
  transform: translateZ(0);
}

That’s it. Your animations will run smoothly in Safari now.

How does translateZ help?

The basic reason translateZ(ANY_VALUE) makes the animation run smoothly is that it tells Safari to render the animation using the GPU instead of the CPU.

Without Without translateZ (or a similar hint like translate3d), Safari will try to animate your blur() filter using the CPU, which isn’t nearly as good at rendering complicated graphics. For graphics-intensive tasks like animating a filter effect, the GPU will always do a much better job.

Help us out, Safari

Chrome and other non-Safari browsers automatically hardware-accelerate animations like this by engaging the GPU, but Safari still requires you to add extra declarations like translateZ() to get the same result.

Hopefully the Safari team will fix that soon so Scott can animate all the CSS filters he likes without needing workarounds like this one.

Until then, translateZ() is your friend.

Related

• CSS Filter Effects landing in WebKit — mentions the need for translateZ() back in 2011 • Chrome Developers
• Increase Your Site’s Performance with Hardware-Accelerated CSS • Treehouse
• Improving HTML5 App Performance with GPU Accelerated CSS Transitions • Urban Insight
• CSS GPU Animation: Doing It Right • Smashing Magazine
• Web Performance Fundamentals — calls out that translateZ() is still needed to get hardware accelerated CSS animations on some platforms • MDN Web Docs
• Andy Ngo tweet — suggests translate3d() version of same trick

A factory function is a function that returns a new object. The key feature of a factory is that its only job is to pump out those items, just like an actual factory.

The place I use factories most often is when I want to return an object with methods but use closures to create private methods and values. Even better, I never have to even think about the this{:js} keyword.

In other words, you can write them instead of classes!

I found a good use case for this technique today while adding extra properties and helper methods to blocks I’d fetched from the Notion API, so I wanted to give Kyle a shout-out for the timely assist.

Check out Kyle's blog for lots of other interesting coding tips.

If an image makes you laugh or cry, ideally the alt text will do the same:

The relevant parts of an image aren't limited to the cold hard facts. Images can make you feel a particular way, and that's something that should be made available to a screen reader user.

It isn't just a head-shot of me. I'm doing a thing. I'm peering from behind a plant and pulling a bit of a silly face. There's humour expressed in the image. I'm not saying that it's going to win any comedy awards, but the image expresses a particular tone, and that matters. So, it should go in the alt: “Head-shot of me, a pale white guy, wearing glasses, grinning slightly, and partially hiding behind a plant”.

Let the creative writing begin!

After that change, the everything in the tests reads at a single level of abstraction (namely, how would a user interact with this component?) rather than alternating between what the computer is doing and what the user is doing. The result is easier to understand.

Basically, you just want to act as if you’re sitting next to the person who is using the system you’re testing…just imagine how you’d explain to them what behaviour you’re testing and then write your tests using similar words.

I think this advice would apply just as well to non-test code. If you notice a code block mixes levels of abstraction, try extracting the lower level bits so the resulting code reads as a continuous train of thought from a single point of view.

For more, check out Sam's complete video.

[Video: https://m.youtube.com/watch?v=G_0yKeh0Sf0&t=303s]

This post originally appeared on the ecobee engineering blog.

My team at ecobee spent months debugging build failures on the staging site for ecobee.com until we finally discovered a way to solve them with automated Slack notifications.

Production vs. staging

ecobee.com is an e-commerce website, which we populate with content from Contentful and Shopify.

Our designers and writers create new pages by building and assembling “blocks” of content in Contentful. As they work, they can preview their changes on a staging version of our site.

Here’s the live version of ecobee.com:

And here’s the staging version of ecobee.com:

The staging version looks very similar to the live version becae, well, that’s the point. It’s generated using the exact same codebase as the live site and all the content for both versions of the site come from Contentful.

If you aren't familiar, here’s what Contentful looks like:

This example is the “Page” entry that creates the homepage for ecobee.com. It’s basically a series of fields: some are required; some are not.

If you mark a field as required, Contentful will confirm that field is populated before it lets you publish the entry. (The same applies to other field validations like max length, number vs. text, etc.) And because the live site only consumes published entries, we know all the fields have been validated (e.g. no required fields are empty), and the data the website receives will match what we expect.

So far, so good.

However, the staging site is a little different: it connects to the same production Contentful data as the live site, but also includes any draft entries that have not yet been published.

That’s by design, so that editors can preview their in-progress changes on staging as they go.

But those draft entries have one major problem: their fields have not been validated.

Broken staging builds

The challenge we faced was that we kept seeing errors like this:

This the deploy log for a build of our staging website on Netlify. And over and over, we encountered errors saying Cannot read property X of null or Cannot read property X of undefined, which indicated that our codebase trusted a value existed (because its Contentful field was required) and tried to use it, but failed because that value was unexpectedly empty.

So, why didn’t Contentful make sure those required fields were populated?

Draft entries cannot be trusted

The basic reason is that unpublished draft entries (which staging consumes) are not validated by Contentful because Contentful only applies validations when an entry is published.

So, while our production builds were stable (since the live site only consumes validated, published entries), our staging builds were constantly exposed to the risk that a field coming from a draft entry could contain invalid data (or no data at all).

This issue was minor when our team only had one or two content editors. But as our team grew to include half a dozen or more editors working separately and constantly triggering new builds, incomplete data began to be hit staging on a daily basis.

Debugging in the dark

When one of these errors occurred, the staging website would stop updating, and one or more devs would have to put aside their work and jump into the build logs to locate an error like the one shown above.

Next came the hard part: trying to figure out which field in Contentful was responsible for the problem.

For example, all the error in the screenshot above says is that the Link component was passed an empty href. It doesn’t say which field in Contentful that empty value came from.

Trying to track down those empty fields was no fun. Each hunt took up a lot of time and involved a lot of head scratching and rooting through Contentful, searching for the root of each problem. Work ground to a halt for editors and developers while we hunted, sometimes for up to an hour.

We were to determined to free ourselves from these thankless debugging sessions.

Custom null checks everywhere

To do that, we started adding defensive null checks all over our codebase.

Even though we didn’t have to worry about the existence of required values in production, for the sake of our editors having a working preview site to look at, we realized we needed to treat required fields as unreliable and start guarding against their lack of existence.

So, we started adding things like this all over our codebase that intercepted empty values and identified exactly which Contentful field to fix:

const ReviewsResolver = ({
  data: { productReference, header, name = '' },
}: BlockResolverProps<ReviewsDataType>) => {
  const logError = useLogError()

  if (!productReference) {
    logError(
      `The Reviews entry named "${name}" cannot be displayed because its required field "Product reference" is empty.`,
    )

    return null
  }

  return <Reviews product={productReference} />
}

Slack to the rescue

We also created a helper, called logError(), to decide whether each error message should appear just in the console or also be posted to Slack:

// If this is prod, report the issue and break the build
if (context === 'PRODUCTION') {
  const prodImpact = 'The live site will not update until this issue is resolved.'

  const prodMessage = formatMessage(
    `🚨 PRODUCTION ERROR 🚨\\n\\n${message}\\n\\n${prodImpact}`,
  )

  postToSlack(prodMessage).then(() => {
    throw new Error(prodMessage)
  })
}

// If this is staging, report the issue and continue the build
if (context === 'STAGING') {
  const stagingImpact =
    'The staging site will continue to update while this issue gets resolved.'

  const stagingMessage = formatMessage(
    `⚠️ STAGING ERROR ⚠️\\n\\n${message}\\n\\n${stagingImpact}`,
  )

  postToSlack(stagingMessage)
  console.error(stagingMessage)

  return
}

// In all other environments, just log the issue and continue the build
console.error(formatMessage(`\\n\\n⚠️ ${message}\\n\\n`))

So, we ask, “Are we on the production site? Or staging? Or just in development?”, and based on the answer we decide if we should to stop the build (production only) and/or post the error to Slack (production and staging only). In all environments, we also log the error to the console.

We then created a dedicated Slack channel and invited all our editors to it, created a Slack bot, and used the Slack Web API to automatically post these Slack messages for us:

import type { WebClient } from '@slack/web-api'

import { isTest } from '../../src/utils/getEnv'

const isSSR = typeof window === 'undefined'

async function postToSlack(text: string, channel = '#dotcom-cms-errors') {
  // Only post during a build that isn't part of a test run
  if (!isSSR || isTest || !global.slackClient) {
    return
  }

  try {
    await global.slackClient.chat.postMessage({ channel, text })
  } catch (error) {
    console.error(`\\n[postToSlack]: ${error}\\n`)
  }
}

Now, whenever a build error occurs and is caused by a Contentful data entry problem we know about, the message goes straight to that editor-focused channel so they can can hop into Contentful and fix the issue without dev assistance:

An endless process

But, we weren’t done.

We started noticing that after we had fixed the 10 or so errors we had seen over and over, there were 10 or so new ones that we hadn’t previously seen because they never had a chance to appear because the other errors were happening first.

It slowly dawned on us that with our current approach of inserting custom null checks and error messages in individual components, this problem would never end. We’d have to null check every single value we considered required and continue doing that whenever we wrote new code.

That was not going to be scalable for us in the future.

Automated validation step

So, we came up with a better idea.

To avoid having to spread null checks all over our codebase and worry about this forever, early in our build we now query all of the data from Contentful that our website will use along with the rules for every field, and then compare the two.

We take each field and its rules and pass it to a function that asks if it’s invalid. For example, for requiredness, we check if the field’s rules say it’s required and if its value is empty. If both are true at the same time, the field is invalid:

function isEmptyRequiredField(fieldRules: ContentFields, fieldValue: unknown) {
  const isRequired = 'required' in fieldRules && !!fieldRules.required

  // We don't want to check falsiness here, since that would flag fields intentionally set to 0
  const isEmpty = fieldValue == null

  return isRequired && isEmpty
}

function detectFieldInvalidity(
  rules: ContentTypeIdsAndFields,
  contentTypeId: string,
  fieldName: string,
  fieldValue: unknown,
): string | null {
  const fieldRules = getFieldRules(rules, contentTypeId, fieldName)

  if (isEmptyRequiredField(fieldRules, fieldValue)) {
    return 'is required but empty'
  }

  if (isInvalidRichTextField(fieldValue)) {
    return 'contains invalid HTML'
  }

  return null
}

Then, we have a single place where we call our logError() helper as many times as necessary:

const isInvalidReason = detectFieldInvalidity(
  rules,
  contentTypeId,
  fieldName,
  entry[fieldName],
)

if (isInvalidReason) {
  const contentTypeName = startCase(contentTypeId)

  const entryName = entry?.name || entry?.title || ''

  const contentTypeAndEntryName = entryName
    ? `The Contentful ${contentTypeName} entry named "${entryName}"`
    : `An unnamed Contentful ${contentTypeName} entry with entry ID "${entry.contentful_id}"`

  const locale = entry?.node_locale
    ? (String(entry.node_locale).toLowerCase() as 'en-us' | 'en-ca')
    : undefined

  const message = `${contentTypeAndEntryName} cannot be displayed because its field "${fieldName}" ${isInvalidReason}.\\n\\nPlease update this field and publish your changes.`

  logError(message, locale, entry.contentful_id)
}

The updated version of these Slack error notifications looks like this:

The updated template tells editors the name of the broken Contentful entry, which field in the entry has the problem, what the problem is, how to fix it, a link to the entry, and context about the environment where the build error occurred.

Happy editors and developers

We've been living with this solution for the past few months and our editors and developers are both much happier!

There are no more confusing debugging sessions. Whenever a staging build breaks, the solution is posted to Slack a second later and an editor can fix it themselves within a minute.

This has eliminated an annoying source context switching and restored hours of developer and editor productivity each week.

Please share your ideas!

So, that’s how the ".com" team at ecobee is currently tackling the challenge of using the same codebase for production and staging but sending that codebase different data in each environment.

If you’ve encountered this issue before and have tips or best practices that might help us, please let us know! We realize this is likely a common problem for sites with a content management system, so we’d love to hear how you approached solving this issue.

Here's a trick I often find helpful.

Bad array. Very, very bad.

You have an array of whatever:

const array = [{ stuff }, { moreStuff }, ...]

But hiding in that array are some unusable null or undefined values:

const array = [{ good }, null, { great }, undefined]

Those empty values might be a sneaky little gift from an API. Or you may have left them there yourself while validating the original data. Either way, you've got a problem.

Looping over null data

If you try to perform actions on every item in the array, you'll run into errors when you hit those null and undefined items:

const newArray = array.map(item => {
  // Of course this will work, wheeee...
  const assumption = item.thing
})

// Oh noooo...
// 🚨 Error: Cannot read property "thing" of undefined.
// 🚨 Error: Cannot read property "thing" of null.

Illegal! Now you're a criminal. Before you interact with an item, you need to make sure it exists.

Null checks?

You could confirm each item exists by performing a null check before you use it:

const newArray = array.map(item => {
  // Life has made me cautious.
  if (!item) {
    return item // Just forget it
  }

  // If we get this far, item exists.
  const assumption = item.thing
})

Buuut, now your code is getting cluttered. And worse, those dangerous empty values will be passed along to newArray. So when newArray is used, another round of suspicious null checks will be needed.

The truth and only the truth

Want something better?

Here's my favourite way to quickly remove all empty items from an array:

const array = [{ good }, null, { great }, undefined]

const truthyArray = array.filter(Boolean)
// truthyArray = [{ good }, { great }]

The filter(Boolean) step does the following:

1. Passes each item in the array to the Boolean() object
2. The Boolean() object coerces each item to true or false depending on whether it's truthy or falsy
3. If the item is truthy, we keep it

Where did item go?

I love how concise filter(Boolean) is, but it might look strange that we aren't explicitly passing item to Boolean().

The main thing to know is that, in JavaScript, this:

array.filter(item => Boolean(item))

is exactly the same as this:

array.filter(Boolean)

The second version is just written in a "tacit" or "point-free" style. We don't name each item and pass it into Boolean(), but JavaScript understands that Boolean() takes one argument, so it takes the argument filter() exposes and passes it to Boolean for you.

If you find the first version easier to understand, use it! Readable code is more important than fancy tricks.

Safer mapping

With our new tool, we can remove the null checks from above and chain a filtering step instead:

const newArray = array.filter(Boolean).map(item => {
  // Item is always truthy!
  const assumption = item.thing
})

Now, our map() can focus on what it's trying to do, and we've removed the empty values from our pipeline forever.

Hope that helps!

This is the tenth video in our beginner series exploring GatsbyJS and how to use it to easily build performant apps and websites:

1. What is Gatsby?
2. Starting a New Gatsby Project
3. Gatsby's Default Files and Folders
4. Writing HTML in a Gatsby Project
5. Adding Pages to a Gatsby Project
6. Wrapping Pages in a Layout Component
7. Writing CSS in a Gatsby Project
8. Writing CSS-in-JS in a Gatsby Project
9. Adding Content to a Gatsby Project
10. Using GraphQL with Gatsby (this post)

In this video, we explore how GraphQL works and practice using it with Gatsby to query our content.

[Video: https://www.youtube.com/watch?v=IaorT4-efuU]

For more, check out the entire playlist on YouTube.

This is the ninth video in our beginner series exploring GatsbyJS and how to use it to easily build performant apps and websites:

1. What is Gatsby?
2. Starting a New Gatsby Project
3. Gatsby's Default Files and Folders
4. Writing HTML in a Gatsby Project
5. Adding Pages to a Gatsby Project
6. Wrapping Pages in a Layout Component
7. Writing CSS in a Gatsby Project
8. Writing CSS-in-JS in a Gatsby Project
9. Adding Content to a Gatsby Project (this post)
10. Using GraphQL with Gatsby

In this video, we explore a few ways we can add content to a Gatsby project and discuss the advantages and disadvantages of each approach.

[Video: https://youtu.be/MPqEU9ywgW8]

For more, check out the entire playlist on YouTube.

This is the eighth video in our beginner series exploring GatsbyJS and how to use it to easily build performant apps and websites:

1. What is Gatsby?
2. Starting a New Gatsby Project
3. Gatsby's Default Files and Folders
4. Writing HTML in a Gatsby Project
5. Adding Pages to a Gatsby Project
6. Wrapping Pages in a Layout Component
7. Writing CSS in a Gatsby Project
8. Writing CSS-in-JS in a Gatsby Project (this post)
9. Adding Content to a Gatsby Project
10. Using GraphQL with Gatsby

In this video, we explore CSS-in-JS and how libraries like CSS Modules and Styled Components can make it easier to add and remove styles from your project.

[Video: https://youtu.be/RwUn0Trhyyk]

For more, check out the entire playlist on YouTube.

This is the seventh video in our beginner series exploring GatsbyJS and how to use it to easily build performant apps and websites:

1. What is Gatsby?
2. Starting a New Gatsby Project
3. Gatsby's Default Files and Folders
4. Writing HTML in a Gatsby Project
5. Adding Pages to a Gatsby Project
6. Wrapping Pages in a Layout Component
7. Writing CSS in a Gatsby Project (this post)
8. Writing CSS-in-JS in a Gatsby Project
9. Adding Content to a Gatsby Project
10. Using GraphQL with Gatsby

In this video, we explore traditional approaches to writing CSS, like inline styles, vanilla CSS, Sass, and PostCSS, and walk through how each one works in a Gatsby project.

[Video: https://youtu.be/hAzt8BIwXiE]

For more, check out the entire playlist on YouTube.

This is the sixth video in our beginner series exploring GatsbyJS and how to use it to easily build performant apps and websites:

1. What is Gatsby?
2. Starting a New Gatsby Project
3. Gatsby's Default Files and Folders
4. Writing HTML in a Gatsby Project
5. Adding Pages to a Gatsby Project
6. Wrapping Pages in a Layout Component (this post)
7. Writing CSS in a Gatsby Project
8. Writing CSS-in-JS in a Gatsby Project
9. Adding Content to a Gatsby Project
10. Using GraphQL with Gatsby

In this video, we learn how to use a Layout component to share common sections and styling between pages.

[Video: https://youtu.be/dvorpyrTI2g]

For more, check out the entire playlist on YouTube.

This is the fifth video in our beginner series exploring GatsbyJS and how to use it to easily build performant apps and websites:

1. What is Gatsby?
2. Starting a New Gatsby Project
3. Gatsby's Default Files and Folders
4. Writing HTML in a Gatsby Project
5. Adding Pages to a Gatsby Project (this post)
6. Wrapping Pages in a Layout Component
7. Writing CSS in a Gatsby Project
8. Writing CSS-in-JS in a Gatsby Project
9. Adding Content to a Gatsby Project
10. Using GraphQL with Gatsby

In this video, we learn how to add new pages to a Gatsby project and how to navigate between them using Gatsby's Link component.

[Video: https://youtu.be/ktHshp6SKXc]

For more, check out the entire playlist on YouTube.

This is the fourth video in our beginner series exploring GatsbyJS and how to use it to easily build performant apps and websites:

1. What is Gatsby?
2. Starting a New Gatsby Project
3. Gatsby's Default Files and Folders
4. Writing HTML in a Gatsby Project (this post)
5. Adding Pages to a Gatsby Project
6. Wrapping Pages in a Layout Component
7. Writing CSS in a Gatsby Project
8. Writing CSS-in-JS in a Gatsby Project
9. Adding Content to a Gatsby Project
10. Using GraphQL with Gatsby

In this video, we take our first look at React and how we can use it to split our HTML markup into reusable components.

[Video: https://youtu.be/MMFvFtpyoIQ]

For more, check out the entire playlist on YouTube.

This is the third video in our beginner series exploring GatsbyJS and how to use it to easily build performant apps and websites:

1. What is Gatsby?
2. Starting a New Gatsby Project
3. Gatsby's Default Files and Folders (this post)
4. Writing HTML in a Gatsby Project
5. Adding Pages to a Gatsby Project
6. Wrapping Pages in a Layout Component
7. Writing CSS in a Gatsby Project
8. Writing CSS-in-JS in a Gatsby Project
9. Adding Content to a Gatsby Project
10. Using GraphQL with Gatsby

In this video, we walk through each file and folder that comes with a new Gatsby project and talk about what each one is for so you can start customizing your site.

[Video: https://youtu.be/o06e7TIJR9Y]

For more, check out the entire playlist on YouTube.

This is the second video in our beginner series exploring GatsbyJS and how to use it to easily build performant apps and websites:

1. What is Gatsby?
2. Starting a New Gatsby Project (this post)
3. Gatsby's Default Files and Folders
4. Writing HTML in a Gatsby Project
5. Adding Pages to a Gatsby Project
6. Wrapping Pages in a Layout Component
7. Writing CSS in a Gatsby Project
8. Writing CSS-in-JS in a Gatsby Project
9. Adding Content to a Gatsby Project
10. Using GraphQL with Gatsby

In this video, we see how easy it is to get a new Gatsby project up and running on your computer.

[Video: https://youtu.be/afeqjonYgs8]

For more, check out the entire playlist on YouTube.

This video kicks off a new beginner series exploring GatsbyJS and how to use it to easily build performant apps and websites:

1. What is Gatsby? (this post)
2. Starting a New Gatsby Project
3. Gatsby's Default Files and Folders
4. Writing HTML in a Gatsby Project
5. Adding Pages to a Gatsby Project
6. Wrapping Pages in a Layout Component
7. Writing CSS in a Gatsby Project
8. Writing CSS-in-JS in a Gatsby Project
9. Adding Content to a Gatsby Project
10. Using GraphQL with Gatsby

The first video explores the Gatsby ecosystem and how Gatsby simplifies web development by combining the best features of React and static HTML.

[Video: https://youtu.be/jAa1wh5ATm0]

For more, check out the entire playlist on YouTube.

Earlier this month, I launched Gatsby Tutorials. 🎉

Gatsby Tutorials is a filterable and sortable directory of all the GatsbyJS learning resources currently available online. It aims to gather every video, audio and written Gatsby tutorial into one place so you can easily find it when you need it.

Oh, and it’s free and open source.

Umm…why did you make this?

Four main reasons:

1. Gatsby is awesome and I want to help other coders learn it. 👍
2. I'd like high-quality Gatsby tutorials to reach a wider audience. 🎙️
3. It can be hard to locate useful resources when you need them. 🧐
4. Subscribing separately to every Gatsby-related Twitter feed, blog and YouTube channel can be overwhelming. 😩

How does it work?

Gatsby Tutorials gathers every Gatsby learning resource in one place so you don’t have to scroll through endless feeds or wander the internet for help.

Instead, just visit the directory and search or filter the list for the topic you’re interested in. The tutorials you need will magically appear:

Gatsby Tutorials also assists new tutorial creators by making their content more discoverable—a win-win for content creators and Gatsby learners alike.

How do I contribute?

If you notice a resource is missing from the list, please add it! This project will work best if the community helps keep it up to date. 🙌

Just click the "Add a tutorial" link and you’ll be taken to the quick instructions.

If you’d like to suggest a tutorial without adding it yourself, that's helpful too! Just copy the link into a new issue and I’ll add it for you.

Roadmap

Gatsby Tutorials currently includes 157 video, audio and written tutorials that can be discovered by browsing, searching, or filtering by format, topic, author and source.

Here are some additional features coming soon:

Better filters 🏷

• Filter tutorials by language
• Filter by multiple topics at once
• Cancel a filter by clicking it again

Better performance ⚡️

• Paginate the tutorials (and search results)
• Generate the filter lists at build time

Better search 🕵️‍♂️

• Implement fuzzy search to enable non-sequential search terms
• Highlight search string in search results

Wrap-up

If you have questions or ideas for making Gatsby Tutorials more useful, please share your thoughts on Twitter or by opening a new issue in the Gatsby Tutorials repo.

Happy coding!

While you can build basic websites with nothing more than a text editor and browser, you may want to up your game by adding a JavaScript framework like React or Vue and useful tools like Git to your workflow.

But wait! Your Mac isn’t ready. Before diving in, you’ll need to install a few items to that will save you from confusing errors later. 😩

This article will guide you through the minimum setup you'll need to get up and running with JavaScript-based web development on your Mac.

Let’s go!

Update Your Mac

Before installing any new software, follow these instructions from Apple to upgrade macOS and your current software to the latest version.

Choose a Terminal App

Since you'll be interacting with your Mac using the command line in this article, you'll need a terminal app.

Any of the following are good options:

• iTerm2
• Kitty
• Alacritty
• WezTerm
• Visual Studio Code's integrated terminal
• Terminal (the default app that comes with your Mac)

If you aren’t sure which one to pick, start with iTerm2.

Command Line Developer Tools

The first thing you'll need to install from the command line are your Mac's command line developer tools. Installing these now will prevent weird errors later.

To check if the tools are already installed, type the following command in your terminal app and hit return:

xcode-select --version

If the result is not a version number, install the tools with this command:

xcode-select --install

A dialog will appear asking if you'd like to install the tools. Click Install and the package will download and install itself.

When the installation finishes, confirm the tools are now installed by rerunning the first command:

xcode-select --version

The result should now be a version number.

Homebrew

Instead of installing the next few tools by going to each tool's website, finding the download page, clicking the download link, unzipping the files, and manually running the installer, we’re going to use Homebrew.

Homebrew is a tool that lets you install, update and uninstall software on your Mac from the command line. This is faster and safer than the manual approach and generally makes your development life easier.

First, check if Homebrew is already installed:

brew --version

If you don't see a version number, install Homebrew with this command:

/bin/bash -c "$(curl -fsSL <https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh>)"

I promise that's the weirdest command you'll see in this article! 😅 Thanks to Homebrew, the rest are simple.

Before moving on, confirm Homebrew is now installed:

brew --version

Node and npm

Node.js is a tool that allows your Mac to run JavaScript code outside of a web browser. If you want to run a JavaScript framework like React or Vue on your Mac, you’ll need to have Node installed.

Node also includes npm (the Node Package Manager), which gives you access to a giant library of free code you can download and use in your projects.

First, check if node is already installed:

node --version

If not, install it with Homebrew:

brew install node

Finally, confirm node and npm are now installed:

node --version

npm --version

Git

Git is a tool that helps you track changes to your code and work with other developers on shared projects.

Using Git on all every project is a great habit to develop and will prepare you for future projects where Git may be required. Some tools (like Gatsby) also depend on Git being installed on your Mac, so you’ll need it even if you don’t plan to add it to your workflow.

Once again, start by checking if git is already installed:

git --version

If not, install it:

brew install git

And confirm the installation worked:

git --version

Update Everything

Once in a while, run the following command and everything you’ve installed with Homebrew will update automatically:

brew update && brew upgrade && brew cleanup && brew doctor

That one command is all you need to keep your system up to date. 🙌 I usually run it each time I start a new project, but feel free to do so whenever you like.

(When you run this command, if Homebrew suggests additional commands for you to run, go ahead and run them. If a command begins with sudo and you are prompted for a password, use your Mac’s admin password.)

That’s it for the command line!

Code Editor

While you can write code in any text editor, using one that highlights and validates your code will make your life much easier.

Any of the following are good options:

• Visual Studio Code
• Atom
• Sublime Text

If you’re just getting started, choose Visual Studio Code.

Browser

As you code, it helps to view the app or website you’re building in a browser to confirm it works. While you can use any browser for this, some include extra developer tools that show you details about your code and how to improve it.

Either of the following are good options:

• Chrome
• Firefox

If you’re just getting started, choose Chrome.

Finder

A quick tip here: you’ll want to show the files your Mac hides by default. (For example, git files are automatically hidden, but sometimes you’ll want to edit them.)

The easiest way to show your Mac’s hidden files is with the keyboard shortcut command-shift-period. This will alternate between showing and hiding these files so you can access them when you need them.

Conclusion

You're all set! 🎉

That’s all you need to get up and running with JavaScript-based frontend development on your Mac.

To prevent confusion, I left out any items that aren’t strictly required. If you'd like to dive deeper into optional ways you can further customize your Mac for web development, check out the links below.

Further Reading

• Setting up a Brand New Mac for Development • Tania Rascia
• Setting up a MacBook for Front-End Development • Ben Honeywill