| title | Prodverbs: Production Proverbs |
|---|---|
| layout | default |
Always have a backup plan.
Luck is not a factor. Hope is not a strategy. Fear is not an option. -- James Cameron
With a sufficient number of users of an API, it does not matter what you promise in the contract: all observable behaviors of your system will be depended on by somebody.
Not only will users depend on all features, they will assume that they have an SLO equivalent to observed behavior.
There's always a baseline expectation of performance. Without an explicit SLO, past performance becomes implicit SLO.
Referred to as Hettich's Epigram (twitter)
See also Hyrum's Law
Unless you restore your data and verify you can read it with your production systems, you don't know if your backup will work.
A global system is a single system or instance that everything else depends on. If it fails, everything that depends on it might as well. Isolate your systems from hard dependecies on global systems, so they can fail independently.